Creating a web application with Goanna

bayt — Sun, 04 Nov 2007 01:59:24 -0800

This is a short tutorial for building web applications with Goanna

It covers the following topics

Installing all the software you need
Getting to know Goanna
A short description on how to develop a web application with Goanna

How it all works

Goanna offers many different services and layers of abstraction.

For this tutorial we will use the Goanna web application library. The Goanna web application library is a framework which allows to build web applications. The framework itself builds upon the Goanna fast cgi interface. And on the base of it all is an eposix server. In order to scale up with the traffic and to be able to host other sites than only the ones created with Goanna we will use an Apache web server.

The Apache web server interacts with our web application via the fast cgi interface.

Installation

These installation instructions are meant for a (k)ubuntu system. At the time of writing ubuntu Feisty Fawn (7.04) was the current stable version.

First thing first, get a version of EiffelStudio. At the time of writing EiffelStudio 5 (5.7.64493 GPL Edition) was the current stable version. You can find EiffelStudio at http://eiffelstudio.origo.ethz.ch/downloads.

After installing EiffelStudio according to the instructions make sure that the following environment variables are set:

export ISE_EIFFEL={location of EiffelStudio in file system}
export ISE_PLATFORM=linux-x86
export PATH=$PATH:$ISE_EIFFEL/studio/spec/$ISE_PLATFORM/bin
export ISE_C_Compiler=gcc

Next thing you need to get is a few Eiffel libraries

Gobo

GOBO: get the latest version from http://www.gobosoft.com
set the following environment variables
- export GOBO=location of gobo in file system
- export GOBO_EIFFEL=ise
- export GOBO_CC=gcc
- export GOBO_OS=unix
for further installation notes read the readme.txt
add the folder '$GOBO/bin' to your path.

Eposix

eposix: http://www.berenddeboer.net/eposix/
set the following environment variables
- export EPOSIX={location of eposix in file system}
- export EPOSIX_LIB=$EPOSIX/lib
for further installation notes read the file $EPOSIX/INSTALL

Log4e

log4e: svn co https://svn.origo.ethz.ch/goanna/trunk/log4e log4e
- set the environment variable $LOG4E to the location of log4e in you file system
- cd $LOG4E
- geant install

Goanna

Goanna: svn co https://svn.origo.ethz.ch/goanna/trunk/goanna goanna
- set the environment variable $GOANNA to the location of Goanna in you file system
- cd $Goanna/library/utility
- geant install

We are still not done

Install the following packages with your favorite package manager or with aptitude.

Apache2
mod_fastcgi

And finally you need to get your fingers dirty

Download DevKit http://www.fastcgi.com/dist/fcgi.tar.gz A how-to-install-source-balls-under-linux can be found here: http://www.tuxfiles.org/linuxhelp/softinstall.html .

The first steps

After you have installed everything open a browser and type localhost onto the location bar. The Apache place holder page should show up. If this is not the case try a good old windows trick, even if we are running (k)ubuntu. Restart. If this doesn't help, you will find a lot of help out in the wild of the WWW.

Now that we have our primary server up and running it is time to get in touch with Goanna.

Create a new directory for the web application. Create an environment variable which points to this directory. For the rest of this tutorial we will refer to this directory as $WEB_APP.

Goanna comes with a template for building web applications. Therefore we don't have to write the whole application form scratch but can start right away with the interesting part. The template can be found at $Goanna/example/application. So copy the content of $Goanna/example/application into your $WEB_APP folder. If you have an SVN version of Goanna remove any .svn directory from the copied template.

If you like you can change the name of the application in the system.xace and the build.gant file. Once you have done this open a console, navigate to $Goanna and then run the command geant build. This will take some time (around 5 minutes). But once it is done, we have a ready to run web application. If you want to work with EiffelStudio you can find a *.ecf file in the $WEB_APP folder. Open the project in EiffelStudio and compile it. It is necessary to compile the project even though we just compiled it with the geant build tool. This is because EiffelStudio will generate a set of meta data about the project during compilation.

Now its time to start the web application. Either you can start it from within EiffelStudio or you can use the binary file which was generated by the geant build tool. It is located in the $WEB_APP folder.

Go back to your browser and type

localhost/fastcgi/demo/go_to.htm?question in you 
browser. If you get an error saying that the URL is not valid everything is ok. Remember, the 
Apache web server communicates with our web application via the fast-cgi interface. But it won't 
do this on its own, we first need to configure it to do so.

The setting for the Apache web server are stored in text files which can be found at 
/etc/apache2

. You will need super user rights in order to edit the settings. For  example you could open a text editor in super user mode and edit the files from within this  editor. Open the file /etc/apache2/httpd.conf and add the following lines:

LoadModule fastcgi_module /usr/lib/apache2/modules/mod_fastcgi.so

FastCgiExternalServer "$WEB_APP/EIFGENs/web_application/W_code/web_application" -host localhost:7878


You need to replace web_application with the name of your project

If you are not using EiffelStudio the fast cgi server is located at  
$WEB_APP/web_application

And finally open the file /etc/apache2/sites-available/default
 and add the  following lines:

Alias /demo "$WEB_APP/EIFGENs/web_application/W_code/web_application"
    <Directory "$WEB_APP/EIFGENs/web_application/W_code/">
    SetHandler fastcgi-script
    Options ExecCGI
    Order allow,deny
    Allow from all
    </Directory>
    TODO


You need to replace web_application with the name of your project

If you are not using EiffelStudio the fast cgi server is located at  $WEB_APP/web_application

Now everything is set up, all that is left to do is to restart the apache server so the new  settings are loaded. Go to a console and type sudo /etc/init.d/apache2 restart.

Head back to the browser and once again type localhost/fastcgi/demo/go_to.htm?question. This time, the page should be displayed.

Congratulations you just finished your first hands on experience with Goanna. Go ahead and play a little with the web page. Try some combinations of input.


A tour around town
This chapter is a guided tour through the example application. The tour will not go into great  depth but still we will see many things so bear with me.

Open your favorite Eiffel editor and open the example web application. The project contains quite  a lot of clusters, but all the user code resides in the root_cluster. Within the root cluster you  find two sub clusters containing the servlets and the parameters.

If you are not familiar with the java servlet, let me give you a brief introduction, for the  rest of you just skip the next paragraph.

The name servlets comes from a mixing of the words server and applets. So servlets are applets  which run on a server. Any servlet must implement three features, 
do_get(request, response), 
do_post(request, response) and do_head(request, response)
. Request is an object which  holds all the data about the requests. Similarly the response is an object which will hold the  generated response which is sent back to the user once the servlet is done with its processing.  The do_get, do_post and do_head features should implement the handling of the HTML  requests GET, POST and HEAD.

Now the servlets in Goanna do exactly the same thing as the java servlets - even the interface  is the same.

We start our tour at the APPLICATION_CONFIGURATION class. As the name suggests this class  holds configurations about the application. Take a look at GOA_APPLICATION_CONFIGURATION  for the documentation of the attributes. Besides the configuration this class also holds the  logic on which servlet is used to generate the response for a request. Take a short look at the  function next_page. The displayable servlet which is returned is used to render the  response.

The different properties which are set in the APPLICATION_CONFIGURATION are described in  the GOA_APPLICATION_CONFIGURATION.

Next we take a look at the APPLICATION_SERVER. The name might be a bit misleading, since  the class does not represent a server but it holds an instance of a eposix server and uses this  instance to communicate with the apache server. The class implements two features  command_line_ok and register_servlets. Command_line_ok does many things but  it does not check if the command_line is correct. Just leaves it as it is, it works even  if it doesn't do what the name suggest. Much more important for you is the  </e>register_servlet</e> method. It is responsible for registering all the servlets to the  SERVLET_MANAGER. I will explain later why it is important to register your servlets.  For now just remember, that you always have to update this feature if you add a new servlet.

A few classes remain in the root_cluster. There is the MESSAGE_CATALOG.  It is used as a STRING repository which allows to use a string in several different  places and only define it once. This also makes it easy to localize your web application by  simply adding another MESSAGE_CATALOG for another language.

The SESSION_STATUS is used to hold informations during a session. Information stored  in an instance of this class will be available to every request during a session. The  REQUEST_PROCESSING_RESULT is used to hold informations on the processing of this request.  It can also be used to hold informations which are only valid during the processing of this  request and not during the entire session. The purpose of the PARAMETER_PROCESSING_RESULT  is very similar to the purpose of the REQUEST_PROCESSING_RESULT except that it holds  information about a parameter rather than about a request. Especially  REQUEST_PROCESSING_RESULT is used heavily in the Goanna application framework. It is  passed as an argument to a lot of functions which makes it very interesting to use as a  container in order to pass information from one parts of the applications to an other during  the processing of a request.

The last class in the root folder is PROGRAMMING_LANGUAGE_SELECTION. It is part of the  example application and not of the Goanna framework.

Now that we came by all the classes in the root_cluster it is time to head into  the parts where the action happens.

Let's first have a look at the servlets. You find three classes in the servlet cluster. Two of  them should remind you of the pages which you have seen while taking a look at the web page.  The third is SHARED_SERVLETS we will come back to this class in a moment. So lets start  with the ANSWER_SERVLET. It inherits from GOA_DISPLAYABLE_SERVLET which means it  must implement the following four methods: ok_to_display, new_xml_document,  add_body, add_footer.

Ok_to_display should be called by the next_page method of the  APPLICATION_CONFIGURATION class. It is there to make sure, that it is ok to display the  servlet to the user. This could for example be used to prevent the displaying of a servlet if a  user is not logged in.

new_xml_document will return a newly created instance of GOA_XML_DOCUMENT. If you  take a closer look at the implementation of ANSWER_SERVLET you notice that  new_xml_document actually returns an instance of GOA_PAGE_XML_DOCUMENT. This is  legal since GOA_PAGE_XML_DOCUMENT is a subclass of GOA_XML_DOCUMENT. Besides just  creating a new instance of GOA_XML_DOCUMENT this feature already adds some elements to  the xml document. In this example only the &lt;html&gt;, &lt;head&gt;...&lt;/head&gt; and  &lt;body&gt; tags, but this can extended to add lets say a menu or a logo. So the  new_xml_document does also an implicit add_header.

The add_body and add_footer do just what they promise by their names. They are used to add the content of the page.

By now you may be wondering why we are creating xml documents if we want to return html as a  response to the request. Well the xml documents which are created will be transformed to html  after the servlet has processed the request. This allows to use the same output of the servlet  for different purposes, all that is needed is a rule to transform the xml into the requested  format.

The last feature is called name and returns a string. This string is used by Goanna to determine  if this servlet is suited to handle a request or not. But more on this later. For now it is ok  to think of the name as being an identifier which allows it for a request to specify which  servlet should be used to handle it. Since name is used as an id, it must be unique within  the application.

Lets move on to the QUESTION_SERVLET. The class looks very similar to the  ANSWER_SERVLET. There are two main differences, besides that the servlet generates  different xml output. The first is, that the last parameter in the call to  start_page_element in the feature new_xml_document is not Void.  This will start a &lt;form&gt; right after the &lt;body&gt; started. So if a page contains a  form, the last parameter in start_page_element must not be Void.

The other difference to the ANSWER_SERVLET is that the QUESTION_SERVLET redefines  make. This is necessary because the servlets wants to receive parameters. In Goanna each servlet  must specify which parameters it can handle and what kind of conditions are involved in  processing this parameters. This is done in the constructor. GOA_APPLICATION_SERVLET  offers you five lists where you can add a parameter. How a parameter is handled during the  processing of the request depends on the list to which it was added. Take a look at the lists  in the GOA_APPLICATION_SERVLET class.

We already reached the last class in this cluster: SHARED_SERVLETS. All this class does  is implementing the singleton pattern for every servlet in the application. Therefore a once  feature must be added for every servlet in the application.

Last but not least we reached the parameters cluster. Similar to the servlet cluster we find a  SHARED_PARAMTERS class. This class does the same for the parameters what the  SHARED_SERVLETS does for the servlets. Now besides the SHARED_PARAMETERS you find  four other classes in this cluster, and if you think back at the web page you should recognize  them. Every parameter which we had on the HTML page has a corresponding Eiffel class in the  cluster parameters.

Goanna provides a large set of deferred parameter base classes from which one can inherit when  implementing the parameters. When taking a look at the different parameters in the example one  can notice that even if each looks different they have several things in common. Each class  offers a method to store and to retrieve the value from the parameter into a persistent storage.  Also each parameter class has a name and a label_string. The name is used to uniquely identify  the parameter, and the label_string will be displayed besides the input field. It is possible to  use the same parameter multiple times on the same page. Goanna parameters offer a suffix  functionality. To use this functionality at least one of the following features needs to be  redefined: min_suffix, max_suffix, is_suffix_valid.


What happens behind the scenes
Now before we dive into writing our own web application let's have a look behind the scene.  Knowing how a request is processed by Goanna is essential in order to be able to write a web  application.

When you send a request the apache server tries to match the requested URI to one of its Aliases. If it finds an alias, it will look what kind of reference is referred to by this alias. In the  case of a Goanna page, the alias will point to a fast-cgi server. So the apache server will  forward the request to the fast-cgi application i.e. the Goanna web application. Now the Goanna  application takes over the control.

Basically the Goanna application will perform the following steps: 

     determine which servlet will handle the request
     validate the parameters which have been submitted with the request
     execute the servlets main logic
     generate a response
     This all sounds very easy and intuitive but there is more to it than it might appear at first.





Which servlet shall process the request
A web application is basically an application which uses servlets to process requests sent to it.  Now if a request reaches the web application it must decide which servlet should process the  request. In order to do this, the application must know all servlets which exist in it and it  also needs a heuristic to decide which one to choose. Every servlet which wants to process  requests must subscribe itself to the SERVLET_MANAGER. Along with its subscription it  must pass a string which represents a path. When receiving a request, Goanna compares the  request URI against the servlets stored in the SERVLET_MANAGER. It searches for the most  specific servlet which matches the URI or a substring of it. For example the request URI is  /test/web/run.htm and there are three servlets stored in the SERVLET_MANAGER  one which subscribed to /test, one which subscribed to /test/web and  one which subscribed to /test/web/walk.htm. In this example /test/web  is chosen, since the /test/web/walk.htm does not match and /test is  less specific than /test/web. If there is no servlet which matches the request URI  or a substring, the default servlet is chosen, and if no default servlet is specified Goanna will  log an error.

When the SERVLET_MANAGER has chosen a suitable servlet, Goanna populates the request  object and calls the do_get or the do_post method of the chosen servlet. Actually  always the do_get is executed because the do_post calls the do_get.


What happens in the servlet
Now that the control flow reached the servlet its getting interesting. The entering point for  the control flow is the do_get feature. So let us take a look at this feature, since most  of the magic happens in this feature understanding it is the key to understand how Goanna can be  used to build web applications. We will not go into every detail, but try to focus on the most  important aspects.

The first thing the feature does is getting the SESSION_STATUS object. The session is  identified by a cookie. If the browser does not yet have a cookie set or there is no session  stored for a cookie a new SESSION_STATUS object is created.

After the session_status object is initialized, a REQUEST_PROCESSING_RESULT is  created. Remember for each servlet there exists only one instance in your application. Therefore  it is not possible to store informations about a certain request in a servlet. Still it is  important to be able to store informations about a request. The solution to this dilemma is the  REQUEST_PROCESSING_RESULT. For every request an instance of REQUEST_PROCESSING_RESULT  is created and passed along until the request has been processed. Within the REQUEST_PROCESSING_RESULT  many references are available which make it easy to access different informations. So if you ever  need a reference to an object the REQUEST_PROCESSING_RESULT is always a good place to look  for it. Remember the REQUEST_PROCESSING_RESULT resides in the root_cluster of  the web application. So feel free to add your own features which are helpful for your application.

Before the parameters are processed the feature ok_to_process_servlet is evaluated. This  is the first hook where you can influence the behavior of the servlet. If the  ok_to_process_servlet feature returns false, the servlet is not processed and the control  flow jumps directly to the place where the displaying servlet is chosen and the answer is  generated.


Handling the parameters
The next step is to get the parameters from the request. This is done by iterating over all  parameters which were passed with the request. Remember, that the parameters which a servlet can  process are specified in the constructor of the servlet. The defined parameters come here into  play. Depending on which collection a parameter has been added to, it will be processed  differently.

In a first step a loop iterates over all parameters and creates an instance of  PARAMETER_PROCESSING_RESULT for each parameter. While doing this it also checks if all  required and all mandatory parameters are present and if there is any parameter which has not  been specified. If either an unexpected parameter has been found or a mandatory or a required  parameter is missing, the processing will abort at this point and the control flow jumps  directly to the place where the displaying servlet is chosen and the answer is generated.

If everything is OK until here, the parameters are split into two categories, mandatory and  non-mandatory. Note that the required parameters are part of the non-mandatory parameters. Then both the mandatory and the non-mandatory parameters are sorted according to their  processing order. The processing order is given as an integer value, default is 3. The smaller  the processing_order, the earlier a parameter is processed, if several parameters have the same processing_order the order in which they are processed is implementation  dependent.


Processing the parameters
Now that the parameters are split and ordered its time to process them. First the mandatory  parameters are processed. Processing means that the feature process of the parameter is called.  After processing a mandatory parameter the application checks the value of feature  is_value_valid. If the value is not valid the processing of the parameters will stop  immediately and the feature perform_invalid_mandatory_parameters_processing is called.

If all mandatory parameters contain valid values, the feature  perform_post_mandatory_parameter_processing is called. After that a second loop will  iterate over the none mandatory parameters. For each of the none mandatory parameters the process  feature of the parameter is executed, but the feature is_value_valid is not evaluated.  This means all parameters are processed even if one or more of them contain invalid values.  After all non-mandatory parameters have been processed the feature perform_final_processing  is called.


Generating the answer
In any case, if the processing was interrupted early or if all parameters contained valid values,  the servlet will call the next_page feature of the APPLICATION_SETTINGS and decide  which servlet is responsible for displaying the answer.

In order to display an answer, a servlet must inherit from goa_displayable_servlet and  implement the deferred features.


Interacting with a database
What would a web application be without a database. If you only plan to do static pages then  don't use Goanna. In order to be independent of the DB a developer chooses, Goanna offers an  interface which allows to control the database accesses. The interface is rather simple and can  be implemented by the developer if he needs a special database. The class which contains the  mentioned interface is called GOA_TRANSACTION_MANAGEMENT. This class offers a hand full of features which are called from within the Goanna web application framework and can be used as  hooks to influence the control flow.

The GOA_TRANSACTION_MANAGMENT allows to distinguish between a state where it is OK to  read from the DB and a state where it is OK to write to the DB. Unfortunately the naming of the  features is not always that clear. If a feature deals with version_access then it is  about reading data. If a feature deals with a transaction it is about writing data. The  feature commit is used to end a transaction.

The state of the system changes in the following way:

some state

                ok_to_start_version_access

            start_version_access

                ok_to_read_data

                ok_to_end_version_access

            end_version_access

                not ok_to_read

                some state



                some state

                ok_to_start_transaction

            start_transaction

                ok_to_write_data

                ok_to_commit

            commit

                not ok_write

                some state



Creating your web application
After all this theory its time to start building your first web application with Goanna. The  following is all just advise, feel free to follow it or make it your way.

My experience with browsers have shown that they are not made to ease the live of a web  application developer. So my first advise is: make a mock-up of your future web application.  This does not have to be anything fancy, just some plain old HTML and a CSS file. Once you are  happy with the design and the layout its still early enough to start messing with Goanna.

I recommend that you start off with the example web application we created in the first part of  this tutorial. A good idea is to subtype the following classes for your application. This gives  you places to add common functionalities of the different servlets.


     GOA_APPLICATION_SERVLET
     GOA_DISPLAYABLE_SERVLET
      GOA_PAGE_XML_DOCUMENT (do not redefine any features)

If you take a look at GOA_PAGE_XML_DOCUMENT you will notice that this class was generated. 


In the same cluster you can also find the following classes which are all generated at the same  time and are made to work together:


     GOA_PAGE_ATTRIBUTE_VALUES
     GOA_PAGE_SCHEMA_CODES


Also in the cluster you find the GOA_PAGE_XML_DOCUMENT_EXTENDED which is meant to contain  all the functionality which is common to all GOA_PAGE_DOCUMENTS.

In any case you should implement the class APPLICATION_CONFIGURATION early on since  without this class nothing works. The documentation of the features of  APPLICATION_CONFIGURATION can be found in the class GOA_APPLICATION_CONFIGURATION.

For every page in your web application repeat the following steps.


     As you do this you probably have to add features to other classes in order to store

intermediate values, or to pass values from one class to an other.
     Values which should persist over several requests, but are not stored in the DB, go

into the class SESSION_STATUS. Values which should be discarded after the request

has been processed go into the class REQUEST_PROCESSING_RESULT. Both classes are 

available from almost all functions and are meant to be containers for such values.
     Make a list of all parameters which your page should be able to handle.
     Mark all parameters which are mandatory. That means that the servlet cannot process a

request if the parameter is missing or invalid.
     Mark all parameters which are required. That means that the servlet cannot process a request

if the parameter is missing, but the processing of the servlet does not depend on the 

correctness of this parameter.
     Mark all parameters which are optional. That means neither the presence nor the correctness is a requirement for the processing of the servlet.
     Mark all parameters which should be added if they are missing.
     Sort the list of parameters in the order they should be processed. Make sure that if

the validation and the processing of parameter A depends on the processing of parameter 

B parameter B is listed before parameter A.
     Give each parameter a processing priority. This can be done in an iterative way. Where 

ever parameter starts with a priority of 3. If parameters A depends on parameter B then 

A.processing_priority = B.processing_priority+1. Repeat this until no more parameters change 

their processing priority.
     Create all parameters which you haven't created for another page. Be careful if a parameter 

is used in more than one servlet. The processing priority is the same for all pages where a 

parameter is used. You don't have to write your parameters from scratch but you are encouraged

to inherit from one of the deferred classes in the goa_parameters cluster.
     The most important task a parameter has to perform is to validate the correctness of its

value. If you ever focused a little on security in programs you know that any input from

the outside has to be considered hostile until it has been validated. And since your web

application is going to be published on the WWW and everybody can access it you should be

even more concerned about validating the values of the parameters which are transmitted with

a request.
     After all parameters are created redefine those features of the servlet which you will need 

for your servlet to work properly. But remember, the parameters should handle themselves. In 

Goanna it is not the responsibility of the servlet to be concerned about parameters. Still the 

framework offers you three features to hook into the processing of the servlet and let you 

influence its behavior. Here is the list of the features which you can use as hooks into the 

processing of the servlet.
     perform_post_mandatory_parameter_processing
     perform_final_processing
     perform_invalid_mandatory_parameters_processing
     I recommend you to separate the logic and the displaying of a servlet. Therefore you should create a 

DISPLAYABLE_SERVLET for every APPLICATION_SERVLET which does not make any 

calculations but focuses solely on generating the output.
     Implement the missing features of the DISPLAYABLE_SERVLET.



A last word
 This guide is only an introduction to Goanna, also is the Goanna framework at the time of  this writing still under heavy development and may change fundamentally. So there is no  guarantee that what is described in the chapters above will hold at the time of reading.Also not all topics have been covered in depth and others have even been touched at all.  But remember code and contracts are the best documentation...

Specially the following topics have not found enough space in this guide.


     SSL
    Parameter Suffix
    The different parameters classes

eiffelroom - Creating a web application with Goanna - Comments

Will update tutorial

Great Tutorial

Creating a web application with Goanna

This is a short tutorial for building web applications with Goanna

How it all works

Installation

Gobo

Eposix

Log4e

Goanna

We are still not done

And finally you need to get your fingers dirty

The first steps

A tour around town

What happens behind the scenes

Which servlet shall process the request

What happens in the servlet

Handling the parameters

Processing the parameters

Generating the answer

Interacting with a database

Creating your web application

A last word