Web Development Status Report

April 2008

Jennifer Oxelson and Tom Yoksas

In this status report:

  1. Web site upgrade to version 3.0
    The Unidata web site is in the process of being 'upgraded' for improvement. The new website, which will be deployed late spring/early summer 2008, will see changes in four following areas:

    1. Site infrastructure
    2. User registration
    3. User account management
    4. General Site design

  2. Spring is here
    In early March, John Caron and I attended a four-day training course on the basics of the Spring Framework. Spring is widely used for J2EE web development by the rest of the planet (it seems we are among the few who are not currently using it). The THREDDS group and I figured it would be a good time for Unidata to see what the hype is about.

  3. Unidata developer's blog
    An interesting conversation with John Caron has resulted in Jen installing blogging software on the Unidata website.

  4. Changes to McIDAS ACRs
    Some recent changes have been made to the Access Control Rules that govern access to McIDAS based on some concerns about distribution of Unidata McIDAS-X to institutions outside of the United States.

  5. RAMADDA
    Jeff McWhirter has been developing a Repository for Archiving, Managing and Accessing Diverse DAta (RAMADDA). Jen and Jeff spent some time in creating a servlet wrapper so that RAMADDA could be run as web application within a servlet container (such as Tomcat) in addition to its normal standalone mode.

  6. Mailing list and forum saga continues
    Alas, the mailing list and forum saga is not over. After a year of running in production mode, the current setup is showing its strengths and weaknesses.

  7. Why Google Analytics is cool
    Jen set up an account with Google Analytics and has had it tracking the Unidata website. By registering with the Google Analytics site and adding some JavaScript to our pages (easy with mod_layout), Unidata essentially now has a free, online website analysis package that gives us just as much information as Sawmill, AWStats, etc.


I. Web site upgrade to version 3.0

The Unidata web site is in the process of being 'upgraded' for improvement. The new website, which will be deployed late spring/early summer 2008, will see changes in four following areas:

  1. Site infrastructure
  2. User registration
  3. User account management
  4. General Site design

Site infrastructure

User registration

The code that handles user registration is being refactored. The following changes are being implemented (listed in no particular order):

User account management

The code that handles website user account information is being refactored. The following changes are being implemented (listed in no particular order):

General Site design

The current Unidata web site design, while adequate, needs updating.

The PV2 group, Linda Miller, and Jo Hansen will meet on April 21th to provide feedback on the possible choices of design for the new site. Over the next couple of months, the design will be refined based on the input from the staff, User's Committee, Policy Committee, and any interested community member.

The new design addresses known usability issues and provides some new features that hopefully the community finds interesting and helpful. Making extensive use of CSS, JavaScript, and AJAX, the new design will provide a richer user experience for site visitors.

Incorporating Web2.0-ish technologies into our site provides almost limitless possibilities for Unidata to provide useful and interesting content to our users not easily accessible before.

Among the planned changes are:

We will hold off on the use of HTML5 until it gains more support in broswers and clients.

The use of CSS-based layout should make it easier for Unidata staff to add/modify website content without having to muck with multiple nested HTML tables. Care is being taken to make sure that users with older browsers with limited CSS and JavaScript support will still be able to access and use the major (important) functions the Unidata site (e.g., software downloads, support, etc.)

Browser stats of site visitors
Google Analytics stats of browser use
Click to enlarge image
Firefox browser version stats
Google Analytics stats of Firefox
Click to enlarge image
IE browser version stats
Google Analytics stats of IE
Click to enlarge image

Unidata is also taking an active role in UOP to provide a website design that displays the UPC's affiliation with UOP and UCAR. (NCAR recently implemented an NCAR-wide redesign of all websites to accommodate a mandate for a common 'branding' from the NCAR President's office. While UOP has no such requirement for a common look across programs, an interest to show a common affiliation has been expressed). Jen and Tina met with Hanne Mauriello (UOP director's office), Kelly Lewis (UOP director's office), and Zhenya Gallon (UCAR Communications) to discuss such site branding. Unidata is taking a leadership role in providing a possible branding solution in the new Unidata website redesign (perhaps to be used by other programs across UOP).


II. Spring is here

Spring

In early March, John Caron and I attended a four-day training course on the basics of the Spring Framework. Spring is widely used for J2EE web development by the rest of the planet (it seems we are among the few who are not currently using it). The THREDDS group and I figured it would be a good time for Unidata to see what the hype is about.

In a buzzword-laden nutshell, Spring is a lightweight, inversion of control and aspect-oriented container framework (bla, bla, bla).

Looking past the buzzwords and hype, does Spring offer something that will make development easier/quicker? Was it worth our time and Unidata's $$$ to send us to this training? John and I think it was.

We experienced several Ah ha! moments when learning of how Spring could easily solve some complex development problems. Spring measures up to the aforementioned buzzwords in a good and practical sense:

Lightweight

This can have a couple of meanings in the Spring context:

One interpretation: Plain Old Java Objects (POJOs) in a Spring application typically have no dependencies on Spring-specific classes. Separation of duties. Your code is the meat and potatoes of the application, while small amounts of Spring is used to bind it all together.

Another interpretation: Just drop a few of small jar files in your classpath and you are ready to go. Spring has a lot of bells and whistles, but you don't necessarily have to use them. (Spring comes bundled in different sets of jar files depending on which features of Spring you want to use.) In the web world, just configure Spring to run like you would any servlet (via the deployment descriptor) and you instantly have access to the Spring framework/container.

Inversion of Control (IoC) - Dependency Injection

Do a Google search of inversion of control and the first hit you will get is an article written by Martin fowler defining the IoC pattern and giving it the more helpful term of dependency injection.

Learning about dependency injection was one of those Ah ha! moments for me. It is cool in a geeky sort of way. Using this pattern I can write modular code in the form of POJOs and POJIs (Plain Old Java Interfaces) which can be swapped out or replaced without having to touch the other POJOs that use/depend on it.

I'm not going to go into depth about IoC or dependency injection. I'll let you RTFM for yourselves (see links below). But here is a 50,000ft view of what it is:

Aspect Oriented Programming (AOP)

Another Ah ha! moment for me. In short, Spring AOP allows the POJOs to only be responsible for the business logic for which they were created while keeping the cross-cutting, non-business logic code (e.g., logging, etc.) out of the POJOs. This cross-cutting code is then applied to the POJOs by Spring using an AspectJ-like nomenclature. While AOP in general can have a bit of a learning curve, Spring does its best to make AOP simple.

Additional features

The core Spring framework contains many other useful features not listed above. The Spring project also has additional modules, add-ons and integration tools that can be used in conjunction with Spring. Some of the more notable framework features and add-ons are:

Spring as a standard?

With the adoption of Spring by Jen and the THREDDS group for the TDS, we are taking a common approach/architecture to our development. The use of standards and conventions to some degree promotes knowledge-sharing, code reuse, peer-review, and better quality of work.

If an application was developed using Spring, certain things are understood: structure, configuration, etc. If someone (e.g., a unidata developer or a user of unidata software) wishes to extend or take over development of a Spring-based app, it wouldn't take him/her much time to spin up on how the application works.

For more information about the Spring framework, visit:


III. Unidata developer's blog

An interesting conversation with John Caron has resulted in Jen installing blogging software on the Unidata website (pebble). Many developers of commercial and opensource software keep blogs to share development ideas, tips, and collect user feedback. Blogging by Unidata developers could providing a unique opportunity to share ideas with the community.

Goals and techniques to maintain a focused discussion via Unidata blogging still needs to be fleshed out. Perhaps the Usercomm, Polcomm, or other members of the community may have ideas and/or a need for contributing to the Unidata blog?

A link to the blog will be published/advertised when posts are being made. Stay tuned.


IV. Changes to McIDAS ACRs

Some recent changes have been made to the Access Control Rules that govern access to McIDAS based on some concerns about distribution of Unidata McIDAS-X to institutions outside of the United States. Tom and Jen will be working to modify the existing Participation Policy in the Unidata community to address these concerns. As it stands now, website access to McIDAS is governed by the following Access Control Rules:

The user must have provided:

OR

These current ACRs should 'catch' all attempted access to McIDAS by institutions outside of the United States so that they may be approved on a case-by-case basis.

For more information about access to McIDAS, please see the McIDAS status report.


V. RAMADDA

Jeff McWhirter has been developing a Repository for Archiving, Managing and Accessing Diverse DAta (RAMADDA). Jen and Jeff spent some time in creating a servlet wrapper so that RAMADDA could be run as web application within a servlet container (such as Tomcat) in addition to its normal standalone mode. Tomcat 'sandboxes' have been created on both andy (development server) and conan (production server) so that Jeff may continue his development of RAMADDA.


VI. Mailing list and forum saga continues

Alas, the mailing list and forum saga is not over. After a year of running in production mode, the current setup is showing its strengths and weaknesses.

Concerns:

Mailman is the probably one of the most popular and widely-used mailing list managers. Out-of-the-box mailman is a easy to use. However, added in customizations and the mailman environment quickly gets complex and unruly.

Similar to mailman, the Jive forums Out-of-the-box are easy to install and use. However, customization of the forum software can quickly lead to complexities that border on the un-maintainable. Worse yet, the actual use of the forums by Unidata staff is no where near the predicted use of the forums (the IDV group is the exception). A lot of $$ and time/effort for customization has gone into the forums - one wonders if it is worth the effort.

Therefore, while it is not a high priority, Jen will continue to investigate and look for other solutions to the forum/mailing list dilemma. Namely, look for lighter-weight fourm packages and mailing list managers that handle customization easier.


VII. Why Google Analytics is cool

Google Analytics

Jen set up an account with Google Analytics and has had it tracking the Unidata website. By registering with the Google Analytics site and adding some JavaScript to our pages (easy with mod_layout), Unidata essentially now has a free, online website analysis package that gives us just as much information as Sawmill, AWStats, etc. (Google's service was modeled upon Urchin Software Corporation's analytics system, Urchin on Demand. Google Analytics runs as a software-as-a-service (SAAS) version of the product.)

The Google Analytics result data can be emailed and exported in a variety of formats. In addition to accessing the information in these formats, Jen has created a user account for Unidata staff to access and query the results directly.

You can do a lot with this, but for a complete list of features, see: Google Analytics Tutorial and visit the Google Analytics site at: http://www.google.com/analytics/indexu.html

One of the cooler features offered by Google Analytics is a Site Overlay of visitor click patterns. The stats are overlay the web page of interest and give the statistics of viewer clicks for each link. This can be a very effective tool in determining how users navigate a site, tracking down usability issues, and revealing the popular vs. un-popular content. See the example screen-capture of the Unidata home page for an example:

Google Analytics site overlay of Unidata's homepage
Click to enlarge image

Subsequent Usercomm and Polcomm status reports will contain the following basic Unidata website stats/information for your examination (please let Jen know if you would like to add any additional stat information to this list):