Unidata - To provide the data services, tools, and cyberinfrastructure leadership that advance Earth system science, enhance educational opportunities, and broaden participation. Unidata
         
  advanced  
 
McIDAS Status

FX-Linux Overview
Winter 2002

Tom Yoksas
March 14, 2002

Impetus for evaluating FX-Linux as a possible Unidata-supported package

Unidata has been tracking the AWIPS software development at FSL for several years. Current software packages being developed by FSL include, but are not limited to:

In the past year, several (12 according to contacts in FSL) Unidata universities have independently approached FSL for access to either or both FX-Net and FX-Linux. Because of the interest shown by these sites, questions were included in the recent User Committee survey that asked the broader community how important it was to them for Unidata to offer display/analysis packages with AWIPS-like (meaning either or both of FX-Net and FX-Linux) capabilities. The results of this survey will be presented in a separate report by the head of the User's Committee.

In partial response to the survey, and in fulfillment of our long stated intention of keeping abreast of software capabilities of FSL offerings, the UPC installed an FX-Linux data server and display workstations in-house. This installation has been used to gain a clearer picture of how large a job it would be to install, run, and maintain FX-Linux in a university context.

What this overview is and is not

This white paper is meant to serve as a focal point for discussion by Unidata User and potentially Policy committee members. The intent is to present findings and opinions about how FX-Linux might function in a Unidata university setting, and what the impacts on UPC technical staff would be if FX-Linux were to be adopted as a supported Unidata offering.

This document is not intended to be a full review of the capabilities of D2D. Most readers would agree that D2D is a fully functional, meteorological display and analysis suite given its adoption as the standard application package at NWS WFOs, and the numerous awards it has already won.

FX-Linux Installation at the UPC

In the fall of 2001, Ron Kahn of NOAA/FSL coached me through a CD-based, binary installation of FX-Linux on two moderately configured (Intel PIII 550 Mhz; 500 MB RAM; 20 GB HD; etc.) UPC PCs that were loaded with fully patched RedHat Linux 6.2. The two machines used for the installation are referred to as Data Server #1, DS1, and WorkStation #1, WS1. For the remainder of this overview, these machines will be referred to as DS and WS.

The DS machine was configured to run an LDM that would ingest NOAAPORT channels 1 (GOES-East), 2 (GOES-West), and 3 (NWSTG) products from a server at the Boulder FSL offices. It is responsible for decoding all decoding activities:

It should be noted that feeding the DS machine with data from an LDM is not the standard mode of operation supported by FSL.

The WS machine accesses data files being produced by DS by NFS mounts of the data directory. Multiple workstations can access the data being produced by one or more DS machines (decoding activities can be split among several machines to equalize loads).

Installation and partial configuration activities consumed about a half day initially, and then a couple of additional hours in a follow-up visit to the UPC by both Ron Kahn and Susan Williams of FSL. At the time of the installation, my impression was that the setup needed to support FX-Linux was more complicated than that required for existing Unidata packages (e.g., GEMPAK and McIDAS). Further reflection, however, has convinced me that part of the impression of complexity stemmed from my unfamiliarity with how FX-Linux works, and what roles were played by the two machines. In short, I did not have a "big picture" overview of what needed to happen where, and how pieces fit together. This situation is not dissimiliar to what Unidata sites feel about our existing packages.

After completing the installation and configuration of an LDM feed from FSL, the ability to view data was as hoped/expected: data flows in; gets decoded into appropriate forms for display; gets filed in appropriate locations; and is easily displayed/analyzed from FX-Linux (D2D) itself.

FX-Linux: Pluses and Minuses

FX-Linux Pluses

FX-Linux is a full-featured, display and analysis package that is optimized for rapid visualization of a wide variety of data. The speed that products can be displayed and the interactivity with those displays is a crutial component included for support of the forecaster at a WFO, and would be welcomed by university users in a lab setting.

AWIPS as an initiative, and D2D as an analysis tool are well documented, and this documentation is expected to continue to be updated as the system evolves. The fact the documentation is maintained outside of the UPC is seen as a significant plus.

FX-Linux is highly flexible through mult-"level" customizations and configurable through definable procedures and bundles. Multiple users can share the same workstation customization while being able to operate in their own environment through use of the procedures that they create themselves. This sort of per-user configurability would allow a site to keep invocations of FX-Linux active in generic (e.g., 'student') accounts.

FX-Linux has, depending on the user's choice, 3 or 5 active display panes. Displays, loops, etc. in each pane are automatically updated upon receipt of new data, and switching a side pane into the main display is easily accomplished with a simple mouse click. Auto update features like this are apparently typical of the features that Unidata users most like about the GEMPAK GUIs, GARP and NMAP/NMAP2. Compared to the briefing frame construct in McIDAS and the Function key multiple loop construct in GEMPAK/NMAP, the multiple pane/loop capability of FX-Linux is much easier to use, and is more visually accessible.

Like the packages currently supported by the UPC, FX-Linux runs on modestly priced PC platforms. FSL has chosen RedHat as its Linux bundle of choice, much the same as the UPC did several years ago.

The FX-Linux UI, developed in Tcl/Tk, presents an easily understood access to a large set of data display and analysis capabilities. A very useful feature of the menus is the time listing (so called green times) of the most recently received data for each type supported by the interface. The green times are even updated while menus are posted for use.

The volume browser interface to calculations made with gridded fields, even though fully functional, may be somewhat daunting to the average user given its use of reverse Polish notation.

Since FX-Linux (actually D2D on HP-UX systems) is in operational use at the NWS, there is a growing body of expertise in how to use the package effectively. Use of FX-Linux at Unidata sites would prepare those individuals aiming to work in the NWS with hands-on experience before they ever apply for jobs. Just how many people this would positively affect is unknown, but it is thought to be small in comparison with the numbers of meteorology students in the over 100 institutions participating in Unidata.

Perhaps the most attractive aspect of FX-Linux from a UPC perspective is the maintenance and update of meteorological decoding software outside of the UPC. Also, decoder updates by those primarily responsible for supporting FX-Linux (e.g., FSL and perhaps PRC) are done typically before changes are made to NOAAPORT datastreams. This is in stark contrast to the reactive mode that UPC staff have always had to operate in.

Lastly, at least for the purposes of this white paper, is the fact that a full FX-Linux installation includes a great deal of the code needed to support other FSL software offerings: FX-Net and FX-Collaborate. Exactly how much additional code would be needed to run things like FX-Net Server on the Linux workstation is unknown to me (it is hard to find a single person at FSL who knows enough about all aspects of their systems to get a definitive answer), but I am left with the impression that it should be limited. (I will gladly stand corrected).

FX-Linux Minuses

Even though FX-Linux represents a fully-functional, advanced weather analysis system, it is lacking some of the capabilities that Unidata has been recently been embracing.

There currently appears to be the need for separate DS and WS machines. Those Unidata sites that do not have the resources to dedicate a machine to data ingest/decode might be not be able to affort to run a "standard" FX-Linux installation. This may not be absolutely true, however. Several FSL representatives have expressed their believe that one could run both the DS and WS activities on a single, suitably powerful Linux workstation. This configuration has apparently not, however, been rigourously tested, so it remains to be seen if it all activities could run on a single machine without appreciably slowing data display and analyses.

There is no capability to perform objective analysis of observational data in FX-Linux/D2D. The approach to display of objective data (contour lines or color filled levels displays) appears to be that one simply uses initialization fields from locally run models. I see this as a weakness for educational use of the system, but this has been argued to not be significant by User Committee members in the past.

There doesn't appear to be any way that individual observational fields (e.g., T, TD, etc.) can be plotted on a display. The approach to display of surface observations is to always display station model plots.

There is no effective way to do time series tracking of parameter changes at display locations. The only way to view period to period changes is to interrogate the display for the different times and perform the calculations oneself. Being able to assess such things as the speed and direction of cloud feature movement interactively would be be a useful addition to existing capabilities.

There does not appear to be any way to script FX-Linux capabilities. Displays must be created within the UI itself. Furthermore, there does not appear to be any easy way to save displays as web-displayable products (one can use freely-available screen capture applications like xwd, however).

FX-Linux does not offer remote access to data sets like McIDAS the Unidata MetApps application, IDV. In this regard, however, it is the same as GEMPAK: all data files must reside on locally accessible disk.

Even though D2D has been successfully ported to Linux, it has not been ported to other, non-HP platforms. Surveys of the Unidata community have consistently shown that the most popular computing platform in universities is Sun Solaris SPARC. A port of FX-Linux to SunOS may turn out to be trivial, but it is a task that would have to be addressed if FX-Linux was to be widely accepted in the Unidata community.

The FX-Linux package is large, even by comparison to the currently supported Unidata packages (GEMPAK and McIDAS). In addition, since FX-Linux uses a good fraction of data files in netCDF, and since netCDF currently has no compression capabilities, the amount of disk space needed at a site would significantly increase from what it is today. And, the need for additional disk space would be in addition to that needed for GEMPAK/McIDAS if a site wanted to be able to run all three packages. The disk space issue may seem overblown given how cheap PC disks have become, but the university budgeting cycle may prevent sites from performing necessary upgrades rapidly enough to benefit from FX-Linux availability in the short term.

If one is to believe the PRC documentation related to planning and implementation, software upgrades are non-trivial and time consuming. Since I have not had the occasion to do an upgrade of FX-Linux, I can't say if the documentation should be taken with a grain of salt, or if it appears overly complex because of the need to be very clear of what has to be taken into account at sites that do extensive customizations. In any case, a quick read of upgrade documents is plain frightening.

Even though D2D/AWIPS is well documented, Unidata would still need to establish its own FX-Linux web site that addressed issues specific to Unidata-site use of the package. This includes Unidata customizations, specific upgrade/bugfix/enhancement upgrades, LDM data delivery, etc. The creation of a usable, user-friendly web site is not a full time occupation, but it is a substantial undertaking nonetheless.

Last, and at the moment least, FX-Linux does not provide for 3D display of data. This capability is, however, provided by the D3D (Vis5D with a well designed Tcl/Tk GUI interface) offering that FSL is demonstrating to others in NOAA, so we may see it become available as time progresses.

FX-Linux Approach to Data Analysis/Display - Localizations

D2D/FX-Linux's approach to data display/analysis is, by nature, WFO centric; specific customizations known as localizations have been created for each US WFO. Inherent in this view of data is the limiting of display of high resolution data sets to a small region around the WFO the localization was created for. This approach to limiting what data can be seen for a particular localization is well suited to forecasters at a WFO, but it is much different from the needs of the research and educational community that wants to be able to perform detailed analysis of weather events that occur anywhere in the US or, increasingly, the world.

Relatively recently, in an effort to address the WFO localization limitations, FSL has created a National Centers localization. This view of the weather is somewhat more flexible than the standard WFO localizations in terms of the data that can be seen in specific regions, but it does not appear to allow for display of high resolution data (e.g., satellite imagery, model output, etc.) anywhere. The exception to this, however, is the provision of displaying individual NEXRAD radar products, but unlike the WFO localizations, this is not at their full resolution.

The fact that a National Centers localization could be created, and the fact that localizations have been created for sites not in the US (e.g., Korea), when coupled with experiences at sites like the Johnson Space Center (JSFC) indicate that FX-Linux may be customized to include capabilities for roaming, high-resolution displays that Unidata sites seem to desire. The job of producing such a localization would naturally fall on the UPC if FX-Linux were to be adopted for support. From my little experience in looking through how customizations are created, it is my opinion that whoever undertook to create such a localization would necessarily have to become an expert in virtually all aspects of FX-Linux before being successful. This is essentially what UPC staff have had to do with the currently-supported packages, so it is not seen as anything different. One thing to remember, however, is that my experience is that there are either no or very few individuals at FSL that have enough of a grasp of the entire FX-Linux package to allow them to create such a localization. A UPC support person would then have to become as much or more of an expert in FX-Linux than most people who work with it on a full time basis. So, the job at hand would be a full time one.

Ramifications of FX-Linux Support

If FX-Linux were to be adopted for full Unidata support, several new projects would have to be undertaken by the UPC:

Recommendations

In reviewing FX-Linux, I have tried to determine what minimum level of support would be required in order to add it to the suite of packages that the UPC supports.

The options for support have always been and remain:

After careful consideration, and even though there are a number of challanges that would have to be overcome in order to fully support FX-Linux, I believe that the package would be welcomed by the existing Unidata community if installation and configuration were greatly simplified. Given this, it seems that some hard decisions need to be made by the UPC and its governing bodies:

A Final Thought

It may well be the case that the majority of Unidata universities would be more interested in the FX-Net application than in FX-Linux itself. If this is the case, then Unidata will need to find the resources (internally or possibly at FSL) to host a set of FX-Net Servers that could provide data display creation to a large number of simultaneous users. If FX-Net were to become very popular in the community, the number of servers needed to service them could be quite large and costly.

This page was prepared by Tom Yoksas.
Send questions/comments to <support@unidata.ucar.edu>.

http://www.unidata.ucar.edu was updated on .


Go to the Unidata Homepage.
 
 
  Contact Us     Site Map     Search     Terms and Conditions     Privacy Policy     Participation Policy
 
National Science Foundation (NSF) UCAR Office of Programs University Corporation for Atmospheric Research (UCAR)   Unidata is a member of the UCAR Office of Programs, is managed by the University Corporation for Atmospheric Research, and is sponsored by the National Science Foundation.
P.O. Box 3000     Boulder, CO 80307-3000 USA     Tel: 303-497-8643     Fax: 303-497-8690