Unidata

IDV for Field Project Data (Don Murray, IDV Developer)

As an early proponent of Unidata's Integrated Data Viewer (IDV), Steve Aulenbach (NCAR/CGD) welcomed the release of Version 1.2. New features in this release that make it suitable for use with data collected during field campaigns are making it much easier for Steve, Teresa Campos (EOL/ACD) and Stephan De Wekker (ASP) to assess the quality of data from the ACME (Airborne Carbon in the Mountains Experiment). The ACME team also value the built-in collaborative analysis capabilities as they use the IDV to visually place airborn CO2 observations within their geospatial context.

Work to develop these enhancements began in fall 2004 in support of the Rain In Cumulus over the Ocean (RICO). This work was funded under a joint proposal between EOL, JOSS, and Unidata for the Real-Time Display and Coordination Center (RDCC) for that project. Some of the new developments include displays of aircraft tracks and SPOL radar imagery in real time. The IDV was used in the RDCC in St. John's, Antigua to help support daily flights and analyze some of the collected data.

But development of IDV enhancements for field projects did not stop after the data gathering segment of RICO ended. Back in the shop, IDV developers discovered that a one-second image from the forward nose camera of the C130 aircraft was among the available datasets. The idea of displaying these as an animation synchronized in time with the display of the flight track was hatched, and this feature was added to the 1.2 release. Now users can correlate radar, satellite, and aircraft data with a picture of the weather in front of the airplane. Users report that it's almost like being there, without the nausea! The same IDV control is used for displaying real-time web cam imagery that is gathered from sites across the country. These can be synchronized with real-time satellite and radar imagery, giving users a "look out the window" at those sites.

The image above shows a C130 flight track from ACME displayed in IDV. The image below is an IDV display showing time synchronized flight track, downward looking camera, and chat log.

Yet another related dataset collected during the field project is the log of conversations between researchers on the ground and scientists on the airplane. A "chat" facility was used to coordinate the flight plan and provide feedback from the airplane to the ground on what types of weather were being encountered. The daily log of the conversation is archived and provides a snapshot of some of the scientific information gathered during the project. These logs can be read into the IDV and displayed time-synched with the other data collected.

Other enhancements were also incorporated in the software to provide better support for the Research Aviation Facility (RAF) netCDF aircraft data convention for RICO. Chris Webster at RAF, Jeff McWhirter and Ethan Davis of Unidata made enhancements to the new netCDF for Java 2.2 Common Data Model (CDM) to support trajectory data (point observations connected in time) in a more efficient manner than that which the IDV was using during RICO. Chris also modified the RAF netCDF convention to add metadata that would facilitate its use in the context of THREDDS catalogs that support the CDM. With these enhancements, the IDV can now read in the one-second data from the aircraft flights gathered by the NCAR C130 and the University of Wyoming King Air during the RICO field project. Work is continuing to refine the netCDF format and support sub-Hertz and vector data collected by the aircraft. This format will be used by NCAR's new HIAPER aircraft when it starts collecting data.

Steve, in collaboration with Michael Burek of SCD, has been creating preliminary THREDDS catalogs of the various ACME flight data. These data include the sensor data collected during the flight, downward looking camera images, and the chat log. The enhancements made for RICO enable Steve to easily put together a time-synched display of the flight track, camera and chat log data.

An additional concern that arises in field projects is data quality. Until the data have been processed and quality controlled, it is important that the datasets be restricted to the scientists working on the project. IDV developers are working with Community Data Portal (CDP) developers to create ways to access restricted CDP data through a secure session.

As soon as the IDV 1.2 release was out the door, work started on developing new features for the 1.3 release that will make the IDV useful for field projects and users in general. A major new feature now being tested is a charting and graphing facility. This new feature will allow users to create time series, scatter plots, histograms, and table readouts of a variety of data types. Initially it is being tested with the aircraft data from the ACME project. This is a major step forward in the general usefulness of the package for data analysis by researchers.

Field project support is an area where the IDV already has made substantial contributions, and its developers are continuing to collaborate with NCAR colleagues to improve its utility in the field.

Be on the lookout for the 1.3 beta 1 release sometime in November.