HYDRO-NEXRAD Report: April 2009

Mohan Ramamurthy and Jeff Weber

To maintain a vibrant program, the UPC, from time to time, participates in certain projects that are tied to its overall mission but funded separately from the core program.  Such synergistic activities are both essential and complementary to the core effort, and both the Policy Committee and NSF encourage them. Such projects have played a vital role in advancing the program in new directions, creating new capabilities for the community, enhancing interoperability of Unidata software, providing new datasets to the community, and entraining and diffusing innovative ideas and technologies into the community.

Continued progress was made to advance HYDRO-NEXRAD ITR project goals during last year. However, the severe flooding in Iowa City, Iowa during the summer months resulted in prolonged interruptions and outages at the University of Iowa and delayed the ongoing work by over two months, with many of the systems and servers used in the project not fully functional until the end of August 2008. [In the wake of the severe flooding, the NSF Director, Dr. Arden Bement, visited the University of Iowa and the project PIs, Profs. Witek Krajewski and Anton Kruger, had an opportunity to present some of their ongoing work to Dr. Bement, including some of the data and products from this project. Unidata provided access to archived data to help fill in gaps due to power and hardware failures to aid in the demonstration.]

 

The Iowa PIs, in collaboration with Unidata, have submitted a supplemental request for the continuation of the project.  As such, it is important to continue the ongoing efforts to provide a smooth transition to the continuation project.

 

In this annual report, we summarize Unidata’s activities pertaining to the HYDRO-NEXRAD ITR project during the past year.

 

1. Continued Provision of NEXRAD Level II Radar Data and Adaptation to Super Resolution Data

 

Unidata continues to supply the University of Iowa with a top-level feed for Level II radar data using the LDM/IDD.  Originally, the data feed included only 10 radars of interest, but during the last year that number was expanded to include 40WSR-88D sites.  Recently, the National Weather Service began a major upgrade to the resolution of the Level II radar data.  The so-called “super resolution” (Build 10) data provides reflectivity data with 0.5 degree resolution in azimuth and 250 m resolution in range. The super resolution data present an important opportunity for the hydrologic community given the high variability of the runoff generation processes in urban environments.  Unidata has already adapted its decoders and applications to ingest the super resolution data format and are currently working with the Iowa team to ensure that the HYDRO-NEXRAD system can use the higher resolution data feed.

 

In addition to providing WSR-88D Level II radar data, Unidata provided 1 km national radar mosaic products to the Iowa PIs to fill in gaps in data caused by format change of Level II data.

 

2. HYDRO-NEXRAD Feed and Generation of Level II Metadata

 

Last year, Unidata created a new type of data feed for the hydrology community.  Unlike traditional data feeds that distribute only the data in real-time, the new feed called NEXRAD4 or HYDRO augments value-metadata for individual radars, basins, and points of interest (rain gauge locations) along with the original radar data.  Metadata include hydrologically meaningful characteristics of rainfall (mean area rainfall, fractional basin coverage, maximum rainfall amount, etc.).  As a result, the HYDRO-NEXRAD feed is a significant step to facilitate data mining and knowledge extraction from the vast volumes of radar data. Below is a schematic of the process:

 

Researchers at the University of Iowa continue to develop additional metadata descriptions as well as new algorithms for metadata generation.  As those algorithms become ready for operational deployment, Unidata has been implementing those algorithms and augmenting the Level II radar data feeds through a new LDM/IDD HYDRO feed.  Unidata and University of Iowa PIs mutually agreed last year that Unidata will begin generating the value-added metadata locally and then distribute the metadata and Level II files in the new HYDRO-NEXRAD feed.

 

Unidata is also working closely with Prof. David Maidment, University of Texas, Austin, and head of CUAHSI-HIS group and his students toward creating a metadata database for use with the CUAHSI long term hydrologic observatories.

 

3. NEXRAD-ITR MapServer at Unidata

 

During the past year, we continue to develop, test and deploy a dedicated MapServer at the Unidata Program Center, enabling the availability of the MapServer to Unidata’s large user community.  Specifically, the UPC staff tested and provided feedback on the MapServer GUI.  Work is underway to prepare the Unidata MapServer to become a node for HYDRO-NEXRAD services in the future.

 

4. Future

 

Jeff Weber visited the University of Iowa in June to discuss project priorities for the next year, especially for Unidata.  Those discussions and conference calls with the Unidata PI resulted in the identification of the following tasks for Unidata for the next year, and a submittal of a supplemental request to NSF to continue the project for another two years.

 

The recent switch by the National Weather Service to the super-resolution data collection by most of the WSR-88D radars has resulted in the re-coding of the ingestors and the development of the databases for radar-based and basin-based metadata.  The codes Iowa researchers provided to Unidata to calculate the metadata no longer work and the migration of those codes for the super-resolution radar data will be a high priority activity during the extension period.

 

The Unidata Program Center will continue to assist the project PIs in that transition by updating the Hydro data feed and aiding in the expansion of the feed to calculate metadata for all 158 radars. The current codes do not scale for efficient use with 150 radars in real-times.  The UPC and Iowa developers are working together to use a C-based code from GEMPAK to perform that task. Simultaneously, we are also exploring ways to organize the Java-based calculations to make them faster and more efficient.

 

The UPC staff are continuing to test and provide feedback on the HYDRO-NEXRAD MapServer GUI.  We anticipate that the MapServer that is maintained at Unidata will become a node for HYDRO-NEXRAD services during the upcoming year. Entrainment of more users for the HYDRO feed and the database is another goal for the project during the next year.

 

In addition to the above work, Unidata will continue to provide other support as necessary toward fulfilling the goals of the project and to facilitate a smooth transition of the project resources toward meeting the needs of the hydrology community. In addition, the results and products from the project will be disseminated to the Unidata community and our governing committees at their regular meetings.

5. Current State

This project is ongoing, yet static, we await possibilities for future funding and new directions.