LDM and IDD Status Report

LDM and IDD Status Report
Fall-Winter 2006/2007

Steve Chiswell
Steve Emmerson
Mike Schmidt
Jeff Weber
Tom Yoksas

March 4, 2007

Real-time, self-managing data flows -- Unidata will foster and support the existence of real-time data flows that encompass a broad range of Earth-system phenomena, can be accessed with ease by all constituents, and are self managing in respect to changing contents and user needs.

--A goal of Unidata 2008: Shaping the Future of Data Use in the Geosciences


LDM Status Report

LDM

Current LDM version is 6.5.1.

Release highlights since 6.4.6 of previous report:

Next Generation LDM

Steve Emmerson's work on the LDM replacement was interrupted by necessary work on the current LDM and also by creation of a replacement for the UDUNITS package, which will be both a stand-alone package and incorporated into the next-generation netCDF package.


IDD Relay Cluster idd.unidata.ucar.edu

The cluster approach to toplevel IDD relay, first reported in the Spring 2005 status report, has been operational at the UPC since early summer 2005. The cluster, described in the June 2005 CommunitE-letter article Unidata's New IDD Cluster, routinely relays data to more than 400 downstream connections. Data input to the cluster nodes is approx. 4 GB/hr (0.97 TB/day); average data output by the cluster is approx. 246 Mbps (~2.7 TB/day); peak rates routinely exceeding 450 Mpbs (~4.9 TB/day).

We still consider the cluster to be an experiment in progress. Configurations change as we learn more about how the system performs, and new operating system releases are tested for performance.

The cluster approach to IDD relay has been adopted by NOAA/GSD (formerly FSL) and Penn State (using funds provided by the Unidata-administered Equipment Awards program). Unidata staff have fielded questions on implementing similar clusters at Unidata community sites (Texas A&M) that have expressed an interest in functioning as toplevel IDD relay nodes.


Next Generation LDM Update

Steve Emmerson continues to work on a replacement system for the LDM (codename Hermes) that will combine the best features of the LDM, INN (i.e., netnews), and more modern peer-to-peer systems such as BitTorrent. The work is currently in the design stage. Design features include


IDD Status Report


For previous information, please refer to the April 2006 Users Committee status report.