•         advanced

Real-time, self-managing data flows -- Unidata will foster and support the existence of real-time data flows that encompass a broad range of Earth-system phenomena, can be accessed with ease by all constituents, and are self managing in respect to changing contents and user needs.

--A goal of Unidata 2008: Shaping the Future of Data Use in the Geosciences

LDM-6 Status Report

• LDM 6.0.14 was released on July 21, 2003. This version is stable and has been deployed throughout most of the IDD system ( 95 out of 144 hosts reporting statistics at http://www.unidata.ucar.edu/staff/chiz/rtstats/siteindex.shtml were running version 6.0.14 on 2003-09-23).

• Improvements to the LDM system since the March 13-14, 2003 User Committee meeting include the following highlights:

  • The LDM package has been ported to the HP-UX B.11 operating system (the NWS can now use LDM 6 on their updated HP systems).

  • The LDM daemon has been improved: shut-down speed has been increased, unnecessary reconnections have been reduced, superfluous processes have been eliminated, and the determination of when to fall-back to the LDM 5 protocol has been improved.

  • A bug in the connection logic that sometimes caused a process to crash has been eliminated.

  • The rtstats(1) utility has been improved: performance has been increased , bugs have been fixed, memory-leaks have been removed, and support for the LDM 6 protocol has been added

  • The ldmsend(1) utility has been improved: a memory-leak has been removed and support for the LDM 6 protocol has been added.

  • Many log messages have been improved.

• The LDM 6 running on thelma.ucar.edu was stress-tested by adding additional downstream, requesting LDM processes to determine the maximum output possible without introducing any data-product latencies. With 18 receiving LDM processes, Thelma was able to run 139 sending LDM processes without introducing additional data-product latency. This test lasted two days (it was terminated only because hurricane Isabel's landfall required adjustments to the IDD topology). The peak output during this test was near 260 Mbps (approx. 2.8 TB/day, if sustained) and the mean output was near 130 Mbps (approx. 1.4 TB/day, sustained). This test is visible in plots of Thelma's outbound traffic.

• A new set of HTML documentation is being developed for the LDM 6 package -- including an LDM 6 tutorial. This documentation will be used at the fall 2003 LDM 6 Training Workshops and will also be made available on-line on the LDM homepage.

• An attempt will be made to simulate NEXRAD level 2 data-reception at the NWS regional headquarters in order to validate the NWS plans for distributing this data. This will involve about 40 LDM processes sending synthetic radar data-products to an LDM system on a single host.

IDD Status Report

• 134 machines at 90 sites are running LDM-6 and reporting real time statistics. Sites continuing to run LDM-5 are being encouraged to upgrade to LDM-6 as quickly as possible. Unidata staff have remotely installed LDM-6 at many sites to expedite the changeover process.

• Real time LDM statistics can be found in the Real Time IDD Statistics webpage. The topology and data flow information reflects changes in feed requests within a few minutes of changes. Rtstats have proven to be most useful in diagnosing reception problems at sites, especially when universities have implemented restrictions on packets volume in and out of the campus without regard for their user needs.

  In order to correctly guage real-time status of the IDD, it is important for all sites participating to accurately maintain their clocks (easily done through Network Time Protocol software and servers).

• The CONDUIT injection host was upgraded to LDM-6 on May 5, 2003 leading to greatly improved CONDUIT timeliness and reliability. The Unidata LDM-6 host at NSF began relaying CONDUIT as a top level node in August. Experience with LDM-6 on hosts conected over I2 has revealed that previously recomended 5-way splitting of CONDUIT requests may not be necessary at most sites.

• NWS will use LDM-6 to move NEXRAD Level II data. LDM-6.0.14 will be in the initial distributed build.

• The NWS has consolidated satellite imagery on the NOAAport broadcast to a single GOES East/West channel by implementing compression of images and moving non-CONUS images to channel 4 (aka OCONUS - dcp channel). The freed channel is now being used to test DVB-S distribution of the NWSTG data. Unidata has obtained a receiver and is working to incorporate this technology into the Unidata NOAAPORT system.

• IDD volumes for thelma.ucar.edu

  • Selected Inbound - September 25, 2003

  • Outbound - September 24, 2003

Contact Us   •   Site Map   •   Search   •   Terms and Conditions   •   Privacy Policy   •   Participation Policy

Unidata is a member of the UCAR Community Programs, is managed by the University Corporation for Atmospheric Research, and is sponsored by the National Science Foundation. P.O. Box 3000   •   Boulder, CO 80307-3000 USA   •   Tel: 303-497-8643   •   Fax: 303-497-8690