LEAD at Unidata

Status Update July 2, 2007



For an overview of the LEAD project please see: LEAD Fact Sheet



Post Year Five Planning


Discussions are continuing regarding the post ITR phase of LEAD and Unidata's role in the project. Mohan and Kelvin held meetings with NCAR director Tim Killeen and Rich Loft, Deputy Director of NCAR's Computational and Information Systems Laboratory (CISL).  Both were supportive of the effort.  Tim Killeen said, "Please consider NCAR as a supporting center in LEAD's future."  However, specifics of NCAR's role in and contribution to LEAD-2 will need to be worked out.


Meanwhile, the PIs continue to dialog with NSF regarding a visit to NSF to discuss the future of the project.  Currently an attempt is underway to schedule a meeting sometime in August with:  Dan Atkins, head of the Office of Cyberinfrastructure (OCI), Steve Meacham, Program Director in OCI and the Directorate for Geosciences, Division of Atmospheric Sciences, Steve Nelson, director of NSF's mesoscale dynamic meteorology program, and Cliff Jacobs.  During the visit, Steve Meacham has requested a presentation by LEAD on the project's impact, thus far, on science.



Beta Users Program


The Beta Users program has been expanded to include all participants who have requested accounts through the LEAD portal.  To support these users while minimizing potential impact on the LEAD TeraGrid allocation, two new workflows have been introduced.  These are the Public NAM initialized workflow and the Public ADAS initialized workflow.  Users who are not part of LEAD, the WxChallenge or the SPC effort are provided only these workflows in their account.  These workflows are limited in forecast duration to a maximum of 12 hours. 


As the LEAD team gears up for the Fall WxChallenge, the LEAD development portal is being enhanced with new capabilities from the RENCI group that are intended to deal with the fault tolerance issues LEAD has faced throughout the Beta Users Program.  When a fault is found within a particular element of a workflow, the software will restart the element on a different TeraGrid resource.  This technique has been used successfully in a manual form to work around problems encountered when supporting the WxChallenge and the SPC effort.



Unidata LEAD Test Bed Status


We continue to improve upon data availability on the Unidata LEAD Test Bed system now boasting at least 6 months worth of the key LEAD datasets housed on the system (see the catalogs at http://lead.unidata.ucar.edu:8080/thredds/catalog.html and http://lead.unidata.ucar.edu:8080/thredds/topcatalog.html).  Overall reliability of the system has improved owing to better distribution of datasets across the RAIDed systems and improvements in the use of the file management configuration to reduce its loading on the system.  We are also working to better distribute and create fault tolerance for key systems including the TDS and the LDM.





The Current State of the TDR


TDR functionality has been expanded to include:

  • Editing of individual metadata elements
  • Generation of interactive HTML pages for an improved user experience
  • SSL encryption of requests to the TDR server


TDR Next Tasks

  • Move and copy operations to allow users to more easily structure and maintain the repository.
  • Enhanced performance, including support for multiple simultaneous clients and very large data volumes