| • |
|
|
|
|||
|
||||
Tom Baltzer
Brian Kelly
Mohan
Ramamurthy
Anne Wilson
At the LEAD all hands meeting in Chicago last October, LEAD project members agreed upon a goal of providing software to support a "LEAD Day" at our Unidata User's Workshop this summer. The intent is to showcase LEAD and demonstrate its initial capabilities to the atmospheric science community as well as obtain their feedback and review of LEAD technologies and ideas. The primary LEAD functional goal is to allow up to 30 workshop participant teams of 1-3 people to discover and visualize atmospheric datasets being stored, cataloged and served by LEAD, and then run WRF forecasts using LEAD data holdings for initial and boundary conditions and NCSA compute facilities and then view the result. The location and start time of the domains for these forecasts will be determined individually by workshop participants.
Towards that end LEAD has been testing and deploying supporting functionality. Once the capabilities have passed internal alpha testing, the capabilities will be handed to a group of volunteer Beta testers external to the LEAD enterprise. Unidata staff members have aided this effort by specifying required degrees of code hardening and robustness, gathering and organizing testing material, and performing developer testing.
Work at Unidata has been to support two different orchestration paradigms being developed within the LEAD team, that of the Indiana University Workflow Architecture and of the NCSA Ensemble Broker Architecture. One or both of these paradigms will be used to support the summer workshop. By working with both paradigms we have a wider variety of use cases for the THREDDS Data Repository (TDR) development. We have also leveraged the steered WRF work at Unidata by getting these capabilities working on the NCSA architecture.
Work has continued to build out the 40 TB storage array on the Unidata LEAD Test Bed system. At present we have over 3 months of IDD data available on the system and being made available to LEAD and the Unidata Community via the THREDDS Data Server (TDS). The original design of 2 tiered RAID systems met with some technical difficulties resulting in the need to restructure our approach. We now have six 8 TB nodes being accessed individually as opposed to them appearing as a single 40 TB node with redundant RAID capabilities. This work is providing an excellent test case for the underlying Unidata technologies.
With Doug's departure from Unidata, Anne has assumed responsibility for the TDR.
TDR development is currently being driven by the two use cases described under "LEAD at the Unidata Summer Workshop", above. The TDR is being integrated into the TDS to provide a way to move and store data served by the TDS. We are currently calling the combined server the THREDDS Data Server and Repository (TDSR).
We have presented a scenario to LEAD depicting the role of the TDR in running the WRF forecasts for the summer workshop. The next steps on integration are under discussion.
We have presented a similar scenario to our colleagues at NCSA that uses their Ensemble Broker for orchestration. That architecture has been approved by the working group, and development is ongoing as details continue to be worked out.
Currently the TDR can store and retrieve individual files to/from a UNIX disk on the Unidata LEAD test bed. It has a command line interface and a web interface.
Unidata had provided and is responsible for the code that generates LEAD metadata from THREDDS metadata. This code is being used as LEAD crawls THREDDS LEAD catalogs on the Unidata LEAD Testbed and generates entries for the LEAD Resource Catalog. We continue to work with LEAD team members to maintain and upgrade this code.
| Contact Us • Site Map • Search • Terms and Conditions • Privacy Policy • Participation Policy | ||||||
|
||||||
