McIDAS, ldm-mcidas, Satellite Data Ingest

Status Report: October 2013 - March 2014

Tom Yoksas

Strategic Focus Areas

McIDAS activities support the following Unidata funding proposal focus areas:

  1. Enable widespread, efficient access to geoscience data
    McIDAS remains the application of choice for the satellite meteorology community.
    The Advanced Data Distribution Environment (ADDE) componet of McIDAS was the first application offered by Unidata to provide remote, programmatic access to a wide variety of data that is important to the atmospheric science community.
  2. Develop and provide open-source tools for effective use of geoscience data The fifth generation of McIDAS, McIDAS-V, unlike its predecessors, is a fully open source application that is is in wide scale and growing use in the satellite meteorology community.
    McIDAS ADDE continues to evolve and provide access to increasing volumes of image and non-image data.
  3. Provide cyberinfrastructure leadership in data discovery, access, and use
    Concepts articulated in ADDE inspired the development of THREDDS (to address the lack of rich metadata available in ADDE) and RAMADDA. ADDE remains one of the most used data services in the Unidata suite serving over 3 TB per month.
  4. Build, support, and advocate for the diverse geoscience community
    McIDAS is sought for use by those interested in satellite meteorology worldwide.

Activities Since the Last Status Report

Unidata McIDAS v2009q released on March 8, 2014

Unidata McIDAS version 2009q includes all SSEC versions up to and including the current release, v2013.1 and Unidata updates and bugfixes. Changes to Unidata McIDAS continue to be made through an addendum process. The current release, v2009q, reflects 17 updates since McIDAS v2009 was first made available in late July, 2009. v2009q is the 2013 release of McIDAS-X.

SSEC McIDAS Advisor Committee (MAC)

The UPC (Yoksas) continues to participate as the Unidata representative to the McIDAS Advisory Committee (MAC) that is operated by SSEC. IDV developers (Yuan Ho) have been participating in the MAC recently mainly to help MAC members understand new features being added to the IDV.

The MAC was assembled by UW/SSEC to advise SSEC on McIDAS-X users needs/concerns/desires for development in the next generation McIDAS, McIDAS-V. The MAC was modeled after the Unidata IDV Steering Committee.

Interest in McIDAS by Non-core Users

The UPC continues to receive requests for McIDAS from international university users, U.S. government agencies and other non-traditional Unidata users (e.g., private businesses, etc.). Government agencies and non-traditional Unidata users are referred to UW/SSEC for access to McIDAS; international educational community user requests are granted on a case-by-case basis after they provide a clear statement of their acceptance of the terms of use provided by SSEC.

Planned Activities

Ongoing Activities

Continued support of existing and new community members.

New Activities

Implement an indexing scheme for ADDE image datasets to speed up access especially in large and archive datasets. A preliminary design for ADDE image dataset indexing has been made. Investigations for how to integrate the new capbilities in to the suite of existing ADDE servers is in progress.

Add support for new types of data when they become available, otherwise McIDAS-X support is in maintenance mode.

Relevant Metrics

  • Internet2 (I2) bandwidth usage by the McIDAS ADDE protocol routinely exceeds several TB/week. This ranks second in Advanced Applications use behind the LDM.
  • McIDAS Inquiry Metrics

ldm-mcidas Decoders Activities

Development

ldm-mcidas releases are made when needed to support changes in software development and operating system environments. ldm-mcidas v2012 was released at the end of September, 2012. This release addresses building on newer OS versions.

Geostationary Satellite Data Ingest

Unidata continues to ingest GOES-East and GOES-West imager data at the UCAR Foothills Lab campus in Boulder. GOES-South (GOES-South America) was decommissioned on August 16, 2013, and there appears to be no current plans for repurposing an existing GOES platform for South American surveillance.

  • Direct, programmatic access to real-time GOES-South (GOES-12) data via McIDAS ADDE had been used by over 820 users in 33 countries who downloaded approx. 520 GB of data per month over the past year.
  • Direct, programmatic access to real-time GOES-East (GOES-13) data via McIDAS ADDE is being used by approx. 35 users who downloaded an average of approx. 1 TB of data per month over the past year.
  • Direct, programmatic access to real-time GOES-West (GOES-15) data via McIDAS ADDE is used by approx. 25 users have downloaded an average of 800 GB of data per month for the past year.

Planned Activities

Ongoing Activities

Continued ingest and serving of GOES-East and GOES-West imagery. This effort sporadic requires maintenance of the satellite ingest and computer data equipment.

New/future Activities

Repurpose former USAN dish at Mesa Lab to operation as a remotely controllable ingester for any of the GOES platforms. This is a moderately low priority activity.

Proposed Activities

Begin planning for the resources it will take to ingest and disseminate GOES-R data (which is currently scheduled to become availble in 2015). This activity will proceed with cooperation/coordination of NCAR/RAL, NCAR/EOL and NOAA. A draft DRAFT Executive Summary and Budget (i.e., a non-proposal "proposal") was developed in cooperation with RAL and EOL, and submitted to Steve Goodman who is in NOAA's GOES-R office. After submittal, NOAA decided to postpone making a decision on funding GOES-R ingest capabilities until the summer of 2013 at the earliest.