The Big Data Project (BDP) is an initiative undertaken by the National Oceanic and Atmospheric Administration (NOAA) to increase public availability of large volumes of environmental data collected and generated by the agency. As part of the Big Data Project, Unidata is working in collaboration with Amazon Web Services (AWS) on a demonstration project to provide access to a more than twenty years of archived NEXRAD Level II radar data — augmented continuously with new, real-time data — stored in Amazon's Simple Storage Service (S3) environment. In addition to assisting AWS with ingesting new data flowing from the NEXRAD sites, Unidata Program Center staff have set up a THREDDS Data Server in the AWS environment to provide services allowing community access to the stored data.

The Unidata Program Center (UPC) is searching for atmospheric science researchers or research groups to participate in a pilot project aimed at designing and implementing robust data management workflows. The project aims to assist at least three community partners representing modest research projects of different scales in the implementation of data management processes that satisfy National Science Foundation and other federal funding agency requirements.

Beyond simply satisfying current funding proposal requirements, the project hopes to test effective methods of collecting, transforming, storing, and sharing atmospheric science data. The methods used will be documented and polished for broad community use as examples serving to guide similar projects. If successful, the project will give researchers tools to satisfy funding agency requirements while making their data more widely discoverable, available, open, and usable by others in the community.

The HIWPP Open Data Initiative's first Users' Meeting will be held on Thursday, September 10th, from 1:00 to 3:00 pm Eastern Daylight Time. This will be a virtual meeting hosted via GoToMeeting.

The High Impact Weather Prediction Project (HIWPP) is a collaboration between a dozen or more organizations led by the NOAA Office of Oceanic and Atmospheric Research (OAR) Earth System Research Laboratory (ESRL) and the OAR/Office of Weather and Air Quality. Funded as part of the Hurricane Sandy Disaster Relief Supplemental Appropriations, the project aims to improve near term (from “now” to several weeks or months in the future) prediction of dangerous weather events including hurricanes, floods, and blizzards.

National Centers for Environmental Prediction (NCEP) Global Forecast System (GFS) model output with 0.25-degree resolution will be added to the Internet Data Distribution (IDD) CONDUIT data stream on July 28, 2015.

Update: GFS 0.25-degree output is now available via the CONDUIT data stream.

NCEP began producing the GFS model output with a 0.25-degree resolution for use in weather forecasting operations in January, 2015. Unidata Program Center staff have tested the 0.25-degree GFS model output internally and have been working with operators of top-level IDD relay sites to ensure that they have the capacity to handle the increased data volume associated with this new data stream. The approximate volume of the 0.25-degree GFS model output is 20 GB per model run, four times each day. For comparison, all of the current GFS model output delivered via CONDUIT (GFS 0.5-degree, 1-degree, and 2.5-degree) total approximately 5 GB per model run, four times each day.

The High Impact Weather Prediction Project (HIWPP) team has announced that output from the experimental, high-resolution NAVGEM model has been added to the HIWPP Open Data Initiative Real-time Data service.