Data Proximate Computing

Data Proximate Computing

The NSF Unidata Science Gateway combines access to compute resources via JupyterHub with Internet2 connectivity and NSF Unidata's expertise in moving and serving large quantities of data. Gateway users can benefit from a variety of services deployed on the Jetstream2 cloud, enabling data proximate computing and rapid data transfer rates. By connecting to the Gateway, users can perform analysis alongside the data and circumvent much of the wait times associated with fetching data.

Internet Data Distribution System Connectivity

NSF Unidata maintains a node of the NSF Unidata Internet Data Distribution (IDD) system on Jetstream2 in order to deliver near real-time data to Science Gateway visitors. Users can browse and access a recent archive of radar, satellite, and gridded data via the Science Gateway's THREDDS Data Server (TDS) installation. If you know of a dataset which is distributed via the IDD but is not accessible on the Gateway, contact the Science Gateway team at support-gateway@unidata.ucar.edu.

AWIPS EDEX

NSF Unidata develops and maintains a version of the National Weather Service's Advanced Weather Interactive Processing System (AWIPS) for educational and research use. Users wishing to use the visual AWIPS CAVE client can connect to the Science Gateway's AWIPS EDEX data server located at edex-cloud.unidata.ucar.edu. Alternatively, data can be accessed programmatically via the python-awips library.

Python-awips notebook example gallery

THREDDS Nexrad

As part of NOAA's Open Data Dissemination (NODD) Program, real-time and historical NEXRAD Level II data is made publicly available on an Amazon Web Services (AWS) S3 bucket. NSF Unidata has exposed this dataset via a THREDDS Data Server installation on the Science Gateway to increase accessibility for those who already integrate the TDS into their data analysis workflows. You can find it at tds-nexrad.scigw.unidata.ucar.edu.

THREDDS Data Server

The THREDDS Data Server found at tds.scigw.unidata.ucar.edu serves data requested from the IDD to Science Gateway visitors via a variety of common protocols, including HTTPS, OPeNDAP, OGC WCS, and OGC WMS. Data can be accessed programmatically via notebooks or scripts using tools that implement these protocols, such as NSF Unidata's Siphon Python library, or can be downloaded to your local machine by visiting the TDS in your browser. The table below lists available datasets.

Data available on tds.scigw.unidata.ucar.edu
SourceDays Available

Forecast Model Data

  • National Centers for Environmental Prediction (NCEP)
    • Global Forecast System (GFS)
    • High Resolution Rapid Refresh (HRRR)
    • North American Model (NAM)
    • Rapid Refresh (RAP)
    • Short Range Ensemble Forecasting (SREF)
    • Wave Watch III (WW3)
  • Canadian Meteorological Centre (CMC)
    • Regional Deterministic Prediction System (RDPS)
1

Forecast Products and Analyses

  • National Centers for Environmental Prediction (NCEP)
    • National Digital Forecast Database (NDFD)
    • Real Time Mesoscale Analysis (RTMA)
    • Multi-Radar Multi-Sensor (MRMS) Analysis
    • NCEP National Blend of Models
1

Radar Data

  • NEXRAD Level III
  • TDWR Level III
  • NEXRAD Composites
1

Satellite Data

  • GOES East Products
  • GOES West Products
  • GOES 17 Products
  • GOES 18 Products
1