Data Formats and Sources

Previous: Downloading and Running the IDV

1.3 Data Formats and Sources

The IDV can read a variety of data formats either from local files or remote data servers (e.g., HTTP, TDS, ADDE)(1). This page contains information about some data sources that work with the IDV.

To connect the IDV to data sources, see Choosing Data Sources.

1.3.0	Supported Data Types and Formats
1.3.1	Remote Data Servers
1.3.2	netCDF files
1.3.3	ASCII Text Point Data

1.3.0 Supported Data Types and Formats

Data Type

Description

Supported Formats

Access method

Gridded data

Numerical weather prediction models, climate analysis, gridded oceanographic datasets, NCEP/NCAR Reanalysis

- netCDF(2)

- GRIB (versions 1&2)

- Vis5D

- GEMPAK

- local files, HTTP, TDS servers

- local files, TDS servers

- local files, HTTP

- local files, TDS servers

Satellite imagery

Geostationary satellite imagery, MODIS, derived satellite products

- ADDE (3)

- McIDAS AREA

- GINI

- ADDE servers

- local files, ADDE servers

- local files, TDS servers

Radar data

NEXRAD Level II and Level III radar data

- Level II

- Level III

- local files or TDS (bzip2 compressed or uncompressed

- ADDE servers, local files or TDS

Point observations

Surface observations (METAR and SYNOP), earthquake observations

- ADDE

- netCDF (Unidata, AWIPS/MADIS formats)

- Text (ASCII, CSV), Excel spreadsheet4

- ADDE servers

- local files

Trajectories

Aircraft observations

- netCDF (RAF convention)

- local files

RAOBs

Global balloon soundings

- ADDE

- netCDF(Unidata, AWIPS/MADIS formats)

- CMA text format

- ADDE

- local files

Profiler

NOAA Profiler Network winds

ADDE

ADDE servers

GIS data

Data typically used in Geographic Information Systems (GIS)

- ESRI Shapefile

- USGS DEM

- local files, HTTP

- local files

QuickTime

QuickTime movies (without extensions)

QuickTime

Local files, HTTP

1.3.1 Remote Data Servers

Extensive meteorological and oceanographic data is available from remote data servers for use in research and education. Some of these data have restrictions on their use, see here for that information.

Abstract Data Distribution Environment (ADDE)

The Unidata community maintains a set of cooperating ADDE servers which serve up real-time and archived atmospheric datasets for use in IDV and McIDAS. Most of the data choosers in the IDV use ADDE as the access method (satellite imagery, Level III radar, surface, profiler and RAOB). The ADDE choosers are pre-configured with a list of these cooperating servers. You can use any of these to access the near-realtime data available to the Unidata community. For more information on accessing data on ADDE servers, see the Choosing Data Sources section.

Image data sets include:

RTIMAGES

GINIEAST

GINIWEST

GINICOMP

CIMSS

NEXRCOMP

You can configure defaults for particular images by creating a custom defaults file. For more information, see Configuring Image Defaults.

THREDDS Data Server (TDS)/OPeNDAP

The IDV can access gridded data (netCDF/GRIB/GEMPAK) and NEXRAD radar data stored on a THREDDS Data Server (TDS) through the OPeNDAP (formerly called DODS) protocol. See Choosing a Cataloged Data Source from a Remote Server for more information.

HTTP

Many of the data sources listed in the table above can read files directly from web servers (e.g. Apache) through the HTTP protocol. In most cases, the server must support the HTTP 1.1 protocol and be configured to set the "Content-Length" and "Accept-Ranges: bytes" headers. See the See Choose a URL for more information.

1.3.2 netCDF files

The Network Common Data Form ( netCDF) provides a common data access method for Unidata applications. This format can be used to store a variety of data types that encompass single-point observations, time series, regular grids, and satellite and radar images. The mere use of netCDF by itself is not sufficient to make data "self-describing" and meaningful to the IDV.

Generally, the IDV requires that datasets in netCDF format use meta data conventions to be able to fully understand and geolocate the dataset. These conventions provide documented "best practices". The whole point of using netCDF is to insure your data is complete and self-describing, and can be used by others. Using conventions helps ensure this. We recommend you use CF, COARDS, or NUWG conventions for netCDF data files for the IDV, and be sure to follow the best practices noted above.

1.3.3 ASCII Text Point Data

The IDV can read point data and trajectories (aircraft tracks) from comma-separated value (CSV) text files. See the documentation on the Text (ASCII) Point Data Format.

Unidata's Integrated Data Viewer > Overview