Articles tagged: caron

Sep 6, 2011

Many GRIB writers are not versioning their files in ways that allow generic readers to find the correct tables. Tales of mystery and mayhem.

Aug 19, 2011
If you are archiving BUFR data, the future needs your help.
Jun 7, 2011

Indexes vs. indexes.  Gray's third requirement:

3. Intelligent indexes: for efficiently subsetting and filtering the data.

Jun 4, 2011

Gray's second desiderata gloria scientia: 

2. Schema language: powerful data definition tools allow one to specify the abstract data formats and to specify how the data is organized.

May 23, 2011

Talking about the first of Gray's criteria for scalable Scientific Data Management Systems:

1. Data independence: the physical data organization can change without breaking existing applications

May 16, 2011

How do we scale up to petabyte datasets? Lessons from the adult in the room, relational databases.

May 10, 2011

Indexed data access exposes the physical schema of datasets and makes violations of the relationship between coordinate and indices commonplace for datasets that change, such as aggregations of rolling archives. Making data requests in coordinate space will be added to the next generation of data access protocols, partly in order to solve this problem.

May 4, 2011

You may be shocked that netCDF coordinates can't always be monotonic. Other embarrassing facts are also revealed.

Apr 25, 2011

When model data is encoded in GRIB, the dataset schema is lost. So is an unambiguous identification of the the schema variables. To deal with these problems, GRIB reading libraries probably make assumptions that may not be valid for all datasets.

Apr 25, 2011

Problems with external tables make BUFR/GRIB not suitable as long-term storage formats. To solve this, there must be a foolproof way for reading software to know what tables the writing software used, and there must be an authoritative registry of tables.