How do we scale up to petabyte datasets? Lessons from the adult in the room, relational databases.
[Read More]What can we learn from Relational Databases?
16 May 2011
How do we scale up to petabyte datasets? Lessons from the adult in the room, relational databases.
[Read More]Indexed data access and coordinate contract violations
10 May 2011
Indexed data access exposes the physical schema of datasets and makes violations of the relationship between coordinate and indices commonplace for datasets that change, such as aggregations of rolling archives. Making data requests in coordinate space will be added to the next generation of data access protocols, partly in order to solve this problem.
[Read More]Do blogs about Coordinates need to be monotonous?
04 May 2011
You may be shocked that netCDF coordinates can't always be monotonic. Other embarrassing facts are also revealed.
[Read More]Dataset Schemas are lost in GRIB datasets
25 April 2011
When model data is encoded in GRIB, the dataset schema is lost. So is an unambiguous identification of the the schema variables. To deal with these problems, GRIB reading libraries probably make assumptions that may not be valid for all datasets.
[Read More]On the suitability of BUFR and GRIB for archiving data
25 April 2011
Problems with external tables make BUFR/GRIB not suitable as long-term storage formats. To solve this, there must be a foolproof way for reading software to know what tables the writing software used, and there must be an authoritative registry of tables.
[Read More]