You might be interested to know about a system we have been working on
for storing large amounts of spatio-temporal data. The system focuses
on ease of data access: after ingesting a dataset, it is available as
one large image from which you can extract subsections. This is done
using GDAL for import and export, and NetCDF-Java for viewing and
processing. The internal storage format is tiled NetCDF 4 classic.
The system was called the Raster Storage Archive during development;
now we call it SpatialCube. It's free software with commercial support
available . Here are some videos explaining what it does and the
motivation behind it:
Short video (2:34)
Long video (13:07) - more detail; includes examples of generated NCML
Perhaps of particular interest to this list: SpatialCube includes an
image processing system built on top of NetCDF-Java called rsaquery.
The videos introduce the query engine; details of the design can be
found in the repository on GitHub . The design of the storage
system is also documented there .
Questions, comments and collaboration are of course welcome.
VPAC Innovations Pty Ltd.