<div class="socmaildefaultfont" dir="ltr" style="font-family:Arial, Helvetica,
sans-serif;font-size:10pt" ><div dir="ltr" ><div><p>On behalf of my team, I’m
happy to announce the first release of HDF5-UDF: user-defined-functions for
HDF5 (and NetCDF4, by extension). The project enables the embedding of Lua
scripts in HDF5 so that users can programmatically define a dataset whose data
is generated on-the-fly each time that dataset is read.</p>
<p>The primary motivation for this project is to dramatically reduce the disk
space used by datasets that are a variation of existing data. We have
successfully used HDF-UDF to virtually eliminate the impact of derived data in
a number of use cases; grids that used to take a few gigabytes on disk,
uncompressed, now require just a couple of kilobytes.</p>
<p>Underneath, the source code is converted to a bytecode representation that
LuaJIT executes when the dataset is read by the application. Through
Just-In-Time compilation the overhead of virtualization is barely noticed:
outputting grids that have no dependency on existing datasets can be an order
of magnitude faster than reaching out to disk for I/O.</p>
<div>HDF5-UDF is shipped as a filter, which means that no modifications are
needed to applications that read from NetCDF4 files.</div>
<div> </div>
<div>The project is hosted at <a href="https://github.com/lucasvr/hdf5-udf";
>https://github.com/lucasvr/hdf5-udf</a></div>
<div> </div>
<div>We invite everyone to try it out and to open pull requests. We hope you
find it as useful as we do.</div>
<p>Thanks,<br>Lucas</p></div></div></div><BR>
