[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[netCDFJava #MQO-415619]: Efficiently serializing NetCDF in memory objects

This question has come up before.
This is currently not possible for netcdf-4 files.
Using the NC_DISKLESS mode flag allows for keeping
the file in memory. The nc_open_mem function allows for the
read only access to treat a chunk of memory as if it was
a netcdf file. Unfortunately, for netcdf-4, we ultimately
depend on the HDF5 operation H5P_set_core
(https://support.hdfgroup.org/HDF5/doc/RM/RM_H5P.html#Property-SetFaplCore).
Upon close(), this can optionally store the memory buffer in a file.
Unfortunately, it does not (currently) provide an option to copy out the memory.
The only solution I can see for now is to build an alternative to the core 
driver
that provides access (somehow) to the memory.
BTW, this is all going on at the netcdf-c library level. Our pure jave HDF5
reader is read-only, hence cannot create or modify files.

I have created an issue for this
(https://github.com/Unidata/netcdf-c/issues/708)
but it is not likely to get implemented anytime soon.

You will have to be content with writing the contents to a file.



> 
> I am using NetCDF Java library 4.6.10.
> 
> My goal is to efficiently send NetcdfFile
> <https://www.unidata.ucar.edu/software/thredds/current/netcdf-java/javadoc/ucar/nc2/NetcdfFile.html>
> objects over the network using a communication library (such as ZeroMQ).
> Because the NetcdfFile
> <https://www.unidata.ucar.edu/software/thredds/current/netcdf-java/javadoc/ucar/nc2/NetcdfFile.html>
> class
> does not implement Serializable, I need to implement that step myself.
> 
> Since NetCDF is a machine independent data format, I would like to access
> the raw binary data.
> 
> This is trivial if the data kas been written to a file on the disk. But,
> what about in memory datasets ? If possible, I would like to access to
> binary data without writing it to disk...
> 
> Can I access the buffer of an in-memory NetcdfFile
> <https://www.unidata.ucar.edu/software/thredds/current/netcdf-java/javadoc/ucar/nc2/NetcdfFile.html>
> object
> from the Java API ? Any pointers will be appreciated.
> 
> Kind regards,
> 
> Michaël
> 
> 

=Dennis Heimbigner
  Unidata


Ticket Details
===================
Ticket ID: MQO-415619
Department: Support netCDF Java
Priority: Normal
Status: Open
===================
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata 
inquiry tracking system and then made publicly available through the web.  If 
you do not want to have your interactions made available in this way, you must 
let us know in each email you send to us.