I like John's suggestion here.
But, any code you add to any applications now will work *only* for files that
were produced post-adoption of this convention.
There are probably a bazillion files out there at this point that don't follow
that convention and you probably still want your applications to be able to
read them.
In VisIt, we support >140 format readers. Over 20 of those are different
variants of HDF5 files (H5part, Xdmf, Pixie, Silo, Samrai, netCDF, Flash, Enzo,
Chombo, etc., etc.) When opening a file, how does VisIt figure out which plugin
to use? In particular, how do we avoid one poorly written reader plugin (which
may be the wrong one for a given file) from preventing the correct one from
being found. Its kinda a hard problem.
Some of our discussion is captured here. . .
http://www.visitusers.org/index.php?title=Database_Format_Detection
Mark
From: Hdf-forum
<hdf-forum-bounces@xxxxxxxxxxxxxxxxxx<mailto:hdf-forum-bounces@xxxxxxxxxxxxxxxxxx>>
on behalf of John Shalf <jshalf@xxxxxxx<mailto:jshalf@xxxxxxx>>
Reply-To: HDF Users Discussion List
<hdf-forum@xxxxxxxxxxxxxxxxxx<mailto:hdf-forum@xxxxxxxxxxxxxxxxxx>>
Date: Wednesday, March 2, 2016 1:02 PM
To: HDF Users Discussion List
<hdf-forum@xxxxxxxxxxxxxxxxxx<mailto:hdf-forum@xxxxxxxxxxxxxxxxxx>>
Cc: "netcdfgroup@xxxxxxxxxxxxxxxx<mailto:netcdfgroup@xxxxxxxxxxxxxxxx>"
<netcdfgroup@xxxxxxxxxxxxxxxx<mailto:netcdfgroup@xxxxxxxxxxxxxxxx>>, Ward
Fisher <wfisher@xxxxxxxx<mailto:wfisher@xxxxxxxx>>
Subject: Re: [Hdf-forum] Detecting netCDF versus HDF5
Perhaps NetCDF (and other higher-level APIs that are built on top of HDF5)
should include an attribute attached to the root group that identifies the name
and version of the API that created the file? (adopt this as a convention)
-john
On Mar 2, 2016, at 12:55 PM, Pedro Vicente
<pedro.vicente@xxxxxxxxxxxxxxxxxx<mailto:pedro.vicente@xxxxxxxxxxxxxxxxxx>>
wrote:
Hi Ward
As you know, Data Explorer is going to be a general purpose data reader for
many formats, including HDF5 and netCDF.
Here
http://www.space-research.org/
Regarding the handling of both HDF5 and netCDF, it seems there is a potential
issue, which is, how to tell if any HDF5 file was saved by the HDF5 API or by
the netCDF API?
It seems to me that this is not possible. Is this correct?
netCDF uses an internal function NC_check_file_type to examine the first few
bytes of a file, and for example for any HDF5 file the test is
/* Look at the magic number */
/* Ignore the first byte for HDF */
if(magic[1] == 'H' && magic[2] == 'D' && magic[3] == 'F') {
*filetype = FT_HDF;
*version = 5;
The problem is that this test works for any HDF5 file and for any netCDF file,
which makes it impossible to tell which is which.
Which makes it impossible for any general purpose data reader to decide to use
the netCDF API or the HDF5 API.
I have a possible solution for this , but before going any further, I would
just like to confirm that
1) Is indeed not possible
2) See if you have a solid workaround for this, excluding the dumb ones,
for example deciding on a extension .nc or .h5, or traversing the HDF5 file to
see if it's non netCDF conforming one. Yes, to further complicate things, it is
possible that the above test says OK for a HDF5 file, but then the read by the
netCDF API fails because the file is a HDF5 non netCDF conformant
Thanks
----------------------
Pedro Vicente
pedro.vicente@xxxxxxxxxxxxxxxxxx<mailto:pedro.vicente@xxxxxxxxxxxxxxxxxx>
http://www.space-research.org/
_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@xxxxxxxxxxxxxxxxxx<mailto:Hdf-forum@xxxxxxxxxxxxxxxxxx>
http://secure-web.cisco.com/1r-EJFFfg6rWlpQsvXstBNTjaHQaKT_NkYRN0Jj_f-Z3EK0-hs6IbYc8XUBRyPsH3mU3CS0iiY7_qnchCA0QxNzQt270d_2HikCwpAWFmuHdacin62eaODutktDSOULIJmVbVYqFVSKWPzoX7kdP0yN9wIzSFxZfTwfhU8ebsN409xRg1PsW_8cvNiWzxDNm9wv9yBf9yK6nkEm-bOx2S0kBLbg9WfIChWzZrkpE3AHU9I-c2ZRH_IN-UF4g_g0_Dh4qE1VETs7tZTfKd1ox1MtBmeyKf7EKUCd3ezR9EbI5tK4hCU5qW4v5WWOxOrD17e8yCVmob27xz84Lr3bCK5wIQdH5VzFRTtyaAhudpt9E/http%3A%2F%2Flists.hdfgroup.org%2Fmailman%2Flistinfo%2Fhdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@xxxxxxxxxxxxxxxxxx<mailto:Hdf-forum@xxxxxxxxxxxxxxxxxx>
http://secure-web.cisco.com/1r-EJFFfg6rWlpQsvXstBNTjaHQaKT_NkYRN0Jj_f-Z3EK0-hs6IbYc8XUBRyPsH3mU3CS0iiY7_qnchCA0QxNzQt270d_2HikCwpAWFmuHdacin62eaODutktDSOULIJmVbVYqFVSKWPzoX7kdP0yN9wIzSFxZfTwfhU8ebsN409xRg1PsW_8cvNiWzxDNm9wv9yBf9yK6nkEm-bOx2S0kBLbg9WfIChWzZrkpE3AHU9I-c2ZRH_IN-UF4g_g0_Dh4qE1VETs7tZTfKd1ox1MtBmeyKf7EKUCd3ezR9EbI5tK4hCU5qW4v5WWOxOrD17e8yCVmob27xz84Lr3bCK5wIQdH5VzFRTtyaAhudpt9E/http%3A%2F%2Flists.hdfgroup.org%2Fmailman%2Flistinfo%2Fhdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
