[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: 20030719: hdf-eos to netcdf conversion

>To: address@hidden
>cc: Suzana Djurcilov <address@hidden>
>From: Suzana Djurcilov <address@hidden>
>Subject: hdf-eos to netcdf conversion
>Organization: UC Santa Cruz
>Keywords: 200307190018.h6J0I0Ld012879 HDF to netCDF

Hi Suzana,

> Sorry to bring up an issue previously mentioned, but it seems like there
> ought to be a reasonable straightforward way to convert hdf-eos files to
> netcdf. I've contacted the support team of MODIS and they gave me a
> pointer to the MODIS API and suggested to contact you.
> 
> So far I've had (almost) no problem converting regular HDF version 4 files
> to netcdf, ncdump and ncgen accomplish that quite nicely. Whereas I am
> able to produce a HUUUGE ascii file from a MODIS dataset using the ncdump
> that comes with the HDF4.1 distribution, I get errors trying to make a
> binary netcdf file with the ncgen command from netcdf3.5. Basically it
> chokes on line 3, which is
> YDim:Grid = 4096 ;
> obviously it doesn't recognize the grid dimension nomenclature used by
> hdf-eos.

This is not enough information to determine whether there is a problem
with ncgen.  Is YDim a properly declared variable?  Dimensions can't
have attributes in the netCDF data model, so if YDim is merely a
dimension and not also a variable, then the above attempt to assign an
attribute to it would be in error.

Is the dataset you are trying to convert larger than 2 Gbytes?
NetCDF-3 can't handle arbitrary large datasets, but only those that
are structured in a particular way:

  http://www.unidata.ucar.edu/packages/netcdf/faq.html#lfs

The ncgen from netCDF 3.5 should recognize any file produced by the
ncdump from netCDF 3.5.  However, the ncdump that comes with the
HDF4.1 distribution is based on a much earlier version of netCDF, and
may contain problems (either in the ncdump code or in the netCDF 2.4
version on HDF 4 code) that have been fixed in later versions.

To diagnose whether there is a problem, we would need to be able to be
able to reproduce it.  Could you make available the output of "ncdump
-c" on the file you are trying to convert?  The "-c" says to print out
the header information as well as the coordinate variables, so it
shouldn't be too large.  If ncgen has trouble with that, then we can
provide a fix for the bug.  Otherwise, we will have to continue
debugging the problem.

> What do you suggest as a course of action ? I don't believe I'm the only
> person dealing with this problem - I'm sure an anwer which would be made
> part of the FAQ would be greatly appreciated by the scientific community.
> 
> Best regards and thanks ahead for any help you might be able to provide...

We didn't write and don't maintain or support HDF-EOS software or
conversion software from HDF-EOS to other forms, including netCDF.
The approach you have taken, using ncdump and ncgen for conversion, is
what I would recommend if you don't have the resources to write custom
software for this purpose.  We'll do what we can to make it work, if
you provide us enough information to diagnose the problems.  Thanks.

--Russ

_____________________________________________________________________

Russ Rew                                         UCAR Unidata Program
address@hidden                      http://my.unidata.ucar.edu