[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[netCDF #YYX-933412]: overwirte nc4 variable issues.

Wei,

I've looked at the destination file you made available,
 
  wrfrst_d01_2010-01-20_00_00_00_destination

and it's a netCDF-4 file (not a netCDF-4 classic model format).

The integer variables in that file are all defined with 
NC_ENDIAN_BIG, defined in "netcdf.h" to be 2.  But all the
floating-point variables in the file, including LU_INDEX,
seem to have been defined with NC_ENDIAN_NATIVE, defined in 
"netcdf.h" to be 0.  Do you know if the function 

  nc_def_var_endian(int ncid, int varid, int endian)

was explicitly called when defining any of the variables? 
Or was the destionation file generated by nccopy?

I'm asking because there may be another bug involved, if the
endianness of floating-point variables isn't properly 
represented in the destination file of nccopy ...

> This sounds like the open bug report, described in this Jira
> ticket:
> 
> https://bugtracking.unidata.ucar.edu/browse/NCF-291
> 
> The byte order for a variable is fixed when the variable is defined,
> but if values of the variable are later written on a platform with a
> different byte order, the values are not converted to the byte order
> of the platform on which the variable was first created.  This is a
> bug in netCDF-4 that is assigned for fixing, but I have just changed
> its priority from "Major" to "Critical", as you are the second person
> to report the bug.
> 
> The only workaround currently is for the user to check the endianness
> of the data in the netCDF-4 file using the nc_inq_var_endian()
> function. If it differs from the endianness of the platform on which
> the data will be overwritten or appended, the user must change the
> endianness of the in-memory array before writing the data.  That's
> clearly not the intended behavior, but a consequence of taking the
> "reader makes write" rule of HDF5 too far.
> 
> > I have two files:
> > met_em.d01.2010-01-20_12_00_00.nc
> > wrfrst_d01_2010-01-20_00_00_00_destination
> > Both have a variable named "LU_INDEX",
> > I want to read from first file, and then overwrite the one in the second.
> >
> > When tried to read back this variable from second file.
> > Then value is not right.
> > Attached is a C program which demonstrates this behavior.
> >
> > The above two file can get from:
> > ftp ftp.cgd.ucar.edu
> > login: anonymous
> > passwd: your-email
> > cd incoming
> > get file-name
> > ...
> > quit
> >
> > We have also found out that the overwritten variable is somehow
> > byte swapped. The following NCL script demonstrates this.
> >
> > f = addfile("met_em.d01.2010-01-20_12_00_00.nc", "r")
> > lu = f->LU_INDEX
> > lug = ndtooned(lu)
> > print(lug(::100))
> >
> > f = addfile("wrfrst_d01_2010-01-20_00_00_00_destination", "r")
> > lu := f->LU_INDEX
> > lut = ndtooned(lu)
> > print(lut(::100))
> >
> > setfileoption("bin","WriteByteOrder","BigEndian")
> > cbinwrite("test.bin",lu)
> > lu1 = cbinread("test.bin",-1,"float")
> > print(lu1(::100))
> >
> >
> > Could please check this issue, and see if we have done anything wrong,
> > or if there is any way we can control the byte order?
> Russ Rew                                         UCAR Unidata Program
> address@hidden                      http://www.unidata.ucar.edu
> 
> 
Russ Rew                                         UCAR Unidata Program
address@hidden                      http://www.unidata.ucar.edu



Ticket Details
===================
Ticket ID: YYX-933412
Department: Support netCDF
Priority: Normal
Status: Closed