[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[netCDF #DKW-240688]: nc_rename_var(), nc_rename_dim() bugs in netCDF4?

> Thanks for the quick update.
> Sorry to hear that this problem affects non-NCO software as well.
> 
> In case it is useful, please know that
> The NCO ncrename documentation summarizes issues with renaming
> objects in netCDF4 datasets that we have encountered over the years:
> 
> http://nco.sf.net/nco.html#bug_nc4_rename
> 
> I've updated that section to include what is known about
> this latest problem.

Here's a partial workaround for renaming dimensions and variables when
the target is a netCDF-4 file, but not netCDF-4 classic model.  It seems
that renaming a dimension and corresponding coordinate variable works
OK in this case, as long as you do not call nc_redef before the rename
and nc_enddef after (though that shouldn't matter):

In other words, this works in my test:

        if (nc_open(FILE_NAME, NC_WRITE, &ncid)) ERR;
        if (nc_inq_dimid(ncid, DIM_NAME, &dimid)) ERR;
        if (nc_inq_varid(ncid, VAR_NAME, &varid)) ERR;
        /* if (nc_redef(ncid)) ERR; /\* not necessary and not allowed ? *\/ */
        if (nc_rename_dim(ncid, dimid, NDIM_NAME)) ERR;
        if (nc_rename_var(ncid, varid, NVAR_NAME)) ERR;
        /* if (nc_enddef(ncid)) ERR; /\* not necessary and not allowed ? *\/ */
        if (nc_get_var(ncid, varid, lats_in)) ERR;

but uncommenting the nc_redef and nc_enddef calls corrupts the variable
values with fill values.  The nc_redef and nc_enddef calls are of course
required for the other 3 format variants.

--Russ

> On 10/09/2014 08:07 AM, Unidata netCDF Support wrote:
> > Hi Charlie,
> >
> >> A user has pointed out a serious problem with ncrename.
> >> As best as I can tell, this traces back to a netCDF4 library bug.
> >> Please investigate and let me know if this is true or not.
> >
> > You're right, this looks fairly serious. We haven't finished
> > determining the scope of the bug, but we'll let you know soon. A
> > posting to netcdfgroup may be needed to warn users not to trust
> > coordinate data in coordinate variables that have been renamed.
> >
> > Thanks for reporting the bug.
> >
> > --Russ
> >
> >> The bug apparently affects (at least) versions 4.3.1--4.3.2 of the
> >> netCDF4 library (and current snapshots).
> >> This bug corrupts values contained in netCDF4 arrays (not scalars)
> >> that are renamed or that utilize dimensions that are renamed.
> >> In other words, renaming variables and dimensions succeeds
> >> yet it corrupts the values contained by the affected array variables.
> >> This bug corrupts affected variables by replacing their values with the
> >> default _FillValue for that variable's type.
> >> Attached is a file whose values are corrupted when either the
> >> dimension or the variables or both are renamed.
> >>
> >> ncks -O -4 -v lat -C -h -m -M ~/nco/data/in.nc ~/bug.nc # Create test
> >> ncrename -O -v lat,tal ~/bug.nc ~/foo.nc # Broken
> >> ncrename -O -d lat,tal ~/bug.nc ~/foo.nc # Broken
> >> ncrename -O -d lat,tal -v lat,tal ~/bug.nc ~/foo.nc # Broken
> >> ncks --cdl ~/foo.nc
> >>
> >> The only known workaround is to convert the file to netCDF3 first,
> >> then rename as intended, then convert back. I checked renaming of
> >> groups and attributes in netCDF4 files and these operations both
> >> perform as intended. So the problem appears to be only with with
> >> renaming dimensions and renaming array variables in netCDF4 files.
> >>
> >> Thanks!
> >> Charlie
> >>
> >> zender@givre:~$ ncks --cdl ~/bug.nc
> >> netcdf bug {
> >> dimensions:
> >> lat = 2 ;
> >>
> >> variables:
> >> float lat(lat) ;
> >>
> >> data:
> >> lat = -90, 90 ;
> >>
> >> } // group /
> >> zender@givre:~$ ncrename -O -v lat,tal ~/bug.nc ~/foo.nc
> >> ncrename: In total renamed 0 attributes, 0 dimensions, 0 groups, and 1
> >> variable
> >> zender@givre:~$ ncks --cdl ~/foo.nc
> >> netcdf foo {
> >> dimensions:
> >> lat = 2 ;
> >>
> >> variables:
> >> float tal(lat) ;
> >>
> >> // global attributes:
> >> :history = "Wed Oct  8 10:24:15 2014: ncrename -O -v lat,tal
> >> /home/zender/bug.nc /home/zender/foo.nc" ;
> >>
> >> data:
> >> tal = 9.96921e+36, 9.96921e+36 ;
> >>
> >> } // group /
> >> zender@givre:~$
> >>
> >> --
> >> Charlie Zender, Earth System Sci. & Computer Sci.
> >> University of California, Irvine 949-891-2429 )'(
> >>
> >>
> >
> > Russ Rew                                         UCAR Unidata Program
> > address@hidden                      http://www.unidata.ucar.edu
> >
> >
> >
> > Ticket Details
> > ===================
> > Ticket ID: DKW-240688
> > Department: Support netCDF
> > Priority: Normal
> > Status: Closed
> >
> 
> --
> Charlie Zender, Earth System Sci. & Computer Sci.
> University of California, Irvine 949-891-2429 )'(
> 
> 

Russ Rew                                         UCAR Unidata Program
address@hidden                      http://www.unidata.ucar.edu



Ticket Details
===================
Ticket ID: DKW-240688
Department: Support netCDF
Priority: Normal
Status: Closed