[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[netCDFJava #QAX-163810]: Data Value Discrepancies Between Netcdf 4.2 and 4.3



Hi Kevin:

There were a number of subtle problems with GRIB reading in 4.2, that we think 
we have fixed in 4.3. The most important was that time intervals coordinates 
were not correctly handled. You should assume that the 4.2 values are wrong. 

4.3 is a complete rewrite of the GRIB handling code, the variable names needed 
to be changed to disambiguate different variables that used the same GRIB 
parameter.

While we think that generally the 4.3 GRIB reading is correct, we have been 
unsatisfied with the way that reference time was handled there; this affects 
files (or collections of files) that have more than one reference time, with 
the same valid time. A coming version of the netcdf-java library (4.5) has 
another refactor in which these datasets expose both reference time and valid 
time. This allows users to see all of the data in the file. This version will 
be out as alpha in the next few weeks.

We would be glad to examine a specific file if you suspect that the 4.3 reading 
is wrong, or to compare with 4.2 to figure out why the data values are 
different. Even better would be if you have some independent verification 
process on those files that we could compare to.

John

> Hello,
> 
> I'm writing to inquire about some data value discrepancies observed when
> working with Netcdf-java versions 4.2 and 4.3.
> 
> We have a collection of RUC/RAP weather data sets in grb2 format reaching
> back to 2005 and have recently attempted to update our Netcdf dependencies
> to 4.3 form 4.2. There have been a series of discrepancies in the data
> values observed when copying a file using the 4.3 jar and then comparing
> the copied files data to the original files data.
> 
> This behavior has been observed in the following use cases
> 
> 1) Utilizing the example for V4.2 here
> 
> <https://www.unidata.ucar.edu/software/thredds/current/netcdf-java/tutorial/NetcdfWriteable.html>
> 
> we copied an entire files contents and then brought the copied file into
> a MATLAB environment using nctoolbox
> 
> <http://code.google.com/p/nctoolbox/downloads/list>
> 
> version 20091112 (underlying V4.2 Netcdf dependency). The data values
> matched exactly. Variable names matched exactly.
> 
> 2) Utilizing the example for V4.3 here
> 
> <http://www.unidata.ucar.edu/software/thredds/v4.4/netcdf-java/tutorial/NetcdfWriting.html>
> 
> we copied an entire files contents and then brought the copied file
> into a MATLAB environment using nctoolbox
> 
> <http://code.google.com/p/nctoolbox/downloads/list>
> 
> version 20091112(underlying V4.2 Netcdf dependency). The data values
> for 'uComponentOfWind' differed by as much as 120 knots compared to
> the original file. Other variables such as the lambertian X,Y grid
> had negligible variation.  Variable names differed between the copy
> and original.
> 
> 3) Utilizing the example for V4.3 here
> 
> <http://www.unidata.ucar.edu/software/thredds/v4.4/netcdf-java/tutorial/NetcdfWriting.html>
> 
> we copied an entire files contents and then brought the copied file
> into a MATLAB environment using nctoolbox
> 
> <http://code.google.com/p/nctoolbox/downloads/list>
> 
> version 20130305(underlying V4.3 Netcdf dependency). The data values
> for 'uComponentOfWind' differed by as much as 120 knots compared to
> the original file. Other variables such as the lambertian X,Y grid
> had negligible variation.  Variable names differed between the copy
> and original.
> 
> 4) We used nctoolbox 20091112(4.2 dependency) and version 20130305(4.3
> dependency) to separately open the same RUC grb2 file. The data values
> and variable names matched exactly
> 
> 5) Utilizing the 4.3 Tools jar we copied an entire file and brought it
> into matlab using both versions of nctoolbox separately. In both cases
> data values differed significantly and variable names differed.
> 
> 6) Utilizing the 4.2 tools jar we copied an entire file and brought it
> into matlab using both versions of nctoolbox separately. In both cases
> data values differed significantly and variable names differed.
> 
> As far as I can tell, anytime the file passes through the 4.3 jar
> (except for when the nctoolbox is involved) it alters the values of
> the underlying data. Upon request I can provide the data files and
> the code we are running.  From the netcdf website it appears that grb2
> handling altered drastically between 4.2 and 4.3, any assistance you
> could provide to help identify which approach provides the correct data
> would be greatly appreciated.
> 
> Thanks!
> 
> Sincerely,
> Kevin Ray
> 
> Kevin Ray
> Computer Science, Sr
> 3N306, MITRE2, McLean, VA
> address@hidden<mailto:address@hidden>
> (703)983-2288
> 
> 
> 

Ticket Details
===================
Ticket ID: QAX-163810
Department: Support netCDF Java
Priority: Normal
Status: Closed