[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[netCDFJava #PYP-475421]: Bug in NetCDF-Java 4.3 exposed by this GRIB2 file



> Lansing & John,
> 
> I can live with that, but it's unclear to me how to fix a collection of
> these in NcML, so if you have an example, that would be great.

One could argue this should be done by experts.
I think the right thing to do is to fix them on the TDS server.
There are several problems with CFSR that we are still working on.

> 
> Speaking of which, it would be very cool if there was a place (a wiki?)
> where the community could contribute annotated NcML that solves different
> problems.  On the IOOS Modeling Testbed, we have several complicated NcMLs
> that others have asked for as examples, and if there was a Unidata wiki
> that we could author, perhaps others would find them useful as well.

good idea.


> 
> -Rich
> 
> 
> address@hidden> wrote:
> 
> > Hi Rich:
> >
> > It is an error in the CFSR files (they have lots). We have a workaround in
> > NcML and if served from the TDS. Can you live with that?
> >
> > John
> >
> > > Lansing,
> > >
> > > I was expecting you to say that this issue was a bug in the GRIB
> > > reader, and it would get a ticket to be fixed.   But I think you are
> > > saying that the data was written wrong, and therefore the issue will
> > > not be fixed?
> > >
> > > This is one of those "big deal" datasets that it would be great to
> > > read correctly.
> > > http://rda.ucar.edu/pub/cfsr.html
> > >
> > > It's good to know that NcML can fix the issue for a file, but there
> > > are hundreds of these, so I would be better to fix the reader if you
> > > can.
> > >
> > > Thanks,
> > > -Rich
> > >
> > > On Tue, Jun 12, 2012 at 12:53 PM, Unidata netCDF Java Support
> > > <address@hidden> wrote:
> > > >> Lansing,
> > > >>
> > > >> It looks like you are using ToolsUI 4.2, not ToolsUI 4.3.   In toolsUI
> > > >> 4.2, you see one variable, but the times are wrong.   In 4.3, you see
> > > >> two variables with correct times, but we want one variable with
> > > >> correct time!
> > > >>
> > > >> Try this:
> > > >>
> > http://www.unidata.ucar.edu/software/netcdf-java/v4.3/webstart/netCDFtools.jnlp
> > > >>
> > > >> On Mon, Jun 11, 2012 at 1:10 PM, Unidata netCDF Java Support
> > > >> <address@hidden> wrote:
> > > >> >> NetCDF-Java folks:
> > > >> >>
> > > >> >> There appears to be a bug in the GRIB2 decoding problem using
> > > >> >> NetCDF-Java 4.3 on some of the surface pressure data we got from
> > > >> >> http://rda.ucar.edu/pub/cfsr.html
> > > >> >>
> > > >> >> This data is supposed to have a single variable (pressure) with
> > > >> >> uniform hourly sampling, but when we bring it up in NetCDF 4.3
> > tools,
> > > >> >> we see two variables instead of one, with the 1st having a uniform
> > > >> >> time base of 6 hours (0,6,12), and the 2nd having a non-uniform
> > time
> > > >> >> base that fills in the rest of the hourly data  (1,2,3,4,5
> > > >> >> 7,8,9,10,11  13,14...)
> > > >> >>
> > > >> >> Screenshot: http://screencast.com/t/DWEgKMfy
> > > >> >>
> > > >> >> I put a sample file here:
> > > >> >>
> > http://geoport.whoi.edu/thredds/catalog/usgs/data1/rsignell/test/catalog.html?dataset=usgs/data1/rsignell/test/pressfc.gdas.199801.grb2
> > > >> >>
> > > >> >> Thanks,
> > > >> >> Rich
> > > >> >> --
> > > >> >> Dr. Richard P. Signell   (508) 457-2229
> > > >> >> USGS, 384 Woods Hole Rd.
> > > >> >> Woods Hole, MA 02543-1598
> > > >> >>
> > > >> >>
> > > >> > Hello Rich,
> > > >> >
> > > >> > I'm not sure what's going on, but I tried your file and saw only
> > one variable reported.  I've attached a screenshot from NetCDF-Java
> > ToolsUI.  Maybe I'm missing something obvious here?
> > > >> >
> > > >> > Regards,
> > > >> >  Lansing Madry
> > > >> >  Unidata
> > > >> >  Boulder, Colorado
> > > >> >
> > > >> > Ticket Details
> > > >> > ===================
> > > >> > Ticket ID: PYP-475421
> > > >> > Department: Support netCDF Java
> > > >> > Priority: Normal
> > > >> > Status: Open
> > > >>
> > > >>
> > > >>
> > > >> --
> > > >> Dr. Richard P. Signell   (508) 457-2229
> > > >> USGS, 384 Woods Hole Rd.
> > > >> Woods Hole, MA 02543-1598
> > > >>
> > > >>
> > > > Rich,
> > > >
> > > > Thanks for educating me on 4.2 vs. 4.3 -- details, details!
> > > >
> > > > I dug a little deeper to see what is going on with the data in the
> > file you sent.  It looks like, at the byte level, there are some
> > differences that 4.2 conveniently ignored and 4.3 notices.  Specifically,
> > if you look at the file in the iosp/GRIB2/GRIB2Collection tab, you'll
> > notice that two variables seem to show up, with separate GDS hash codes.
> >  In that view, if you look down to the Grib2GridDefinitionSection,
> > highlight a row, right-click on the row, and select "Show GDS", you'll see
> > the information for Lo2 differs for the two "different" variables:
> > > >
> > > > Lo2 - longitude of last grid point == 359687000
> > > > Lo2 - longitude of last grid point == 359688000
> > > >
> > > > This is really a defect in how the data was written out - perhaps a
> > rounding error somewhere, or some other loss of precision.
> > > >
> > > > In the meantime, you can "correct" the issue using a bit of NcML to
> > change the GDS hashcode.  Details are here:
> > > >
> > > >
> > http://www.unidata.ucar.edu/software/netcdf-java/formats/GribFiles.html
> > > >
> > > > and I've tried this bit of NcML and found it to work in this case:
> > > >
> > > > <?xml version="1.0" encoding="UTF-8"?>
> > > > <netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2";
> > location="C:/Users/madry/rsignell/pressfc.gdas.199801.grb2">
> > > >  <iospParam>
> > > >   <gdsHash from="1450192070" to="1450218978"/>
> > > >  </iospParam>
> > > > </netcdf>
> > > >
> > > > However, you will need to delete the associated .ncx file before
> > opening this NcML file in the toolsUI-4.3 viewer.
> > > >
> > > > Let me know if this doesn't work for you, and I'll take another crack
> > at it.
> > > >
> > > > Cheers,
> > > >  Lansing
> > > >
> > > > Ticket Details
> > > > ===================
> > > > Ticket ID: PYP-475421
> > > > Department: Support netCDF Java
> > > > Priority: Normal
> > > > Status: Open
> > > >
> > >
> > >
> > >
> > > --
> > > Dr. Richard P. Signell   (508) 457-2229
> > > USGS, 384 Woods Hole Rd.
> > > Woods Hole, MA 02543-1598
> > >
> > >
> >
> >
> > Ticket Details
> > ===================
> > Ticket ID: PYP-475421
> > Department: Support netCDF Java
> > Priority: Normal
> > Status: Open
> >
> >
> 
> 
> --
> Dr. Richard P. Signell   (508) 457-2229
> USGS, 384 Woods Hole Rd.
> Woods Hole, MA 02543-1598
> 
> 


Ticket Details
===================
Ticket ID: PYP-475421
Department: Support netCDF Java
Priority: Normal
Status: Closed


NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publicly available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us.