[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[THREDDS #GJF-200148]: TDS 4.3 GRIB names different from TDS 4.2



Hi Fan:

We have a bug when a datasetScan has NcML that has an iospParam. We are working 
on a fix for version 4.4.
At the moment, the only workaround i can think of is to add the table into our 
standard tables, see

  
http://www.unidata.ucar.edu/software/thredds/current/netcdf-java/formats/GribTables.html

thanks for reporting the problem,
John


> Hi Lansing,
> 
> Thanks and we just installed the 4.3.19 version from your link.  Seems my 
> problem remains - the variables in the grib table are not showing up with 
> their correct names.  Wonder if you checked the fix.  The data and the grib 
> table are still at my ftp site:
> 
> ftp://hydro1.sci.gsfc.nasa.gov/private/ffang/GLDAS_CLM10SUBP_3H.A1979002.0000.001.2008086151108.grb
> ftp://hydro1.sci.gsfc.nasa.gov/private/ffang/gribtab_clm.tab
> 
> and my TDS config has these lines:
> 
> <datasetScan name="GLDAS_CLM10SUBP_3H" ID="GLDAS_CLM10SUBP_3H" 
> path="GLDAS_CLM10SUBP_3H" 
> location="/ftp/data/s4pa_TS2/TEST_OPENDAP/GLDAS_CLM10SUBP_3H/">
> <netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2"; 
> iospParam="gribParameterTable=/home/ffang/gribtab_clm.tab">
> </netcdf>
> <metadata inherited="true">
> <serviceName>odapncss</serviceName>
> <dataType>Grid</dataType>
> </metadata>
> <filter>
> <include wildcard="*.grb"/>
> </filter>
> </datasetScan>
> 
> It is possible we did something wrong.  If it works for you, could you send 
> us the configuration lines?
> 
> Thanks.
> 
> -Fan
> 
> ________________________________________
> From: Unidata THREDDS Support address@hidden
> Sent: Tuesday, September 03, 2013 2:14 PM
> To: Fang, Fan (GSFC-610.2)[ADNET SYSTEMS INC]
> Cc: address@hidden; Fang, Fan (GSFC-610.2)[ADNET SYSTEMS INC]; 
> address@hidden; Fang, Fan (GSFC-610.2)[ADNET SYSTEMS INC]
> Subject: [THREDDS #GJF-200148]: TDS 4.3 GRIB names different from TDS 4.2
> 
> Fan,
> 
> You can download a 4.3.19 snapshot here:
> 
> https://docs.google.com/file/d/0B7jP7ooGP2ZzdFh5NUprdEowMkE/edit?usp=sharing
> 
> Enjoy!
> 
> -Lansing
> 
> On 9/3/2013 11:24 AM, Fan Fang wrote:
> > New Client Reply: TDS 4.3 GRIB names different from TDS 4.2
> >
> > Hi Lansing,
> >
> > I notice that the bug is fixed in version 4.3.18.  Is it downloadable
> > from somewhere?
> >
> > Thanks.
> >
> > -Fan
> >
> > On 05/23/2013 04:37 PM, Unidata THREDDS Support wrote:
> >> Fan,
> >>
> >> I reproduced the behavior locally, and it is a bug.  Thanks for finding 
> >> and reporting it!  I've set up bug-ticket in jira, our issue tracking 
> >> system, which is here:
> >>
> >> https://bugtracking.unidata.ucar.edu/browse/TDS-435
> >>
> >> The developer who really knows grib is out for a long weekend, but I'll 
> >> make sure he sees it next week.
> >>
> >> -Lansing
> >>
> >>> Hi Lansing,
> >>>
> >>> You can get as many as you want from here:
> >>>
> >>> ftp://hydro1.sci.gsfc.nasa.gov/data/s4pa/GLDAS_V1/GLDAS_CLM10SUBP_3H/1979/002/
> >>>
> >>> My sample was the first file (at top) of that day.  The rest .grb files 
> >>> represent the hourly data for that day.  For more than one-day worth of 
> >>> data, go to 1979/003, 1979/004, etc. for year 1979.
> >>>
> >>> -Fan
> >>> ________________________________________
> >>> From: Unidata THREDDS Support address@hidden
> >>> Sent: Thursday, May 23, 2013 1:37 PM
> >>> To: Fang, Fan (GSFC-610.2)[ADNET SYSTEMS INC]
> >>> Cc: address@hidden; Fang, Fan (GSFC-610.2)[ADNET SYSTEMS INC]; 
> >>> address@hidden; Fang, Fan (GSFC-610.2)[ADNET SYSTEMS INC]
> >>> Subject: [THREDDS #GJF-200148]: TDS 4.3 GRIB names different from TDS 4.2
> >>>
> >>> Fan,
> >>>
> >>> Can you put a few more of the grb files up?
> >>>
> >>> -Lansing
> >>>
> >>>> Hello Lansing,
> >>>>
> >>>> Thanks for the tip.  You can get the sample data and grib table from:
> >>>>
> >>>> ftp://hydro1.sci.gsfc.nasa.gov/private/ffang/GLDAS_CLM10SUBP_3H.A1979002.0000.001.2008086151108.grb
> >>>> ftp://hydro1.sci.gsfc.nasa.gov/private/ffang/gribtab_clm.tab
> >>>>
> >>>> My TDS configuration for the dataset is:
> >>>>
> >>>> <datasetScan name="GLDAS_CLM10SUBP_3H" ID="GLDAS_CLM10SUBP_3H"
> >>>> path="GLDAS_CLM10SUBP_3H"
> >>>> location="/ftp/data/s4pa_TS2/TEST_OPENDAP/GLDAS_CLM10SUBP_3H/">
> >>>> <netcdf
> >>>> xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2";
> >>>> iospParam="gribParameterTable=/home/ffang/gribtab_clm.tab">
> >>>> </netcdf>
> >>>> <metadata inherited="true">
> >>>> <serviceName>odapncss</serviceName>
> >>>> <dataType>Grid</dataType>
> >>>> </metadata>
> >>>> <filter>
> >>>> <include wildcard="*.grb"/>
> >>>> </filter>
> >>>> </datasetScan>
> >>>>
> >>>> -Fan
> >>>>
> >>>> On 05/22/2013 03:55 PM, Unidata THREDDS Support wrote:
> >>>>> Fan,
> >>>>>
> >>>>> Two things - first, don't serve the ncx files.  It doesn't matter where 
> >>>>> they are located (in the data directory or elsewhere), but it's not 
> >>>>> meant to be served as data.
> >>>>>
> >>>>> Second, it sounds like you are describing a bug in how user tables are 
> >>>>> applied.  I'd like to set up a tds locally in a debugger with your 
> >>>>> files, collection spec, and local grib table.  Please bundle them up 
> >>>>> and send them to me, or point me to where I can get them.
> >>>>>
> >>>>> Thanks,
> >>>>>    Lansing
> >>>>>
> >>>>>> Hello Lansing,
> >>>>>>
> >>>>>> Any comments/findings about my questions?
> >>>>>>
> >>>>>> With or without featureCollection we need to serve grib data in their 
> >>>>>> granule form, so how to make grib table work for individual granules 
> >>>>>> is remaining question number one.  So far our grib table works with 
> >>>>>> aggregated featureCollection dataset, but not for granules under 
> >>>>>> 'files' folder.  I also tried the 'iospParam' attribute of the 
> >>>>>> 'netcdf' element outside featureCollection, like
> >>>>>>
> >>>>>> <datasetScan name="GLDAS_CLM10SUBP_3H" ID="GLDAS_CLM10SUBP_3H" 
> >>>>>> path="GLDAS_CLM10SUBP_3H" 
> >>>>>> location="/ftp/data/s4pa_TS2/GLDAS_CLM10SUBP_3H/">
> >>>>>> <netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2"; 
> >>>>>> iospParam="gribParameterTable=/home/ffang/gribtab_clm.tab">
> >>>>>> </netcdf>
> >>>>>> ...
> >>>>>>
> >>>>>> and TDS didn't seem to read the grib table.
> >>>>>>
> >>>>>> The other main question is about the collection .ncx file usage.  I 
> >>>>>> found accessing that file very speedy in TDS and wonder if it should 
> >>>>>> be used to serve our extremely long time series datasets (~30 years of 
> >>>>>> hourly data), i.e. shall we let TDS make the file at the data location 
> >>>>>> instead of under cache/cdm, and treat it as the timeseries dataset?
> >>>>>>
> >>>>>> -Fan
> >>>>>>
> >>>>>> ________________________________________
> >>>>>> From: Fang, Fan (GSFC-610.2)[ADNET SYSTEMS INC]
> >>>>>> Sent: Friday, May 17, 2013 2:50 PM
> >>>>>> To: Fang, Fan (GSFC-610.2)[ADNET SYSTEMS INC]; address@hidden
> >>>>>> Cc: address@hidden
> >>>>>> Subject: RE: [THREDDS #GJF-200148]: TDS 4.3 GRIB names different from 
> >>>>>> TDS 4.2
> >>>>>>
> >>>>>> I take it back - the .tab extension apparently works.  We previously 
> >>>>>> made an error renaming the grib table.
> >>>>>>
> >>>>>> I also changed '/s' to '.s' but both came out the same.  For example, 
> >>>>>> kg/m^2/s or kg/m^2.s would show up as kgm^2s, which is wrong.  I 
> >>>>>> suppose we have to change it to kgm^-2s^-1 in the table?  I hope it is 
> >>>>>> consistent with software like 'wgrib'.
> >>>>>>
> >>>>>> It seems the variable names are essentially the long names and units 
> >>>>>> in grib table plus something like '_surface', with space chars 
> >>>>>> replaced by underscores.  I guess we can survive with that.
> >>>>>>
> >>>>>> These show up in the collection 'Best Timeseries' (is there a way to 
> >>>>>> re-configure this collection name?), but not for individual files 
> >>>>>> under 'files' folder in TDS (again, is there a way to configure for 
> >>>>>> the folder name?).  Is this expected?
> >>>>>>
> >>>>>> One final question: if I drop the collection index file .ncx at the 
> >>>>>> data location, TDS seems to be able to serve it like a dataset, and 
> >>>>>> for time series it is surprisingly fast to serve the .ncx file 
> >>>>>> instead.  Is this intended - in other words we shall direct user to 
> >>>>>> use .ncx, especially for long time series?  I know in default it's 
> >>>>>> under cache/cdm and not exposed to users.  What's the catch here?
> >>>>>>
> >>>>>> Thanks.
> >>>>>>
> >>>>>> -Fan
> >>>>>> ________________________________________
> >>>>>> From: Fang, Fan (GSFC-610.2)[ADNET SYSTEMS INC]
> >>>>>> Sent: Thursday, May 16, 2013 6:13 PM
> >>>>>> To: address@hidden for
> >>>>>> Cc: address@hidden
> >>>>>> Subject: RE: [THREDDS #GJF-200148]: TDS 4.3 GRIB names different from 
> >>>>>> TDS 4.2
> >>>>>>
> >>>>>> I can confirm changing the grib table name to have .tab as extension 
> >>>>>> still does not work.
> >>>>>>
> >>>>>> I'll try the modifications in the units, but wonder why it fails for 
> >>>>>> all since some of them have legit units, like 'K' for temperature.
> >>>>>>
> >>>>>> -Fan
> >>>>>> ________________________________________
> >>>>>> From: Unidata THREDDS Support address@hidden
> >>>>>> Sent: Thursday, May 16, 2013 2:37 PM
> >>>>>> To: Fang, Fan (GSFC-610.2)[ADNET SYSTEMS INC]
> >>>>>> Cc: address@hidden; Fang, Fan (GSFC-610.2)[ADNET SYSTEMS INC]; 
> >>>>>> address@hidden
> >>>>>> Subject: [THREDDS #GJF-200148]: TDS 4.3 GRIB names different from TDS 
> >>>>>> 4.2
> >>>>>>
> >>>>>> Fan,
> >>>>>>
> >>>>>> I did a little more digging, and I don't think the file name extension 
> >>>>>> will matter - that was my misinterpretation of how the table was 
> >>>>>> getting picked up.  However, you will probably have the best result 
> >>>>>> directly embedding the table information in your featureCollection 
> >>>>>> element using the xml notation.  It looks like you tried this already, 
> >>>>>> so I am not sure why it did not work for you.  It is possible that it 
> >>>>>> broke because you did not follow udunits standard -- for instance, 
> >>>>>> square meters is notated as m^2, rather than m(superscript)2.  The 
> >>>>>> code you are asking about is just the number that identifies the grib 
> >>>>>> parameter.
> >>>>>>
> >>>>>> Inside your <featureCollection> element:
> >>>>>>
> >>>>>> <gribConfig>
> >>>>>> <parameterMap>
> >>>>>> <parameter code="2">  <<-- This will override the standard table 
> >>>>>> values for parameter 2 (code < 128 are standard, > 128 are local)
> >>>>>> <description>Pressure reduced to MSL</description>
> >>>>>> <units>Pa</units>
> >>>>>> <name>PRMSL</name>
> >>>>>> </parameter>
> >>>>>> <parameter code="131">
> >>>>>> <description>Snowfall rate</description>
> >>>>>> <units>kg/m^2.s</units> <<--Compound units are notated with a dot, so 
> >>>>>> /m^2/s is just /m^2.s
> >>>>>> <name>Snof</name>
> >>>>>> </parameter>
> >>>>>> ...
> >>>>>>
> >>>>>> </parameterMap>
> >>>>>>
> >>>>>> </gribConfig>
> >>>>>> -Lansing Madry
> >>>>>> Unidata
> >>>>>> Boulder, Colorado
> >>>>>>
> >>>>>> Ticket Details
> >>>>>> ===================
> >>>>>> Ticket ID: GJF-200148
> >>>>>> Department: Support THREDDS
> >>>>>> Priority: Low
> >>>>>> Status: Closed
> >>>>>>
> >>>>>>
> >>>>> -Lansing Madry
> >>>>>    Unidata
> >>>>>    Boulder, Colorado
> >>>>>
> >>>>> Ticket Details
> >>>>> ===================
> >>>>> Ticket ID: GJF-200148
> >>>>> Department: Support THREDDS
> >>>>> Priority: Low
> >>>>> Status: Open
> >>>>>
> >>> -Lansing Madry
> >>> Unidata
> >>> Boulder, Colorado
> >>>
> >>> Ticket Details
> >>> ===================
> >>> Ticket ID: GJF-200148
> >>> Department: Support THREDDS
> >>> Priority: Low
> >>> Status: Open
> >>>
> >>>
> >> -Lansing Madry
> >>    Unidata
> >>    Boulder, Colorado
> >>
> >> Ticket Details
> >> ===================
> >> Ticket ID: GJF-200148
> >> Department: Support THREDDS
> >> Priority: Low
> >> Status: Open
> >>
> >
> >
> > Ticket Details
> > ===================
> > Ticket ID: GJF-200148
> > Department: Support THREDDS
> > Priority: Low
> > Status: Open
> > Link:  
> > https://www.unidata.ucar.edu/esupport/staff/index.php?_m=tickets&_a=viewticket&ticketid=22083
> 
> 
> 
> Ticket Details
> ===================
> Ticket ID: GJF-200148
> Department: Support THREDDS
> Priority: Low
> Status: Open
> 
> 
> 

Ticket Details
===================
Ticket ID: GJF-200148
Department: Support THREDDS
Priority: Low
Status: Closed


NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publicly available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us.