[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[THREDDS #GJF-200148]: TDS 4.3 GRIB names different from TDS 4.2



Fan,

You can download a 4.3.19 snapshot here:

https://docs.google.com/file/d/0B7jP7ooGP2ZzdFh5NUprdEowMkE/edit?usp=sharing

Enjoy!

-Lansing

On 9/3/2013 11:24 AM, Fan Fang wrote:
> New Client Reply: TDS 4.3 GRIB names different from TDS 4.2
>
> Hi Lansing,
>
> I notice that the bug is fixed in version 4.3.18.  Is it downloadable
> from somewhere?
>
> Thanks.
>
> -Fan
>
> On 05/23/2013 04:37 PM, Unidata THREDDS Support wrote:
>> Fan,
>>
>> I reproduced the behavior locally, and it is a bug.  Thanks for finding and 
>> reporting it!  I've set up bug-ticket in jira, our issue tracking system, 
>> which is here:
>>
>> https://bugtracking.unidata.ucar.edu/browse/TDS-435
>>
>> The developer who really knows grib is out for a long weekend, but I'll make 
>> sure he sees it next week.
>>
>> -Lansing
>>
>>> Hi Lansing,
>>>
>>> You can get as many as you want from here:
>>>
>>> ftp://hydro1.sci.gsfc.nasa.gov/data/s4pa/GLDAS_V1/GLDAS_CLM10SUBP_3H/1979/002/
>>>
>>> My sample was the first file (at top) of that day.  The rest .grb files 
>>> represent the hourly data for that day.  For more than one-day worth of 
>>> data, go to 1979/003, 1979/004, etc. for year 1979.
>>>
>>> -Fan
>>> ________________________________________
>>> From: Unidata THREDDS Support address@hidden
>>> Sent: Thursday, May 23, 2013 1:37 PM
>>> To: Fang, Fan (GSFC-610.2)[ADNET SYSTEMS INC]
>>> Cc: address@hidden; Fang, Fan (GSFC-610.2)[ADNET SYSTEMS INC]; 
>>> address@hidden; Fang, Fan (GSFC-610.2)[ADNET SYSTEMS INC]
>>> Subject: [THREDDS #GJF-200148]: TDS 4.3 GRIB names different from TDS 4.2
>>>
>>> Fan,
>>>
>>> Can you put a few more of the grb files up?
>>>
>>> -Lansing
>>>
>>>> Hello Lansing,
>>>>
>>>> Thanks for the tip.  You can get the sample data and grib table from:
>>>>
>>>> ftp://hydro1.sci.gsfc.nasa.gov/private/ffang/GLDAS_CLM10SUBP_3H.A1979002.0000.001.2008086151108.grb
>>>> ftp://hydro1.sci.gsfc.nasa.gov/private/ffang/gribtab_clm.tab
>>>>
>>>> My TDS configuration for the dataset is:
>>>>
>>>> <datasetScan name="GLDAS_CLM10SUBP_3H" ID="GLDAS_CLM10SUBP_3H"
>>>> path="GLDAS_CLM10SUBP_3H"
>>>> location="/ftp/data/s4pa_TS2/TEST_OPENDAP/GLDAS_CLM10SUBP_3H/">
>>>> <netcdf
>>>> xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2";
>>>> iospParam="gribParameterTable=/home/ffang/gribtab_clm.tab">
>>>> </netcdf>
>>>> <metadata inherited="true">
>>>> <serviceName>odapncss</serviceName>
>>>> <dataType>Grid</dataType>
>>>> </metadata>
>>>> <filter>
>>>> <include wildcard="*.grb"/>
>>>> </filter>
>>>> </datasetScan>
>>>>
>>>> -Fan
>>>>
>>>> On 05/22/2013 03:55 PM, Unidata THREDDS Support wrote:
>>>>> Fan,
>>>>>
>>>>> Two things - first, don't serve the ncx files.  It doesn't matter where 
>>>>> they are located (in the data directory or elsewhere), but it's not meant 
>>>>> to be served as data.
>>>>>
>>>>> Second, it sounds like you are describing a bug in how user tables are 
>>>>> applied.  I'd like to set up a tds locally in a debugger with your files, 
>>>>> collection spec, and local grib table.  Please bundle them up and send 
>>>>> them to me, or point me to where I can get them.
>>>>>
>>>>> Thanks,
>>>>>    Lansing
>>>>>
>>>>>> Hello Lansing,
>>>>>>
>>>>>> Any comments/findings about my questions?
>>>>>>
>>>>>> With or without featureCollection we need to serve grib data in their 
>>>>>> granule form, so how to make grib table work for individual granules is 
>>>>>> remaining question number one.  So far our grib table works with 
>>>>>> aggregated featureCollection dataset, but not for granules under 'files' 
>>>>>> folder.  I also tried the 'iospParam' attribute of the 'netcdf' element 
>>>>>> outside featureCollection, like
>>>>>>
>>>>>> <datasetScan name="GLDAS_CLM10SUBP_3H" ID="GLDAS_CLM10SUBP_3H" 
>>>>>> path="GLDAS_CLM10SUBP_3H" 
>>>>>> location="/ftp/data/s4pa_TS2/GLDAS_CLM10SUBP_3H/">
>>>>>> <netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2"; 
>>>>>> iospParam="gribParameterTable=/home/ffang/gribtab_clm.tab">
>>>>>> </netcdf>
>>>>>> ...
>>>>>>
>>>>>> and TDS didn't seem to read the grib table.
>>>>>>
>>>>>> The other main question is about the collection .ncx file usage.  I 
>>>>>> found accessing that file very speedy in TDS and wonder if it should be 
>>>>>> used to serve our extremely long time series datasets (~30 years of 
>>>>>> hourly data), i.e. shall we let TDS make the file at the data location 
>>>>>> instead of under cache/cdm, and treat it as the timeseries dataset?
>>>>>>
>>>>>> -Fan
>>>>>>
>>>>>> ________________________________________
>>>>>> From: Fang, Fan (GSFC-610.2)[ADNET SYSTEMS INC]
>>>>>> Sent: Friday, May 17, 2013 2:50 PM
>>>>>> To: Fang, Fan (GSFC-610.2)[ADNET SYSTEMS INC]; address@hidden
>>>>>> Cc: address@hidden
>>>>>> Subject: RE: [THREDDS #GJF-200148]: TDS 4.3 GRIB names different from 
>>>>>> TDS 4.2
>>>>>>
>>>>>> I take it back - the .tab extension apparently works.  We previously 
>>>>>> made an error renaming the grib table.
>>>>>>
>>>>>> I also changed '/s' to '.s' but both came out the same.  For example, 
>>>>>> kg/m^2/s or kg/m^2.s would show up as kgm^2s, which is wrong.  I suppose 
>>>>>> we have to change it to kgm^-2s^-1 in the table?  I hope it is 
>>>>>> consistent with software like 'wgrib'.
>>>>>>
>>>>>> It seems the variable names are essentially the long names and units in 
>>>>>> grib table plus something like '_surface', with space chars replaced by 
>>>>>> underscores.  I guess we can survive with that.
>>>>>>
>>>>>> These show up in the collection 'Best Timeseries' (is there a way to 
>>>>>> re-configure this collection name?), but not for individual files under 
>>>>>> 'files' folder in TDS (again, is there a way to configure for the folder 
>>>>>> name?).  Is this expected?
>>>>>>
>>>>>> One final question: if I drop the collection index file .ncx at the data 
>>>>>> location, TDS seems to be able to serve it like a dataset, and for time 
>>>>>> series it is surprisingly fast to serve the .ncx file instead.  Is this 
>>>>>> intended - in other words we shall direct user to use .ncx, especially 
>>>>>> for long time series?  I know in default it's under cache/cdm and not 
>>>>>> exposed to users.  What's the catch here?
>>>>>>
>>>>>> Thanks.
>>>>>>
>>>>>> -Fan
>>>>>> ________________________________________
>>>>>> From: Fang, Fan (GSFC-610.2)[ADNET SYSTEMS INC]
>>>>>> Sent: Thursday, May 16, 2013 6:13 PM
>>>>>> To: address@hidden for
>>>>>> Cc: address@hidden
>>>>>> Subject: RE: [THREDDS #GJF-200148]: TDS 4.3 GRIB names different from 
>>>>>> TDS 4.2
>>>>>>
>>>>>> I can confirm changing the grib table name to have .tab as extension 
>>>>>> still does not work.
>>>>>>
>>>>>> I'll try the modifications in the units, but wonder why it fails for all 
>>>>>> since some of them have legit units, like 'K' for temperature.
>>>>>>
>>>>>> -Fan
>>>>>> ________________________________________
>>>>>> From: Unidata THREDDS Support address@hidden
>>>>>> Sent: Thursday, May 16, 2013 2:37 PM
>>>>>> To: Fang, Fan (GSFC-610.2)[ADNET SYSTEMS INC]
>>>>>> Cc: address@hidden; Fang, Fan (GSFC-610.2)[ADNET SYSTEMS INC]; 
>>>>>> address@hidden
>>>>>> Subject: [THREDDS #GJF-200148]: TDS 4.3 GRIB names different from TDS 4.2
>>>>>>
>>>>>> Fan,
>>>>>>
>>>>>> I did a little more digging, and I don't think the file name extension 
>>>>>> will matter - that was my misinterpretation of how the table was getting 
>>>>>> picked up.  However, you will probably have the best result directly 
>>>>>> embedding the table information in your featureCollection element using 
>>>>>> the xml notation.  It looks like you tried this already, so I am not 
>>>>>> sure why it did not work for you.  It is possible that it broke because 
>>>>>> you did not follow udunits standard -- for instance, square meters is 
>>>>>> notated as m^2, rather than m(superscript)2.  The code you are asking 
>>>>>> about is just the number that identifies the grib parameter.
>>>>>>
>>>>>> Inside your <featureCollection> element:
>>>>>>
>>>>>> <gribConfig>
>>>>>> <parameterMap>
>>>>>> <parameter code="2">  <<-- This will override the standard table values 
>>>>>> for parameter 2 (code < 128 are standard, > 128 are local)
>>>>>> <description>Pressure reduced to MSL</description>
>>>>>> <units>Pa</units>
>>>>>> <name>PRMSL</name>
>>>>>> </parameter>
>>>>>> <parameter code="131">
>>>>>> <description>Snowfall rate</description>
>>>>>> <units>kg/m^2.s</units> <<--Compound units are notated with a dot, so 
>>>>>> /m^2/s is just /m^2.s
>>>>>> <name>Snof</name>
>>>>>> </parameter>
>>>>>> ...
>>>>>>
>>>>>> </parameterMap>
>>>>>>
>>>>>> </gribConfig>
>>>>>> -Lansing Madry
>>>>>> Unidata
>>>>>> Boulder, Colorado
>>>>>>
>>>>>> Ticket Details
>>>>>> ===================
>>>>>> Ticket ID: GJF-200148
>>>>>> Department: Support THREDDS
>>>>>> Priority: Low
>>>>>> Status: Closed
>>>>>>
>>>>>>
>>>>> -Lansing Madry
>>>>>    Unidata
>>>>>    Boulder, Colorado
>>>>>
>>>>> Ticket Details
>>>>> ===================
>>>>> Ticket ID: GJF-200148
>>>>> Department: Support THREDDS
>>>>> Priority: Low
>>>>> Status: Open
>>>>>
>>> -Lansing Madry
>>> Unidata
>>> Boulder, Colorado
>>>
>>> Ticket Details
>>> ===================
>>> Ticket ID: GJF-200148
>>> Department: Support THREDDS
>>> Priority: Low
>>> Status: Open
>>>
>>>
>> -Lansing Madry
>>    Unidata
>>    Boulder, Colorado
>>
>> Ticket Details
>> ===================
>> Ticket ID: GJF-200148
>> Department: Support THREDDS
>> Priority: Low
>> Status: Open
>>
>
>
> Ticket Details
> ===================
> Ticket ID: GJF-200148
> Department: Support THREDDS
> Priority: Low
> Status: Open
> Link:  
> https://www.unidata.ucar.edu/esupport/staff/index.php?_m=tickets&_a=viewticket&ticketid=22083



Ticket Details
===================
Ticket ID: GJF-200148
Department: Support THREDDS
Priority: Low
Status: Open


NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publicly available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us.