forecast aggregations

[John Caron]

Steven.Anthony wrote:
> Hey John,
> 
> The dev server with the FMRC on it is not accessible outside of the 
> building,
> however, I have attached the FMRC catalog & the directory listing.
> If you need anything else, feel free to ask.


> 
> Thanks for the quick response.
> - Steven
> 
> John Caron said the following on 2/22/2007 1:21 PM:
> 
>> Hi Steven:
>>
>> Can you send me a directory listing and the resulting FMRC catalog?
>>
>> If the server is running and I can access, just send me the catalog URL
>>
>> thanks
>>
>> Glenn.Rutledge wrote:
>>
>>> The following from Steve Anthony- Glenn
>>>
>>> Hey,
>>> Our problems with the FMRC Aggregation are from a date issue.  It 
>>> appears that when the data is read, the dates are getting thrown off 
>>> by an offset.
>>>
>>> In the catalog.xml file, we have copied (almost verbatim) the syntax 
>>> that UCAR uses for its Motherlode server.
>>> from catalog.xml:
>>>
>>> <datasetFmrc name="gfs-test-FMRC" collectionType="ForecastModelRuns" 
>>> harvest="true" path="fmrc/gfs_3">
>>> <netcdf 
>>> xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2" 
>>> enhance="true">
>>>    <aggregation dimName="run" type="forecastModelRunSingleCollection" 
>>> fmrcDefinition="NCEP-GFS-Global_onedeg.fmrcDefinition.xml" 
>>> timeUnitsChange="true" >
>>>          <scanFmrc 
>>> location="/nomads3_data/raid2/noaaport/merged/gfs-hi/200611/20061129"
>>>        regExp=".*\.grb$"
>>>        runDateMatcher="#gfs_3_#yyyyMMdd_HH"
>>>        forecastOffsetMatcher="HHH#.grb#"/>
>>>          </aggregation>
>>> </netcdf>
>>>    <fmrcInventory 
>>> location="/nomads3_data/raid2/noaaport/merged/gfs-hi/200611/20061129" 
>>> suffix=".grb" 
>>> fmrcDefinition="NCEP-GFS-Global_onedeg.fmrcDefinition.xml" />
>>> </datasetFmrc>
>>>
>>> When I open up the "Forecast Model Run" folder in the FMRC dataset, I 
>>> see 4 links (like I should) but the run dates are horribly off, as 
>>> you can see below, the run dates are jumping by almost a month! - 
>>> these should only be 6 hours apart.
>>> gfs-test-FMRC_RUN_2007-02-12T00:00:00Z
>>> gfs-test-FMRC_RUN_2007-01-18T00:00:00Z
>>> gfs-test-FMRC_RUN_2006-12-24T00:00:00Z
>>> gfs-test-FMRC_RUN_2006-11-29T00:00:00Z

i dont see anything obviously wrong. Are there any error or debug messages in catalina.out, or in thredds/content/logs/threddsServlet.log?

Do you know how to change log4j debugging on ucar.nc2.ncml to DEBUG (from thredds debug page is easiest) ? Else from WEB-INF/log4j.xml, add

  <logger name="ucar.nc2.ncml">
    <level value="DEBUG"/>
    <appender-ref ref="threddsServlet"/>
  </logger>

then look at threddsServlet.log after accessing the fmrc dataset.

>>>
>>> The "Constant Forecast Offset" folder appears as it should, forecast 
>>> offsets out to 180 hours, and at a 3 hour interval.
>>>
>>> The "Constant Forecast Date" folder, however, also has problems, but 
>>> they are a combination of the 2 previous ones.
>>> gfs-test-FMRC_ConstantForecast_2006-11-29T00:00:00Z
>>> gfs-test-FMRC_ConstantForecast_2006-11-29T03:00:00Z
>>> gfs-test-FMRC_ConstantForecast_2006-11-29T06:00:00Z
>>> ...
>>> gfs-test-FMRC_ConstantForecast_2006-12-06T12:00:00Z  << looks normal 
>>> up to this point
>>> gfs-test-FMRC_ConstantForecast_2006-12-24T00:00:00Z  << then it jumps 
>>> 18 days, and does this 2 more times in the dataset.
>>> gfs-test-FMRC_ConstantForecast_2006-12-24T03:00:00Z
>>> gfs-test-FMRC_ConstantForecast_2006-12-24T06:00:00Z
>>> ...
>>>
>>> Questions:
>>> Is my syntax wrong in the catalog.xml file? (we have one grib1 file 
>>> per forecast hour per model run)
>>> What is in the NCEP-GFS-Global_onedeg.fmrcDefinition.xml file, and 
>>> where is it?

why dont you remove this for now, and we can come back to it later. It may be interfering.
So remove the <fmrcInventory> element and the fmrcDefinition attribute.

also, what version of TDS are you running? 

is there any way you can run TDS on a public server, with enough data files to duplicate the proble?

otherwise, can you make accessible to me as many as the files as needed to where the problem starts to occur. I see from my testing i only had 3 files, all from the same run.

thanks