[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: forecast aggregations





Steven.Anthony wrote:
Hey John,

The dev server with the FMRC on it is not accessible outside of the building,
however, I have attached the FMRC catalog & the directory listing.
If you need anything else, feel free to ask.



Thanks for the quick response. - Steven

John Caron said the following on 2/22/2007 1:21 PM:

Hi Steven:

Can you send me a directory listing and the resulting FMRC catalog?

If the server is running and I can access, just send me the catalog URL

thanks

Glenn.Rutledge wrote:

The following from Steve Anthony- Glenn

Hey,
Our problems with the FMRC Aggregation are from a date issue. It appears that when the data is read, the dates are getting thrown off by an offset.


In the catalog.xml file, we have copied (almost verbatim) the syntax that UCAR uses for its Motherlode server.
from catalog.xml:


<datasetFmrc name="gfs-test-FMRC" collectionType="ForecastModelRuns" harvest="true" path="fmrc/gfs_3">
<netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2"; enhance="true">
<aggregation dimName="run" type="forecastModelRunSingleCollection" fmrcDefinition="NCEP-GFS-Global_onedeg.fmrcDefinition.xml" timeUnitsChange="true" >
<scanFmrc location="/nomads3_data/raid2/noaaport/merged/gfs-hi/200611/20061129"
regExp=".*\.grb$"
runDateMatcher="#gfs_3_#yyyyMMdd_HH"
forecastOffsetMatcher="HHH#.grb#"/>
</aggregation>
</netcdf>
<fmrcInventory location="/nomads3_data/raid2/noaaport/merged/gfs-hi/200611/20061129" suffix=".grb" fmrcDefinition="NCEP-GFS-Global_onedeg.fmrcDefinition.xml" />
</datasetFmrc>


When I open up the "Forecast Model Run" folder in the FMRC dataset, I see 4 links (like I should) but the run dates are horribly off, as you can see below, the run dates are jumping by almost a month! - these should only be 6 hours apart.
gfs-test-FMRC_RUN_2007-02-12T00:00:00Z
gfs-test-FMRC_RUN_2007-01-18T00:00:00Z
gfs-test-FMRC_RUN_2006-12-24T00:00:00Z
gfs-test-FMRC_RUN_2006-11-29T00:00:00Z

i dont see anything obviously wrong. Are there any error or debug messages in catalina.out, or in thredds/content/logs/threddsServlet.log?

Do you know how to change log4j debugging on ucar.nc2.ncml to DEBUG (from 
thredds debug page is easiest) ? Else from WEB-INF/log4j.xml, add

 <logger name="ucar.nc2.ncml">
   <level value="DEBUG"/>
   <appender-ref ref="threddsServlet"/>
 </logger>

then look at threddsServlet.log after accessing the fmrc dataset.


The "Constant Forecast Offset" folder appears as it should, forecast offsets out to 180 hours, and at a 3 hour interval.


The "Constant Forecast Date" folder, however, also has problems, but they are a combination of the 2 previous ones.
gfs-test-FMRC_ConstantForecast_2006-11-29T00:00:00Z
gfs-test-FMRC_ConstantForecast_2006-11-29T03:00:00Z
gfs-test-FMRC_ConstantForecast_2006-11-29T06:00:00Z
...
gfs-test-FMRC_ConstantForecast_2006-12-06T12:00:00Z << looks normal up to this point
gfs-test-FMRC_ConstantForecast_2006-12-24T00:00:00Z << then it jumps 18 days, and does this 2 more times in the dataset.
gfs-test-FMRC_ConstantForecast_2006-12-24T03:00:00Z
gfs-test-FMRC_ConstantForecast_2006-12-24T06:00:00Z
...


Questions:
Is my syntax wrong in the catalog.xml file? (we have one grib1 file per forecast hour per model run)
What is in the NCEP-GFS-Global_onedeg.fmrcDefinition.xml file, and where is it?

why dont you remove this for now, and we can come back to it later. It may be interfering. So remove the <fmrcInventory> element and the fmrcDefinition attribute.

also, what version of TDS are you running?

is there any way you can run TDS on a public server, with enough data files to 
duplicate the proble?

otherwise, can you make accessible to me as many as the files as needed to 
where the problem starts to occur. I see from my testing i only had 3 files, 
all from the same run.

thanks


NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publicly available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us.