[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Feature Collections for RTOFS data?

0) are the horizontal (orthogonal curvililinear) grids constant for all the data in the dataset, or do they change? 

1) Seems like

.f#HHH#*\

is a bit fishy

H = hour in day, does this actually work?

in file ofs_atl.t00z.f120.20110531.grb.grib2

f120 = 120th forecast hour?
t00z = 0z run ? Are all runs at 0 Z ?

so run time= 20110531 0Z ?


2) In 4.3 you will use something like:
<featureCollection name="RTOFS Aggregation" featureType="GRIB" harvest="true" path="rtofs_fc_agg">
<collection name="RTOFS_Aggregation" spec="C:/charles_local/thredds_test/RTOFS/**/.*#yyyyMMdd#\.grib2$" /> 

</featureCollection>
id have to understand your data better to be exact. Something more elaborate if you want to keep surface and 3d fields seperate.
3) i expect to have an alpha version out in a few weeks if you would like to test it. Note that this only works on Grib2 files. 

On 6/24/2011 12:21 PM, Rich Signell wrote:

Charles,
As I understand it, in 4.3 the FMRC is both simplified and made a lot
more efficient by only allowing best time series, and basically just
dropping inidices in time slots as you plow through the list of files.
   I think John said it would only work initially for GRIB files
although now that I think about it, I'm not sure why that would be.
He set this up after working with NCDC on some 100,000 file GRIB2
aggregation, or something like that.

-Rich

On Fri, Jun 24, 2011 at 10:14 AM, Charles Carleton
<address@hidden>  wrote:

Believe it or not, I got it to work by not using the date-stamped folders; I
just skipped them with the /**/ directive and only defined the block with
hours. How will things work in 4.3?  When is it coming out?

The victorious code looks like this:
  <dataset name="RTOFS Nowcast/Forecast">
    <metadata inherited="true">
      <serviceName>allServices</serviceName>
      <dataType>Grid</dataType>
      <dataFormat>NetCDF</dataFormat>
      <creator>
        <name vocabulary="DIF">Dr. Avichal Mehra</name>
        <contact  email="address@hidden"/>
      </creator>

      <documentation xlink:href="http://polar.ncep.noaa.gov/ofs/";
        xlink:title="RTOFS Real-Time Ocean Forecast System (Atlantic)
Project Page"/>
      <documentation type="Summary">  Real-Time Ocean Forecast System
(Atlantic)</documentation>
      <documentation type="Rights">    The data is provided free of charge,
without warranty of any kind.</documentation>

    </metadata>


    <featureCollection name="RTOFS Aggregation"
        featureType="FMRC"  harvest="true" path="rtofs_fc_agg">
      <metadata inherited="true">
        <documentation type="summary">Aggregation of Real Time Ocean
Forecast data</documentation>
        <serviceName>agg</serviceName>

</metadata><!--/data3T/www/ncom/Mid_Atlantic_Bight/ncom_glb_mab_#yyyyMMddHH#
\.nc$20101231-->
      <!--<collection
spec="C:/charles_local/thredds_test/**/ocnh01\.gdas\.#yyyyMMddHH#\.grb2$"-->
      <collection
spec="C:/charles_local/thredds_test/RTOFS/surface/**/ofs_atl\.t00z\.f#HHH#*\
.grib2$"
           recheckAfter="5 min" olderThan="5 min"/>
      <protoDataset>
      </protoDataset>
      <fmrcConfig regularize="true" datasetTypes="Best" />
    </featureCollection>
  </dataset>

-----Original Message-----
From: address@hidden [mailto:address@hidden] On Behalf Of Rich
Signell
Sent: Thursday, June 23, 2011 9:42 PM
To: Charles Carleton
Cc: address@hidden
Subject: Re: Feature Collections for RTOFS data?

Charles,
I think the short answer is that you can't do it in 4.2 but it will
really rock when  4.3 comes out!
-RIch

On Thu, Jun 23, 2011 at 5:59 PM, Charles Carleton
<address@hidden>  wrote:

I’m trying to do an aggregation of RTOFS data.  A non-aggregated version

can

be found here:

http://data.nodc.noaa.gov/thredds/catalog/ncep/rtofs/catalog.html



The date-stamp is in the directory name, for example ‘ofs.20110531’, while
the hour, which can exceed 24, because these are forecasts, is in the
filename, like this:
‘/ofs.20110531/surface/ofs_atl.t00z.n000.20110531.grb.grib2’.



There is a ‘time’ variable, could I just leave the date-stamp stuff out of
the spec an just replace it with wildcards?

The time variable has these attributes:

long_name: "forecast time"

units: "hour since 2011-05-31T00:00:00Z"

coordinates: "Latitude_of_Pressure_Point Longitude_of_Pressure_Point"



I need to know what the ‘spec’ parameter in the ‘collection’ tag inside of
‘featureCollection’ should look like.

Here’s what I’m guessing the Feature Collection should look like, but the
spec string is wrong; THREDDS can’t find the data using the one I tried:

     <featureCollection name="RTOFS Aggregation"

         featureType="FMRC"  harvest="true" path="rtofs_fc_agg">

       <metadata inherited="true">

             <documentation type="summary">Aggregation of Real Time Ocean
Forecast data</documentation>

         <serviceName>agg</serviceName>

       </metadata>

       <collection


spec="C:/charles_local/thredds_test/RTOFS/surface/ofs.#yyyyMMdd#/ofs_atl.t00
z.f#HH#*\.grib2$"

          recheckAfter="5 min" olderThan="5 min"/>

       <protoDataset>

       </protoDataset>

       <fmrcConfig regularize="true" datasetTypes="Best" />

     </featureCollection>

   </dataset>





--
Dr. Richard P. Signell   (508) 457-2229
USGS, 384 Woods Hole Rd.
Woods Hole, MA 02543-1598