Re: [thredds] Problems with large aggregations on JPL THREDDS?

  • To: Roy Mendelssohn - NOAA Federal <roy.mendelssohn@xxxxxxxx>
  • Subject: Re: [thredds] Problems with large aggregations on JPL THREDDS?
  • From: "Signell, Richard" <rsignell@xxxxxxxx>
  • Date: Sun, 17 Feb 2013 11:32:44 -0500
Roy,

>From this page:
http://www.unidata.ucar.edu/projects/THREDDS/tech/tds4.3/reference/collections/FeatureCollections.html

It looks like the only "featureType" values currently allowed in a
FeatureCollection are
TDS 4.2: FMRC, Point, and Station
TDS 4.3: FMRC, Point,  Station and GRIB

We've been doing FeatureCollection FMRC featureTypes with netcdf files
and it works fine.

But I don't think you can use FeatureCollection for joinNew or
joinExisting aggregations of netcdf files yet.

-Rich

On Sun, Feb 17, 2013 at 10:26 AM, Roy Mendelssohn - NOAA Federal
<roy.mendelssohn@xxxxxxxx> wrote:
> Hi Rich:
> On Feb 17, 2013, at 4:10 AM, "Signell, Richard" <rsignell@xxxxxxxx> wrote:
>
>> "If you have joinExisting Aggregations, coordinate information will be
>> written to a cache directory....Every scour amount of time, any item
>> that hasnt been changed since maxAge time will be deleted. Set scour
>> to -1 to not scour if you have aggregations that never change."
>>
>> The OSCAR dataset is not changing, so JPL could modify their
>> threddsConfig.xml to be:
>>
>> ...
>> <AggregationCache>
>> ...
>> <scour>-1 hours</scour>
>> ...
>> </AggregationCache>
>>
>> I'm cc'ing the THREDDS list as it seems a common issue.
>
> But then the scour value is set for all datatsets.  I do not know of a way to 
> set dataset specific rates.  We run into this problem a lot, though not as 
> extreme as the example given.  Recently I have had to start setting up 
> featureCollections for some GRIB data, and that has a lot of desired 
> properties - you can set collection specific times, and you can have an 
> external process force the update.   But, I have not had any success using 
> featureCollections for just regular gridded netcdf datasets to replace the 
> older dataset aggregation.  I have seen others on this list imply that they 
> haven't quite been able to get it working either.
>
> If someone is using featureCollection to aggregate gridded netcdf files  (not 
> GRIB files) can they post the URL and include a copy of the relevant portions 
> of the catalog.xml?
>
> Thanks,
>
> -Roy M.
>
>
> **********************
> "The contents of this message do not reflect any position of the U.S. 
> Government or NOAA."
> **********************
> Roy Mendelssohn
> Supervisory Operations Research Analyst
> NOAA/NMFS
> Environmental Research Division
> Southwest Fisheries Science Center
> 1352 Lighthouse Avenue
> Pacific Grove, CA 93950-2097
>
> e-mail: Roy.Mendelssohn@xxxxxxxx (Note new e-mail address)
> voice: (831)-648-9029
> fax: (831)-648-8440
> www: http://www.pfeg.noaa.gov/
>
> "Old age and treachery will overcome youth and skill."
> "From those who have been given much, much will be expected"
> "the arc of the moral universe is long, but it bends toward justice" -MLK Jr.
>



-- 
Dr. Richard P. Signell   (508) 457-2229
USGS, 384 Woods Hole Rd.
Woods Hole, MA 02543-1598



  • 2013 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the thredds archives: