Re: [thredds] NcML aggregation strangely pulling files from multiple datasets

  • To: John Maurer <jmaurer@xxxxxxxxxx>
  • Subject: Re: [thredds] NcML aggregation strangely pulling files from multiple datasets
  • From: Roy Mendelssohn <Roy.Mendelssohn@xxxxxxxx>
  • Date: Thu, 19 May 2011 15:48:13 -0700
Hi John:

We have run into this too, particularly when adding new datasets.  Whenever we 
do, we always do the following and find it clears up the problem:

1.  Delete all of $TOMCAT_HOME/content/thredds/cache/agg

2. Delete all of $TOMCAT_HOME/content/thredds/cache/ehcache

not certain ehculprit - it looks like the ehcache might assosciate the agg 
cache with certain datasets, and if that gets changed the list gets out of 
whack, but that is a wild guess.  What I do know is that when we have that 
problem, those two steps clear it up.

The downside is that access to large aggregate datasets is slow the first time 
they are accessed.

-Roy

On May 19, 2011, at 2:24 PM, John Maurer wrote:

> Hi all,
> I'm seeing some weird behavior today with one of our joinExisting NcML 
> aggregations for high-frequency radio (HFR). The OPeNDAP page lists variables 
> that are from a totally different dataset (e.g. HFR does not measure water 
> temperature): 
> http://oos.soest.hawaii.edu/thredds/idd/hfr.html?dataset=kokagg. I looked at 
> "content/thredds/cache/agg/" at the aggregation cache file for the dataset 
> (hioos-hfr-kokagg) and it confirms that it's strangely pulling files from 3 
> unrelated datasets: a glider, an NCOM global FMRC, and the HFR data. Since 
> nothing has changed recently in my config file for the dataset and I haven't 
> recently upgraded TDS, this has me worried about a possible disk problem from 
> recent power outages? The three datasets are not subdirectories of each 
> other, so I don't see how TDS can be lumping them together. Any ideas what 
> could be causing this?
> 
> In addition to the above, 5 of my FMRC's are suddenly inaccessible today as 
> well and were recently working just fine. It would seem their file 
> aggregations may have similarly gone haywire as well, but their cache files 
> under "content/thredds/collection/" (right?) are binary (.jdb) so I can't 
> confirm whether they're combining multiple datasets. The catalogs give an 
> error in threddsServlet.log with the message:
>         at 
> ucar.nc2.ft.fmrc.FmrcDataset.getNetcdfDataset2D(FmrcDataset.java:166)
>         at ucar.nc2.ft.fmrc.Fmrc.getDataset2D(Fmrc.java:183)
> 
> What's the best way for me to clear all TDS caches? Is it safe to remove 
> "content/thredds/cache/" entirely?, and then restart Tomcat/TDS? Will it get 
> re-built then automatically after I access each dataset?
> 
> I've copied the relevant agg and collection caches, config files, and 
> threddsServlet.log to my anonymous ftp server at:
> 
> ftp.soest.hawaii.edu
> jmaurer/thredds/
> 
> Thanks for your help!,
> John Maurer
> Pacific Islands Ocean Observing System (PacIOOS)
> University of Hawaii
> _______________________________________________
> thredds mailing list
> thredds@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe,  visit: 
> http://www.unidata.ucar.edu/mailing_lists/

**********************
"The contents of this message do not reflect any position of the U.S. 
Government or NOAA."
**********************
Roy Mendelssohn
Supervisory Operations Research Analyst
NOAA/NMFS
Environmental Research Division
Southwest Fisheries Science Center
1352 Lighthouse Avenue
Pacific Grove, CA 93950-2097

e-mail: Roy.Mendelssohn@xxxxxxxx (Note new e-mail address)
voice: (831)-648-9029
fax: (831)-648-8440
www: http://www.pfeg.noaa.gov/

"Old age and treachery will overcome youth and skill."
"From those who have been given much, much will be expected" 



  • 2011 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the thredds archives: