Re: [thredds] NcML aggregation strangely pulling files from multiple datasets

Hi John and Roy,

Yes, I see this problem too with a joinExisting aggregation of mooring data
where the last file in the aggregation is clobbered and recreated every two

To fix it I follow Roy's step 1 AND I reload the app with Tomcat's manager.
(I've found that I need to do both of these steps to correct the problem.)

If I've reloaded the app more than a few times I see the permgen error and
will need to restart Tomcat - most of the time I just restart tomcat after
removing the cache/agg files.

I¹ve struggled to consider a more permanent fix to this problem.  If anyone
has some suggestions my ears are wide open.


Mike McCann 
Software Engineer 
Monterey Bay Aquarium Research Institute
7700 Sandholdt Road
Moss Landing, CA 95039-9644
Voice: 831.775.1769  Fax: 831.775.1736

On 5/19/11 3:59 PM, "John Maurer" <jmaurer@xxxxxxxxxx> wrote:

> MANY THANKS, ROY! That cleared up all the problems. I'm hugely relieved that I
> and/or my disk weren't going haywire. :) OK, catching my breath... phew.
> Cheers,
> John Maurer
> On Thu, May 19, 2011 at 12:48 PM, Roy Mendelssohn <Roy.Mendelssohn@xxxxxxxx>
> wrote:
>> Hi John:
>> We have run into this too, particularly when adding new datasets.  Whenever
>> we do, we always do the following and find it clears up the problem:
>> 1.  Delete all of $TOMCAT_HOME/content/thredds/cache/agg
>> 2. Delete all of $TOMCAT_HOME/content/thredds/cache/ehcache
>> not certain ehculprit - it looks like the ehcache might assosciate the agg
>> cache with certain datasets, and if that gets changed the list gets out of
>> whack, but that is a wild guess.  What I do know is that when we have that
>> problem, those two steps clear it up.
>> The downside is that access to large aggregate datasets is slow the first
>> time they are accessed.
>> -Roy
>> On May 19, 2011, at 2:24 PM, John Maurer wrote:
>>> > Hi all,
>>> > I'm seeing some weird behavior today with one of our joinExisting NcML
>>> aggregations for high-frequency radio (HFR). The OPeNDAP page lists
>>> variables that are from a totally different dataset (e.g. HFR does not
>>> measure water temperature):
>>> I looked at
>>> "content/thredds/cache/agg/" at the aggregation cache file for the dataset
>>> (hioos-hfr-kokagg) and it confirms that it's strangely pulling files from 3
>>> unrelated datasets: a glider, an NCOM global FMRC, and the HFR data. Since
>>> nothing has changed recently in my config file for the dataset and I haven't
>>> recently upgraded TDS, this has me worried about a possible disk problem
>>> from recent power outages? The three datasets are not subdirectories of each
>>> other, so I don't see how TDS can be lumping them together. Any ideas what
>>> could be causing this?
>>> >
>>> > In addition to the above, 5 of my FMRC's are suddenly inaccessible today
>>> as well and were recently working just fine. It would seem their file
>>> aggregations may have similarly gone haywire as well, but their cache files
>>> under "content/thredds/collection/" (right?) are binary (.jdb) so I can't
>>> confirm whether they're combining multiple datasets. The catalogs give an
>>> error in threddsServlet.log with the message:
>>> >         at 
>>> ucar.nc2.ft.fmrc.FmrcDataset.getNetcdfDataset2D(
>>> >         at ucar.nc2.ft.fmrc.Fmrc.getDataset2D(
>>> >
>>> > What's the best way for me to clear all TDS caches? Is it safe to remove
>>> "content/thredds/cache/" entirely?, and then restart Tomcat/TDS? Will it get
>>> re-built then automatically after I access each dataset?
>>> >
>>> > I've copied the relevant agg and collection caches, config files, and
>>> threddsServlet.log to my anonymous ftp server at:
>>> >
>>> > <>
>>> > jmaurer/thredds/
>>> >
>>> > Thanks for your help!,
>>> > John Maurer
>>> > Pacific Islands Ocean Observing System (PacIOOS)
>>> > University of Hawaii
>>> > _______________________________________________
>>> > thredds mailing list
>>> > thredds@xxxxxxxxxxxxxxxx
>>> > For list information or to unsubscribe,  visit:
>> **********************
>> "The contents of this message do not reflect any position of the U.S.
>> Government or NOAA."
>> **********************
>> Roy Mendelssohn
>> Supervisory Operations Research Analyst
>> Environmental Research Division
>> Southwest Fisheries Science Center
>> 1352 Lighthouse Avenue
>> Pacific Grove, CA 93950-2097
>> e-mail: Roy.Mendelssohn@xxxxxxxx (Note new e-mail address)
>> voice: (831)-648-9029
>> fax: (831)-648-8440
>> www:
>> "Old age and treachery will overcome youth and skill."
>> "From those who have been given much, much will be expected"
> _______________________________________________
> thredds mailing list
> thredds@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe,  visit: