[thredds] over written file has old global attributes
Jean.C.Newman at noaa.gov
Jean.C.Newman at noaa.gov
Thu Nov 8 13:28:32 MST 2007
It looks like turning off the caching has solved the "old attribute"
problem. Thanks, Jean
___________________________________________________________
Jean Newman Tel: 206-526-6531
On Wed, 7 Nov 2007, Ethan Davis wrote:
> Hi Jean,
>
> The TDS data file cache holds a pointer to each cached file along with all
> the metadata for the file. The attributes get pulled from the cache while
> data requests access the file on disk. The cache is supposed to check the
> last modified date for the file and re-read if it has been modified since the
> cache was last updated. But what you are reporting sure makes it sound like
> that capability (checking last modified) might be broken.
>
> Just to make sure, next time you see this behavior, could you take a look at
> the cache on your server and, if the file is listed, clear the cache and see
> if the problem goes away? You have to have TDS Remote Management setup (see
> the TDS Reference for details,
> http://www.unidata.ucar.edu/projects/THREDDS/tech/reference/).
>
> It will be a few weeks before we will be able to look at this in depth. For
> now, we have a new 3.16 release with a minor tweak that turns off caching if
> the max number of cached items is set to zero. You set the max values by
> editing your threddsConfig.xml file which is in the content/thredds
> directory. There are three types of caching you can adjust: NetcdfFileCache,
> NetcdfDatasetCache, and HTTPFileCache. You may only need to modify the
> NetcdfFileCache settings but NetcdfDatasetCache may come into play if you
> have datasets described with NcML. To change the settings, find the elements
> of the same name in your threddsConfig.xml and set max to 0.
>
> You probably are not seeing this behavior every time because the cache
> occasionally scours out files that aren't as active as others.
>
> Hope that helps,
>
> Ethan
>
> Jean.C.Newman at noaa.gov wrote:
>> On the sift machine, because we were doing testing, we have "opendap" with
>> & without thredds and "looking" at the file - ncdump & dncdump with
>> "opendap no thredds" both give correct attributes while dcdump with
>> "thredds" gives old attribute information. Jean
>>
>> ncdump -h crescent_runup_ha.nc
>> ...
>> // global attributes:
>> :history = "MOST v2.080 11-05-2007 16:09:22" ;
>> :title = "Aleutian-Cascadia: Mwt 8.4, 21.00*b65" ;
>>
>> dncdump -h
>> http://sift.pmel.noaa.gov/thredds/dodsC/data/crescent_runup_ha.nc
>> ...
>> // global attributes:
>> :history = "MOST v2.080 11-05-2007 11:49:33" ;
>> :title = "Aleutian-Cascadia: Mwt 8.5, 29.00*b25" ;
>>
>> dncdump -h http://sift.pmel.noaa.gov/opendap/crescent_runup_ha.nc
>> ...
>> // global attributes:
>> :history = "MOST v2.080 11-05-2007 16:09:22" ;
>> :title = "Aleutian-Cascadia: Mwt 8.4, 21.00*b65" ;
>> ___________________________________________________________
>> Jean Newman Tel: 206-526-6531
>> NOAA Center for Tsunami Research
>> NOAA/PMEL/OERD2 - UW/JISAO FAX: 206-526-6485
>> 7600 Sand Point Way NE, Bldg. 3 jean.c.newman at noaa.gov
>> Seattle, WA 98115-6349 jcn at u.washington.edu
>> _________________________. URL: http://nctr.pmel.noaa.gov/
>> (__________________________________
>>
>> On Tue, 6 Nov 2007, Jean.C.Newman at noaa.gov wrote:
>>
>>
>>> One other thing, the "old attributes" lasts. I made a run yesterday at
>>> 11:23 am and it was over written several time the last at 16:09 and today
>>> if I look at that output file via LAS/"dncdump -h" it still shows the "old
>>> stuff" from the 11:23 run. "ncdump -h crescent_runup_ha.nc" has the
>>> correct 16:09 information. Thanks, Jean
>>> ___________________________________________________________
>>> Jean Newman Tel: 206-526-6531
>>>
>>> On Tue, 6 Nov 2007, Jean.C.Newman at noaa.gov wrote:
>>>
>>>
>>>> The question:
>>>> Is there some way to turn off caching of metadata so the file is checked
>>>> every time it's accessed? Jean
>>>> ___________________________________________________________
>>>> Jean Newman Tel: 206-526-6531
>>>>
>>>> On Tue, 6 Nov 2007, Jean.C.Newman at noaa.gov wrote:
>>>>
>>>>
>>>>> It's not a browser caching problem. I have a very old LAS that accesses
>>>>> these files. From the LAS, input for a model run is set up then a
>>>>> servlet
>>>>> is run on 1 of several possible machines that starts a FORTRAN model run
>>>>> that overwwites files so that the LAS interface has static files/"DODs"
>>>>> URLs for the output. We are moving to different code but need to keep
>>>>> this
>>>>> working until the new is up and reliable. The data updates, I can see
>>>>> that in the plots, but I use global attributes in labeling and I get the
>>>>> old information sometimes, 2 out of 31 SIM runs. The example I put out
>>>>> on
>>>>> the sift machine was run on an internal machine and then sftp-ed to
>>>>> sift.
>>>>> Then I made another run and again sftp-ed it to the sift machine and saw
>>>>> the same "old attributes" problem, which I really didn't think I would.
>>>>> If tomcat is restarted then I get the new information so I imagine it is
>>>>> some caching problem. I did try removing the file before rerunning the
>>>>> SIM and that did not solve the problem. Is there some way to turn off
>>>>> caching of metadata so the file is checked every time it's accessed?
>>>>> Jean
>>>>> ___________________________________________________________
>>>>> Jean Newman Tel: 206-526-6531
>>>>>
>
> --
> Ethan R. Davis Telephone: (303) 497-8155
> Software Engineer Fax: (303) 497-8690
> UCAR Unidata Program Center E-mail: edavis at ucar.edu
> P.O. Box 3000
> Boulder, CO 80307-3000 http://www.unidata.ucar.edu/
> ---------------------------------------------------------------------------
>
>
>
More information about the thredds
mailing list