[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Updating Aggregations in TDS



Are files also deleted, or only added? Do files ever get appended to or 
modified?

Roy Mendelssohn wrote:
> When a file is added to a directory, a flag is set in a table that says
> it has been added.   We would rather poll that table  (much faster as it
> doesn't need to open a read a slew of netcdf files).  I can have Bob
> write a program in Java to do the required recheck operations if you can
> point us to the appropriate steps to be taken for the netcdf-java library.
> 
> In the near future, both the information to generate our catalog.xml and
> what files are present and/or have been added will be in a database.
> 
> -Roy
> 
> On Oct 30, 2008, at 9:55 AM, John Caron wrote:
> 
>> We are going to add this kind of functionality to 4.0, currently it
>> doesnt exist.
>>
>> How do you detect when data changes?
>>
>> Roy Mendelssohn wrote:
>>> Hi John:
>>>
>>> I am glad we understand how recheckEvery works.  It is consistent with
>>> what we were experiencing.
>>>
>>>> basically you want to do the recheck in the background, and then
>>>> accept whatever is there when a request comes in? Or you want to
>>>> detect whenever the directory changes?
>>>>
>>>
>>> Because other things, such as web page displays, need to be updated when
>>> data is added, we already have a system where we know when data have
>>> been added to a dataset.  What we would like to do is at that point
>>> force a recheck for that dataset, and hopefully only for that dataset,
>>> rather than the entire catalog.  If we can do that, we probably will
>>> turn off the recheckEvery because we know every dataset will be up to
>>> date.
>>>
>>> Does this makes sense?  If need be, we can write a script in Java to do
>>> the above, if we can get advice on the best way to do this.
>>>
>>> Thanks,
>>>
>>> -Roy
>>>
>>> **********************
>>> "The contents of this message do not reflect any position of the U.S.
>>> Government or NOAA."
>>> **********************
>>> Roy Mendelssohn
>>> Supervisory Operations Research Analyst
>>> NOAA/NMFS
>>> Environmental Research Division
>>> Southwest Fisheries Science Center
>>> 1352 Lighthouse Avenue
>>> Pacific Grove, CA 93950-2097
>>>
>>> e-mail: address@hidden (Note new e-mail address)
>>> voice: (831)-648-9029
>>> fax: (831)-648-8440
>>> www: http://www.pfeg.noaa.gov/
>>>
>>> "Old age and treachery will overcome youth and skill."
>>> "From those who have been given much, much will be expected"
> 
> **********************
> "The contents of this message do not reflect any position of the U.S.
> Government or NOAA."
> **********************
> Roy Mendelssohn
> Supervisory Operations Research Analyst
> NOAA/NMFS
> Environmental Research Division
> Southwest Fisheries Science Center
> 1352 Lighthouse Avenue
> Pacific Grove, CA 93950-2097
> 
> e-mail: address@hidden (Note new e-mail address)
> voice: (831)-648-9029
> fax: (831)-648-8440
> www: http://www.pfeg.noaa.gov/
> 
> "Old age and treachery will overcome youth and skill."
> "From those who have been given much, much will be expected"


NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publicly available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us.