[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Updating Aggregations in TDS



When a file is added to a directory, a flag is set in a table that says it has been added. We would rather poll that table (much faster as it doesn't need to open a read a slew of netcdf files). I can have Bob write a program in Java to do the required recheck operations if you can point us to the appropriate steps to be taken for the netcdf-java library.

In the near future, both the information to generate our catalog.xml and what files are present and/or have been added will be in a database.

-Roy

On Oct 30, 2008, at 9:55 AM, John Caron wrote:

We are going to add this kind of functionality to 4.0, currently it doesnt exist.

How do you detect when data changes?

Roy Mendelssohn wrote:
Hi John:

I am glad we understand how recheckEvery works. It is consistent with
what we were experiencing.

basically you want to do the recheck in the background, and then
accept whatever is there when a request comes in? Or you want to
detect whenever the directory changes?


Because other things, such as web page displays, need to be updated when
data is added, we already have a system where we know when data have
been added to a dataset.  What we would like to do is at that point
force a recheck for that dataset, and hopefully only for that dataset,
rather than the entire catalog.  If we can do that, we probably will
turn off the recheckEvery because we know every dataset will be up to date.

Does this makes sense? If need be, we can write a script in Java to do
the above, if we can get advice on the best way to do this.

Thanks,

-Roy

**********************
"The contents of this message do not reflect any position of the U.S.
Government or NOAA."
**********************
Roy Mendelssohn
Supervisory Operations Research Analyst
NOAA/NMFS
Environmental Research Division
Southwest Fisheries Science Center
1352 Lighthouse Avenue
Pacific Grove, CA 93950-2097

e-mail: address@hidden (Note new e-mail address)
voice: (831)-648-9029
fax: (831)-648-8440
www: http://www.pfeg.noaa.gov/

"Old age and treachery will overcome youth and skill."
"From those who have been given much, much will be expected"

**********************
"The contents of this message do not reflect any position of the U.S. Government or NOAA."
**********************
Roy Mendelssohn
Supervisory Operations Research Analyst
NOAA/NMFS
Environmental Research Division
Southwest Fisheries Science Center
1352 Lighthouse Avenue
Pacific Grove, CA 93950-2097

e-mail: address@hidden (Note new e-mail address)
voice: (831)-648-9029
fax: (831)-648-8440
www: http://www.pfeg.noaa.gov/

"Old age and treachery will overcome youth and skill."
"From those who have been given much, much will be expected"