Hi John:while it is a little more complicated then that, for now we can assume we are just doing additions in time (basically we do deletes infrequently enough that we can do a restart even if needed).
-Roy On Oct 31, 2008, at 1:09 PM, John Caron wrote:
Are files also deleted, or only added? Do files ever get appended to or modified?Roy Mendelssohn wrote:When a file is added to a directory, a flag is set in a table that says it has been added. We would rather poll that table (much faster as itdoesn't need to open a read a slew of netcdf files). I can have Bobwrite a program in Java to do the required recheck operations if you can point us to the appropriate steps to be taken for the netcdf-java library.In the near future, both the information to generate our catalog.xml andwhat files are present and/or have been added will be in a database. -Roy On Oct 30, 2008, at 9:55 AM, John Caron wrote:We are going to add this kind of functionality to 4.0, currently it doesnt exist. How do you detect when data changes? Roy Mendelssohn wrote:Hi John:I am glad we understand how recheckEvery works. It is consistent withwhat we were experiencing.basically you want to do the recheck in the background, and then accept whatever is there when a request comes in? Or you want to detect whenever the directory changes?Because other things, such as web page displays, need to be updated when data is added, we already have a system where we know when data havebeen added to a dataset. What we would like to do is at that pointforce a recheck for that dataset, and hopefully only for that dataset, rather than the entire catalog. If we can do that, we probably will turn off the recheckEvery because we know every dataset will be up todate.Does this makes sense? If need be, we can write a script in Java to dothe above, if we can get advice on the best way to do this. Thanks, -Roy **********************"The contents of this message do not reflect any position of the U.S.Government or NOAA." ********************** Roy Mendelssohn Supervisory Operations Research Analyst NOAA/NMFS Environmental Research Division Southwest Fisheries Science Center 1352 Lighthouse Avenue Pacific Grove, CA 93950-2097 e-mail: address@hidden (Note new e-mail address) voice: (831)-648-9029 fax: (831)-648-8440 www: http://www.pfeg.noaa.gov/ "Old age and treachery will overcome youth and skill." "From those who have been given much, much will be expected"********************** "The contents of this message do not reflect any position of the U.S. Government or NOAA." ********************** Roy Mendelssohn Supervisory Operations Research Analyst NOAA/NMFS Environmental Research Division Southwest Fisheries Science Center 1352 Lighthouse Avenue Pacific Grove, CA 93950-2097 e-mail: address@hidden (Note new e-mail address) voice: (831)-648-9029 fax: (831)-648-8440 www: http://www.pfeg.noaa.gov/ "Old age and treachery will overcome youth and skill." "From those who have been given much, much will be expected"
**********************"The contents of this message do not reflect any position of the U.S. Government or NOAA."
********************** Roy Mendelssohn Supervisory Operations Research Analyst NOAA/NMFS Environmental Research Division Southwest Fisheries Science Center 1352 Lighthouse Avenue Pacific Grove, CA 93950-2097 e-mail: address@hidden (Note new e-mail address) voice: (831)-648-9029 fax: (831)-648-8440 www: http://www.pfeg.noaa.gov/ "Old age and treachery will overcome youth and skill." "From those who have been given much, much will be expected"