Re: [thredds] wget and thredds server

On 5/11/2012 7:33 AM, Patrick Brockmann wrote:
Hi all,

I would like to synchonize some netcdf files available from
a THREDDS Data Server with a local repository.

I have used the wget command and HTTPServer access from
the TDS but it seems that the server does not provide enough
or correct informations to use properly the --timestamp option
of wget (very useful to synchronize).
Read http://www.gnu.org/software/wget/manual/wget.html#Time_002dStamping

The method works for an old OPeNDAP server but not with 2 different Thredds Data Servers

Try:
$ wget -N http://dods.ipsl.jussieu.fr/brocksce/tmp/tyty.txt
If you retry you get the message
----> Server file no newer than local file `tyty.txt' -- not retrieving.
if the 2 files (input and local) are not the same from date or size, the
download is done again.
If you modify the file by adding a character, you get
----> The sizes do not match (local 43) -- retrieving.
If you apply a touch -t 201106230000 on the file to see what happens
----> Remote file is newer, retrieving.

I would like to get the same feature (synchro on date and size) when I download from a TDS/HTTPServer.

Any help welcome on this synchronization.

Regards
Patrick


Hi Patrick:

When compression is enabled, which it often is on a TDS, the Content-Length header is not sent, since the response is variable length. Apparently wget needs that to synch (?) Seems like it should just use Last-Modified.

I dont see a workaround without turning off compression.

John



  • 2012 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the thredds archives: