Re: [thredds] How to download bulk datasets?

Hi Jerry,

Thanks for the suggestion. We've heard this type of request a few times.
I have added it to our enhancement wish list so we can try to fit it in
to our plans.



On 7/1/2010 1:36 PM, Pan, Jerry Yun wrote:
> I too wish there is a simple link on the server side to allow
> our users to use wget for downloading all files of a collection.
> I think it might be the case that every user would want to do a
> bulk file download from a thredds server, and I think it might
> be the case that a provider should have the burden to have a
> simple link for bulk download all granules.
> We can do some configurations or add a servlet for this, like
> Heiko has done, although I think it would be a nice to have
> feature directly from TDS software. It would seem to me that
> this is can be implemented as a dynamic URL at collection
> level to return a list of HTTP download URLs of the files
> ('fielServer') for the simplest case.
> Comments?
> Thanks,
> -Jerry
>> -----Original Message-----
>> From: thredds-bounces@xxxxxxxxxxxxxxxx 
>> [mailto:thredds-bounces@xxxxxxxxxxxxxxxx] On Behalf Of Heiko Klein
>> Sent: Monday, May 10, 2010 5:18 AM
>> To: John Caron
>> Cc: thredds@xxxxxxxxxxxxxxxx
>> Subject: Re: [thredds] How to download bulk datasets?
>> Hi John,
>> I played a bit more with the catalog.xml. This works well 
>> with wget. I managed now to download all the netcdf-files 
>> from a directory:
>> wget -nc -r -l2   -I /thredds/fileServer/,/thredds/catalog/
>> 'http://dev-vm188/thredds/catalog/osisaf/'
>> I use here the existing datasetScan catalog.xml file, and 
>> fetch all nc-files up to two links away. Beside the nc-file, 
>> I get the catalog-file of the nc-file (e.g.
>> http://dev-vm188/thredds/catalog/osisaf/
>> l?,
>> too.
>> A catalog-file in the fileServer would be saver, since the 
>> 2-levels (parent and child) might include other information, 
>> but at least I can offer our users something already now.
>> Best regards,
>> Heiko
>> On 2010-05-06 21:31, John Caron wrote:
>>> Hi Heiko:
>>> We use catalog.xml exactly because theres no standard html 
>> index format.
>>> A simple java GUI app could make this easy to do, but Im 
>> not clear if 
>>> that would help your case.
>>> John

  • 2010 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the thredds archives: