I am just resending a copy of an email I wrote a month or so ago regarding easier access to large data downloads for the PATMOS dataset. I would be grateful if you could give me some guidance on the matters raised please?
Dear Sir / Madam,
I am currently investigating the possibilities of using the AVHRR visible channel record to put together a proposal for investigating how a species of highly reflective marine phytoplankton have shifted in their global distribution over the past 30 years. Mike Foster and Andrew Heidinger kindly pointed me towards the AVHRR Pathfinder Atmospheres- Extended (PATMOS-x) Reflectance data set available from the catalog:
Dr John Caron suggested that I contacted NCDC directly. It seems with some testing and code writing on my part, that the dataset is ideal (I require Channel 1 and 2 reflectances as well as the attendant geometries to do other corrections).
However, each daily global file at 0.1 degree resolution (which is required to look at the features of interest) and the parameters required results in a netCDF file of approximately 155 Mb. Each of these takes about a minute to put together the request on the website and then a further 8 minutes to download.
Putting all of these numbers together, for the 35 year time series, results in:
2 Tb of data to download
90 days of download time(!)
I was wondering whether or not there are some alternative options that we could investigate to:
(1) Speed up the data subsetting and;
(2) Transfer the data across.
Please? One option we have looked at in the past is sending discs over to agencies so that the data can be physically couriered across.
I would be very grateful if you could help me in this matter please?
Dr Tim Smyth
Plymouth Marine Laboratory
Latest news: www.pml.ac.uk and @PlymouthMarine Plymouth Marine Laboratory (PML) is a company limited by guarantee registered in England & Wales, company number 4178503. Registered Charity No. 1091222. Registered Office: Prospect Place, The Hoe, Plymouth PL1 3DH, UK. This message is private and confidential. If you have received this message in error, please notify the sender and remove it from your system. You are reminded that e-mail communications are not secure and may contain viruses; PML accepts no liability for any loss or damage which may be caused by viruses.
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publicly available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us.