_From owner-netcdf-java@xxxxxxxxxxxxxxxx Tue Apr 5 23:09:54 2005
Received: (from majordo@localhost)
by unidata.ucar.edu (UCAR/Unidata) id j3657t0r012120
for netcdf-java-out; Tue, 5 Apr 2005 23:07:55 -0600 (MDT)
Received: from gale.ho.bom.gov.au (gale.ho.bom.gov.au [126.96.36.199])
by unidata.ucar.edu (UCAR/Unidata) with ESMTP id j3657Vv2012108;
Tue, 5 Apr 2005 23:07:37 -0600 (MDT)
Received: from [188.8.131.52] (kahless.ho.bom.gov.au [184.108.40.206])
by gale.ho.bom.gov.au (8.9.3 (PHNE_29774)/8.9.3) with ESMTP id FAA17501;
Wed, 6 Apr 2005 05:07:24 GMT
User-Agent: Mozilla Thunderbird 1.0 (X11/20041206)
X-Accept-Language: en-us, en
References: <42423D15.3080204@xxxxxxxxxx> <4242E000.4020807@xxxxxxxxxxxxxxxx>
X-Enigmail-Supports: pgp-inline, pgp-mime
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
For anyone who's interested, I am now quite happily serving large data
sets via HTTP to thredds. The key was to re-write my java servlet in
python. I think object creation overhead was killing me because of the
sheer number of independant requests made by thredds. I used a default
window size of 65,536 bytes, and was able to get 1.1Mb/s throughput
downloading from thredds. My test file was 590Mb.
I like this approach, because my custom data source is neatly de-coupled
from thredds code. All I have to do is deal with the HTTP protocol, and
everything is handled for me.
If anyone else is having difficulty integrating a new data source into
thredds, feel free to contact me for my code.
Thanks for everyone's help...