Re: Thredds out of memory

Tennessee Leeuwenburg wrote:

For anyone who's interested, I am now quite happily serving large data sets via HTTP to thredds. The key was to re-write my java servlet in python. I think object creation overhead was killing me because of the sheer number of independant requests made by thredds. I used a default window size of 65,536 bytes, and was able to get 1.1Mb/s throughput downloading from thredds. My test file was 590Mb.

I like this approach, because my custom data source is neatly de-coupled from thredds code. All I have to do is deal with the HTTP protocol, and everything is handled for me.

If anyone else is having difficulty integrating a new data source into thredds, feel free to contact me for my code.

Thanks for everyone's help...

Cheers,
-Tennessee

Are you sure the key was moving to python, and not moving to a 65K buffer?


  • 2005 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the netcdf-java archives: