[thredds] THREDDS 3.16: Memory issues

[John Caron]

Hi Carlos:

We are looking at removing these kinds of limitations in the TDS 4.0, but it wont be for a while. Im wondering what client you use that can handle such big output?

Carlos Valiente wrote:
> John Caron wrote:
>>> We're starting up Tomcat with -Xmx406m, in order to give it 4 GB of 
>>> memory, but that does not seem to be enough. We've tried raising that 
>>> limit to 6 GB, which is the total memory (physicall + virtual) 
>>> available on that box, but it did not make any difference.
>> -Xmx406m = 406Mbytes, i assume you mean -Xmx4096m ??
> 
> Yep, typo
> 
>> Are you running 64-bit linux and JVM? Otherwise the JVM maxes out at 
>> around 2 Gb.
> 
> Yep, it's the 64-bit one.
> 
>>> Are THREDDS memory requirements for aggregation higher than those 
>>> figures, or are we perhaps doing something not too clever with our setup?
>> the aggregation itself should not be the problem. But how big is the 
>> request?
> 
> For the variable that does not work (geopotential), these are the NetCFD 
> files:
> 
> $ du -sch *
> 178M    MM_129_mon_1980.nc
> 178M    MM_129_mon_1981.nc
> 178M    MM_129_mon_1982.nc
> 178M    MM_129_mon_1983.nc
> 178M    MM_129_mon_1984.nc
> 178M    MM_129_mon_1985.nc
> 178M    MM_129_mon_1986.nc
> 178M    MM_129_mon_1987.nc
> 178M    MM_129_mon_1988.nc
> 178M    MM_129_mon_1989.nc
> 178M    MM_129_mon_1990.nc
> 178M    MM_129_mon_1991.nc
> 178M    MM_129_mon_1992.nc
> 178M    MM_129_mon_1993.nc
> 178M    MM_129_mon_1994.nc
> 178M    MM_129_mon_1995.nc
> 178M    MM_129_mon_1996.nc
> 178M    MM_129_mon_1997.nc
> 178M    MM_129_mon_1998.nc
> 178M    MM_129_mon_1999.nc
> 178M    MM_129_mon_2000.nc
> 178M    MM_129_mon_2001.nc
> 3.9G    total
> $
> 
> The files for the one that does work are smaller:
> 
> $ -sch *
> 60M     MM_228_mon_1980.nc
> 60M     MM_228_mon_1981.nc
> 60M     MM_228_mon_1982.nc
> 60M     MM_228_mon_1983.nc
> 60M     MM_228_mon_1984.nc
> 60M     MM_228_mon_1985.nc
> 60M     MM_228_mon_1986.nc
> 60M     MM_228_mon_1987.nc
> 60M     MM_228_mon_1988.nc
> 60M     MM_228_mon_1989.nc
> 60M     MM_228_mon_1990.nc
> 60M     MM_228_mon_1991.nc
> 60M     MM_228_mon_1992.nc
> 60M     MM_228_mon_1993.nc
> 60M     MM_228_mon_1994.nc
> 60M     MM_228_mon_1995.nc
> 60M     MM_228_mon_1996.nc
> 60M     MM_228_mon_1997.nc
> 60M     MM_228_mon_1998.nc
> 60M     MM_228_mon_1999.nc
> 60M     MM_228_mon_2000.nc
> 60M     MM_228_mon_2001.nc
> 1.3G    total
> $
> 
>  > The current opendap implementation requires the response to be
>  > built in memory.
> 
> OK, I see. So I guess we need more memory in order to serve that kind of 
> requests...
> 
> Thanks for your quick response, John!
> 
> C
> 
> _______________________________________________
> thredds mailing list
> thredds at unidata.ucar.edu
> For list information or to unsubscribe,  visit: http://www.unidata.ucar.edu/mailing_lists/