Re: [thredds] Large aggregated datasets, WMS memory issue

On 3/31/2011 2:21 AM, Jon Blower wrote:
Hi Lin, all,

Memory issues in Java are often mysterious because the
garbage-collection is hidden from view.  But I can't see any reason
why the request below (which is small) should cause memory problems.
The size of the dataset doesn't matter for a GetMap request, and the
size of the image doesn't matter much (it matters a bit, but should
be trivial for a 100x100 image).

So I don't really know what's going on here.  If you start from a
fresh reboot and make the same GetMap request, do you consistently
get the error?  Or only after a period of use?


To aid in diagnosing memory problems, you can add "-verbose:gc" to JAVA_OPTS= in your [tomcat]/bin/startup.sh or setenv.sh file.

Jon

-----Original Message-----


Hi John,

Thanks for the information. My large datasets will eventually be
hosted on a 64bit JVM. At the moment, I'm testing a 1.3GB dataset
with -Xmx1536m on a dev machine. The following WMS request results in
HTTP Status 500 - Internal Server Error (java.lang.OutOfMemoryError:
Java heap space):

http://localhost:8090/thredds/wms/aggregated/magmap_V5_2010_s.nc?service=WMS&version=1.3.0&request=GetMap&Layers=magmap_V5_2010&CRS=EPSG:4326&BBOX=152,-39,154,-37&width=100&height=100&Styles=BOXFILL/occam_pastel-30&format=image/jpeg

 The dataset has an bounding box:
<westBoundLongitude>112.50461544244448</westBoundLongitude>
<eastBoundLongitude>154.66376524558447</eastBoundLongitude>
<southBoundLatitude>-39.560232620003625</southBoundLatitude>
<northBoundLatitude>-35.19773436500001</northBoundLatitude>

I guess the request is faily small (BBOX=152,-39,154,-37) .

I   In addtion, the NetCDFSubsetService works fine for the same
dataset with the same amount of memory.

I'm wondering if in general case, the maximum heap is determined by
the dataset size for Thredds WMS service, and if there is an
configuration option to limit the WMS request size.

Regards and thanks,

Lin

________________________________ From:
thredds-bounces@xxxxxxxxxxxxxxxx
[mailto:thredds-bounces@xxxxxxxxxxxxxxxx] On Behalf Of John Caron
Sent: Thursday, 31 March 2011 5:21 AM To: thredds@xxxxxxxxxxxxxxxx
Subject: Re: [thredds] Large aggregated datasets, WMS memory issue

Hi Lin:

It doesnt (usually) matter how big the dataset is, just how big the
request is.

Can you send a typical WMS request that causes this problem? Do you
know what size of data you are requesting? What file format?

-Xmx1536m is around the max for 32 bit JVMs. I strongly advise you to
use a 64bit JVM with more like 4 Gbyte heap.

John

_______________________________________________ thredds mailing list
thredds@xxxxxxxxxxxxxxxx For list information or to unsubscribe,
visit: http://www.unidata.ucar.edu/mailing_lists/

--
Sincerely,

Bob Simons
IT Specialist
Environmental Research Division
NOAA Southwest Fisheries Science Center
1352 Lighthouse Ave
Pacific Grove, CA 93950-2079
Phone: (831)658-3205
Fax:   (831)648-8440
Email: bob.simons@xxxxxxxx

The contents of this message are mine personally and
do not necessarily reflect any position of the
Government or the National Oceanic and Atmospheric
Administration.
<>< <>< <>< <>< <>< <>< <>< <>< <><