Re: [thredds] Large aggregated datasets, WMS memory issue

• To: "thredds@xxxxxxxxxxxxxxxx" <thredds@xxxxxxxxxxxxxxxx>
• Subject: Re: [thredds] Large aggregated datasets, WMS memory issue
• From: Jon Blower <j.d.blower@xxxxxxxxxxxxx>
• Date: Thu, 31 Mar 2011 09:21:11 +0000

Hi Lin, all,
 
Memory issues in Java are often mysterious because the garbage-collection is 
hidden from view.  But I can't see any reason why the request below (which is 
small) should cause memory problems.  The size of the dataset doesn't matter 
for a GetMap request, and the size of the image doesn't matter much (it matters 
a bit, but should be trivial for a 100x100 image).

So I don't really know what's going on here.  If you start from a fresh reboot 
and make the same GetMap request, do you consistently get the error?  Or only 
after a period of use?

Jon

-----Original Message-----


Hi John,

Thanks for the information. My large datasets will eventually be hosted on a 
64bit JVM. At the moment, I'm testing a 1.3GB dataset with -Xmx1536m on a dev 
machine. The following WMS request results in HTTP Status 500 - Internal Server 
Error (java.lang.OutOfMemoryError: Java heap space):

http://localhost:8090/thredds/wms/aggregated/magmap_V5_2010_s.nc?service=WMS&version=1.3.0&request=GetMap&Layers=magmap_V5_2010&CRS=EPSG:4326&BBOX=152,-39,154,-37&width=100&height=100&Styles=BOXFILL/occam_pastel-30&format=image/jpeg

The dataset has an bounding box:
<westBoundLongitude>112.50461544244448</westBoundLongitude>
    <eastBoundLongitude>154.66376524558447</eastBoundLongitude>
    <southBoundLatitude>-39.560232620003625</southBoundLatitude>
    <northBoundLatitude>-35.19773436500001</northBoundLatitude>

    I guess the request is faily small (BBOX=152,-39,154,-37) .

I   In addtion, the NetCDFSubsetService works fine for the same dataset with 
the same amount of memory.

I'm wondering if in general case, the maximum heap is determined by the dataset 
size for Thredds WMS service, and if there is an configuration option to limit 
the WMS request size.

Regards and thanks,

Lin

________________________________
From: thredds-bounces@xxxxxxxxxxxxxxxx 
[mailto:thredds-bounces@xxxxxxxxxxxxxxxx] On Behalf Of John Caron
Sent: Thursday, 31 March 2011 5:21 AM
To: thredds@xxxxxxxxxxxxxxxx
Subject: Re: [thredds] Large aggregated datasets, WMS memory issue

Hi Lin:

It doesnt (usually) matter how big the dataset is, just how big the request is.

Can you send a typical WMS request that causes this problem? Do you know what 
size of data you are requesting? What file format?

-Xmx1536m is around the max for 32 bit JVMs. I strongly advise you to use a 
64bit JVM with more like 4 Gbyte heap.

John

• Follow-Ups:
  • Re: [thredds] Large aggregated datasets, WMS memory issue
    • From: Bob Simons