Re: [thredds] THREDDS Garbage Collection Tuning

Bob,

This is great info.   Two questions:

1.  So what do you guys actually do on your tomcat TDS?
Is the only thing you actually do is to bring -Xms up to the same
value as -Xmx?   No other tricky stuff?

2.  What would be involved with running the TDS on multiple servers
with load balancing?
Is anyone doing that?

Thanks,
Rich

On Wed, Jun 1, 2011 at 12:27 PM, Bob Simons <Bob.Simons@xxxxxxxx> wrote:
> Three options:
>
> Tweak it: Java Garbage Collectors (GC's), and the recommendations for using
> them, have changed several times. I encourage you to find a more recent
> article (e.g.,
> http://www.oracle.com/technetwork/java/javase/gc-tuning-6-140523.html) so
> you get the recommendations for the version of Java that you are using. It's
> worth a try.
>
> Fix it: The ideal approach is to identify why the problem is occurring, not
> just deal with the consequences of the problem. Some of the issues are
> programming related and can be solved or minimized via code changes, but you
> have to have a clear idea of what the problem is. You can use hprof to
> identify what lines of code are generating the most, persistent objects.
> (There are other profilers.)  And you can use -verbose:gc to monitor the GC.
>  I'm sure John Caron has investigated and will continue to investigate
> thoroughly. But you are in a better position to investigate the problems on
> your TDS.
>
> Restart it: It is hard/impossible to prevent these problems from occurring.
>  The garbage collectors aren't perfect. Sometimes, increasing MaxPermSize
> and Xmx and tweaking the GC settings just forestall the inevitable.
> Sometimes, you just need to acquiesce and restart tomcat or restart the
> computer.  Although the big web services do as much as possible to avoid
> problems like this, they rely on multiple servers and load balancing so that
> a given computer can be restarted when needed.
>
> Good luck.
>
> On 6/1/2011 8:24 AM, Rich Signell wrote:
>>
>> THREDDS folk,
>>
>> Have folks played around with Garbage Collection tuning parameters
>> besides "-XX:MaxPermSize" to help tomcat/TDS permgen errors?
>>
>> Our TDS tomcat is throwing "GC overhead limit exceeded" errors again.
>> I know on the Unidata TDS page
>>
>> http://www.unidata.ucar.edu/projects/THREDDS/tech/tds4.2/tutorial/BasicTomcatAndTDSSecurity.html
>> it says the only thing you can do is increase -XX:MaxPermSize, which
>> only delays the problem, solved only by restarting tomcat.
>>
>> But in this Garbage Collection tuning article
>> http://www.petefreitag.com/articles/gctuning/
>> they describe many different options, and I'm wondering whether folks
>> have tried using these, or whether they just don't do any good.
>>
>> Thanks,
>> Rich
>
> --
> Sincerely,
>
> Bob Simons
> IT Specialist
> Environmental Research Division
> NOAA Southwest Fisheries Science Center
> 1352 Lighthouse Ave
> Pacific Grove, CA 93950-2079
> Phone: (831)658-3205
> Fax:   (831)648-8440
> Email: bob.simons@xxxxxxxx
>
> The contents of this message are mine personally and
> do not necessarily reflect any position of the
> Government or the National Oceanic and Atmospheric
> Administration.
> <>< <>< <>< <>< <>< <>< <>< <>< <><
>



-- 
Dr. Richard P. Signell   (508) 457-2229
USGS, 384 Woods Hole Rd.
Woods Hole, MA 02543-1598