[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[IDV #SRS-784269]: PermGen error and memory issues



Thanks for fixing this Don!

Sean

> There is already an item up there on permgen.  I've working on a branch
> that I hope to check in today.
> 
> Don
> 
> On 4/4/12 8:22 AM, Unidata IDV Support wrote:
> > I think that sounds reasonable. Would you like me to post this conversation 
> > to readmine?
> >
> > Sean
> >
> >> Okay, I just checked and I'm running with 128m even on the session where
> >> I run with 10G of max heap on the mac.  We could have the default as 128
> >> and let the user set it to as low as 64 if they want.
> >>
> >> Don
> >>
> >> On 4/3/12 11:41 AM, Unidata IDV Support wrote:
> >>> I run 128 MB on my mac and haven't ran into any issues yet, but I'm sure 
> >>> you abuse your IDV a bit more than me :-)
> >>>
> >>> Sean
> >>>
> >>>> I'd have to run some tests to make sure 128 works on the mac.  It might
> >>>> need to be 256.
> >>>>
> >>>> Don
> >>>>
> >>>> On 4/3/12 11:01 AM, Unidata IDV Support wrote:
> >>>>> I wouldn't have a problem with 128 MB - I don't think it would hurt 
> >>>>> anything, although we should do some checks to make sure that the 
> >>>>> automagic memory management does not allocate too much memory for the 
> >>>>> edge cases when a user puts something like "Use 99% of available 
> >>>>> memory" in their system preferences.
> >>>>>
> >>>>> Sean
> >>>>>
> >>>>>> 64m does not work on a mac, so the default has to be 128.  What if that
> >>>>>> was the default for all OS's?
> >>>>>>
> >>>>>> Don
> >>>>>>
> >>>>>> On 4/3/12 10:00 AM, Unidata IDV Support wrote:
> >>>>>>> Hi Don,
> >>>>>>>
> >>>>>>> The default minimum for the JVM is 64 MB. I think as long as the IDV 
> >>>>>>> can startup at 64 MB, we set it to that. We can recommend that Mac 
> >>>>>>> users bump it up to 128 MB (or maybe we can edit the Mac installer to 
> >>>>>>> use 128 MB by default). I just think it would be nice to get this in 
> >>>>>>> the release. As far as git, if you'd like to sit down and go through 
> >>>>>>> a few "branch, hack, commit, push" cycles, I'd be happy to do so!
> >>>>>>>
> >>>>>>> Sean
> >>>>>>>
> >>>>>>>> Sean-
> >>>>>>>>
> >>>>>>>> The tricky think is what to use as a minimum.  I think that varies 
> >>>>>>>> from
> >>>>>>>> system to system.  On the mac, it seems like 128 is a bare minimum.  
> >>>>>>>> And
> >>>>>>>> then you have to worry about how much total memory the user has.  I 
> >>>>>>>> have
> >>>>>>>> versions of runIDV and runIDV.bat which go along with the user
> >>>>>>>> preference.  I can send those to you if you want.  I'm not having fun
> >>>>>>>> with Git - otherwise, I'd make a branch. ;-)
> >>>>>>>>
> >>>>>>>> Don
> >>>>>>>>
> >>>>>>>> On 4/3/12 9:01 AM, Unidata IDV Support wrote:
> >>>>>>>>> Hi Don!
> >>>>>>>>>
> >>>>>>>>> I was (finally) looking into the PermGen stuff and noticed that you 
> >>>>>>>>> checked in some code back in December to handle this in the System 
> >>>>>>>>> Preference Tab, as well as code to handle saving the preference - 
> >>>>>>>>> awesome! I did some testing with jvisualvm to see how the PermGen 
> >>>>>>>>> space changed while abusing the IDV, and with 6 GB I was able to 
> >>>>>>>>> get 108 MB (I loaded about 20 or so bundles, has 12 views as well 
> >>>>>>>>> as data panels, etc.), so I would say 512 MB is a good upper limit. 
> >>>>>>>>> If you want, I can open a ticket on Redmine for discussion on the 
> >>>>>>>>> topic so that we can get your work in the tree...this will really 
> >>>>>>>>> help windows users, as they get a bit finicky when they have to 
> >>>>>>>>> edit the runIDV.bat script using admin privileges.
> >>>>>>>>>
> >>>>>>>>> Thanks!
> >>>>>>>>>
> >>>>>>>>> Sean
> >>>>>>>>>
> >>>>>>>>>> Hi Sean-
> >>>>>>>>>>
> >>>>>>>>>> Thanks for looking into this.  It's becoming more and more of an
> >>>>>>>>>> annoyance (especially on MacOS) and I'm surprised that no one has
> >>>>>>>>>> written in to complain (besides me).
> >>>>>>>>>>
> >>>>>>>>>> I've been using 256 and 512 (depending on how much memory I 
> >>>>>>>>>> allocate to
> >>>>>>>>>> the IDV which varies between 2 and 8 GB).
> >>>>>>>>>>
> >>>>>>>>>> I think we need to allow the user to set it through the system tab 
> >>>>>>>>>> of
> >>>>>>>>>> the user preferences in case the default doesn't work.  I think 
> >>>>>>>>>> starting
> >>>>>>>>>> small (i.e. 128) would be fine and the UI could just be an editable
> >>>>>>>>>> combo box of 128, 256, 512 (editable so the user can type in their 
> >>>>>>>>>> own
> >>>>>>>>>> amount if they want).  The selected value would be used in the
> >>>>>>>>>> runIDV/runIDV.bat scripts.
> >>>>>>>>>>
> >>>>>>>>>> Let me know if you have any questions about how to implement this.
> >>>>>>>>>>
> >>>>>>>>>> Don
> >>>>>>>>>>
> >>>>>>>>>> On 10/7/11 2:02 PM, Unidata IDV Support wrote:
> >>>>>>>>>>> Don,
> >>>>>>>>>>>
> >>>>>>>>>>> What would be the downside of adding -XX:MaxPermSize=128m to 
> >>>>>>>>>>> runIDV? 128 MB seems to keep things under control on my system (I 
> >>>>>>>>>>> think the default is 64MB).
> >>>>>>>>>>>
> >>>>>>>>>>> Sean
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>> Hi-
> >>>>>>>>>>>>
> >>>>>>>>>>>> During the recent AAP workshop, we were running the IDV under 
> >>>>>>>>>>>> CentOS in
> >>>>>>>>>>>> a virtual machine set up under Oracle's Virtual Box.  The native 
> >>>>>>>>>>>> system
> >>>>>>>>>>>> had 2 GB of memory and the VM's were set up to allocate 1.5 GB.
> >>>>>>>>>>>>
> >>>>>>>>>>>> The first problem was that the IDV only allocated 512 MB of the 
> >>>>>>>>>>>> 1.5
> >>>>>>>>>>>> available.  This caused problems loading satellite images and 
> >>>>>>>>>>>> any other
> >>>>>>>>>>>> raster data.
> >>>>>>>>>>>>
> >>>>>>>>>>>> The second issue was getting PermGen errors.  This seems to be 
> >>>>>>>>>>>> cropping
> >>>>>>>>>>>> up more and more.  At the very least, there should be an FAQ 
> >>>>>>>>>>>> item on
> >>>>>>>>>>>> this, but it might be useful to have a permgen setting in the 
> >>>>>>>>>>>> System tab
> >>>>>>>>>>>> of the Preferences so the users don't have to edit runIDV to 
> >>>>>>>>>>>> account for
> >>>>>>>>>>>> this.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Don
> >>>>>>>>>>>> --
> >>>>>>>>>>>> Don Murray
> >>>>>>>>>>>> NOAA/ESRL/PSD and CIRES
> >>>>>>>>>>>> 303-497-3596
> >>>>>>>>>>>> http://www.esrl.noaa.gov/psd/people/don.murray/
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> Ticket Details
> >>>>>>>>>>> ===================
> >>>>>>>>>>> Ticket ID: SRS-784269
> >>>>>>>>>>> Department: Support IDV
> >>>>>>>>>>> Priority: Critical
> >>>>>>>>>>> Status: Open
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> --
> >>>>>>>>>> Don Murray
> >>>>>>>>>> NOAA/ESRL/PSD and CIRES
> >>>>>>>>>> 303-497-3596
> >>>>>>>>>> http://www.esrl.noaa.gov/psd/people/don.murray/
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> Ticket Details
> >>>>>>>>> ===================
> >>>>>>>>> Ticket ID: SRS-784269
> >>>>>>>>> Department: Support IDV
> >>>>>>>>> Priority: Critical
> >>>>>>>>> Status: Open
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>> Don Murray
> >>>>>>>> NOAA/ESRL/PSD and CIRES
> >>>>>>>> 303-497-3596
> >>>>>>>> http://www.esrl.noaa.gov/psd/people/don.murray/
> >>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> Ticket Details
> >>>>>>> ===================
> >>>>>>> Ticket ID: SRS-784269
> >>>>>>> Department: Support IDV
> >>>>>>> Priority: Critical
> >>>>>>> Status: Open
> >>>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> Don Murray
> >>>>>> NOAA/ESRL/PSD and CIRES
> >>>>>> 303-497-3596
> >>>>>> http://www.esrl.noaa.gov/psd/people/don.murray/
> >>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>> Ticket Details
> >>>>> ===================
> >>>>> Ticket ID: SRS-784269
> >>>>> Department: Support IDV
> >>>>> Priority: Critical
> >>>>> Status: Open
> >>>>>
> >>>>
> >>>> --
> >>>> Don Murray
> >>>> NOAA/ESRL/PSD and CIRES
> >>>> 303-497-3596
> >>>> http://www.esrl.noaa.gov/psd/people/don.murray/
> >>>>
> >>>>
> >>>
> >>>
> >>> Ticket Details
> >>> ===================
> >>> Ticket ID: SRS-784269
> >>> Department: Support IDV
> >>> Priority: Critical
> >>> Status: Open
> >>>
> >>
> >> --
> >> Don Murray
> >> NOAA/ESRL/PSD and CIRES
> >> 303-497-3596
> >> http://www.esrl.noaa.gov/psd/people/don.murray/
> >>
> >>
> >
> >
> > Ticket Details
> > ===================
> > Ticket ID: SRS-784269
> > Department: Support IDV
> > Priority: Critical
> > Status: Open
> >
> 
> --
> Don Murray
> NOAA/ESRL/PSD and CIRES
> 303-497-3596
> http://www.esrl.noaa.gov/psd/people/don.murray/
> 
> 


Ticket Details
===================
Ticket ID: SRS-784269
Department: Support IDV
Priority: Critical
Status: Open


NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publicly available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us.