Re: [ldm-users] Running your LDM queues out of memory really helps!


On Wed, 27 Feb 2008, Gilbert Sebenste wrote:

On Wed, 27 Feb 2008, Arthur A. Person wrote:


Yes... it's a great way to speed things up and save wear-and-tear on a disk
if your queue is relatively small and you have enough memory.  The primary
disadvantage is if you reboot you lose everything in the queue, which is a
problem for relay sites and a potential problem for the local system if the
decoders are running behind in queue processing.

Since I have multiple machines, I avoid that problem by rebooting one
machine at a time, and try to do it in low-volume hours.

The solid state drives, I have been told, have a nasty habit of going out
on you gradually after about 300,000 writes (which ain't much when you
have something like NOAAport).

Actually, for 50 GB/day in a 2 GB queue, that would be 25 re-writes/day or 300000/25/365=32 years. Some of the newer drives have write cycles in the 1-5 million range, so I think it might work okay for an LDM queue.

Also, right now they're about 15
times more expensive than standard hard drives, have much lower capacity
than hard drives,

True, but we should be comparing them to system memory, not hard drives. 8 GB of system memory for one of my servers is selling for ~$600 right now. I can buy a RiDATA 32 GB SATA SSD for ~$680, which is ~ 1/4 the price/GB for my server. Depending on your server and its memory costs, SSD could be more expensive, but SSD's are advancing rapidly and I expect costs to drop quickly.

and as I see on Wikipedia, really bad things can happen
if your power goes out while it's running, even worse than with a hard
drive. See:

Hmmm, not sure about the power issue. If it's a significant problem, I would expect the designers would fix it pretty quickly since these things are going to be in everyone's laptops pretty soon. Short of breaking (such that it's not revivable after a power failure/reboot cycle), I don't see how that would be any worse than using main memory... you would still have to remake the LDM queue. Extra main memory is also used extensively by the OS for file system caching, so before grabbing up a lot of it for use by an LDM queue, you need to be sure you have enough to spare.


Gilbert Sebenste                                                     ********
(My opinions only!)                                                  ******
Staff Meteorologist, Northern Illinois University                      ****
E-mail: sebenste@xxxxxxxxxxxxxxxxxxxxx                                  ***
web:                                      **
ldm-users mailing list
For list information or to unsubscribe,  visit:

Arthur A. Person
Research Assistant, System Administrator
Penn State Department of Meteorology
email:  person@xxxxxxxxxxxxx, phone:  814-863-1563

  • 2008 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the ldm-users archives: