[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[IDD #FFC-953121]: Main system memory v. LDM queue size



Hi Justin,

re:
> Sounds like you might need some more details.

More information is always good :-)

> We have 6GB of data being
> generated every 6 hours that we need to get out to about 5 other LDM
> servers and about 500MB we need to get in from about 5 other LDM servers.

Questions:

- are the downstream LDM servers accessible by high speed network connections
  (e.g., >= 100 Mbps), or are they on the other end of a relatively slow 
connection

- are the upstream servers accessible by high speed network connections
  (e.g., >= 100 Mbps), or are they on the other end of a relatively slow 
connection

Given your scenario, the size of the queue will depend on how long it takes to
send the data to the slowest of your downstreams.  The same sort of comment is
applicable in the case of the data you are trying to ingest.

> Because its conceivable that all 6GB of data could be put into the
> queue at the same time, we believe we will need something larger to ensure
> that all the receivers have time to get the data, so we estimate around
> an 8GB queue.

I agree depending on the estimated time it will take to send the data to
the downstreams.

> Given that we are not getting nor receiving from a large number of hosts,
> what amount of memory would you recommend for the LDM server?

In your situation, I would recommend that the memory be on the order of the
size of your queue. If your data insertion needs were greatly exaggerated,
this rule of thumb would not hold (e.g., if you wanted to inject 400 GB of data,
it is highly unlikely that you could affors a machine with 400 GB of RAM, at 
least,
not today).

> Does LDM
> actually require twice as much memory as the size of the queue for our
> situation?  or would 10-12GB be sufficient?

No, it does not require twice as much memory as the size of the queue, and
yes 10-10 GB should be more than sufficient.

Question:

- what kind of machine are we talking about and what OS will it be running


Cheers,

Tom
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: FFC-953121
Department: Support IDD
Priority: Normal
Status: Closed