[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[LDM #WYF-519931]: LDM - Future use of LDM for next generation GOES



Dan,

re:
> Thank you for all of your previous responses.  I have another request, I
> was hoping that you could give me the current processing requirements
> for the LDM server.  I can't seem to find the information on the
> website.

The reason that there are no pages dealing with system recommendations
for machines running the LDM is the equipment needed depends on how
much data a site wants to ingest and relay.

> In particular the number of CPUs & type, memory requirements,
> I/O size.

For a site just getting started in Unidata, we typically recommend a
well configured, dual 64-bit processor machine with an ample amount
of RAM.  For a site with modest data needs this might be a dual
Xeon EM64T machine with at least 2 GB of RAM.  For a site that wants
to be able to ingest AND relay all of the data in the IDD, we typically
recommend implementing a cluster approach like the one we built here
at the UPC.  The following is a short description of our toplevel
IDD relay node cluster, idd.unidata.ucar.edu:

idd.unidata.ucar.edu, is a Linux cluster composed of:

2 - accumulators, machines that request data feeds from upstream sites
1 - director, a machine that sends feed requests to back end data servers
4 - data servers, these are the machines that feed downstream sites

For a couple of years the accumulator machines consisted of a dual 1.8 Ghz
Opteron PC w/2 GB of RAM running Fedora Core Linux (1, then 3, then 4) and
a dual 1 Ghz P4 box w/3 GB of ram running FreeBSD 4.x. We have recently
upgraded the Opteron box to a dual 2.8 Ghz Xeon EM64T (64-bit) box w/6 GB
of RAM running 64-bit Fedora Core 5 Linux. We have plans to upgrade the
FreeBSD box in the not too distant future.

The director is currently a dual 2.8 Ghz, 32-bit Xeon machine, but it really
doesn't need to be. We could get run a much less richly configured machine
as the director.

The dataservers are all dual 2 Ghz Opteron boxes with 14 or 16 GB of RAM.
The large amount of memory allows us to keep over two hours of ALL of the
data being relayed in the IDD in the LDM queue.

The cluster has been show to be able to relay up to 900 Mbps of data in
stress tests almost a year ago. It currently relays over 230 Mbps of
data _on average_ to about 400 downstream connections, and routinely
has peak relay rates of 440 Mbps. Having 4 dataservers in the cluster
and a Gpbs network allows us to act as the failover for any/all IDD
connections in the world.

The entire cluster cost under $25-30K to put together. A more modest cluster
could be put together for under $15K. The sizing of the cluster would
depend on how many downstream sites one desired to be able to feed and
how much data one wanted to have available in the LDM queue.

> Thanks,

I hope that this helps...


Cheers,

Tom
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: WYF-519931
Department: Support LDM
Priority: High
Status: Closed