[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

20041209: ldm queue size

>From: Eirh-Yu Hsie <address@hidden>
>Organization: CU/CIRES
>Keywords: 200412091453.iB9ErOlI022584 LDM queue CONDUIT

Hi Hsie,

re: 0.5 degree GFS added to CONDUIT stream
>What kind of machine are you running LDM on?

We run LDMs on Sun SPARCs, Fedora Core Linux [12], and FreeBSD 4.[89].

>On a Intel 32-bit machine, there
>is a 2 GB file size limits.

FreeBSD has a 2 GB limit on queue size since the maximum size of
a file that can be memory mapped is 2 GB.  Linux, on the other hand
allows one to create a queue up to 4 GB in size as long as the
LDM is built with large file support.

Here are the settings we use to build queues larger than 2 GB on
Solaris SPARC and Linux:

Solaris SPARC:

  Environment setting(s):

  CFLAGS=-O -xarch=v9

  Modification to LDM's configure:

  change:

               *)  libs="-R/usr/ucblib -L/usr/ucblib -lrpcsoc -lnsl"

  to:

               *)  libs="-R/usr/ucblib/sparcv9 -L/usr/ucblib/sparcv9 -lrpcsoc 
-lnsl"


Linux:

  Environment variable(s):

  CPPFLAGS=-D_FILE_OFFSET_BITS=64 -D_LARGEFILE64_SOURCE

If you are using LDM-6.1.0 you can set the queue size in ldmadmin
using:

$pq_size="4G"

If you are using LDM-6.0.14, you should use:

$pq_size="4000M"

We are running on a couple of "PC" platforms that are not 32-bit limited:

- dual AMD Opteron running Fedora Core 2 64 bit
- dual AMD Opteron running Sun Solaris x86 v10

We have not run these systems hard enough to recommend them to users yet,
but preliminary results look very good on both as IDD leaf nodes.  I
have seen some performance problems on the FC2 machine when trying to
use it as an IDD relay node.

Please let me know if you would like more information on building a
large queue under Linux.

Cheers,

Tom
--
NOTE: All email exchanges with Unidata User Support are recorded in the
Unidata inquiry tracking system and then made publicly available
through the web.  If you do not want to have your interactions made
available in this way, you must let us know in each email you send to us.