[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[LDM #SXI-868914]: Data product maximum size

Hi Leonard,

> We're running LDM 6.10.1 on a mix of Linux CentOS 5 and 6 systems. All
> are 64-bit.  Builds were done on the systems the software runs on.  The
> product queues are 5GB.
> Upstream has started sending data products (files) that are 2.4GB each.
> Some come through, some do not.
> I've seen mention of a 2GB limit in a couple of support tickets. Since
> it is working for some products which are larger than than, I'd like to
> ask what the maximum product size is.

The metadata that accompanies a data-product has a size parameter that's 
interpreted as a 32-bit unsigned integer, so the maximum size of a data-product 
is (2^32)-1 bytes.

The approximately 2GB limit stems from the type of an "off_t" in the C runtime 
library. "off_t"s are used to randomly access the product-queue. On many 32-bit 
systems, they are a 32-bit signed integer, in which case the maximum size of 
the product-queue is limited to (2^31)-1 bytes. Because you have a 64-bit 
system, this doesn't apply and you shouldn't have any problems sending (2^32)-1 
byte products.

> Thanks!
> --
> ==Leonard E. Sitongia
> High Altitude Observatory
> National Center for Atmospheric Research
> P.O. Box 3000 Boulder CO 80307  USA
> address@hidden  voice: (303)497-2454  fax: (303)497-1589

Steve Emmerson

Ticket Details
Ticket ID: SXI-868914
Department: Support LDM
Priority: Normal
Status: Closed

NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publicly available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us.