The LDM Product-Queue

The product-queue is a memory-mapped file that temporarily stores data-products. Access to the product-queue is via the pq module, which coordinates access by the reading and writing processes to ensure that the product-queue isn't corrupted.

The product-queue has the following characteristics:

• The number of data-products that the queue can contain is fixed when the queue is created. This is the maximum number of data-product slots.
• A data-product is stored in the product-queue in contiguous space.
• Reading a data-product from the queue creates a read-lock on that product. Read-locks can be shared by reading processes.
• Writing a data-product into the queue creates a write-lock on that product. Write-locks can't be shared.
• If a new data-product can't be added, either because no space is available for the data-product or because no slot is available, then unlocked data-products are deleted from the queue -- begining with the oldest data-product and continuing to the youngest -- until addition is possible.

The canonical pathname of the LDM product-queue, relative to the LDM home-directory, is data/ldm.pq

A paper on implementation-details can be found here.