The LDM Product-Queue

The product-queue is a memory-mapped file that temporarily stores data-products. Access to the product-queue is via the pq module, which coordinates access by the reading and writing processes to ensure that the product-queue isn't corrupted.

The product-queue has the following characteristics:

• The number of data-products that the queue can contain is fixed when the queue is created. This is the maximum number of data-product slots.
• A data-product is inserted into the product-queue in contiguous space and indexed by to the time of insertion according to the system clock.
• Reading a data-product from the queue creates a read-lock on that product. Read-locks can be shared by reading processes.
• Data-products read from the product-queue sequentially are returned in time-sorted order (oldest first) according to the insertion-time. Therefore, data-products can be skipped if the system clock is set backwards.
• Writing a data-product into the queue creates a write-lock on that product. Write-locks can't be shared.
• If a new data-product can't be added, either because no space is available for the data-product or because no slot is available, then unlocked data-products are deleted from the queue -- begining with the oldest data-product and continuing to the youngest -- until addition is possible.
• A writer-counter field in the product-queue keeps track of the number of times the product-queue has been opened for writing. Opening the product-queue for writing increments the counter; closing the product-queue decrements the counter. If no process has the product-queue open for writing, then the writer-counter should be 0. This counter is used to ensure that the product-queue has been properly closed by all processes that could modify it.

A paper on implementation-details can be found here.