One trick would be make your pqact.conf entries pipe the
bulletin to an executable script. The script can take its
STDIN and write it to a file, then touch another file or
whatever you want to use as a signal. Or if your
application is small enough, pqact.conf can pipe the
bulletin directly to your application. This eliminates the
need to configure pqact.conf to write the bulletin to the
filesystem directly.
Andy Huhn
DataSwitch Information Services, Inc.
http://www.stormwarn.com
-----Original Message-----
[mailto:owner-ldm-users@xxxxxxxxxxxxxxxx]On
Behalf Of Anne Wilson
Sent: Friday, October 05, 2001 5:34 PM
Cc: ldm-users@xxxxxxxxxxxxxxxx
"Jason J. Levit" wrote:
Hi all,
I'm writing an application that will process
some real-time data I'm
ingesting through LDM. This application will
run around the clock in
the cron, so it needs to be fairly robust.
Here's my question - say I
ingest a file via LDM. How do I know that file
is complete? Does LDM
place some sort of lock or other information on
the file before it's
completed writing? If my application tries to
access a file that does
not contain all the necessary data (a file that
still has yet to be
completely transmitted, for example), then I
need to have my application
not process the file.
Does anyone else out there have experience
with this? I know there's
tricks you can perform, such as checking to see
if the file has been
modified in the past 30 seconds before opening,
etc., etc., but I was
just wondering if LDM actually places some sort
of information with the
file that says "hey, this file isn't done yet".
Thanks for any help!
Jason
--
--------------------------------------------------
--------------------------
Jason J. Levit, N9MLA
Research Scientist,
jlevit@xxxxxx Center for Analysis
and Prediction of Storms
Room 1022
University of Oklahoma
405/325-3503
http://www.caps.ou.edu/
Hi Jason,
I'm not sure if you're talking about when a
product is received in the
queue, or when a product is being written out to
disk from the queue via
pqact, although I'm guessing it's the latter.
When the LDM receives a product in the queue,
that product is not
available to pqact or for retransmission to
another site until the
product is complete. For the purpose of
retransmission or actions by
pqact, you can consider the product to be an atomic unit.
When pqact is writing the product out to file, it
is unknown as to when
the product is complete. (I am assuming that
you're not talking about
appending to a file.) This is a frequently asked
question. You would
have to devise some trick to figure that out.
For example, if you know
the structure of the product, you could write a
program that simply
writes out the product, but also identifies when
the end of the product
is reached and touches some other file to serve
as a flag. You would
then PIPE the product to your program via pqact.
But, to answer your
question, the LDM does not have any way to signal
that pqact has or has
not completely written the product to disk.
Hope this helps!
Anne
--
***************************************************
Anne Wilson UCAR Unidata Program
anne@xxxxxxxxxxxxxxxx P.O. Box 3000
Boulder, CO 80307
----------------------------------------------------
Unidata WWW server http://www.unidata.ucar.edu/
****************************************************
