Unidata - To provide the data services, tools, and cyberinfrastructure leadership that advance Earth system science, enhance educational opportunities, and broaden participation. Unidata
         
  advanced  
 

20020415: PIPE processing for large products

>From: Joe Van Andel <vanandel@xxxxxxxxxxxx>
>Organization: National Center for Atmospheric Research
>Keywords: 200204151621.g3FGLRa22170 LDM PIPE

Joe,

>If I call a processing routine via 'PIPE -close' in pqact.conf, am I 
>guarenteed that my 'decoder' is only invoked once, no matter what the 
>size of the product?

No.  If, for some reason, the decoding process does not read all of the
product from the PIPE before it exits, the LDM process 'pqact' will think
that there was an error and will try to run the process one more time.

>I'd like to know that when my decoder gets end-of-file from the pipe 
>that the entire product is available.

It should be, yes.

>This would allow my decoder to invoke other processing on the completed file.

We do this kind of processing all of the time:

o PIPE the product to an executable shell script
o read all of the product in the shell script; I typically use 'cat' to
  redirect STDIN to a disk file
o finally, run one or more processes after successfully reading all of the
  product and writing it to disk.

>Thanks!

Tom Yoksas
**************************************************************************** <
Unidata User Support                                    UCAR Unidata Program <
(303)497-8644                                                  P.O. Box 3000 <
support@xxxxxxxxxxxxxxxx                                   Boulder, CO 80307 <
---------------------------------------------------------------------------- <
Unidata WWW Service                        http://www.unidata.ucar.edu/      <
**************************************************************************** <

 
 
  Contact Us     Site Map     Search     Terms and Conditions     Privacy Policy     Participation Policy
 
National Science Foundation (NSF) UCAR Community Programs   Unidata is a member of the UCAR Community Programs, is managed by the University Corporation for Atmospheric Research, and is sponsored by the National Science Foundation.
P.O. Box 3000     Boulder, CO 80307-3000 USA     Tel: 303-497-8643     Fax: 303-497-8690