Gerry, Good question. But not as simple as "/E" to determine- which has its own flaws if you received the parts out of order from multiple upstream hosts. First off, every file inserted in CONDUIT has a .status file associated with it, which is inserted as the very last product of the file, however, due to the possibility of using multiple feed requests, you can't guarantee that it will be the last fLDM product received. You have the following options: 1) On receipt of the 126 hour status product, you can use the sequence number at the end of the product (eg the 6 digit number following the !.*! pattern, or (......)$ ) to tell you how many pieces there should be (since the counter starts at 0, the number of the status product will also be the number of grib messages to be received). You can then wait a short amount of time until either- a) all pieces have been received b) you haven't gotten any more pieces for a given timeout The check in "b" would prevent waiting for ever for something that never arrives. I'd suggest 5 minutes as a max, but probably as little as 30 seconds would suffice. 2) Use the inserted size included within the .status file to check for the number of bytes files for the aggregate 126 hour file, and wait until either: a) the file size is >= to the inserted size b) no new data has been added for a given timeout In application, I have to file the individual pieces to separate files for performing the reception stats as in: http://motherlode.ucar.edu/cgi-bin/ldm/conduit_reception_new.csh?/data/nccf/com/gfs/prod/gfs.2007082912 however, most sites would probably find it better off to file by forecast hour as in (2). A last option, if you were keying on decoded output rather than FILE's grib parts, would be to check the decoded file for completion/inactivity (eg just "b" in number 2 without "a". Attached is a example csh script that takes 3 arguments (directory, filename, and size_in_bytes extracted from the .status file inserted string). The script will sleep a specified interval and recheck. It will proceed if either: 1) The specified file is greater than or equal to the size in bytes 2) It has looped 10 times (eg 5 minutes at 30 seconds) without the file being touched (eg the file is no longer actively being updated) 3) It has looped 10 times without the file existing Just in case you find it useful..... Steve Chiswell Unidata User SUpport On Wed, 2007-08-29 at 13:47 -0500, Gerry Creager wrote: > Chiz, > > Donna and I are scheming about using a pqact trigger to start > preprocessing when GFS 0.5 deg hour 126 comes in. Is there a flag when > the last of the data is received we can trigger on? Is it something > similar to the end of a volume scan in Level II data > > Thanks, gerry -- Steve Chiswell <address@hidden> Unidata
Attachment:
donetest.csh
Description: C-Shell script