[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: LDM update

Carl Sinclair wrote:
> 
> All,
> 
> There has been a problem recently with those who are receiving all of
> the data (41 radars currently) on one machine and uncompressing the data
> on that machine.  The problem has been solved thanks to some great
> investigative work by Jason Humphress.  The problem stems from the pqact
> binary.  The code only allows 32 child processes to be spawned.  With 41
> radars, that causes problems.  If you will look in the
> ~ldm/runtime/src/pqact/filel.c source, you will find this entry:
> 
> #define MAXENTRIES 32
> 
> Change that 32 to 64 or higher.  This should prevent these problems from
> re-occurring.  You'll have to then do this:
> 
> make clean
> make
> copy the new pqact file to ~ldm/bin
> 
> There is also new decompression code (nexradII_bz) that is now called
> 'nexradII'.  I've attached the source.  If your ldm home directory is
> not '/usr/local/ldm' you'll have to edit the Makefile accordingly.
> 
> If you have any questions, please let me know.
> 
> Carl
> --
> _______________________
> Carl Sinclair
> OU/CAPS Research Associate
> (405)325-5938
> 
>   ------------------------------------------------------------------------
>                           Name: nexradII.tar
>    nexradII.tar           Type: Unix Tape Archive (application/x-tar)
>                       Encoding: base64
>                Download Status: Not downloaded with message

Hi Carl,

First, has the decompression code changed?  I'd rather not upgrade my
nexradII_bz if I don't need to.

Second, it's my understanding that each invocation of nexradII_bz only
handles one product.  Please correct me if I'm wrong about this, or
maybe it's been changed.  Assuming it does handle only one product,
that's a lot of overhead to start up and clean up after every
invocation.  It would be very worthwhile to have it to wait for new
input instead of dying and being reinvoked for each new product.  (This
is how we write our decoders.)  Then, in pqact.conf, you could have
multiple lines using patterns that divide up the products hopefully
somewhat evenly, say, by site.  This way you could have some fixed
number of nexradII_bzs running all the time and you won't have to deal
with having too many processes open.   Ultimately you'll run into some
system limit on the number of open handles anyway - increasing the
number just defers the problem.  Ultimately you expect to have around
160 sites participating, right?  For even the current volume of
products, I suspect the savings in overhead would be significant.  
Here's a page from the LDM workshop with guidelines for writing
decoders: 
http://www.unidata.ucar.edu/packages/ldm/workshop2001/Decoders.html#s8. 
FYI.

Anne
-- 
***************************************************
Anne Wilson                     UCAR Unidata Program            
address@hidden                 P.O. Box 3000
                                  Boulder, CO  80307
----------------------------------------------------
Unidata WWW server       http://www.unidata.ucar.edu/
****************************************************