[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

20000731: McIDAS pqact for png products

>From: James Murakami <address@hidden>
>Organization: UCLA
>Keywords: 200007312200.e6VM0CT17187 ldm-mcidas pnga2area pqact.conf

James,

>I've set up the ldm pqact.conf as follows (showing only 
>mcidas portion)--
>
># AREA files:
>#
>MCIDAS ^(LWTOA3 .*)
>#      PIPE    -close  /unidata/ldm/bin/lwtoa3
>#      -d /unidata/mcidas/data
>#
># AREA files (will replace LWTOA3 datastream)
>MCIDAS ^pnga2area Q. U. .... (.*) (.*) (.*) (........) (....)
>       PIPE    -close  /unidata/ldm/bin/pnga2area
>       -d /unidata/mcidas/data

The question is whether or not you want to use McIDAS file routing
concepts (use of ROUTE.SYS to set the name space for your AREA files),
or if you want to specify the names for the output images (AREA file
format, but not AREA file name.  I am assuming from your comment at the
end of this message (things were working that you were used to naming
the decoded imagery by using McIDAS routing concepts.  My example(s)
will assume that you want to continue to use the McIDAS routing
concepts.

Three things need to be changed in your pqact.conf file:

o specify explicitly 'Q1' for the broadcast time delimiter; if you don't
  do this, the action will also match CIMSS product headers in addition
  to the old LWTOA3 images.  (You will see a little further down that
  this is OK if you want to process both 'LWTOA3' and CIMSS images
  using routing techniques).

o add regular expression specifications for the product code and default
  AREA number

o add the '-r' flag specification to the pnga2area action

With these changes your pqact.conf entry would then look like:

# AREA files (will replace LWTOA3 datastream)
MCIDAS  ^pnga2area Q1 (..) (*.) (.*) (.*) (.*) (........) (....)
        PIPE    -close  /unidata/ldm/bin/pnga2area
        -d /unidata/mcidas/data -r \1,\2

(mind the tabs!)

The '-r' flag and regular expression specifications are also missing
from your CIMSS entries.

#
# All CIMSS Products
#
MCIDAS  ^pnga2area Q0 (..) (.*) (.*) (.*) (.*) (........) (....)
        PIPE    -close  /unidata/ldm/bin/pnga2area
        -d /unidata/mcidas/data -r \1,\2

You will see that the only difference in this pattern and the one above
for 'LWTOA3' images is the 'Qx' specification.  If you really are using
McIDAS routing concepts for the ouput file naming, you would need to have
only one entry in your pqact.conf file:

#
# All Unidata-Wisconsin PNG compressed imagery
#
MCIDAS  ^pnga2area Q[01] (..) (.*) (.*) (.*) (.*) (........) (....)
        PIPE    -close  /unidata/ldm/bin/pnga2area
        -d /unidata/mcidas/data -r \1,\2

The difference is in the broadcast time specification regular
expression: 'Q[01]'.  Specifying this explicitly for the images coming
in now will prevent new images from begin decoded in the future, but
that may be a good thing if you need to adjust your McIDAS routing table
entries to account for the higher frequence of images of a type

For example, if you want to keep 10 hours of GOES-West IR images and we
double the temporal frequence, you would need to keep 20 iamges instead
of the 10 that you may be keeping now.  This would require a change
to your McIDAS routing table entries.  It is probably best to leave
this entry this way until you have a chance to think about this
eventuality.)

If you adopt the last example entry  above, then you should delete all
of the rest that you now have in your pqact.conf file:

>#
># CIMSS Cloud Top Pressure - McIDAS product code CA
>#
>MCIDAS ^pnga2area Q. CA .... (.*) (.*) (.*) (........) (....)
>       PIPE    -close  /unidata/ldm/bin/pnga2area
>       -d /unidata/mcidas/data
># CIMSS Total Column Precipitable Water - McIDAS product code CB
>#
>MCIDAS ^pnga2area Q. CB .... (.*) (.*) (.*) (........) (....)
>       PIPE    -close  /unidata/ldm/bin/pnga2area
>       -d /unidata/mcidas/data
># CIMSS Sea Surface Temperature - McIDAS product code CC
>#
>MCIDAS ^pnga2area Q. CC .... (.*) (.*) (.*) (........) (....)
>       PIPE    -close  /unidata/ldm/bin/pnga2area
>       -d /unidata/mcidas/data
># CIMSS Lifted Index - McIDAS product code CD
>#
>MCIDAS ^pnga2area Q. CD .... (.*) (.*) (.*) (........) (....)
>       PIPE    -close  /unidata/ldm/bin/pnga2area
>       -d /unidata/mcidas/data
># CIMSS CAPE - McIDAS product code CE
>#
>MCIDAS ^pnga2area Q. CE .... (.*) (.*) (.*) (........) (....)
>       PIPE    -close  /unidata/ldm/bin/pnga2area
>       -d /unidata/mcidas/data
># CIMSS Ozone - McIDAS product code CF
>#
>MCIDAS ^pnga2area Q. CF .... (.*) (.*) (.*) (........) (....)
>       PIPE    -close  /unidata/ldm/bin/pnga2area
>       -d /unidata/mcidas/data


>I based the paths on the top one (for the soon to be obsolete
>lwtoa3). Now we were getting satellite imagery fine before the
>attempted change. Now we get no products from the mcidas datastream.

OK, the pnga2area decode entries were not quite correct.

>No AREA files get updated (new CIMSS area files aren't created either).

OK.

>I saw the followinglog statements--
>
>Jul 31 21:24:44 typhoon pqact[12589]: ReReading configuration file /home/ldm/e
> tc/pqact.conf
>Jul 31 21:26:31 typhoon pqexpire[12587]: > Recycled  91767.685 kb/hr ( 12111.6
> 91 prods per hour)
>Jul 31 21:28:27 typhoon pnga2area[19303]: Starting Up
>Jul 31 21:28:27 typhoon pqact[12589]: pbuf_flush (6) write: Broken pipe
>Jul 31 21:28:27 typhoon pqact[12589]: pipe_dbufput: -close/unidata/ldm/bin/png
> a2area-d/unidata/mcidas/data write error
>Jul 31 21:28:27 typhoon pqact[12589]: pipe_prodput: trying again
>Jul 31 21:28:27 typhoon pnga2area[19304]: Starting Up
>Jul 31 21:28:28 typhoon pqact[12589]: pbuf_flush (6) write: Broken pipe
>Jul 31 21:28:28 typhoon pqact[12589]: pipe_dbufput: -close/unidata/ldm/bin/png
> a2area-d/unidata/mcidas/data write error
>Jul 31 21:28:28 typhoon pqact[12589]: child 19304 terminated by signal 11
>Jul 31 21:28:28 typhoon pqact[12589]: child 19303 terminated by signal 11
>Jul 31 21:30:06 typhoon pnga2area[19320]: Starting Up
>Jul 31 21:30:06 typhoon pqact[12589]: pbuf_flush (6) write: Broken pipe
>Jul 31 21:30:06 typhoon pqact[12589]: pipe_dbufput: -close/unidata/ldm/bin/png
> a2area-d/unidata/mcidas/data write error

This is to be expected since the decoder is exiting before the LDM
could send it the entire product.  As soon as you make your pqact.conf
change and then send a HUP to pqact, things should resume working.

>Since I'm not a programmer by profession, any help would be appreciated.

No problem.

>Thanks in advance.

You are wlecome.  Please let me know if the above isn't clear enough.

>James Murakami
>--------------------------------------
>James Murakami
>Staff Meteorologist/Student Affairs
>Department of Atmospheric Sciences
>University of California, Los Angeles
>405 Hilgard Ave.
>Los Angeles, CA  90095-1565
>   e-mail:  address@hidden
>telephone:  310-825-2418
>      Fax:  310-206-5219
>---------------------------------------

Tom Yoksas