[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

20010530: LDM ingest of NLDN data; ldm-mcidas decoding of same

>From: alan anderson <address@hidden>
>Organization: UCAR/Unidata
>Keywords: 200105302052.f4UKq2p15034 ldm-mcidas LDM pqact.conf


>Hope summer is treating you ok.

It has been pretty hectic for me so far (too many things going on at
the moment).

>I am trying to get the NLDN lightning
>data into our ldm ingester and seem to be stuck.   I have obtained
>permission from Albany, and they have added us with an 'allow'
>statement on their server.  When our ldm starts, I see a request to
>striker.atmos.albany.edu  and then a     feedme OK  so I think that
>part is working ok.

OK.  This sounds promising.  A quick check of your LDM ingest status
using 'notifyme' shows that you are receiving the data from SUNYA:

/local/ldm% notifyme -vxl- -f NLDN -o 3600 -h waldo.stcloudstate.edu
Jun 01 00:03:36 notifyme[21880]: Starting Up: waldo.stcloudstate.edu: 
20010531230336.975 TS_ENDT {{NLDN,  ".*"}}
        NOTIFYME(waldo.stcloudstate.edu) returns OK
Jun 01 00:03:38 notifyme[21880]: NOTIFYME(waldo.stcloudstate.edu): OK
Jun 01 00:03:38 notifyme[21880]: 36b03f8fa3cd0004a735249bd245a29b    18228 
20010531230636.292    NLDN 000  2001151230005
Jun 01 00:03:39 notifyme[21880]: 61101470ff0fda9de7ec141981ffc230    17388 
20010531231236.837    NLDN 000  2001151230611
Jun 01 00:03:39 notifyme[21880]: 517b0b1a7f2ca77aaedc1f9d1c0c0fd3    16632 
20010531231835.657    NLDN 000  2001151231217
Jun 01 00:03:40 notifyme[21880]: 076aa5b7085a23e7b977eed59c85f742    16828 
20010531232437.165    NLDN 000  2001151231823
Jun 01 00:03:40 notifyme[21880]: 93d04775d5df866f536858eef0ddce6d    13692 
20010531233036.820    NLDN 000  2001151232429
Jun 01 00:03:41 notifyme[21880]: a2a5e0e65e543b6360efec398da4aaf3    14308 
20010531233639.239    NLDN 000  2001151233035
Jun 01 00:03:41 notifyme[21880]: 8b16ba07c7f7a2de30993c45a75f7b88    13412 
20010531234236.203    NLDN 000  2001151233641
Jun 01 00:03:41 notifyme[21880]: e4b17c492bbc52d9506eda83e1eb0e5d    11872 
20010531234837.751    NLDN 000  2001151234247
Jun 01 00:03:41 notifyme[21880]: 693fd0aef2dba30f84d02499c8971c36    12460 
20010531235437.415    NLDN 000  2001151234853
^CJun 01 00:03:42 notifyme[21880]: Interrupt
Jun 01 00:03:42 notifyme[21880]: exiting

>My pqact entry for the NLDN data is just below; basically just a copy
>from what has been there since last year some time.  I did change the
>first 2 entries as I assume I need to list something for
>       YYJJJHHMbMe  as listed in the pqact.conf discussion on your web
>       pg.
>My leading items in the pattern are  0 and 1-9  to match  year 01  -
>09 (looking ahead) I note that the example on your web site starts
>with   [12][0-9][0-9][0-9] | which I don't understand for a  2 digit
>listing of year   YY.

This pattern will match 1???, 2???, and ??: examples are 1999, 2000, 2001,
99, etc.

>Anyway, I am not getting any data;  no MDXX0071, which is the file for
>the data.  Also, doing a Route List in MCIDAS  shows   'none' for NLDN
>data received.

The existence of a lightning MD file and an entry in ROUTE.SYS showing
such a file are two different ways of looking at the same thing.
You are getting data, but it isn't being decoded.  More on that below.

>One more question about my LDM.  Looking today, I see lots of error
>lines regarding a broken pipe in my ldmd.log.  Samples listed at
>bottom.  I did not see these yesterday when I was editing pqact.conf,
>but maybe I created another error.   I did run pqactcheck and it said

OK.  The errors seem to indicate a problem in FSL wind profiler decoding.
Did you stop and restart the LDM after doing the pqact.conf modifications?
I am thinking that the answer is yes, and further that the environment
from which you stopped and started the LDM had a PATH that did not include
the ~ldm/decoders directory where proftomd and nldn2md live.

>The machine is waldo, and Tom knows how to get in if needed.

Got it.

>NLDN    ^(0[1-9])([0-3][0-9][0-9])([0-2][0-9])([0-5][0-9])([0-5][0-9])
>        PIPE    -close
>        nldn2md -d /var/data/mcidas 70 NLDN DIALPROD=LD \1\2 \3\400 DEV=CCN

This should be:

        PIPE    -close
        nldn2md -d /var/data/mcidas 70 NLDN DIALPROD=LD \1\2 \3\400 DEV=CCN

>May 30 20:32:43 waldo pqact[1126]: pipe_prodput: trying again
>May 30 20:32:43 waldo pqact[1126]: pbuf_flush (6) write: Broken pipe
>May 30 20:32:43 waldo pqact[1126]: pipe_dbufput:
>dasU6WPR691 write error
>May 30 20:32:43 waldo pqact[1126]: child 6205 exited with status 127
>May 30 20:32:43 waldo pqact[1126]: child 6203 exited with status 127   

This is failing for proftomd for some reason.  I will login and poke around.


OK, based on my hunch the the environment in which you stopped and restarted
your LDM did not have a PATH that contained the ~ldm/decoders directory, 
I did the following:

o login to waldo as ldm
o edit pqact.conf to change the NLDN entry
o stopped the LDM
o restarted the LDM

I now see that both FSL wind profiler and NLDN lightning data are being

Jun 01 00:32:42 waldo proftomd[14176]: Starting up
Jun 01 00:32:43 waldo proftomd[14176]: Making /var/data/mcidas/MDXX0092; may 
take some time...
Jun 01 00:32:47 waldo proftomd[14176]: Decoding 2001152.0018 data into 
Jun 01 00:32:47 waldo proftomd[14176]: Exiting
Jun 01 00:33:58 waldo nldn2md[14178]: NLDN2MD -- BEGIN
Jun 01 00:33:58 waldo nldn2md[14178]: PRODUCT CODE=LD          2001152     
Jun 01 00:33:58 waldo nldn2md[14178]: Created MD file: 72
Jun 01 00:33:58 waldo nldn2md[14178]: NLDN2MD - Flash events in file: 385
Jun 01 00:33:59 waldo nldn2md[14178]: NLDN2MD -- DONE

The corresponding entries in the McIDAS routing table are in agreement with

<as 'mcidas'>
cd workdata
route.k LIST
  LD NLDN Lightning Flashes      71-71   MDXX0072     01152   33     none     3
  U6 FSL2 6-minute Wind profil  default  MDXX0092     01152   32     none     3
route.k: Done

Looks like things are working.  Please let me know if you find anything
else that looks amiss.


NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publicly available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us.