[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[IDD #AIZ-345619]: LDM: pattern in ldmd.conf and pact.conf no longer works for CONDUIT??

Hi Christian,

re: extended regular expression in a 'notifyme' invocation
> > I just ran this from the 'ldm' account on my personal development machine
> > where I am running LDM v6.13.11, and the pattern matches lots of products.
> 
> I also tested the same and it also matched lots of products.

OK.  The next step that I would do is to have the 'notifyme' invocation
using the ERE pointing at your upstream feed site running in one terminal
and have an 'ldmamdin watch -f CONDUIT' running in a second terminal.  This way
you could see if your REQUEST is actually resulting in the receipt
of the products that you want.

Assuming that the above test shows that you are, in fact, receiving the
products you want, the next step is to:

- verify that you have an LDM configuration file EXEC line that will cause
  the CONDUIT products received to be acted on by the pattern-action file
  actions that don't appear to be working

- assuming that the received products are being received, and there is a
  'pqact' instance running that should process the products, I would
  increase the logging level of the 'pqact' instance

  This is done by sending a USR2 signal to the 'pqact' instance that is
  running the actions that should process the products.

  NB: the effects of running 'kill -s USR2 <pqact pid>' is incremental and
  cyclical: sending one signal increases the log level from notice to info
  and sending a second will cause the 'pqact' instance to issue debug
  log messages.  Sending a third signal will return the 'pqact' instance
  back to its original logging level.
 
re: the pattern-action file actions that I used in my test were:
> > ~ldm/etc/pqact.conf_conduit:
> >
> > CONDUIT prod/gfs\.(........)/(..)/gfs.*pgrb2\.1p00\.f(.*) !grib2
> >         FILE    /data/ldm/pub/models/conduit/gfs/\1\2_\3_gfs.grib
> > CONDUIT prod/gfs\.(........)/(..)/gfs.*pgrb2\.1p00\.a.* !grib2
> >         FILE    /data/ldm/pub/models/conduit/gfs/\1_anl_gfs.grib
> >
> 
> I also redid the same in my installation.

OK.

Can you send your LDM configuration file so I can verify that you are, in
fact, trying to process the received products via an 'EXEC "pqact..."'
entry?

re: my test produced the disk files specified
> 
> I did not have any file created... it is weird because I am migrating to
> new servers, and the LDM instance there has the same ldmd.conf and
> pqact.conf and the files are being created.

Hmm... So, there is something different about the setup on your older
LDM machine(s).  Quite frankly, the only thing that comes to mind is
there not being an LDM configuration file EXEC line that will tell a
'pqact' instance to process the products.

re:
> So I checked that the directory permissions were fine, and they are... but
> I still need those older servers to still work for the next few weeks
> because the migration is not over yet.

I understand.  I want to solve the mystery no matter what :-)

re: my testing worked
> 
> I re-typed those 2 pattern-action and it did not change the results, with
> no file being created.

What is the output of the following run while your LDM is running:

<as 'ldm'>
ps -eaf | grep pqact

re: LDM configuration or pattern-action file(s) edited with a Windows editor?

> I only edit directly using emacs on the CentOS linux system...

OK.

re: relevant lines from your LDM log file (feed starting up
line and any errors)?
> 
> 
> 20200716T134650.077475Z                 ldmd[18482] ldmd.c:main() NOTE  
> Starting Up (version: 6.13.10; built: Mar  8 2019 10:41:08)
> 20200716T134650.078550Z                 ldmd[18482] 
> ldmd.c:create_ldm_tcp_svc() NOTE  Using local address 0.0.0.0:388
> 20200716T134650.103817Z    idd.meteo.psu.edu[18498] 
> LdmConfFile.c:requester_exec() NOTE  Starting Up(6.13.10): 
> idd.meteo.psu.edu:388 20200716134432.103690 TS_E NDT {{NEXRAD3, 
> "/p(N0Z)(CXX)"}}
> 20200716T134650.104574Z                pqact[18487] pqact.c:main() NOTE  
> Starting Up
> ...
> 20200716T134650.115367Z    idd.ssec.wisc.edu[18504]  requester6.c:req6_new() 
> NOTE  LDM-6 desired product-class: 20200716134432.115283 TS_ENDT {{CONDUIT, 
> "status"},{NONE, "SIG=dd53be508c5457add3ab29bf89b86c1a"}}
> 20200716T134650.107628Z striker.atmos.albany.edu[18495]  
> requester6.c:req6_new() NOTE  LDM-6 desired product-class: 
> 20200716134432.107552 TS_ENDT {{LIGHTNING, ".*"},{NONE, 
> "SIG=e5e9dce70e99b19eacaeced2bd199906"}}
> 20200716T134650.116877Z iddb.unidata.ucar.edu[18503] 
> LdmConfFile.c:requester_exec() NOTE  Starting Up(6.13.10): 
> iddb.unidata.ucar.edu:388 20200716134432.116801 TS_ENDT {{CONDUIT, 
> "(gfs.*grb2.1p00.(f|a).*).*(PMSL|GUST|HGHT|P..M|RELH/2.m|RELH.*hPa|WX|OMEG.*hPa|TMPK|TM[XN]K03|[UV]REL|CLD|FXSH|CIN|CAPE|LIFT|PWTR|PRES/0.*NONE)"}}
> 20200716T134650.117428Z iddb.unidata.ucar.edu[18503] requester6.c:req6_new() 
> NOTE  LDM-6 desired product-class: 20200716134432.117255 TS_ENDT {{CONDUIT, 
> "(gfs.*grb2.1p00.(f|a).*).*(PMSL|GUST|HGHT|P..M|RELH/2.m|RELH.*hPa|WX|OMEG.*hPa|TMPK|TM[XN]K03|[UV]REL|CLD|FXSH|CIN|CAPE|LIFT|PWTR|PRES/0.*NONE)"},{NONE,"SIG=523332ba202d0b22560d67494aeffb4c"}}
> 20200716T134650.121885Z                pqact[18491] pqact.c:main() NOTE  
> Starting Up
> 20200716T134650.125146Z                pqact[18491] pqact.c:main() NOTE  
> Starting from insertion-time 2020-07-16 13:46:46.910676 UTC
> 20200716T134650.126128Z              rtstats[18488] rtstats.c:main() NOTE  
> Starting Up (18482)
 ...

OK, it is hard to tell from the listing sent if there is a 'pqact' instance that
is trying to run actions on the CONDUIT data.  The contents of your LDM 
configuration
file and/or the 'ps -eaf | grep pqact' output should tell us/me if this is, in 
fact
the case.

re:
> 
> >
> > Can you run the 'notifyme' that I included above and send the first "few"
> > (say 30 or so) lines of output (please don't send a large listing as it
> > is not needed).

> [ldm@vkepler logs]$ notifyme -vl- -f CONDUIT -h iddb.unidata.ucar.edu -p 
> "(gfs.*grb2.1p00.(f|a).*).*(PMSL|GUST|HGHT|P..M|RELH/2.m|RELH.*hPa|WX|OMEG.*hPa|TMPK|TM[XN]K03|[UV]REL|CLD|FXSH|CIN|CAPE|LIFT|PWTR|PRES/0.*NONE)"
>  -o 10800
> 20200716T134929.130382Z             notifyme[27581] notifyme.c:main() NOTE  
> Starting Up: iddb.unidata.ucar.edu:
> 20200716104929.129996 TS_ENDT {{CONDUIT, 
> "(gfs.*grb2.1p00.(f|a).*).*(PMSL|GUST|HGHT|P..M|RELH/2.m|RELH.*hPa|WX|OMEG.*hPa|TMPK|TM[XN]K03|[UV]REL|CLD|FXSH|CIN|CAPE|LIFT|PWTR|PRES/0.*NONE)"}}
> 20200716T134929.130497Z             notifyme[27581] ldm5_clnt.c:forn5() NOTE  
> LDM-5 desired product-class:
> 20200716104929.129996 TS_ENDT 
> {{CONDUIT,"(gfs.*grb2.1p00.(f|a).*).*(PMSL|GUST|HGHT|P..M|RELH/2.m|RELH.*hPa|WX|OMEG.*hPa|TMPK|TM[XN]K03|[UV]REL|CLD|FXSH|CIN|CAPE|LIFT|PWTR|PRES/0.*NONE)"}}
> 20200716T134929.132425Z             notifyme[27581] error.c:err_log() INFO  
> Resolving iddb.unidata.ucar.edu to 128.117.130.3 took 0.001793 seconds
> 20200716T134929.263367Z             notifyme[27581] ldm5_clnt.c:forn_signon() 
> NOTE  NOTIFYME(iddb.unidata.ucar.edu): OK
> 20200716T134933.838864Z             notifyme[27581] 
> notifyme.c:notifymeprog_5() INFO       46159 20200716122819.498183 CONDUIT 
> 017  
> data/nccf/com/gfs/prod/gdas.20200716/06/gdas.t06z.pgrb2.1p00.anl!grib2/ncep/SSIGFS/#000/202007160600F000/UREL/3
>  hPa PRES! 000017
> 20200716T134933.902886Z             notifyme[27581] 
> notifyme.c:notifymeprog_5() INFO       60923 20200716122819.543308 CONDUIT 
> 037  data/nccf/com/gfs/prod/gdas.20200716/06/gdas.t06z.pgrb2.1p00.anl 
> !grib2/ncep/SSIGFS/#000/202007160600F000/HGHT/20 hPa PRES! 000037
> 20200716T134934.068210Z             notifyme[27581] 
> notifyme.c:notifymeprog_5() INFO       64054 20200716122819.690882 CONDUIT 
> 097  data/nccf/com/gfs/prod/gdas.20200716/06/gdas.t06z.pgrb2.1p00.anl 
> !grib2/ncep/SSIGFS/#000/202007160600F000/HGHT/200 hPa PRES! 000097
> 20200716T134934.131771Z             notifyme[27581] 
> notifyme.c:notifymeprog_5() INFO       52132 20200716122819.759393 CONDUIT 
> 117  data/nccf/com/gfs/prod/gdas.20200716/06/gdas.t06z.pgrb2.1p00.anl 
> !grib2/ncep/SSIGFS/#000/202007160600F000/VREL/250 hPa PRES! 000117
> 20200716T134934.297414Z             notifyme[27581] 
> notifyme.c:notifymeprog_5() INFO       63707 20200716122819.477299 CONDUIT 
> 005  data/nccf/com/gfs/prod/gdas.20200716/06/gdas.t06z.pgrb2.1p00.anl 
> !grib2/ncep/SSIGFS/#000/202007160600F000/HGHT/1 hPa PRES! 000005
> 20200716T134934.360755Z             notifyme[27581] 
> notifyme.c:notifymeprog_5() INFO       78081 20200716122819.939182 CONDUIT 
> 237  data/nccf/com/gfs/prod/gdas.20200716/06/gdas.t06z.pgrb2.1p00.anl 
> !grib2/ncep/SSIGFS/#000/202007160600F000/UREL/700 hPa PRES! 000237
> ...

This listing shows that the upstream is getting the products you want.

re:
> > re:
> > > But nothing gets through. I checked with ldmadmin watch -f CONDUIT and
> > > I get no headers. So it points out a problem in ldmd.conf, but I cannot
> > > figure it out, as I tried many variations in the regex. I also checked
> > > the regex and it is validated.

If this is still true, try switching your CONDUIT feed REQUEST to a
different upstream relay.  We have on rare occasion seen upstream processes
stop sending data to the downstream to which it has an active connection.
Changing the REQUEST to a different upstream would definitely result
in a different machine feeding you.  NB: simply restarting your LDM will
not result in your feed REQUEST(s) going to a different real-server backend
machine of the cluster/machine that you have been/are REQUESTing from.

Last question for this email:

Is the machine you are having problems with autan.sca.uqam.ca?
If yes, I can say that the real-time stats that it is reporting back to us
indicates that CONDUIT data is being received now:

https://rtstats.unidata.ucar.edu/cgi-bin/rtstats/iddstats_vol_nc?CONDUIT+autan.sca.uqam.ca

re:
> > Lets figure out one problem before attacking the other...
> 
> Yes, thanks!!

No worries.

Cheers,

Tom
--
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: AIZ-345619
Department: Support IDD
Priority: Normal
Status: Open
===================
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata 
inquiry tracking system and then made publicly available through the web.  If 
you do not want to have your interactions made available in this way, you must 
let us know in each email you send to us.