[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[LDM #WAK-808745]: CONDUIT data (more beers ...)

Hi Terri,

Sorry for the slowish reply;  I was out yesterday (having oral surgery, which
was no fun).

re: 
> I'm just now starting the process we talked about last summer where we
> wanted to move any feeds we got from RAL to Unidata. The one I'm starting
> with is some CONDUIT data since it appears RAL is not distributing it
> anymore, at least not to us, since it's not coming in to our servers.

Hmm... that's odd.  I was under the impression that RAL was/is REQUESTing
all of CONDUIT from us.

re:
> So in my ldmd.conf file, this is what existed (before me, I'm assuming
> it worked at some point), and what I changed it to:
> 
> # request       CONDUIT "/ndfd/Y[CEF]U" 128.117.200.218
> # request       CONDUIT "/ndfd/Y[CEF]U" 128.117.200.218
> request CONDUIT "/ndfd/Y[CEF]U"         idd.unidata.ucar.edu
> 
> The pqact.conf file:
> 
> CONDUIT data/ndfd/Y[CEF]U..._KWBN.*/(........)(....)F([0-9][0-9][0-9])/
> FILE    data/raw_data/model/ndfd/\1/ndfd.\1.i\2.f\3.grb2
> 
> I see from http://www.unidata.ucar.edu/data/conduit_ndfd_20100629.html
> that this data should be available, but nothing is coming in. Notifyme
> (aren't you proud of me?)  shows:
> 
> shavano:ldm>notifyme -vl- -f CONDUIT -h idd.unidata.ucar.edu
> Nov 11 17:53:42 notifyme[29588] NOTE: Starting Up: idd.unidata.ucar.edu: 
> 20141111175342.098 TS_ENDT {{CONDUIT,  ".*"}}
> Nov 11 17:53:42 notifyme[29588] NOTE: LDM-5 desired product-class: 
> 20141111175342.098 TS_ENDT {{CONDUIT,  ".*"}}
> Nov 11 17:53:42 notifyme[29588] INFO: Resolving idd.unidata.ucar.edu to 
> 128.117.140.3 took 0.003235 seconds
> Nov 11 17:53:42 notifyme[29588] NOTE: NOTIFYME(idd.unidata.ucar.edu): OK ...
> Nov 11 17:55:16 notifyme[29588] INFO:    25678 20141111175443.679 CONDUIT 249 
>  data/nccf/com/rap/prod/rap.20141111/rap.t17z.awp252pgrbf07.grib2 !grib2/nc
> ...
> 
> There are lots of lines like the one between the "..."'s.

Yes, I am very proud of you for immediately thinking of using 'notifyme' :-)
Top marks would have been yours if you had included the '-p' pattern and '-o'
offset flags when looking for NDFD data.  Here is an example:

% notifyme -vl- -f CONDUIT -h idd.unidata.ucar.edu -p ndfd -o 1000000
Nov 13 22:00:09 notifyme[28283] NOTE: Starting Up: idd.unidata.ucar.edu: 
20141102081329.378 TS_ENDT {{CONDUIT,  "ndfd"}}
Nov 13 22:00:09 notifyme[28283] NOTE: LDM-5 desired product-class: 
20141102081329.378 TS_ENDT {{CONDUIT,  "ndfd"}}
Nov 13 22:00:09 notifyme[28283] INFO: Resolving idd.unidata.ucar.edu to 
128.117.140.3 took 0.00042 seconds
Nov 13 22:00:09 notifyme[28283] NOTE: NOTIFYME(idd.unidata.ucar.edu): OK
Nov 13 22:00:10 notifyme[28283] INFO:   310239 20141113195309.793 CONDUIT 003  
data2/ndfd/YSUZ98_KWBN_131951.grib2 
!grib2/nwstg/NWS_0/#000/201411132000F022/SNOW/0 - NONE! 000003
Nov 13 22:00:10 notifyme[28283] INFO:   284898 20141113195309.830 CONDUIT 006  
data2/ndfd/YSUZ98_KWBN_131951.grib2 
!grib2/nwstg/NWS_0/#000/201411132000F040/SNOW/0 - NONE! 000006
Nov 13 22:00:10 notifyme[28283] INFO:   667494 20141113195307.880 CONDUIT 000  
data2/ndfd/YHUZ98_KWBN_131951.grib2 
!grib2/nwstg/NWS_0/#000/201411132000F016/TMNK/0 - NONE! 000000
Nov 13 22:00:10 notifyme[28283] INFO:   317037 20141113195309.763 CONDUIT 000  
data2/ndfd/YSUZ98_KWBN_131951.grib2 
!grib2/nwstg/NWS_0/#000/201411132000F004/SNOW/0 - NONE! 000000
Nov 13 22:00:10 notifyme[28283] INFO:     1179 20141113195309.866 CONDUIT 009  
.status.data2/ndfd/YSUZ98_KWBN_131951.grib2 000009
Nov 13 22:00:10 notifyme[28283] INFO:   609800 20141113195322.959 CONDUIT 000  
data2/ndfd/YGUZ98_KWBN_131951.grib2 
!grib2/nwstg/NWS_0/#000/201411132000F004/TMXK/0 - NONE! 000000
 ...

re:
> So ... any ideas why I'm not getting any data stored to disk?

Let's take a look at the extended regular expression in your pattern-action
file action:

  'data/ndfd/Y[CEF]U..._KWBN.*/(........)(....)F([0-9][0-9][0-9])/'

I believe that what is keeping your pattern from working is the use of 'data'
instead of 'data2'.  Let's test to see if this is true:

% notifyme -vl- -f CONDUIT -h idd.unidata.ucar.edu -o 1000000 -p 
"data2/ndfd/Y[CEF]U..._KWBN.*/(........)(....)F([0-9][0-9][0-9])"
Nov 13 22:04:47 notifyme[28962] NOTE: Starting Up: idd.unidata.ucar.edu: 
20141102081807.730 TS_ENDT {{CONDUIT,  
"data2/ndfd/Y[CEF]U..._KWBN.*/............F([0-9][0-9][0-9])"}}
Nov 13 22:04:47 notifyme[28962] NOTE: LDM-5 desired product-class: 
20141102081807.730 TS_ENDT {{CONDUIT,  
"data2/ndfd/Y[CEF]U..._KWBN.*/............F([0-9][0-9][0-9])"}}
Nov 13 22:04:47 notifyme[28962] INFO: Resolving idd.unidata.ucar.edu to 
128.117.140.3 took 0.000456 seconds
Nov 13 22:04:47 notifyme[28962] NOTE: NOTIFYME(idd.unidata.ucar.edu): OK
Nov 13 22:04:48 notifyme[28962] INFO:   810882 20141113195539.724 CONDUIT 004  
data2/ndfd/YCUZ98_KWBN_131952.grib2 
!grib2/nwstg/NWS_0/#000/201411132000F005/SPED/0 - NONE! 000004
Nov 13 22:04:48 notifyme[28962] INFO:   813935 20141113195539.746 CONDUIT 005  
data2/ndfd/YCUZ98_KWBN_131952.grib2 
!grib2/nwstg/NWS_0/#000/201411132000F006/SPED/0 - NONE! 000005
Nov 13 22:04:49 notifyme[28962] INFO:   800809 20141113195540.091 CONDUIT 014  
data2/ndfd/YCUZ98_KWBN_131952.grib2 
!grib2/nwstg/NWS_0/#000/201411132000F015/SPED/0 - NONE! 000014

 ...

Comments:

- I seem to recall that the change from 'data/' to 'data2/' was announced
  to the address@hidden email list when the change was made at
  NCEP

- from http://www.unidata.ucar.edu/data/conduit_ndfd_20100629.html is out of 
date

  I just updated it to reflect the 'data/' to 'data2/' change.

This was a good example of how 'notifyme' can be used to not only find out
what current Product IDs really look like, but also how to test an extended
regular expression to see if it matches the subset of a feed desired.
'notifyme' is your friend :-)

Request:

In the future, please send these sorts of inquiries directly to
address@hidden, address@hidden or
address@hidden.  This way, they will be forwarded
into our inquiry tracking system where multiple Unidata staffers
can see the question and jump in with an answer.

Cheers,

Tom
--
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: WAK-808745
Department: Support CONDUIT
Priority: Normal
Status: Closed