[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: wunderground.com NOAAPORT feed

On Mon, 8 Nov 1999, Jeff Masters wrote:

> 
> 
>   Hi Robb, we've got data flowing from our NOAAPORT dish into the ldm
> system at wunderground.com, and I was starting to think about how (if?) we
> will be feeding the IDD with this info. Questions:

Jeff, 

Glad to hear that the system is up and running. Good job.  At this time,
lets get the configuration before you start to feed downstream nodes.
Also, I need some time to see what the arrangements should look like.


> 
> 1) It seems to me that we will have to make the software mod you outline
> in the forwarded email below, unless it is already a part of ldm 5.0.8.
> Do you have a special mod I need to add to pqing to add pil-like
> identifiers?

Noop

> 
> 2) Also, how do I construct the filter to get rid of the SDUS97 products? 
> You mentioned use "^[^S]|^S[^D]|^SD..[^59]|^SD..9[^7]" in a previous post,
> does this mean do a : 
> 
> pqact -p "^[^S]|^S[^D]|^SD..[^59]|^SD..9[^7]"
             ^^^^

This means to accept all products that don't start with an S, the ^ is a
negation inside [].  Outside the [] it means start with.


> but it seems like we want the inverse of this. I don't understand your
> regular expression, it looks to me like it matches everything beginning
> with S. Why all the extra "^" characters? You don't have to answer this
> question, I am happy with a black box that works. 
> 
Actually there is a weakness in the current ldm architecture. The pqing
receives all the products including the encrypted radars, etc. So our
setup consists of an Np ingestor machine (noaaport) and another secondary
machine(thelma) that has a request like:

request WMO "(^[^S]|^S[^D]|^SD..[^59]|^SD..9[^7])"  
        noaaport.unidata.ucar.edu

This is how the filtering mechanism works.  If you plan on letting
outsiders accessing your ingest machine, then each one of the downstream
nodes needs to have an request line similar to above.


> 3) Alden hasn't been very helpful about providing us with documentation 
> or technical support for our system, so I need to find some additional
> contacts. Who would be the best person at SSEC to ask this question:

You can send questions to Chad Johnson <address@hidden>.  UPC
actually did the Np setup for Alden, they don't have anybody dedicated to
running it.

> 
> Is there a special mod needed to the SSEC code used to ingest NOAAPORT
> GINI format GOES imagery and convert it to McIDAS format, thanks to the

GINI data comes in on channel 1,2 and 4.  SSEC Np solution only receives 
channel 3 that is a superset of FOS as you know. At this point, you can't 
receiver the GINI data with the SSEC system.


> recent change to GINI2 format? Is there a McIDAS mailing list we should be
> on to receive alerts about code changes at SSEC?


You might want to talk to Chad about that.

> 

Robb...




> Thanks,
> Jeff
> ------------------------------------------------------------------------------
>  Dr. Jeff Masters (address@hidden)                           (  )  
>  Chief Meteorologist                              /\ Home of the       (    ) 
>  The Weather Underground, Inc.               /\  /  \  /\       /\    (      )
>  P.O. Box 3605                              /  \/    \/  \ /\  /  \    ------
>  Ann Arbor, MI 48106-3605            ______/              /  \/    \_   \\\\\
>  734-994-8824                                   Weather Underground      \`\`\
>                                             http://www.wunderground.com
> 
> 
> ---------- Forwarded message ----------
> Date: Tue, 22 Jun 1999 10:34:34 -0600
> From: Steve Chiswell <address@hidden>
> To: Tom Grzelak <address@hidden>
> Cc: address@hidden, address@hidden
> Subject: 19990622: Matching PIL id's in pqact
> 
> 
> 
> Tom,
> 
> Several weeks ago, we implemented a change to the ingest software
> which now adds the pil-like identifier to the DDPLUS|IDS NOAAport
> datastream product headers. Text products which contain a line that
> conforms to a PIL have the string appended to the LDM product
> identifier as "/pPIL_ID" Note that this change only affects the 
> product identifier as seen in the LDM, and does not modify the
> data product in any way. Two of the three top tier sites have
> already upgraded to the new ingest software, so almost all 
> products available on the IDD have the pil-like identifiers
> appended. With the additional product identifier appended to the 
> end of the WMO identifier, we have been able to implement
> this change basically without anyone noticing it. Of course,
> we will notify everyone when all top tier ingest sites are
> using the updated ingester so that all data on the IDD
> will have the tag regardless of where it was ingested.
> 
> In the example you show below, the product that begins with:
> 
>  742
>  SRUS51 KPHI 221222
>  RR1PHL
>  ...
> 
> will be identified in the LDM data stream as:
> 
>  SRUS51 KPHI 221222 /pRR1PHL
> 
> So, you can use a pqact.conf line such as:
> 
>  DDPLUS       ^SRUS51 KPHI .* /pRR1PHL
> 
> The "/p" identifier is analogous to the "/m" identifier which
> is appended to the model data products in the HDS feed,
> and can be used to differentiate between products issued under
> common WMO headers. Note that WMO headers frequently have some
> communitaction identifiers following the DDHHMM group, such as
> RTDxx, COR, AAA, etc. which is generally not useful for most IDD
> data processing- but in general, the IDD products will look like:
> 
> TTAAii CCCC DDHHMM [COM] [/pPIL_ID]
> 
> 
> Since not all products have PILs, the "/p" tag will only be placed on
> those products which contain a pil-like line. Some products, such
> as some METAR bulletins, contain second lines that look like AFOS PILS
> while other METAR bulletins do not. As a result, you will see some
> METAR bulletins with /pMETAR appended, while some do not. While
> the METAR string is not an actual PIL, it does conform to what
> a PIL looks like, so the /p tag represents a "pil-like" tag.
> Thus, for surface METAR decoding, you should still use the ^SA.*
> pattern and not the /p tag. 
> 
> To date, nobody has complained about any problems caused by the addition
> of the pil tags, so we plan to have the final top tier site upgrade
> as soon as they can.
> 
> We hope that the community finds this addition to be a valuable
> resource for use in the IDD. If you want to observe the pil tags
> in action, you can use the "ldmadmin watch" command or "notifyme"
> command to show the datastream headers.
> 
> As always, support questions can be addressed to address@hidden.
> 
> Good luck-
> 
> Steve Chiswell
> Unidata User Support
> 
> 
> On Tue, 22 Jun 1999, Tom Grzelak wrote:
> 
> > 
> > Hi!
> > 
> > I would like to set up my pqact.conf file to route certain reports to
> > specifc programs/people.  As long as the reports have unique headers, it is
> > straight-forward.  But I am not sure how to handle the following situation:
> > 
> > I have a single header which is used for SEVERAL different reports.  It is
> > the second and/or third line which makes the report unique.  How would I
> > set up a pqact.conf line item to do pattern matching on more than one line?
> > 
> > For example:
> > Here's a desired product with a header which is used for several other
> > undesired products.
> > 
> > 742 
> > SRUS51 KPHI 221222
> > RR1PHL
> > .BR PHL  0622 DH12/PPP
> > PHL .03,ABE .01,ACY .14,MIV .06,ILG .05
> > WRI .07,NXX .04,DOV .11,PNE .10,EWR T
> > TTN .08,SMQ .03,MPO .01,GED .08,VAY .26
> > 12N .02,RDG .02,BLM .05,WWD .05
> > CDW .05
> > .END
> > 
> > In this example, a pattern match on the header line AND the second line
> > would allow this DESIRED product to be sent to the right person/program.  
> > 
> > THE BIG QUESTION: How do you do a pattern match for the second line?
> > 
> > DDPLUS  ^SRUS51 KPHI
> >         PIPE
> >         -close -strip /usr/bin/mailx address@hidden
> > 
> > Thanks!
> > Tom
> > ===============================================================
> > Tom Grzelak                      address@hidden
> > Manager - Computing Services
> >  ----> Center for Environmental Prediction
> >  ----> Department of Environmental Sciences  
> > 
> > Rutgers University
> > 14 College Farm Road                        Voice: 732-932-4923     
> > New Brunswick, NJ 08901-8551        Fax: 732-932-8644
> >                 http://www.envsci.rutgers.edu
> > ===============================================================
> > 
> 
> 
> 
> 

===============================================================================
Robb Kambic                                Unidata Program Center
Software Engineer III                      Univ. Corp for Atmospheric Research
address@hidden             WWW: http://www.unidata.ucar.edu/
===============================================================================