[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

20040223: LDM on weather3.admin.niu.edu



Gilbert,

> To: Unidata Support <address@hidden>
> cc: address@hidden
> From: Gilbert Sebenste <address@hidden>
> Subject: Re: 20040221: Help! LDM on weather3.admin.niu.edu going crazy... 
> Organization: UCAR/Unidata
>Keywords: 200402212235.i1LMZlrV009065

The above message contained the following:

> You see, I was testing out a NWWS feed from David Magee, just to see if I 
> could ingest it. Bingo! It worked. He sends it out under the "PPS" feed.
> 
> Problem.
> 
> Whenever a zone forecast comes in on my machine, it runs a script called 
> "prepend_file" to append the incoming forecast to a particular file, such 
> as "TX.feb23". Basically, it just does a "cat" and appends the output to 
> the end of the file. No biggie.
> 
> BUT...
> 
> When the NWWS feed was running, I was getting *two* of those hitting the 
> script at the same time...one from NOAAPORT, the other from NWWS.

Odd.  The product-queue module of the LDM won't insert a data-product
with the same signature (i.e., MD5 checksum) as one that's already in
the product-queue.

Could it be that David Magee's ingester isn't computing the MD5 checksum
the same way as the NOAAPORT ingester?

> Thus, the "prepend_file" script went into an endless loop.  I finally
> discovered that this morning, along with 6 billion copies of the state
> zone forecast in there. It just did a repeated dump of the forecast
> until disk space was full.

Assuming that you only have one pqact(1) process active, then the two
identical data-products should be sent serially to the standard input of
the "prepend_file" script (assuming their signatures differed) rather
than simultaneously.  This should result in two data-products being
appended to the file rather than the script going into an endless loop.

> Lovely.
> 
> Now, I am not sure if this IS the cause of the problem. But:
> 
> 1. It only happened with state zone forecasts, as that is the only data 
> that uses that script
> 
> 2. The other larger file sizes I saw with the hourly weather roundups 
> were due to multiple copies of those products being sent over NWWS, a 
> known problem

Assuming that these data-products all have the same signatures, then
only one should be inserted into your product-queue and, subsequently,
retrieved by the pqact(1) process.

> 3. The problem began on the day when I changed my NWWS feed to read PPS, 
> not GPSSRC. My guess is that PPS *is* the same as "IDS|DDPLUS",

PPS is a subset of IDS|DDPLUS because DDPLUS is a composite of DDS and PPS.

The NOAAPORT textual data-products have the feedtype IDS|DDPLUS because
we can't tell which specific feedtype should be associated with each
data-product -- unlike the old days when each feed each had its own
channel.  Hence, we give the data-product the union of all possible
feedtypes and postpone full identification to pqact(1) and the LDM
decoders.

> which might throw the product into an endless loop, especially if
> you're running a script on it. In other words, if you request PPS, you
> also get anything under IDS|DDPLUS.

For the reason given above, all textual NOAAPORT data-products will match 
the PPS feedtype (as well as any combination of the feedtypes DDS, PPS,
and IDS).

> So, that script had to deal with 4 copies from NOAAPort, 
> plus one from the true PPS feed, all at once. Mee-yow.

I don't understand.  Why four copies?  And duplicate products should be
eliminated by the product-queue module.

> Yep, it just worked. I see that my Texas zone file, just erased, has come 
> back with 7 KB instead of 2 GB like it was doing on the very first 
> product.
> 
> Wowsers! Now, here's the fun part: this morning, for the first time, it 
> also filled up the hard drive on weather.admin.niu.edu, BUT weather2 was 
> fine. Huh? I thought duplicate products weren't sent? 

The product-queue module only allows one data-product with a given
(MD5) signature.

> Also, I was NOT ingesting PPS on weather. But if my hunch above was
> right, yes I was.  Which doesn't explain why weather bombed and
> weather2 was fine. Oy.

Because the amount of data doesn't seem consistent with even a five-fold
increase due to duplicate data-products, I suspect that the problem lies
with the decoder script.

> Anyway, thanks for your help over the weekend. I certainly wasn't 
> expecting that to happen. The funny thing is, when I did an "ldmadmin 
> watch", I couldn't see "PPS". I guess it was being renamed "IDS|DDPLUS" as 
> it was coming in.

If the IDS|DDPLUS NOAAPORT data-products were inserted into your
product-queue before the PPS data-products from David Magee and the MD5
signatures are computed correctly, then you wouldn't see any "PPS"
data-products from an "ldmadmin watch" because the product-queue module
eliminates duplicates and the pqmon(1) utility that underlies the
"ldmadmin watch" command reads from the product-queue.

> I'll watch it to make sure nothing else happens. But I think it's OK now.
> 
> *******************************************************************************
> Gilbert Sebenste                                                     ********
> (My opinions only!)                                                  ******
> Staff Meteorologist, Northern Illinois University                      ****
> E-mail: address@hidden                                               ***
> web: http://weather.admin.niu.edu                                      **
> Work phone: 815-753-5492                                                *
> *******************************************************************************

Regards,
Steve Emmerson
LDM Developer


NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publicly available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us.