I actually put a fix into the LDM to handle this.
NOAAPort data does not come with the standard WMO header sequence. The
NOAAPort ingester that comes with the LDM adds the WMO header and trailer
sequence to make it compatible with the LDM pqing program. The problem is
that the WMO product layout requires a CR-CR-LF-ETX on the end of every
product. Since binary products don't come with a trailing CR-CR-LF, the
NOAAPort code adds one. But it goes one step too far. It adds a CR-CR-LF
to all products.
This is OK it you're not mixing feeds. At the AWC, we get a feed of data
from the TOC. It is formatted with the WMO header the trailer. What I
found out when I started feeding both NOAAPort and TOC data into the LDM,
all the text products were duplicated since the NOAAPort code added a
CR-CR-LF to every product. If the product already ends in a CR-CR-LF, no
one needs to be added.
So I modified the NOAAPort ingest code to add a CR-CR-LF to the binary
products but not for the text products that already ended with a CR-CR-LF.
This solved the problem and NOAAPort and the TOC feed could be ingested
simultaneously without duplicates.
My suspicion is that the EMWIN feed is identical in format to what we were
seeing from the TOC. So the NOAAPort fix would solve this. I made a
recommendation to the LDM group at Unidata to provide this as an option to
the NOAAPort code.
Dan.
On Sun, Feb 24, 2013 at 8:15 PM, daryl herzmann <akrherz@xxxxxxxxxxx> wrote:
> Hi Jeff,
>
> Two things are going on here.
>
> 1) You can't mix noaaport and EMWIN with LDM, you will get near duplicates
> that are only missing 1 CRCRLF between the products.
>
> 2) The two products you see within IDS are different by one byte, this is
> the fun control character that SPC puts into some products to denote they
> have aviation significance (see character infront of SPC MCD) Its \x1e and
> is it documented in product directive?
>
> daryl
>
>
> On Sun, 24 Feb 2013, Jeff Lake wrote:
>
> Greetings..
>> Is anybody else seeing double messages out of the SPC ??
>>
>> Feb 25 00:48:30 ldm pqact[31868] INFO: 1836 20130225004830.210
>> IDS|DDPLUS 94267906 ACUS11 KWNS 250048 /pSWOMCD
>> Feb 25 00:48:39 ldm pqact[31868] INFO: 1835 20130225004839.163
>> EXP 999 EMWIN ACUS11 KWNS 250048 /pSWOMCD
>> Feb 25 00:48:50 ldm pqact[31868] INFO: 1835 20130225004850.231
>> IDS|DDPLUS 94268206 ACUS11 KWNS 250048 /pSWOMCD
>> Feb 25 00:50:12 ldm pqact[31868] INFO: 1840 20130225005012.570
>> EXP 999 EMWIN ACUS11 KWNS 250048 /pSWOMCD
>> Feb 25 00:50:25 ldm pqact[31868] INFO: 1836 20130225005025.144
>> EXP 999 EMWIN ACUS11 KWNS 250048 /pSWOMCD
>>
>> as you can see, I'm getting 2 via IDS|DDPLUS and at least 2 via EMWIN
>> Only reason these stick out, is I have them being mailed and the double
>> email messages caught my attention...
>>
>> what is the best way to find where the error sits ??
>>
>>
>>
> --
> /**
> * Daryl Herzmann
> * Assistant Scientist -- Iowa Environmental Mesonet
> * http://mesonet.agron.iastate.**edu <http://mesonet.agron.iastate.edu>
> */
>
>
> ______________________________**_________________
> ldm-users mailing list
> ldm-users@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe, visit:
> http://www.unidata.ucar.edu/**mailing_lists/<http://www.unidata.ucar.edu/mailing_lists/>
>
--
*Dan Vietor*
*Senior Research Meteorologist*
CIRA, Colorado State Univ
Aviation Weather Center
Kansas City, MO
816.584.7211
