Re: [ldm-users] Duplicate Products

On Thu, 16 Dec 2010, Gerry Creager wrote:

I'll try to figure out what we did wrong here. We're snagging the same EMWIN feed Gilbert is, but could have fumble-fingered something, since I'm still short of calibrated fingers since shoulder surgery...

Unfortunately, one can not seamlessly plex together the EMWIN/WeatherWire with the NOAAPORT/SBN dataset using baseline software. I forget which system does it, but one of them arbitrarily tacks on an extra DOS line feed to every product, thus LDM sees the same product twice, since the line feeds are used to compute the product checksums.

For NWSChat, I get a direct socket feed from the Gateway/TOC that contains the plexing of, I think, 4 different internal NWS systems. This feed has duplicates all day long. So when Gilbert was hoping for a direct backup IDD feed from the TOC, this problem would have caused lots of pain for others... Anyway, I ended up writing a pre-conditioner to my decoders that striped extraneous line feeds and computed product checksums that way. This works for 99% of the products, but there are a few more products that have extra control characters inserted (namely from SPC) into the non SBN feeds.

My impression is that the private sector weather folks have been dealing with this issue for a very long time and it has been frustrating for them. I've been bugging the TOC/Gateway folks about it, but they are extremely short of resources and dealing with ancient IBM computer systems. This issue is on their TODO list though.

I wish LDM had an option to arbitrarily strip trailing line feeds before checksum computation. Before somebody says this would be a performance killer, it isn't. :)

daryl



  • 2010 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the ldm-users archives: