Daryl,
Afraid the company lawyer would fire me and kill me, not necessarily in
that order. However, here are a couple that I'm sure are very much "public"
knowledge...
deleting NIL= lines
changing OOOOOKT (capital oh's) to 00000KT (zeros)
changing 0/0KT and 000/00KT to 00000KT (zeros)
changing G//KT to KT
replacing ttXdd or tt-dd temp/dewpt groups with tt/dd
replacing "<" with "/" (mostly Canadian SA format stations)
I've noticed the "future obs" problem as well. That's one that's awaiting
my attention...
Steve
On Wed, Aug 9, 2017 at 12:07 PM, daryl herzmann <akrherz@xxxxxxxxxxx> wrote:
> Steve,
>
> In the vane of data-curation, would you mind sharing that list of 25
> things? :)
>
> daryl
>
> On Wed, 9 Aug 2017, Stephen Adams wrote:
>
> Greg,
>>
>> Ah... the joys of processing the metar "standard". I made a C program that
>> makes corrections to certain obvious errors in the metars. Over the years
>> I've been adding more and more "events". It's up to 25 now, with 2 more
>> waiting for my attention... The international sites are the most error
>> prone, but good ole USA sites are not immune. Quality control at data
>> entry
>> sounds like a wonderful and logical idea. Therefore, according to Murphy's
>> Law, it'll never happen!
>>
>> Steve
>>
>>
>> On Wed, Aug 9, 2017 at 10:21 AM, Greg Thompson <gthompsn@xxxxxxxx> wrote:
>>
>>
>>> Daryl,
>>>
>>> The number of *exceptions* for METARs, TAFs, and many other NWS products
>>> is insane. My software will ingest the incorrect MR0C just fine, and my
>>> program that seeks out unknown 4-letter IDs found in the METAR DB I
>>> maintain but are not known lat/lon will reveals many sites that were
>>> typos.
>>>
>>> When will someone ever employ software checking upon data entry to
>>> prevent
>>> things like this? Never at the rate we're going. The worse condition in
>>> my opinion is when the time stamp is completely wrong - such as
>>> POST-dated
>>> METAR data. It happens all the time and they merrily end up in final
>>> files
>>> as supposedly good info.
>>>
>>> --Greg
>>>
>>>
>>> On Tue, Aug 8, 2017 at 10:14 PM, daryl herzmann <akrherz@xxxxxxxxxxx>
>>> wrote:
>>>
>>>
>>>> Oh the fun of processing METARs. Behold the METAR site MROC Juan
>>>> Santamaria, Costa Rica. Here's the 0z ob
>>>>
>>>> SACS31 MROC 090000
>>>> METAR
>>>> MROC 090000Z 25005KT 5000 BR FEW005 SCT015 23/23 A2993 NOSIG=
>>>>
>>>> and here's the 1z ob:
>>>>
>>>> SACS31 MROC 090100
>>>> METAR
>>>> MR0C 090100Z 24005KT 3000 BR FEW002 BKN010 23/22 A3000 NOSIG=
>>>>
>>>> Notice what's wrong? :) So the Oh became a zero! So the KWBC
>>>> collective
>>>> then missed it.
>>>>
>>>> SACA35 KWBC 090100
>>>> METAR
>>>> ...snipped...
>>>> MROC NIL=
>>>>
>>>> So should I account for this in my METAR ingesting code somehow or alas,
>>>> the 1z ob on 9 August 2017 is lost forever for MROC :(
>>>>
>>>> daryl
>>>>
>>>> --
>>>> /**
>>>> * daryl herzmann
>>>> * Systems Analyst III -- Iowa Environmental Mesonet
>>>> * https://mesonet.agron.iastate.edu
>>>> */
>>>>
>>>> _______________________________________________
>>>> NOTE: All exchanges posted to Unidata maintained email lists are
>>>> recorded in the Unidata inquiry tracking system and made publicly
>>>> available through the web. Users who post to any of the lists we
>>>> maintain are reminded to remove any personal information that they
>>>> do not want to be made public.
>>>>
>>>> Data-curation mailing list
>>>> Data-curation@xxxxxxxxxxxxxxxx
>>>> For list information, to unsubscribe, or change your membership options,
>>>> visit: http://www.unidata.ucar.edu/mailing_lists/
>>>>
>>>>
>>>
>>> _______________________________________________
>>> NOTE: All exchanges posted to Unidata maintained email lists are
>>> recorded in the Unidata inquiry tracking system and made publicly
>>> available through the web. Users who post to any of the lists we
>>> maintain are reminded to remove any personal information that they
>>> do not want to be made public.
>>>
>>> Data-curation mailing list
>>> Data-curation@xxxxxxxxxxxxxxxx
>>> For list information, to unsubscribe, or change your membership options,
>>> visit: http://www.unidata.ucar.edu/mailing_lists/
>>>
>>>
>>>
>>
> --
> /**
> * daryl herzmann
> * Systems Analyst III -- Iowa Environmental Mesonet
> * https://mesonet.agron.iastate.edu
> */
>
