bufrtables mailing list is no longer active. The list archives are made available for historical reasons.
Hi Stan, et al:I previously reported changes in various versions of the mel-bufr tables which led to thinking BUFR descriptors in the WMO master tables were changing between versions (previous posts copied below, for completeness). Now Im looking at the ECMWF tables used in the BUFRDC encoding/decoding software. They fail to confirm the differences in the mel-bufr table. This leads me to suspect the accuracy of the mel-bufr tables. So perhaps (hopefully), Stan, you are right that elements cant change once they become operational. Still, I wonder why mel-bufr and ECMWF keep versioned copies of the tables?
Also what to make of element 0-22-39 in the latest Table B document (http://www.wmo.int/pages/prog/www/WMOCodes/Operational/BUFR/BufrTabB-11-2007.pdf) which has changed the bit width to 13, when it has been 12 all along? Must be a typo; is anyone using that value(!) ?
ECMWF tables have a few changes of their own between versions. It could be that they are typos, that im looking at "pre-operational" values, or things that are local to ECMWF.
The lack of table management is quite a burden on developing a general reading package.
To reiterate what we are thinking here at Unidata:In the absence of a better mechanism, we will publish on the Unidata web site the BUFR tables we are using for our BUFR reading software, in various standard formats like XML and ASCII, in the hopes that any mistakes in them will be quickly caught and fixed. We will try to obtain canonical sources and have tools to compare representations to check for errors. We will encourage anyone else producing BUFR messages intended for data exchange (as opposed to internal consumption only) to send us their tables or links to their tables, which we will make available on our web site.
As soon as WMO or any other authoritative organization is willing and able to take over these tasks, and publish canonical machine-readable BUFR tables, we will point everyone to that site(s) instead.
John PS: Please let me know if you'd like to be removed from this list, or go to http://www.unidata.ucar.edu/mailing_lists/
-----Original Message-----From: John Caron [mailto:caron@xxxxxxxxxxxxxxxx] Sent: 05 August 2008 20:03To: Kellett, Stanley Cc: Wright, Bruce; Jeff Ator; Ross, Gil; Michelle Mainelli; Scott Jacobs; Paul Hamer; Chris Macdermaid; Robb Kambic Subject: Re: table versions Hi Stan: Thanks so much for your reply. I hope the flies arent biting as hard as in Wyoming where I was camping last week. If new versions of the WMO master table can only add new elements, this would be very good news. It would mean that one really only needs the latest version of the table. Is there somewhere that is explicitly stated in the WMO BUFR specification? The worry that I have is that if a mistake is made in a published version of a table, and is used to code a BUFR message, that message will be incorrectly read by a decoder using the corrected version of thetable. Theres no way to unambiguously know what table the coder used.We have been studying the differences between versions of the tables, using the mel-bufr tables. While we are not sure of their exactprovenance, they seem to be widely used.Below is a summary of the differences in scale,offset and/or bit width among mel-bufr versions 2 through 13. The difference is always with the current version (13); "B3M-000-003-B.diff" for example means it is thedifference with mel-bufr table version 3. For example the line0-2-42 width 6 != 2 (B2M-000-002-B.diff) means that the bit width for field 0-2-42 was 6 for version 2, and 2 for version 13. -------------------------------- 0-2-42 width 6 != 2 (B2M-000-002-B.diff) width 6 != 2 (B3M-000-003-B.diff) width 6 != 2 (B3M-000-004-B.diff) width 6 != 2 (B3M-000-005-B.diff) 0-2-91 scale 0 != 4 (B3M-000-008-B.diff) refer 4 != 0 (B3M-000-008-B.diff) scale 0 != 4 (B3M-000-009-B.diff) refer 4 != 0 (B3M-000-009-B.diff) scale 0 != 4 (B3M-000-010-B.diff) refer 4 != 0 (B3M-000-010-B.diff) 0-2-140 scale 2 != 0 (B2M-000-002-B.diff) scale 2 != 0 (B3M-000-003-B.diff) scale 2 != 0 (B3M-000-004-B.diff) scale 2 != 0 (B3M-000-005-B.diff) 0-2-151 width 4 != 11 (B3M-000-006-B.diff) width 4 != 11 (B3M-000-007-B.diff) width 4 != 11 (B3M-000-008-B.diff) width 4 != 11 (B3M-000-009-B.diff) 0-7-9 refer 0 != -1000 (B3M-000-008-B.diff) 0-10-9 refer 0 != -1000 (B3M-000-008-B.diff) 0-11-52 width 14 != 13 (B3M-000-008-B.diff) 0-11-82 width 13 != 14 (B3M-000-008-B.diff) 0-13-60 refer -10 != -1 (B3M-000-008-B.diff) refer -10 != -1 (B3M-000-009-B.diff) refer -10 != -1 (B3M-000-010-B.diff) 0-21-85 refer -3000 != 0 (B2M-000-002-B.diff) refer -3000 != 0 (B3M-000-003-B.diff) refer -3000 != 0 (B3M-000-004-B.diff) refer -3000 != 0 (B3M-000-005-B.diff) refer -3000 != 0 (B3M-000-006-B.diff) 0-22-39 width 12 != 13 (B2M-000-002-B.diff) width 12 != 13 (B3M-000-003-B.diff) width 12 != 13 (B3M-000-004-B.diff) width 12 != 13 (B3M-000-005-B.diff) width 12 != 13 (B3M-000-006-B.diff) width 12 != 13 (B3M-000-007-B.diff) width 12 != 13 (B3M-000-008-B.diff) width 12 != 13 (B3M-000-009-B.diff) width 12 != 13 (B3M-000-010-B.diff) width 12 != 13 (B3M-000-011-B.diff) width 12 != 13 (B3M-000-012-B.diff) 0-24-1 scale -1 != -11 (B2M-000-002-B.diff) scale -1 != -11 (B3M-000-003-B.diff) scale -1 != -11 (B3M-000-004-B.diff) scale -1 != -11 (B3M-000-005-B.diff) 0-24-2 scale -1 != -11 (B2M-000-002-B.diff) scale -1 != -11 (B3M-000-003-B.diff) scale -1 != -11 (B3M-000-004-B.diff) scale -1 != -11 (B3M-000-005-B.diff) 0-25-50 scale 0 != 4 (B2M-000-002-B.diff) refer 0 != -131072 (B2M-000-002-B.diff) width 7 != 18 (B2M-000-002-B.diff) scale 0 != 4 (B3M-000-003-B.diff) refer 0 != -131072 (B3M-000-003-B.diff) width 7 != 18 (B3M-000-003-B.diff) scale 0 != 4 (B3M-000-004-B.diff) refer 0 != -131072 (B3M-000-004-B.diff) width 7 != 18 (B3M-000-004-B.diff) scale 0 != 4 (B3M-000-005-B.diff) refer 0 != -131072 (B3M-000-005-B.diff) width 7 != 18 (B3M-000-005-B.diff) scale 0 != 4 (B3M-000-006-B.diff) refer 0 != -131072 (B3M-000-006-B.diff) width 7 != 18 (B3M-000-006-B.diff) 0-25-51 width 12 != 7 (B2M-000-002-B.diff) width 12 != 7 (B3M-000-003-B.diff) width 12 != 7 (B3M-000-004-B.diff) width 12 != 7 (B3M-000-005-B.diff) width 12 != 7 (B3M-000-006-B.diff) 0-29-2 width 2 != 3 (B2M-000-002-B.diff) width 2 != 3 (B3M-000-003-B.diff) width 2 != 3 (B3M-000-004-B.diff) width 2 != 3 (B3M-000-005-B.diff) If we believe these mel-bufr tables (and we havent made some mistake in analysing them), scale/reference/width seems to be allowed to change. For example, the last field, 0-29-2, bit width was changed from 2 to 3 at version 6. If so, then we have no choice but to maintain versioned code tables, and assume that the version number (BUFR section 1 octet 11) is correct. Of course the real question is, what values did the coders use? We have some hope of automatically detecting when bit width is wrong by counting bits in the message and comparing to the data size. If scale/offset is wrong, it will take humans and/or more complex algorithms to detectthat.It would be very helpful to hear what the actual practice is with tables at the various generating centers. Or to hear that the mel-bufr tables are wrong, or some other flaw in our conclusion. Thanks for any thoughts, John Kellett, Stanley wrote:operational.John,First of all I am sorry for late response as I have just gotten back from a camp trip with my family.No scale, width and reference values can't be changed onceWhat has probably happened is there was a mistake in the last published tables which was corrected.Stan -----Original Message----- From: John Caron [mailto:caron@xxxxxxxxxxxxxxxx] Sent: 24 July 2008 18:20To: Wright, Bruce; Jeff Ator; Ross, Gil; Michelle Mainelli; Scott Jacobs; Kellett, Stanley; Paul Hamer; Chris Macdermaid; Robb KambicSubject: table versionsWe are trying to understand how the different versions of the WMO master table differ from each other. We are looking at the "mel-bufr" tables, which seem to be the current best representations of the different versions of the WMO master tables, although we are not sure how they were generated.We compare these tables against our version of the WMO master table version 13, which we extracted from the Word doc. Typical are the following difference of table 12 with table 13:1) *** WARNING **** Discrepency Scale Ref Width Units for key 0-13-81 3;0;14;S m-1 from B3M-000-012-B 3;0;14;Siemens m-1 from B4M-000-013-B 2) *** WARNING **** Discrepency Scale Ref Width Units for key 0-15-11 3;14000;13;log from B3M-000-012-B 3;14000;13;log(1/m2) from B4M-000-013-B 3) *** WARNING **** Discrepency Scale Ref Width Units for key 0-22-39 3;-5000;12;m from B3M-000-012-B 3;-5000;13;m from B4M-000-013-B Difference 1) and 2) seems to be clarifications of the units. In 1) "Sm-1" has been changed to "Siemens m-1" and in 2) "log" was changed to"log(1/m2)". It seems likely that those changes are clarifications or corrections that should be used in version 12 BUFR messages.Difference 3) is a change in bit width of field 0-22-39. Should this be understood that version 12 messages use bit width 12, while version13 uses bit width 13? Obviously this is a critical thing to know. When we compare table 10 with 13, we see: 4) *** WARNING **** Discrepency Scale Ref Width Units for key 0-2-91 0;4;10;a from B3M-000-010-B 4;0;10;a from B4M-000-013-B 5) *** WARNING **** Discrepency Scale Ref Width Units for key 0-13-60 1;-10;17;kg m-2 from B3M-000-010-B 1;-1;17;kg m-2 from B4M-000-013-BFor difference 4) I might guess that the scale and reference were mistakenly flipped, that is, it corrects a typo in version 10. Possibly difference 5), which changes the reference value from -10 to -1, corrects a typo also?Are new table versions really allowed to change Scale,Ref,Width or Units of existing table entries? Because if they are, we really have to carefully track what changes with each version.Are there archives of previous versions of WMO master tables? Are the mel-bufr tables accurate?Any thoughts or advice would be appreciated.