Re: [bufrtables] More on table versions


Hello Jeff,

You are right that scale, reference value and data width of a descriptor
cannot be changed between table versions, but these parameters may
be changed with introducing of a new version of the Master Table!!!
It is the reason, why the number of version is included in Section 1.
The information on the time of introduction of the descriptor may be
useful, but it is  not essential at all.

I would like to refer to my Doc. 3.1.1(1), available at
http://www.wmo.int/pages/prog/www/ISS/Meetings/CT-MTDCF-ET- 
DRC_Geneva2008/DocPlan.html, where a change of data
width and reference value of several descriptors from Class 14 is
proposed to be introduced with a new table version. And this document
has been already discussed and you did not express any objections to
the proposed approach.

Up to now, however, this ability of BUFR has not been much used,
i.e. the out-dated descriptors were deprecated and replaced
by new ones. Using this approach, the management of the tables
is simplified and only the last tables may be implemented in the SW.
But the deprecated element descriptors may be included in several
D-descriptors and these would have to be also deprecated, and soon
we would end up with saturated tables full of deprecated descriptors.

With regards,
Eva

Dr Eva Cervena
Czech Hydrometeorological Institute


On 12 Aug 2008 at 12:23, Jeff Ator wrote:

Hello Everyone,

I'm now back from a restful vacation and can add my proverbial $0.02 to
the discussion.

First of all, Stan is correct that scale, reference and bit width values
should never change between table versions.  Once a descriptor has been
included in an operational version it is considered static.  This means
that you theoretically only ever need the latest version of Table B in
order to be able to decode everything, and any discrepancies for a
particular descriptor between successive table versions are indeed typos.

Having said that, I do believe it is useful to maintain previous table
versions, because they do allow you to go back and determine when
certain descriptors became available for use.  For example, if you
received a message that said it was using version 8 of the tables, but
it contained a particular descriptor that only became available
beginning with version 10, having all of the previous versions available
would allow you to easily diagnose such an error.

Please note that I'm only sharing with you what is the current WMO
practice, and I'm not making any personal statement of opinion one way
or the other as to whether this is a good thing.  While the current
practice, whereby each successive Table B is always a superset of all
the previous versions, does simplify the task of decoding, it also has
caused some of the descriptor classes to fill up much more quickly than
would otherwise be necessary.  A good example is Class 12, where we
initially had a descriptor for every conceivable type of temperature
value, but with scale 1 which only allowed accuracy to one digit beyond
the decimal.  After these were in use for a while, it was discovered
that we had a problem, because most instruments report temperatures in
Celsius whereas BUFR uses the SI unit of Kelvin, but the conversion
factor between Celsius and Kelvin is 273.15, and different architectures
didn't always handle the rounding the same way.  This meant you could
have one center take an observation in Celsius, then convert it to
Kelvin for encoding into BUFR, then send that BUFR message to a
different center, and the second center could, when converting back to
Celsius, obtain a value that was off by as much as two-tenths of a
degree from the original observed value.  With hindsight, we realized
that the best solution was to just store all temperatures using scale 2,
but the practice was (and still is!) to never modify the characteristics
of an existing descriptor once it has been in the table.  So we ended up
creating a second copy of each existing temperature descriptor, each
with it's own new reference number and a scale factor of 2, when an
alternative solution could have been to simply increase the scale of
each existing temperature descriptor effective with the next version of
the tables, and then rely on everyone's encoder/decoder software to
differentiate properly between the old and new versions of each
descriptor based on the table version number encoded within the message.

I realize that was a somewhat long-winded historical example, but an
important point is that WMO could possibly change their practice at some
point in the future (especially as some of the Table B classes are now
close to completely full!), and this is another reason why it would be a
good idea for us to hold on to all previous versions of these tables,
even though right now it's not technically necessary.   The easiest way
to do this would be to include the version number somewhere in the
filename within our collective archive.

As for BUFR edition numbers (about which I've also sensed some confusion
in the ongoing email thread), these are a different animal entirely. Whereas table version number changes do not require any corresponding
change to BUFR encoder/decoder software, the edition number is
incremented only when changes are introduced which require corresponding
changes to actual encoding and decoding software (for example, if new
Table C operators are introduced, or the format of BUFR Section 1 is
modified).  In this case, the software needs to be able to
simultaneously handle multiple editions (i.e. formats) of BUFR messages,
which of course adds to programming complexity, and so such changes are
made much less frequently and with much more advance notice.

I hope this helps clarify any confusion!

Best regards,
-Jeff