Re: [bufrtables] More on table versions

Dear Eva and Jeff,

I think that there is no problem to change data width, reference value and scale for different Master table version numbers of tables used.

If we could not do this the whole concept of master table version numbers and local table versions would be meaningless. The only requirement is that once the entry is in any of the tables and used,
it must stay as such in that version of the tables. However the only
setback is that Secretariat must keep track of all versions even if
most entries in the next version is the same as in the previous.

That is the way we are going to correct radiation elements in the version 14.

Regards

Milan


Milan Dragosavac

ECMWF
Shinfield Park, Reading, Berkshire, RG2 9AX, UK

Tel: (+44 118) 949 9403
Fax: (+44 118) 986 9450
Telex: 847908 ECMWF G
E-mail: milan.dragosavac@xxxxxxxxx


Eva Cervena wrote:
Hello Jeff,

You are right that scale, reference value and data width of a descriptor
cannot be changed between table versions, but these parameters may
be changed with introducing of a new version of the Master Table!!!
It is the reason, why the number of version is included in Section 1.
The information on the time of introduction of the descriptor may be
useful, but it is  not essential at all.

 I would like to refer to my Doc. 3.1.1(1), available at
http://www.wmo.int/pages/prog/www/ISS/Meetings/CT-MTDCF-ET- DRC_Geneva2008/DocPlan.html, where a change of data
width and reference value of several descriptors from Class 14 is
proposed to be introduced with a new table version. And this document
has been already discussed and you did not express any objections to
the proposed approach.

Up to now, however, this ability of BUFR has not been much used,
i.e. the out-dated descriptors were deprecated and replaced
by new ones. Using this approach, the management of the tables
is simplified and only the last tables may be implemented in the SW.
But the deprecated element descriptors may be included in several
D-descriptors and these would have to be also deprecated, and soon
we would end up with saturated tables full of deprecated descriptors.

With regards,
Eva

Dr Eva Cervena
Czech Hydrometeorological Institute


On 12 Aug 2008 at 12:23, Jeff Ator wrote:

 Hello Everyone,

 I'm now back from a restful vacation and can add my proverbial $0.02 to
 the discussion.

 First of all, Stan is correct that scale, reference and bit width values
 should never change between table versions.  Once a descriptor has been
 included in an operational version it is considered static.  This means
 that you theoretically only ever need the latest version of Table B in
 order to be able to decode everything, and any discrepancies for a
 particular descriptor between successive table versions are indeed typos.

 Having said that, I do believe it is useful to maintain previous table
 versions, because they do allow you to go back and determine when
 certain descriptors became available for use.  For example, if you
 received a message that said it was using version 8 of the tables, but
 it contained a particular descriptor that only became available
 beginning with version 10, having all of the previous versions available
 would allow you to easily diagnose such an error.

 Please note that I'm only sharing with you what is the current WMO
 practice, and I'm not making any personal statement of opinion one way
 or the other as to whether this is a good thing.  While the current
 practice, whereby each successive Table B is always a superset of all
 the previous versions, does simplify the task of decoding, it also has
 caused some of the descriptor classes to fill up much more quickly than
 would otherwise be necessary.  A good example is Class 12, where we
 initially had a descriptor for every conceivable type of temperature
 value, but with scale 1 which only allowed accuracy to one digit beyond
 the decimal.  After these were in use for a while, it was discovered
 that we had a problem, because most instruments report temperatures in
 Celsius whereas BUFR uses the SI unit of Kelvin, but the conversion
 factor between Celsius and Kelvin is 273.15, and different architectures
 didn't always handle the rounding the same way.  This meant you could
 have one center take an observation in Celsius, then convert it to
 Kelvin for encoding into BUFR, then send that BUFR message to a
 different center, and the second center could, when converting back to
 Celsius, obtain a value that was off by as much as two-tenths of a
 degree from the original observed value.  With hindsight, we realized
 that the best solution was to just store all temperatures using scale 2,
 but the practice was (and still is!) to never modify the characteristics
 of an existing descriptor once it has been in the table.  So we ended up
 creating a second copy of each existing temperature descriptor, each
 with it's own new reference number and a scale factor of 2, when an
 alternative solution could have been to simply increase the scale of
 each existing temperature descriptor effective with the next version of
 the tables, and then rely on everyone's encoder/decoder software to
 differentiate properly between the old and new versions of each
 descriptor based on the table version number encoded within the message.

 I realize that was a somewhat long-winded historical example, but an
 important point is that WMO could possibly change their practice at some
 point in the future (especially as some of the Table B classes are now
 close to completely full!), and this is another reason why it would be a
 good idea for us to hold on to all previous versions of these tables,
 even though right now it's not technically necessary.   The easiest way
 to do this would be to include the version number somewhere in the
 filename within our collective archive.

 As for BUFR edition numbers (about which I've also sensed some confusion
in the ongoing email thread), these are a different animal entirely. Whereas table version number changes do not require any corresponding
 change to BUFR encoder/decoder software, the edition number is
 incremented only when changes are introduced which require corresponding
 changes to actual encoding and decoding software (for example, if new
 Table C operators are introduced, or the format of BUFR Section 1 is
 modified).  In this case, the software needs to be able to
 simultaneously handle multiple editions (i.e. formats) of BUFR messages,
 which of course adds to programming complexity, and so such changes are
 made much less frequently and with much more advance notice.

 I hope this helps clarify any confusion!

 Best regards,
 -Jeff