Re: [bufrtables] BUFR and GRIB2 table information in relational DB?

Dear Eizi,

Some quick answers.

Eizi TOYODA wrote:
Dear Enrico,

Looks great.  Some quick thoughts:

(1) PROGRAMMERS' INTERFACE

The web you mentioned is good for end-user browsing, while Jeff (and
me) also wants programmer-friendly data source.  I guess such users
are suggested to download grib_api package (such as following URL).
Did you perhaps have other interface?

http://www.ecmwf.int/products/data/software/download/software_files/grib_api-1.9.9.tar.gz


My duty is to provide a grib decoder for ECMWF and ECMWF Member States. I cannot provide any programmer-frendly data source. This is WMO responsibility. I can share the experience on the use of a database for a GRIB decoder and in the near future for a BUFR decoder. I would love to have exchange of ideas regarding technical implementations with other in my similar condition. This is not related with WMO work which is at a different level and in a different context.

(2) UPDATE POLICY

The latest grib_api is dated March 22, 2011.  But it includes the last
fast track which got in force on May 4.  It looks like you have
practical update policy that incorporates any decided feature before
becoming operational.  In other words, it is very useful for decoder
software, while additional care is necessary for validation software
or documentation of operational feature.

My users are very demanding regarding the update of the software. We have a disclaimer regarding this.

http://www.ecmwf.int/publications/manuals/d/gribapi/fm92/grib1/


Colleagues in IPET-MDI should know Atsushi at the WMO Secretariat is
also working on computer-readable table.
http://www.wmo.int/pages/prog/www/WMOCodes/WMO306_vI2/2010edition/latestTables.zip
This should contain only operational entries.

This is very useful and we are planning to use these tables from the next release/update. However it would be appropriate to have the full set of BUFR and GRIB tables and possibly organised by version. We should be able to load a version from a file taken from WMO web site. This is a very good starting point and for us is the end of parsing word documents with our perl scripts.
I am very grateful for this.


(3) STRUCTURE

Many people have many tools.  Maybe perhaps somebody has more request
than your database.  But I have to reserve any comment until I see
your xml export.

Our database was not built with the purpose of solving the problems of everyone, only our problems of data modelling. For this purpose is working well and provides a very friendly interface. We also have an internal interface in which we can edit the items. However this is for our internal use. As Jeff asked if someone has got some experience. I have just answered that we have some experience and we would like to share it and possibly to get back some good suggestion for improving our database.

I think this is not a discussion for the team. We should continue this discussion as developers.

Best regards


Best,
--
Eiji (aka Eizi) TOYODA
http://www.google.com/profiles/toyoda.eizi



On Fri, Jun 17, 2011 at 16:39, Enrico Fucile <Enrico.Fucile@xxxxxxxxx> wrote:
Hello Jeff,

this is what I did with grib_api. We have a central repository (database)
which is visible on the web

for GRIB edition 1

http://www.ecmwf.int/publications/manuals/d/gribapi/fm92/grib1/

and GRIB edition 2

http://www.ecmwf.int/publications/manuals/d/gribapi/fm92/grib2/

This database is used to build the decoding rules inside grib_api. In this
way we keep coherence between specifications/documentation/decoder.

I am working at the moment on a similar database/decoder for BUFR.
We would be able to provide xml format out of the database for general
exchange purposes.

I would be happy to share our experience and collaborate to build a common
framework.

Best regards

Enrico

Jeff Ator wrote:
Hi Everyone,

This is more of a general survey question.  Here at NCEP, we currently
have our BUFR and GRIB2 master table information scattered across a bunch of
ASCII tables, web pages and system files.  We're thinking about designing
and implementing a relational database to store all of this information in
one place, and then developing methods to allow the information to be easily
exported from the database into the specific formats we need for all of our
APIs and other operational tasks.  We believe the BUFR and GRIB2 tables lend
themselves well to relational database design (especially when multiple
versions of tables including local entries need to be stored), and having
one master repository would alleviate problems we're currently having
keeping information synchronized across multiple system files and documents.

My question is, has anyone out there ever done or thought about doing this
type of thing?  If so, we'd be interested to correspond with you about your
experiences, issues encountered, best practices, etc. and explore any
possible avenues for collaboration.  It's hard for us to believe that nobody
else has ever thought of this before, so if at all possible we'd like to
benefit from any existing experience in the community and avoid re-inventing
the proverbial wheel.

Please let me know if you have any thoughts or experience in this area
that you'd be willing to share.  Either way, thanks for your time and
consideration!

With best regards,
-Jeff