Robb Kambic wrote:
On Fri, 17 Nov 2006, Simon Elliott wrote:Hi all, WHen I look at the file attached by HansPeter (HP GRIB.001) it looks fine to me. The GRIB part can be found after the first 899 bytes. The remaining part (GRIB to 7777, inclusive) is 803521 bytes long, which is as indicated in octets 9 to 16 of Section 0.Hi Simon, First, let me thank you for your time. There's no problem decoding the (HP GRIB.001) file with my decoder that is 803521 bytes long. The file that was originally sent was not 803521 bytes, but 804420 bytes long. When the file was edited, the editor inserted extra bytes into the file. This caused the decoder to get confused and it could not extract the data. As you know the GRIB format is very rigid, so one extra byte can change the course of action of the decoder. There was some confusion about the acquistion of the file and I was unawared that the file was edited, so I assumed that there was some EUMETSAT changes in the file. Since the EUMETSAT datasets are very important to our community, I made an effort to decode the file with the extra bytes. When debugging, I added some code to accomodate the extra bytes but the decoder still got confused about the data extraction process. When I got an uncorrupted file, my decoder work without a problem. There is one question, the level information is using level 0, is that correct? First Surface Type : 0 Reserved First Surface value : 0.0 Second Surface Type : 0 Reserved Second Surface value : 0.0 I have never run across a model using these values before, can you give some insight? Thanks, Robb... Here's a dump of the Header information of the file, is there an problem here? -------------------------------------------------------------------- Header : GRIB2 Discipline : 3 Space products GRIB Edition : 2 GRIB length : 803521 Originating Center : 254 EUMETSAT Operation Centre Originating Sub-Center : 0 Significance of Reference Time : 3 Observation time Reference Time : 2006-01-02T11:45:00Z Product Status : 1 Operational test products Product Type : 6 Processed satellite observations Number of data points : 1530169 Grid Name : 90 Space View Perspective or Orthographic Grid Shape: 3 Earth oblate spheroid with axes specified by producer Oblate earth major axis: 6378.14 Oblate earth minor axis: 6356.7554 Number of points along parallel: 1237 Number of points along meridian: 1237 Latitude of sub-satellite point: 0.0 Longitude of sub-satellite pt: 0.0 Resolution & Component flags : 0 i direction increment : 1188.0 j direction increment : 1187.0 X-coordinate of sub-satellite : 619.0 Y-coordinate of sub-satellite : 619.0 Scanning mode : 192 Basic angle : 0 Altitude of the camera : 7.4533146E8 X-coordinate of origin : 0.0 Y-coordinate of origin : 0.0 Product Definition : 30 Satellite product Parameter Category : 1 Quantitative_products Parameter Name : 2 Cloud_Top_Height Parameter Units : m Generating Process Type : 8 Observation ForecastTime : 0 First Surface Type : 0 Reserved First Surface value : 0.0 Second Surface Type : 0 Reserved Second Surface value : 0.0 -------------------------------------------------------------------- Header : GRIB2 Discipline : 3 Space products GRIB Edition : 2 GRIB length : 803521 Originating Center : 254 EUMETSAT Operation Centre Originating Sub-Center : 0 Significance of Reference Time : 3 Observation time Reference Time : 2006-01-02T11:45:00Z Product Status : 1 Operational test products Product Type : 6 Processed satellite observations Number of data points : 1530169 Grid Name : 90 Space View Perspective or Orthographic Grid Shape: 3 Earth oblate spheroid with axes specified by producer Oblate earth major axis: 6378.14 Oblate earth minor axis: 6356.7554 Number of points along parallel: 1237 Number of points along meridian: 1237 Latitude of sub-satellite point: 0.0 Longitude of sub-satellite pt: 0.0 Resolution & Component flags : 0 i direction increment : 1188.0 j direction increment : 1187.0 X-coordinate of sub-satellite : 619.0 Y-coordinate of sub-satellite : 619.0 Scanning mode : 192 Basic angle : 0 Altitude of the camera : 7.4533146E8 X-coordinate of origin : 0.0 Y-coordinate of origin : 0.0 Product Definition : 30 Satellite product Parameter Category : 1 Quantitative_products Parameter Name : 3 Cloud_Top_Height_Quality_Indicator Parameter Units : Generating Process Type : 8 Observation ForecastTime : 0 First Surface Type : 0 Reserved First Surface value : 0.0 Second Surface Type : 0 Reserved Second Surface value : 0.0The users who have been looking at the data have received it via EUMETCast, and also via ftp from me. Maybe some got data from the UMARF, but I don't get to hear about them. I think that as long as a decoder looks through a file and pulls out GRIB (or BUFR) messages before decoding, then it doesn't matter how they are packed into files. Simon Dr Simon Elliott Product Implementation Manager Operations Department EUMETSAT tel: (+49) 6151 807385 fax: (+49) 6151 807304HansPeter Roesli <address@hidden> 17/11/2006 09:37:39 >>>Hi Simon and Robb After having read Simon's first reaction I got a slight doubt about what I did. Actually, the file Robb is mentioning is not the original file as retrieved from EUMETSAT's archive U-MARF (you find that now attached). The original file is refused right away by IDV. Therefore, I chopped off the ASCII header with an editor and that is the file Robb is using. Might be that my editor changed things inside the GRIB-7777 clause when I was pruning the file. So, Robb please check again with the attached file which has leading bytes (EUMETSAT's ASCII header) before the bytes spelling GRIB. Also, Simon, when you are saying that other centres are not having decoding problems, are they getting the data from EUMETCast/GTS or from U-MARF. I am using exclusively U-MARF retrieved products. Might be there is a problem with archived data? HP Simon Elliott wrote:Hello Robb, HansPeter has asked me to contact you directly as I am responsibleforthe generation of the GRIB2 data from EUMETSAT. I am interested toreadabout the extra characters you found inside hte message. As I'msureyou can imagine, we have exchanged the data with other centres using different software and they have yet to spot this problem. I suspect either (i) the GRIB data have been chopped up for dissemination and not been properly re-assembled into the original message, or (ii) at some stage the data have been transferred withASCIIFTP ... this happens quite often. If you could send me the file you mention, or make it availabel forFTPtransfer, I would be happy to investigate. Regards, Simon Dr Simon Elliott Product Implementation Manager Operations Department EUMETSAT tel: (+49) 6151 807385 fax: (+49) 6151 807304Robb Kambic <address@hidden> 14/11/2006 22:50:35 >>>HansPeter, i investigated the problem with Grib2 decoder not being able todecodethe space products, ie MSG1-SEVI-MSGCLTH-0100-0100-20060102114500.000000000Z-12774_GRIB.grb at first it seemed to be a table problem but that's not the case. the products do not follow the standard Grib encoding, there are extra characters inserted between the Grib format sections that'stheproblem. i had the same problem with the Grib 1 format from some of the european centers. i tried the Grib 1 format fix then the decoder was able to decode the first 5 sections of the product but then the 6th section didn't begin at the correct position. at this time, i could somehelpin finding out the format of the product, so i can modify the decoder.idid a quick look for the product information but was unable to get the information i needed. Could you contact the data provider to get the structureofthe product? or get a contact person for this product? thanks, robb...=============================================================================== Robb Kambic Unidata Program Center Software Engineer III Univ. Corp for Atmospheric Research address@hidden WWW: http://www.unidata.ucar.edu/ ===============================================================================
begin:vcard fn:HP's Satellite Meteorology n:Roesli;HansPeter adr;quoted-printable:;;via Monte Br=C3=A8 5;Locarno Monti;;6605;Switzerland email;internet:address@hidden tel;cell:+41 079 292 0706 version:2.1 end:vcard