[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[LDM #ITG-685793]: PQING question

Carissa,

> I have 2 more test files. I also noticed some buffer errors in the ldmd.log
> when running these again. Not sure which comes first here, or if they are
> related.
> 
> I'm having trouble sending these attachments. I have put them here -  ????
> 
> ftp://ftp.ncep.noaa.gov/pub/data1/test/
> 
> Sep 15 18:19:27 vm-lnx-ncosrv11 pqing[6171] NOTE: YETZ98 KWBN 151658: no
> end of product
> Sep 15 18:19:27 vm-lnx-ncosrv11 pqing[6171] ERROR: Not a WMO GRIB format
> message.       8374         002  YETZ98 KWBN 151658
> Sep 15 18:19:27 vm-lnx-ncosrv11 pqing[6171] NOTE: YESZ98 KWBN 151658: no
> end of product
> Sep 15 18:19:27 vm-lnx-ncosrv11 pqing[6171] ERROR: Not a WMO GRIB format
> message.     149139         003  YESZ98 KWBN 151658
> Sep 15 18:19:27 vm-lnx-ncosrv11 pqing[6171] INFO: Expanding input buffer
> size to 524288
> Sep 15 18:19:27 vm-lnx-ncosrv11 pqing[6171] INFO: Expanding input buffer
> size to 1048576
> Sep 15 18:19:27 vm-lnx-ncosrv11 pqing[6171] ERROR: Buffer would exceed
> 1048576, resetting input

I the last error-message above might indicate a problem.

How are you invoking pqing(1)? If you're not using the "-s <size>" option, then 
only data-products that are 1048576 bytes or smaller will be accepted.

> Sep 15 18:19:27 vm-lnx-ncosrv11 pqing[6171] INFO: Expanding input buffer
> size to 2097152
> Sep 15 18:19:27 vm-lnx-ncosrv11 pqing[6171] ERROR: Not a WMO format
> message.          139238         -01      00      151819
> Sep 15 18:19:27 vm-lnx-ncosrv11 pqing[6171] ERROR: Buffer would exceed
> 1048576, resetting input
> Sep 15 18:19:27 vm-lnx-ncosrv11 pqing[6171] INFO: Expanding input buffer
> size to 4194304
> Sep 15 18:19:27 vm-lnx-ncosrv11 pqing[6171] ERROR: Buffer would exceed
> 1048576, resetting input
> Sep 15 18:19:27 vm-lnx-ncosrv11 pqing[6171] INFO: Expanding input buffer
> size to 8388608
> Sep 15 18:19:27 vm-lnx-ncosrv11 pqing[6171] ERROR: Buffer would exceed
> 1048576, resetting input
> Sep 15 18:19:27 vm-lnx-ncosrv11 pqing[6171] INFO: Expanding input buffer
> size to 16777216
> Sep 15 18:19:28 vm-lnx-ncosrv11 pqing[6171] ERROR: Not a WMO format
> message.        16540351         -01      00      151819

Regarding the "Not a WMO format message" log-message, here's a character dump 
of the beginning of the 1.2 MB file "testfile.20140912152254":

~/Projects/ldm/ldm/src/noaaport$ od -c ~/Downloads/testfile.20140912152254 | 
head
0000000 001  \r  \r  \n   0   0   0      \r  \r  \n   Y   S   U   Z   9
0000020   8       K   W   B   N       1   0   1   5   5   2  \r  \r  \n
0000040   *   *   *   *   0   0   0   0   1   9   7   5   2   9   *   *
0000060   *   *  \n   Y   S   U   B   1   8       K   W   B   N       1
0000100   0   1   5   5   2  \r  \r  \n   G   R   I   B  \0  \0  \0 002
0000120  \0  \0  \0  \0  \0 003 003 204  \0  \0  \0 025 001  \0  \b 377
0000140 377 001  \0 001  \a 336  \t  \n 020  \0  \0 001 001  \0  \0  \0
0000160   Q 003  \0  \0   - 021 301  \0  \0  \0 036 001  \0  \0   a   7
0000200 200  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \b   a  \0
0000220  \0 005   a 001   4 032 377 016   6   e 257  \0 001   }   x   @

pqing(1) expects that the length of the GRIB message, in bytes, is encoded in 
the 3 bytes following the string "GRIB". Those bytes are all zero in the above.

> Carissa Klemmer
> NCEP Central Operations
> Production Management Branch Dataflow Team
> 301-683-3835

Regards,
Steve Emmerson

Ticket Details
===================
Ticket ID: ITG-685793
Department: Support LDM
Priority: Normal
Status: Closed