Justin, > I was able recompile pqing using a higher value for MAX_CIRCBUFSIZE and the > previously problematic data was successfully ingested into the LDM queue. Excellent! > Can you tell me what pqing uses to determine how large the buffer needs to > be? Is this something defined in the message or does it use the actual size > of the data? The buffer grows dynamically, as needed, in order to accommodate whatever WMO message is being read. Unfortunately for you, there's a limit on how large it can grow. The limit was put in place by the late Glenn Davis. I don't know why he chose the value he did. > The problematic data caused the input buffer size to reach 4194304: > > ldmd.log.3:Aug 14 20:01:31 s2n5 local0:info pqing INFO: Expanding > input buffer size to 4194304 > > But the file itself is only 2100691 bytes: > > $ ls -l WMOMSG.stratusdev.WB79_WMOMSG.377.824082.1344010210.0 > -rw-r--r-- 1 dbndev dcddev 2100691 Aug 13 16:25 > WMOMSG.stratusdev.WB79_WMOMSG.377.824082.1344010210.0 > > Any idea why would the buffer have to be nearly twice the size? When the buffer needs to grow, it doubles its size. I agree that it seems odd for it to grow beyond MAX_CIRCBUFSIZE. > Thanks for your help with this. > > Justin Regards, Steve Emmerson Ticket Details =================== Ticket ID: WMK-256465 Department: Support LDM Priority: Normal Status: Closed
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publicly available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us.