[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: 20050113: dcmetr... input file length limit?



Mike,

Yes, thats the way operational data arrives- as bulletins, or collectives
of METAR reports.

Steve Chiswell
Unidata User Support

On Thu, 13 Jan 2005, Mike Hardiman wrote:

> So, I can breakup the file being fed into dcmetr into multiple
> bulletins and it will still feed all the data into the same gempak file?
>
>
> ----- Original Message -----
> From: Steve Chiswell <address@hidden>
> Date: Thursday, January 13, 2005 7:06 am
> Subject: 20050113: dcmetr... input file length limit?
>
> > Mike,
> >
> > Since the decoders read from STDIN (eg the piped or redirected
> > file input),
> > they don't see the file on disk, so are not effected by input file
> > size.
> > However, the decoders such as dcmetr have a maximum bulletin
> > length, which
> > is the maximum length of a product  between the ZCZC and NNNN
> > delimeters.For dcmetr, this limit is 100K (eg DCMXBF  = 102400 ).
> > You will need to
> > create bulletin delimeters which signal to the decoder that the
> > currentproduct has been read, and can be decoded, after which the
> > decoder will search
> > for the next product ZCZC/NNNN pair.
> >
> > Steve Chiswell
> > Unidata User Support
> >
> > On Thu, 13 Jan 2005, Mike Hardiman wrote:
> >
> > > Hi Folks,
> > >
> > > We recently acquired the Surface Archives CDs from Weather Graphics
> > > Tecnhologies down here for use in a local research project.
> > >
> > > Since I'd like to create a few billion surface plots, I need to
> > get the
> > > data into a gempak-friendly format.
> > >
> > > So I wrote a script to read an input "laundry list" then dig
> > into the
> > > archive CDs, pull out the desired data, unzip it, reformat it
> > with the
> > > proper AFOS headers that the GEMPAK decoders so love, spit it
> > out in
> > > text files for each day/hour requested, and then create a GEMPAK
> > > surface file for each day/hour using dcmetr.
> > >
> > > However, I noticed it was only producing the GEMPAK surface
> > files for a
> > > few scattered days and hours.
> > >
> > > Lo and behold, there was plenty of data in the text files that
> > weren't> being decoded with dcmetr.
> > >
> > > So I tried it manually... dcmetr fails.  Log just says it did not
> > > decode any bulletins.
> > >
> > > If I manually chomp out a huge chunk of obs from the files, they
> > decode> just fine.... so this leads me to the following
> > conclusion:  There must
> > > be a input file length-limit in dcmetr.  Once it reaches this
> > limit, it
> > > reads no further (and therefore doesn't see the "NNNN" it needs
> > to work
> > > properly) and therefore fails.
> > >
> > > Can anyone confirm my fears and/or suggest a solution?
> > >
> > > If you need more info, like sample files, log file output, etc,
> > please> let me know and I'll post them on the web.
> > >
> > > Thanks much!
> > >
> > > -Mike Hardiman
> > > NWS Santa Teresa, NM
> > >
> > >
> > >
> >
>