[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: missing SPECI reports and the db file entries



hiya,

if there was a decoder that stored the individual reports in a gdbm
database then no need for pqsurf?  How about a dir on a daily basis that
has one file/stn with all the reports for the day?  it would probably be
faster  the gdbm.

robb...

On Mon, 20 Sep 2004, Neil R. Smith wrote:

> Our only requirement for pqsurf is the DBFILE action to generate output
> for Peter Neilly's weather program. Our ldm was set up that way when I
> took this job, and I've just left it that way. I see that DBFILE is an
> available pqact.conf action, so I don't know why I couldn't move it into
> a pqact.conf file.
>
> Often times I've wondered about pqsurf and what it's advantage is but
> never found a discussion of it in online ldm documentation. All I have
> is the description I have in printout of the LDM Workshop I attended in
> '98:
>
> pqsurf: "Bursts surface products into products containing individual
> observations".
>
> My margin note says "splits 'bulletins' into reports".
>
> The only use I find in creating a DBFILE output is that using it as
> input to the weather program generates output with only the METAR
> station reports that have a Remarks section. So we don't see both
> reports for that station at that time: the one with Remarks and the one
> without. Since I can find both reports in the output db file, it must be
> a filtering done in Peter's programming for db file processing, because
> if I do a weather program 'FLATMETAR' query from standard metar decoded
> files, the weather program will output both reports, which is something
> we don't want. The speed of db index'd lookups is really not needed
> anymore with the power of 2.x GHz Xeon processors these days, at least
> with these file sizes.
>
>
> On Mon, 2004-09-20 at 16:39, Robb Kambic wrote:
> > hiya,
> >
> > i did a quick grep of the last 3 days for reports that have ".... SPECI"
> > as the first part. the only station that came up was:
> >
> > 2004091718_sao.wmo:SBBE SPECI  ...
> >
> > is this the only station or are there other?  this is not the standard
> > format of according to the WMO 306  on codes.  unidata has been trying to
> > deprecate pqsurf processing because it's a fragile piece of code and it
> > really shouldn't be part of the ldm because it's a decoder. With that
> > being said, is there some other filter to replace the pqsurf code? thus
> > eliminating pqsurf.conf and the pqsurf queue. i would like to open a
> > discussion to find out the available options and the main reasons for
> > using pqsurf.
> >
> > thanks,
> > robb...
> >
> >
> > On Mon, 20 Sep 2004, Kevin R. Tyle wrote:
> >
> > > I have noticed this too; it appears to be a bug in the way METARs
> > > are displayed when read from a METAR file that is in the gdbm database
> > > format.  I have never investigated it further, but have typically
> > > fallen back to using "flatmetar" which reads from METARs written
> > > in ASCII format.  You sacrifice a bit of speed, but make up for
> > > it in the completeness of the data.
> > >
> > > ______________________________________________________________________
> > > Kevin Tyle, Systems Administrator               **********************
> > > Dept. of Earth & Atmospheric Sciences           address@hidden
> > > University at Albany, ES-235                    518-442-4578 (voice)
> > > 1400 Washington Avenue                          518-442-5825 (fax)
> > > Albany, NY 12222                                **********************
> > > ______________________________________________________________________
> > >
> > > On Mon, 20 Sep 2004, Neil R. Smith wrote:
> > >
> > > > All,
> > > > It was pointed out to me recently that our weather program(v4.10) was
> > > > not printing out all SPECI special reports that are available in the
> > > > bulletins.
> > > >
> > > > While looking at both a text and dbfile output from pqsurf, where both
> > > > pqsurf.conf entries have the same header request, and comparing with
> > > > output from weather queries, it seems that reports that are preceded
> > > > with a line that looks like xxxxSPECI, where xxxx is the 4-character
> > > > station id, are not output by METAR, METARDECODE, or METARFULLDECODE,
> > > > whereas those preceded with just a simple line that has only the
> > > > characters SPECI are output by these weather commands.
> > > >
> > > > Is this a known issue with weather, or is it a problem with the DBFILE
> > > > action in the LDM, or perhaps a bad header request entry in pqsurf.conf?
> > > >
> > > > Is there a known solution?
> > > >
> > > > (I can't find a way to search the weatherbud mailing list at unidata, so
> > > > I don't know if this has been treated before.)
> > > >
> > > > Our pqsurf ldmd.conf entry looks like
> > > >
> > > > exec    "pqsurf -p ^S(A....|P....|XUS8.|XUS91) -Q /que/pqsurf.pq
> > > > /unidata/ldm/etc/pqsurf.conf"
> > > >
> > > > Our pqsurf.conf looks like
> > > >
> > > > WMO     ^metar (....) ([0-3][0-9])([0-2][0-9])
> > > >         DBFILE  /data/ddplus/db_metar/(\2:yy)(\2:mm)\2\3.METAR_DB \1
> > > >
> > > > WMO     ^speci (....) ([0-3][0-9])([0-2][0-9])
> > > >         DBFILE  /data/ddplus/db_metar/(\2:yy)(\2:mm)\2\3.METAR_DB \1
> > > >
> > > > Thanks,
> > > > -Neil
> > > > --
> > > > Neil R. Smith, Comp. Sys. Mngr.         address@hidden
> > > > Dept. Atmospheric Sci., Texas A&M Univ. 979/845-6272 FAX:979/862-4466
> > > >
> > > >
> > >
> >
> > ===============================================================================
> > Robb Kambic                            Unidata Program Center
> > Software Engineer III                          Univ. Corp for Atmospheric 
> > Research
> > address@hidden                 WWW: http://www.unidata.ucar.edu/
> > ===============================================================================
> >
> --
> Neil R. Smith, Comp. Sys. Mngr.               address@hidden
> Dept. Atmospheric Sci., Texas A&M Univ.       979/845-6272 FAX:979/862-4466
>

===============================================================================
Robb Kambic                                Unidata Program Center
Software Engineer III                      Univ. Corp for Atmospheric Research
address@hidden             WWW: http://www.unidata.ucar.edu/
===============================================================================


NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publicly available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us.