[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

RE: ldm decoder

On Wed, 7 Apr 2004, Tom Baltzer wrote:

> Hi again Charlie,
>
> I'm glad to hear that we're seeing the same things now and you've got metar
> data coming in as netCDF files.
>
> You're right that metar2nc is a nontrivial script about which I know little
> at this juncture.  METAR form is pretty darned cryptic, but I know that some
> information about METAR can be obtained here:
>
> http://my.unidata.ucar.edu/content/staff/awilson/public/LEAD/dataDesc/index.
> html
>
> It has references to documents that describe the form in more detail.
>
> I've cc'd Rob Kambic, our local decoders expert who developed the metar2nc
> script. Any insight/suggestions as to how to best/most quickly understand
> the metar2nc script Robb?

Hiya,

Official METAR web site: http://metar.noaa.gov

Doesn't include the manual on how to decode a metar, one has to buy the
manual. I have a copy of the manual and some other hard copy printed
documentation if one would like it.

Another method would be to look up METAR decoder using google.


Robb...



>
> Thanks,
> Tom.
>
> > -----Original Message-----
> > From: Charles Moad [mailto:address@hidden]
> > Sent: Tuesday, April 06, 2004 10:43 PM
> > To: Tom Baltzer
> > Cc: 'Beth A Plale'; 'Tom Baltzer'; 'Doug Lindholm';
> > address@hidden
> > Subject: RE: ldm decoder
> >
> >     I get identical output as well for all the mentioned feedtypes.
> > They
> > all say IDS|DDPLUS.
> >     You were right about the missing tab on the first line, so the
> > syntax
> > passes now.  I am also seeing the netcdf files being created so all
> > seems to be working well in that aspect.
> >     The challenge we face now is turning these netcdf files into a
> > stream.
> > The metar2nc script is not exactly a trivial, so any input as to how it
> > works might help.  We are currently using a c++ api to register a dquob
> > stream source, but I think we are going to try to wrap this api in perl
> > and modify the metar2nc script.
> >     Thanks for you help so far.
> >
> > - Charlie
> >
> > On Tue, 2004-04-06 at 17:02, Tom Baltzer wrote:
> > > Hi Charlie,
> > >
> > > Well that's very interesting.  My understanding is that DDPLUS is just a
> > > combination of DDS and PPS (Public Products Service) so if you're able
> > to
> > > get DDPLUS I'd have thought you could get DDS...
> > >
> > > I just spoke with a Jeff Weber (a local expert) who tells me that
> > IDS|DDPLUS
> > > is a single feed composed of DDS, PPS and IDS and you cannot separate
> > these
> > > by feed type. And in fact when I run the two ldmadmin watch -f commands
> > you
> > > show (in two separate windows) I get identical results. I even went so
> > far
> > > as to try each (PPS, DDS, PPS and DDPLUS) individually in 4 windows and
> > got
> > > identical results.  What result do you get when you run the ldmadmin
> > watch
> > > with the IDS|DDS feedtype?
> > >
> > > As far as the error message you're getting from checking the pqact.conf
> > > file, I suspect that there is yet another TAB required.  Check that the
> > > white space between IDS|DDPLUS and ^S in your pqact.conf file is in fact
> > a
> > > tab rather than a space (sorry I didn't clarify this earlier).
> > >
> > > Let me know how it goes and/or if I can provide any other help.
> > >
> > > Best regards,
> > > Tom.
> > >
> > >
> > > > -----Original Message-----
> > > > From: Charles W Moad [mailto:address@hidden]
> > > > Sent: Tuesday, April 06, 2004 1:23 PM
> > > > To: Tom Baltzer
> > > > Cc: 'Beth A Plale'; 'Charles W Moad'; 'Tom Baltzer'; 'Doug Lindholm'
> > > > Subject: RE: ldm decoder
> > > >
> > > > As a first step I am trying run the standard pqact | metar2nc and save
> > > > the data files.  When I run "ldmadmin pqactcheck" I get the following
> > > > error:
> > > >
> > > > ---
> > > > Apr 06 19:20:17 pqact[26196]: feedtype error at line 121: unknown feed
> > > > name in feedtype expression: "IDS|DDPLUS ^S(A....|P....|XUS8.) ....
> > > > ([0-3][0-9])"
> > > > ---
> > > >
> > > > I don't think we get the DDS|IDS feed because it does not show up as a
> > > > valid feedtype for "ldmadmin watch -f "DDS\|IDS"", however "ldmadmin
> > > > watch -f "IDS\|DDPLUS"" does work.
> > > >
> > > > Any help would be greatly appreciated.
> > > >
> > > > Thanks,
> > > >         Charlie
> > > >
> > > > On Tue, 2004-04-06 at 12:43, Tom Baltzer wrote:
> > > > > Hi Beth and Charlie,
> > > > >
> > > > > It sounds like you have found the Unidata netCDF decoder package.
> > In my
> > > > > short time around here I've discovered that there are more decoders
> > than
> > > > > just those found in that package and have been working on
> > documenting
> > > > those
> > > > > that I see used by LDM systems around Unidata here:
> > > > > http://my.unidata.ucar.edu/content/projects/LEAD/Decoders.htm
> > > > >
> > > > > The pqact.conf configuration lines for pulling out metar data and
> > > > placing it
> > > > > in netcdf files in the ~ldm/data/pub/decoded/netcdf/surface
> > directory
> > > > are
> > > > > (note the first 5 lines are comments):
> > > > >
> > > > > #
> > > > > # metar decoder, creates NetCDF files formatted
> > > > > #    data/decoded/netcdf/surface/yymmddhh_metar.nc
> > > > > #
> > > > > # All aviation reports including metar tests
> > > > > DDS|IDS ^S(A....|P....|XUS8.) .... ([0-3][0-9])
> > > > >         PIPE    decoders/metar2nc
> > > > >         etc/metar.cdl
> > > > >         data/pub/decoded/netcdf/surface
> > > > >         (\2:yy)(\2:mm)
> > > > >
> > > > > Be advised that those are <tab>s at the beginning of lines 2-5 (very
> > > > > important!).  Also be advised that it appears the decoder expects
> > the
> > > > > directory structure to exist and be writable before it will work.
> > I've
> > > > > submitted this to the decoder folks as a fix.
> > > > >
> > > > > As Doug mentioned the pqact application will stream the raw data
> > reports
> > > > to
> > > > > stdin and the metar2nc perl script then takes that streaming data
> > and
> > > > writes
> > > > > it to a netCDF file.  So if you wished to you could take the
> > metar2nc
> > > > script
> > > > > and modify it to act as a dQUOB interface. Or if dQUOB takes stdin
> > then
> > > > you
> > > > > could use it to help you parse the incoming metar data. My
> > understanding
> > > > is
> > > > > that you all have been trying to access the LDM product queue to do
> > your
> > > > > dQUOB work.  It seems that using pqact/decoder type functions would
> > be
> > > > much
> > > > > easier and effectively the same.
> > > > >
> > > > > Well I hope this helps.  Feel free to contact me directly with
> > questions
> > > > or
> > > > > comments.
> > > > >
> > > > > Cheers,
> > > > > Tom.
> > > > >
> > > > > Tom Baltzer
> > > > > Software Engineer
> > > > > UCAR Office of Programs
> > > > > Unidata Program Center
> > > > > P.O. Box 3000
> > > > > Boulder, CO 80307-3000
> > > > > address@hidden
> > > > > 303-497-8637
> > > > >
> > > > >
> > > > > > -----Original Message-----
> > > > > > From: Doug Lindholm [mailto:address@hidden]
> > > > > > Sent: Friday, April 02, 2004 11:52 AM
> > > > > > To: Beth A Plale
> > > > > > Cc: Charles W Moad; Tom Baltzer
> > > > > > Subject: Re: ldm decoder
> > > > > >
> > > > > > Hi Beth,
> > > > > >
> > > > > > Tom is spearheading the LEAD related LDM and decoders setup here.
> > I'm
> > > > > > CC-ing him.
> > > > > >
> > > > > > Do you want to work with data streamed from the LDM or decoded
> > files?
> > > > I
> > > > > > suspect you want a decoded stream, though I don't know of any
> > decoders
> > > > > > that do that (but then by breadth of decoder knowledge is
> > limited).
> > > > > >
> > > > > > Since I don't understand the nature of dQUOB I'm not sure which
> > data
> > > > > > product would provide the path of least resistance. METAR data is
> > > > simple
> > > > > > conceptually but the decoding of the raw reports isn't trivial.
> > You
> > > > > > could modify the metar2nc decoder (perl script) to spit out
> > whatever
> > > > > > dQUOB wants.
> > > > > >
> > > > > > BTW, the METAR data comes over the IDS|DDPLUS feed. I'll let Tom
> > help
> > > > > > you figure out how to modify the pqact.conf file to redirect the
> > METAR
> > > > > > data.
> > > > > >
> > > > > > Let me know if you have any questions. There are a lot of
> > resources at
> > > > > > Unidata we could point you to.
> > > > > >
> > > > > > Cheers,
> > > > > > Doug
> > > > > >
> > > > > > Beth A Plale wrote:
> > > > > > > Hi Doug,
> > > > > > >
> > > > > > > My student, Charlie, is looking at piping data to a process, as
> > you
> > > > > > > described it last week, where it will be decoded then used in
> > dQUOB.
> > > > > > > We're looking for a good data product to use, and by "good" I
> > more
> > > > mean
> > > > > > > that we're getting it via our LDM peer, it is relatively simple,
> > and
> > > > > > that
> > > > > > > there's a decoder for it.  What would you suggest?  The data
> > > > products
> > > > > > > we're receiving are below.
> > > > > > >
> > > > > > > thanks!
> > > > > > >
> > > > > > > beth
> > > > > > >
> > > > > > >
> > > > > > >>I don't see METAR data in the queue.  I am running "ldmadmin
> > watch",
> > > > and
> > > > > > >>I only see CRAFT, NNEXRAD, IDS|DDPLUS, CONDUIT, HDS.
> > > > > > >>
> > > > > > >>- Charlie
> > > > > > >
> > > > > > >
> > > > > > >>> I think I found the decoder on the unidata web site.  They
> > > > have
> > > > > > netcdf
> > > > > > >>>decoders for GRIB, METAR, Upperair, Synoptic, Buoy, Ascii, and
> > > > > > >>>ldmConnect.  What data do you want me to pull from the ldm
> > queue?
> > > > All
> > > > > > >>>of it?
> > > > > > >>>
> > > > > > >>>- Charlie
> > > > > > >>>
> > > > > > >>>
> > > > > >
> > > > > > --
> > > > > > *-----------------------------------------------------------------
> > ----
> > > > -*
> > > > > > | Doug Lindholm, Software Engineer          |  E-mail:
> > address@hidden
> > > > |
> > > > > > | Unidata Program Center                    |   Phone: 303-497-
> > 8659
> > > > |
> > > > > > | UCAR Office of Programs                   |
> > > > |
> > > > > > | P.O. Box 3000                             |     There's no place
> > > > |
> > > > > > | Boulder, Colorado 80307-3000              |        like $HOME
> > > > |
> > > > > > *-----------------------------------------------------------------
> > ----
> > > > -*
> > > > >
> > >
>
>

===============================================================================
Robb Kambic                                Unidata Program Center
Software Engineer III                      Univ. Corp for Atmospheric Research
address@hidden             WWW: http://www.unidata.ucar.edu/
===============================================================================