NOTE: The decoders mailing list is no longer active. The list archives are made available for historical reasons.
On Wed, 7 Apr 2004, Tom Baltzer wrote:
Hi again Charlie, I'm glad to hear that we're seeing the same things now and you've got metar data coming in as netCDF files. You're right that metar2nc is a nontrivial script about which I know little at this juncture. METAR form is pretty darned cryptic, but I know that some information about METAR can be obtained here: http://my.unidata.ucar.edu/content/staff/awilson/public/LEAD/dataDesc/index. html It has references to documents that describe the form in more detail. I've cc'd Rob Kambic, our local decoders expert who developed the metar2nc script. Any insight/suggestions as to how to best/most quickly understand the metar2nc script Robb?
Hiya, Official METAR web site: http://metar.noaa.gov Doesn't include the manual on how to decode a metar, one has to buy the manual. I have a copy of the manual and some other hard copy printed documentation if one would like it. Another method would be to look up METAR decoder using google. Robb...
Thanks, Tom. > -----Original Message----- > From: Charles Moad [mailto:cmoad@xxxxxxxxxxx] > Sent: Tuesday, April 06, 2004 10:43 PM > To: Tom Baltzer > Cc: 'Beth A Plale'; 'Tom Baltzer'; 'Doug Lindholm'; > machrist@xxxxxxxxxxxxxx > Subject: RE: ldm decoder > > I get identical output as well for all the mentioned feedtypes. > They > all say IDS|DDPLUS. > You were right about the missing tab on the first line, so the > syntax > passes now. I am also seeing the netcdf files being created so all > seems to be working well in that aspect. > The challenge we face now is turning these netcdf files into a > stream. > The metar2nc script is not exactly a trivial, so any input as to how it > works might help. We are currently using a c++ api to register a dquob > stream source, but I think we are going to try to wrap this api in perl > and modify the metar2nc script. > Thanks for you help so far. > > - Charlie > > On Tue, 2004-04-06 at 17:02, Tom Baltzer wrote: > > Hi Charlie, > > > > Well that's very interesting. My understanding is that DDPLUS is just a > > combination of DDS and PPS (Public Products Service) so if you're able > to > > get DDPLUS I'd have thought you could get DDS... > > > > I just spoke with a Jeff Weber (a local expert) who tells me that > IDS|DDPLUS > > is a single feed composed of DDS, PPS and IDS and you cannot separate > these > > by feed type. And in fact when I run the two ldmadmin watch -f commands > you > > show (in two separate windows) I get identical results. I even went so > far > > as to try each (PPS, DDS, PPS and DDPLUS) individually in 4 windows and > got > > identical results. What result do you get when you run the ldmadmin > watch > > with the IDS|DDS feedtype? > > > > As far as the error message you're getting from checking the pqact.conf > > file, I suspect that there is yet another TAB required. Check that the > > white space between IDS|DDPLUS and ^S in your pqact.conf file is in fact > a > > tab rather than a space (sorry I didn't clarify this earlier). > > > > Let me know how it goes and/or if I can provide any other help. > > > > Best regards, > > Tom. > > > > > > > -----Original Message----- > > > From: Charles W Moad [mailto:cmoad@xxxxxxxxxxx] > > > Sent: Tuesday, April 06, 2004 1:23 PM > > > To: Tom Baltzer > > > Cc: 'Beth A Plale'; 'Charles W Moad'; 'Tom Baltzer'; 'Doug Lindholm' > > > Subject: RE: ldm decoder > > > > > > As a first step I am trying run the standard pqact | metar2nc and save > > > the data files. When I run "ldmadmin pqactcheck" I get the following > > > error: > > > > > > --- > > > Apr 06 19:20:17 pqact[26196]: feedtype error at line 121: unknown feed > > > name in feedtype expression: "IDS|DDPLUS ^S(A....|P....|XUS8.) .... > > > ([0-3][0-9])" > > > --- > > > > > > I don't think we get the DDS|IDS feed because it does not show up as a > > > valid feedtype for "ldmadmin watch -f "DDS\|IDS"", however "ldmadmin > > > watch -f "IDS\|DDPLUS"" does work. > > > > > > Any help would be greatly appreciated. > > > > > > Thanks, > > > Charlie > > > > > > On Tue, 2004-04-06 at 12:43, Tom Baltzer wrote: > > > > Hi Beth and Charlie, > > > > > > > > It sounds like you have found the Unidata netCDF decoder package. > In my > > > > short time around here I've discovered that there are more decoders > than > > > > just those found in that package and have been working on > documenting > > > those > > > > that I see used by LDM systems around Unidata here: > > > > http://my.unidata.ucar.edu/content/projects/LEAD/Decoders.htm > > > > > > > > The pqact.conf configuration lines for pulling out metar data and > > > placing it > > > > in netcdf files in the ~ldm/data/pub/decoded/netcdf/surface > directory > > > are > > > > (note the first 5 lines are comments): > > > > > > > > # > > > > # metar decoder, creates NetCDF files formatted > > > > # data/decoded/netcdf/surface/yymmddhh_metar.nc > > > > # > > > > # All aviation reports including metar tests > > > > DDS|IDS ^S(A....|P....|XUS8.) .... ([0-3][0-9]) > > > > PIPE decoders/metar2nc > > > > etc/metar.cdl > > > > data/pub/decoded/netcdf/surface > > > > (\2:yy)(\2:mm) > > > > > > > > Be advised that those are <tab>s at the beginning of lines 2-5 (very > > > > important!). Also be advised that it appears the decoder expects > the > > > > directory structure to exist and be writable before it will work. > I've > > > > submitted this to the decoder folks as a fix. > > > > > > > > As Doug mentioned the pqact application will stream the raw data > reports > > > to > > > > stdin and the metar2nc perl script then takes that streaming data > and > > > writes > > > > it to a netCDF file. So if you wished to you could take the > metar2nc > > > script > > > > and modify it to act as a dQUOB interface. Or if dQUOB takes stdin > then > > > you > > > > could use it to help you parse the incoming metar data. My > understanding > > > is > > > > that you all have been trying to access the LDM product queue to do > your > > > > dQUOB work. It seems that using pqact/decoder type functions would > be > > > much > > > > easier and effectively the same. > > > > > > > > Well I hope this helps. Feel free to contact me directly with > questions > > > or > > > > comments. > > > > > > > > Cheers, > > > > Tom. > > > > > > > > Tom Baltzer > > > > Software Engineer > > > > UCAR Office of Programs > > > > Unidata Program Center > > > > P.O. Box 3000 > > > > Boulder, CO 80307-3000 > > > > tbaltzer@xxxxxxxx > > > > 303-497-8637 > > > > > > > > > > > > > -----Original Message----- > > > > > From: Doug Lindholm [mailto:lind@xxxxxxxx] > > > > > Sent: Friday, April 02, 2004 11:52 AM > > > > > To: Beth A Plale > > > > > Cc: Charles W Moad; Tom Baltzer > > > > > Subject: Re: ldm decoder > > > > > > > > > > Hi Beth, > > > > > > > > > > Tom is spearheading the LEAD related LDM and decoders setup here. > I'm > > > > > CC-ing him. > > > > > > > > > > Do you want to work with data streamed from the LDM or decoded > files? > > > I > > > > > suspect you want a decoded stream, though I don't know of any > decoders > > > > > that do that (but then by breadth of decoder knowledge is > limited). > > > > > > > > > > Since I don't understand the nature of dQUOB I'm not sure which > data > > > > > product would provide the path of least resistance. METAR data is > > > simple > > > > > conceptually but the decoding of the raw reports isn't trivial. > You > > > > > could modify the metar2nc decoder (perl script) to spit out > whatever > > > > > dQUOB wants. > > > > > > > > > > BTW, the METAR data comes over the IDS|DDPLUS feed. I'll let Tom > help > > > > > you figure out how to modify the pqact.conf file to redirect the > METAR > > > > > data. > > > > > > > > > > Let me know if you have any questions. There are a lot of > resources at > > > > > Unidata we could point you to. > > > > > > > > > > Cheers, > > > > > Doug > > > > > > > > > > Beth A Plale wrote: > > > > > > Hi Doug, > > > > > > > > > > > > My student, Charlie, is looking at piping data to a process, as > you > > > > > > described it last week, where it will be decoded then used in > dQUOB. > > > > > > We're looking for a good data product to use, and by "good" I > more > > > mean > > > > > > that we're getting it via our LDM peer, it is relatively simple, > and > > > > > that > > > > > > there's a decoder for it. What would you suggest? The data > > > products > > > > > > we're receiving are below. > > > > > > > > > > > > thanks! > > > > > > > > > > > > beth > > > > > > > > > > > > > > > > > >>I don't see METAR data in the queue. I am running "ldmadmin > watch", > > > and > > > > > >>I only see CRAFT, NNEXRAD, IDS|DDPLUS, CONDUIT, HDS. > > > > > >> > > > > > >>- Charlie > > > > > > > > > > > > > > > > > >>> I think I found the decoder on the unidata web site. They > > > have > > > > > netcdf > > > > > >>>decoders for GRIB, METAR, Upperair, Synoptic, Buoy, Ascii, and > > > > > >>>ldmConnect. What data do you want me to pull from the ldm > queue? > > > All > > > > > >>>of it? > > > > > >>> > > > > > >>>- Charlie > > > > > >>> > > > > > >>> > > > > > > > > > > -- > > > > > *----------------------------------------------------------------- > ---- > > > -* > > > > > | Doug Lindholm, Software Engineer | E-mail: > lind@xxxxxxxx > > > | > > > > > | Unidata Program Center | Phone: 303-497- > 8659 > > > | > > > > > | UCAR Office of Programs | > > > | > > > > > | P.O. Box 3000 | There's no place > > > | > > > > > | Boulder, Colorado 80307-3000 | like $HOME > > > | > > > > > *----------------------------------------------------------------- > ---- > > > -* > > > > > >
============================================================================== Robb Kambic Unidata Program Center Software Engineer III Univ. Corp for Atmospheric Research rkambic@xxxxxxxxxxxxxxxx WWW: http://www.unidata.ucar.edu/ ==============================================================================
decoders archives: