[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: 20000727: Raw Radar reports


Chris,

You can filter several ways.

Currently, you are using WMO as your type in the pqact.conf pattern
If you only want text, then you can use DDPLUS. The NIDS and most RCM products
will be binary and come across in the HDS stream.

You can also use the /pxxxxxx pil tags. The ROBS will match with /pROB,
the rcm's with /pRCM. The nids have their own pills too.

I believe the filter needs to be in place for the top level feeds.

Steve Chiswell
Unidata User SUpport


On Thu, 27 Jul 2000, C. Vandersip wrote:

> Steve,  
> 
> One example...~23MB file size:
> 
> 22824107 Jul 25 18:14 2000072520.RAD
> 
> Result of grep SDUS5:
> 
> SDUS53 KJKL 252007
> SDUS53 KJKL 252007
> SDUS53 KJKL 252007
> SDUS53 KJKL 252007
> SDUS53 KJKL 252007
> SDUS53 KJKL 252007
> SDUS53 KJKL 252016
> SDUS53 KJKL 252016
> SDUS53 KJKL 252016
> SDUS53 KJKL 252016
> SDUS53 KJKL 252016
> SDUS53 KJKL 252016
> SDUS53 KJKL 252016
> SDUS53 KJKL 252016
> SDUS53 KJKL 252026
> SDUS53 KJKL 252026
>  and so on, and so on...
> 
> I guess I need to filter out SDUS5* reports in the pqact.conf to resolve
> this, or will yo do this at top-level? Actually, for the text files for
> weather, do I just need to bring in the SDUS4* headers?  Sorry, all these
> SDUS header descriptions are making me dizzy :).
> 
> Chris
> 
> On Thu, 27 Jul 2000, Steve Chiswell wrote:
> 
> > 
> > 
> > 
> > 
> > Chris,
> > 
> > In order to improve reliability for outages (such as we had this week),
> > Robb Kambic and Jeff Weber have been working to add weather underground's
> > noaaport reception into the top level ring of NOAAport receivers. This
> > will give more backup/failover choices.
> > 
> > My guess is that we need to filter out the SDUS5x products which
> > we have done for the rest of the top level sites. These are the encrypted
> > nids products that you can't view yet.
> > 
> > You might want to:
> > grep '[A-Z][A-Z][A-Z][A-Z][0-9].* [A-Z][A-Z][A-Z][A-Z] [0-3][0-9][0-2][0-9]'
> > on your .RAD files to see what headers are in the big files.
> > 
> > This will probably see a lot of SDUS8x products which are DPA products
> > but should be very small. The SDUS4x prods are the RCM and ROB.
> > 
> > If you see SDUS5x's, then we have to get these out of the data stream since
> > many sites don't have the bandwidth to handle these.
> > But...they will eventually be unencrypted so we look forward to
> > providing these to everyone! Probably as a new feedtype so that users
> > can be more selective on what they get.
> > 
> > Let me know what you find for headers.
> > 
> > Steve Chiswell
> > Unidata User Support
> > 
> > On Thu, 27 Jul 2000, C. Vandersip wrote:
> > 
> > > Tonight, I had a problem with a data disk filling up, and found out the
> > > culprit was the raw radar reports (MDR- with SDUS header) we ingest for
> > > use in the "weather" program.  Though these files usually average a few MB
> > > in size, some of them have been tens to hundreds of MB in size recently.  
> > > Is this just due to the amount of radar-detected convection, or has there
> > > been some change in the files being fed?  Thanks in advance for any clues
> > > anyone can provide.
> > > 
> > > Chris
> > > 
> > > Relevant pqact.conf section (now commented out! :) )
> > > 
> > >                         -- RADAR Raw Reports --
> > > #
> > > #WMO    ^SDUS.* .... ([0-3][0-9])([0-2][0-9])
> > > #       FILE    data/weather/RAD/(\1:yyyy)(\1:mm)\1\2.RAD
> > > 
> > > 
> > >    ###############################################################
> > >    #                      Chris Vandersip                        #
> > >    #        Computer Research Specialist/Dept. Sysadmin          #
> > >    #  Rm. 024, Dept. of Meteorology, Florida State University    #
> > >    #          address@hidden   (850)644-2522                     #
> > >    ###############################################################
> > > 
> > > 
> > > 
> > 
> 
>        ###############################################################
>        #                      Chris Vandersip                        #
>        #        Computer Research Specialist/Dept. Sysadmin          #
>        #  Rm. 024, Dept. of Meteorology, Florida State University    #
>        #          address@hidden   (850)644-2522                     #
>        ###############################################################
> 
>