[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: 20051115:missing HDS parameters

Wen,

If you are missing decoded fields in your file, and your data volume
and latency for the GDS feed checks out, then the likely problem is that
you are not processing the data out of the queue by pqact before
it gets overwritten by new data. 

What is the size of your queue?
You are receiving approximately 1.2GB an hour on average, and up
to 1.8GB per hour during the hours of maximum model data. If your
queue is smaller than 2GB, that would be the first suspect.

The pqmon program from LDM will tell you what the age of the oldest
product is in your queue. That combined with the amount of IO
you may have with the level2 radar feed might cause the pqact processing
to be slower than is needed.

Are you running separate pqact processes for the GEMPAK decoders apart
from other pqact processing of the Level2 radar? The gen_pqact.csh
script I provide in $NAWIPS/ldm/etc asks whether you want a single
or multiple pqact files, and will generate separate pqact files
and provide you with the suggested ldmd.conf entries as needed.

Steve Chiswell
Unidata User Support




On Tue, 2005-11-15 at 14:38, Jeff Weber wrote:
> Hi Wen,
> 
> I suspect it is a table or reg exp issue, as you are running a fairly old
> version of GEMPAK...
> 
> The latest version is 5.8.4
> 
> I will pass this by our GEMPAK developer to make sure there are not other
> issues at play.
> 
> I checked the volume of your data ingest on the HDS (HRS) feed and it
> matched identically with ours, so the data is making it to your queue, how
> it is being processed from there is where a problem may exist.
> 
> Cheers,
> 
> Jeff
> ---------------------------------------------------------------------
> Jeff Weber                                    address@hidden        :
> Unidata Program Center                        PH:303-497-8676        :
> University Corp for Atmospheric Research      3300 Mitchell Ln       :
> http://www.unidata.ucar.edu/staff/jweber      Boulder,Co 80307-3000  :
> ---------------------------------------------------------------------
> 
> On Tue, 15 Nov 2005, Lu wrote:
> 
> >
> > Hi Jeff,
> >
> > I believe it is version 5.6
> >
> > Wen
> >
> >
> > On Tue, 15 Nov 2005, Jeff Weber wrote:
> >
> > > Hi Wen,
> > >
> > > What version of GEMPAK are you running?
> > >
> > > Thank you,
> > >
> > > Jeff
> > > ---------------------------------------------------------------------
> > > Jeff Weber                                    address@hidden        :
> > > Unidata Program Center                        PH:303-497-8676        :
> > > University Corp for Atmospheric Research      3300 Mitchell Ln       :
> > > http://www.unidata.ucar.edu/staff/jweber      Boulder,Co 80307-3000  :
> > > ---------------------------------------------------------------------
> > >
> > > On Mon, 14 Nov 2005, Lu wrote:
> > >
> > > >
> > > > Hi Jeff,
> > > >
> > > > Let's specify to talk about 211 grids.
> > > > We receive files for
> > > >
> > > > ETA, 2 files daily at hours 00 and 12
> > > > RUC, 24 files, one file per hour
> > > > NGM, 2 files daily at hours 00 and 12
> > > > NOGAPS, 4 files a day at hours 00,06,12,18, without specifing the grid
> > > >
> > > > Also let's display pressure at level 500 of images Nov 13 and 14, errors
> > > > for:
> > > >
> > > > ETA (41 images in all)
> > > > GEMPAK: [DG -7]  Input grid TMPC ^051113/0000F012 @500 %PRES cannot be 
> > > > found.
> > > > GEMPAK: [DG -7]  Input grid TMPC ^051113/0000F042 @500 %PRES cannot be 
> > > > found.
> > > > GEMPAK: [DG -7]  Input grid TMPC ^051113/0000F054 @500 %PRES cannot be 
> > > > found.
> > > > GEMPAK: [DG -7]  Input grid TMPC ^051113/1200F018 @500 %PRES cannot be 
> > > > found
> > > > GEMPAK: [DG -7]  Input grid TMPC ^051113/1200F030 @500 %PRES cannot be 
> > > > found.
> > > > GEMPAK: [DG -7]  Input grid TMPC ^051113/1200F042 @500 %PRES cannot be 
> > > > found.
> > > > GEMPAK: [DG -7]  Input grid TMPC ^051114/0000F012 @500 %PRES cannot be 
> > > > found.
> > > > GEMPAK: [DG -7]  Input grid TMPC ^051114/0000F024 @500 %PRES cannot be 
> > > > found
> > > > GEMPAK: [DG -7]  Input grid TMPC ^051114/0000F054 @500 %PRES cannot be 
> > > > found.
> > > > GEMPAK: [DG -7]  Input grid TMPC ^051114/1200F030 @500 %PRES cannot be 
> > > > found.
> > > > GEMPAK: [DG -7]  Input grid TMPC ^051114/1200F036 @500 %PRES cannot be 
> > > > found.
> > > >
> > > > RUC ( 72 images for Nov 13, 60 till 1900UTC
> > > > GEMPAK: [DG -7]  Input grid TMPC ^051113/0700F003 @500 %PRES cannot be 
> > > > found.
> > > > GEMPAK: [DG -7]  Input grid TMPC ^051114/0200F003 @500 %PRES cannot be 
> > > > found.
> > > >
> > > > NGM ( 36 images in all)
> > > > GEMPAK: [DG -7]  Input grid TMPC ^051113/1200F036 @500 %PRES cannot be 
> > > > found.
> > > > GEMPAK: [DG -7]  Input grid TMPC ^051114/0000F018 @500 %PRES cannot be 
> > > > found.
> > > > GEMPAK: [DG -7]  Input grid TMPC ^051114/1200F042 @500 %PRES cannot be 
> > > > found.
> > > >
> > > >
> > > > NOGAPS (124 images Nov 13,
> > > > GEMPAK: [DG -7]  Input grid WND ^051113/0000F051 @500 %PRES in AVOR 
> > > > cannot be found.
> > > > GEMPAK: [DG -7]  Input grid WND ^051113/0000F075 @500 %PRES in AVOR 
> > > > cannot be found.
> > > > GEMPAK: [DG -7]  Input grid WND ^051113/0000F081 @500 %PRES in AVOR 
> > > > cannot be found.
> > > > GEMPAK: [DG -7]  Input grid WND ^051113/0000F120 @500 %PRES in AVOR 
> > > > cannot be found.
> > > > GEMPAK: [DG -7]  Input grid WND ^051113/0000F132 @500 %PRES in AVOR 
> > > > cannot be found.
> > > > GEMPAK: [DG -7]  Input grid WND ^051113/0600F078 @500 %PRES in AVOR 
> > > > cannot be found.
> > > > GEMPAK: [DG -7]  Input grid WND ^051113/0600F090 @500 %PRES in AVOR 
> > > > cannot be found.
> > > > GEMPAK: [DG -7]  Input grid WND ^051113/1200F051 @500 %PRES in AVOR 
> > > > cannot be found.
> > > > GEMPAK: [DG -7]  Input grid WND ^051113/1200F075 @500 %PRES in AVOR 
> > > > cannot be found.
> > > > GEMPAK: [DG -7]  Input grid WND ^051113/1200F081 @500 %PRES in AVOR 
> > > > cannot be found.
> > > > GEMPAK: [DG -7]  Input grid WND ^051113/1200F084 @500 %PRES in AVOR 
> > > > cannot be found.
> > > > GEMPAK: [DG -7]  Input grid WND ^051113/1200F093 @500 %PRES in AVOR 
> > > > cannot be found.
> > > > GEMPAK: [DG -7]  Input grid WND ^051113/1200F132 @500 %PRES in AVOR 
> > > > cannot be found.
> > > > GEMPAK: [DG -7]  Input grid WND ^051113/1200F144 @500 %PRES in AVOR 
> > > > cannot be found.
> > > > GEMPAK: [DG -7]  Input grid WND ^051113/1800F078 @500 %PRES in AVOR 
> > > > cannot be found.
> > > > GEMPAK: [DG -7]  Input grid WND ^051113/1800F090 @500 %PRES in AVOR 
> > > > cannot be found.
> > > >
> > > > GEMPAK: [DG -7]  Input grid WND ^051114/0000F051 @500 %PRES in AVOR 
> > > > cannot be found.
> > > > GEMPAK: [DG -7]  Input grid WND ^051114/0000F066 @500 %PRES in AVOR 
> > > > cannot be found.
> > > > GEMPAK: [DG -7]  Input grid WND ^051114/0000F069 @500 %PRES in AVOR 
> > > > cannot be found.
> > > > GEMPAK: [DG -7]  Input grid WND ^051114/0000F072 @500 %PRES in AVOR 
> > > > cannot be found.
> > > > GEMPAK: [DG -7]  Input grid WND ^051114/0000F075 @500 %PRES in AVOR 
> > > > cannot be found.
> > > > GEMPAK: [DG -7]  Input grid WND ^051114/0000F078 @500 %PRES in AVOR 
> > > > cannot be found.
> > > > GEMPAK: [DG -7]  Input grid WND ^051114/0000F081 @500 %PRES in AVOR 
> > > > cannot be found.
> > > > GEMPAK: [DG -7]  Input grid WND ^051114/0000F108 @500 %PRES in AVOR 
> > > > cannot be found.
> > > > GEMPAK: [DG -7]  Input grid WND ^051114/0000F132 @500 %PRES in AVOR 
> > > > cannot be found.
> > > > GEMPAK: [DG -7]  Input grid WND ^051114/0600F078 @500 %PRES in AVOR 
> > > > cannot be found.
> > > > GEMPAK: [DG -7]  Input grid WND ^051114/0600F090 @500 %PRES in AVOR 
> > > > cannot be found.
> > > > GEMPAK: [DG -7]  Input grid WND ^051114/1200F051 @500 %PRES in AVOR 
> > > > cannot be found.
> > > > GEMPAK: [DG -7]  Input grid WND ^051114/1200F075 @500 %PRES in AVOR 
> > > > cannot be found.
> > > > GEMPAK: [DG -7]  Input grid WND ^051114/1200F081 @500 %PRES in AVOR 
> > > > cannot be found.
> > > > GEMPAK: [DG -7]  Input grid WND ^051114/1200F090 @500 %PRES in AVOR 
> > > > cannot be found.
> > > > GEMPAK: [DG -7]  Input grid WND ^051114/1200F093 @500 %PRES in AVOR 
> > > > cannot be found.
> > > > GEMPAK: [DG -7]  Input grid WND ^051114/1200F114 @500 %PRES in AVOR 
> > > > cannot be found.
> > > >
> > > > Attached please find are our files of ldmm.conf, pqact.conf, 
> > > > pqact.gempak,
> > > > and pqact.nexrad
> > > >
> > > > Thanks,
> > > >
> > > > Wen
> > > >
> > > >
> > > > On Mon, 14 Nov 2005, Jeff Weber wrote:
> > > >
> > > > > Hi Wen,
> > > > >
> > > > > What grids (model numbers) of the ETA (NAM), RUC, NGM, MRF (GFS), and
> > > > > NOGAPS are you referring to?
> > > > >
> > > > > Are these all the 211 (CONUS) grids?
> > > > >
> > > > > Some models have 6 hour runs, while other are every 12...and NOGAPS, 
> > > > > is
> > > > > spotty with some parameters..
> > > > >
> > > > > No worries about rtstats, that will not affect your data reception, 
> > > > > but
> > > > > thank you for the heads up...I will check on that.
> > > > >
> > > > > Please send me your ldmd.conf if you can, and your pqact.conf files.
> > > > >
> > > > > Also, if you could be more specific about what parameters are missing 
> > > > > and
> > > > > when, and if there is a repeatable pattern to these data gaps, so we 
> > > > > can
> > > > > determine if in fact you are missing data, or if it is in fact a 
> > > > > reality
> > > > > in data availability.
> > > > >
> > > > > Cheers,
> > > > >
> > > > > Jeff
> > > > > ---------------------------------------------------------------------
> > > > > Jeff Weber                                    address@hidden        :
> > > > > Unidata Program Center                        PH:303-497-8676        :
> > > > > University Corp for Atmospheric Research      3300 Mitchell Ln       :
> > > > > http://www.unidata.ucar.edu/staff/jweber      Boulder,Co 80307-3000  :
> > > > > ---------------------------------------------------------------------
> > > > >
> > > > > On Mon, 14 Nov 2005, Lu wrote:
> > > > >
> > > > > >
> > > > > > Hi Jeff,
> > > > > >
> > > > > > I have a couple quentions for you.
> > > > > > Recently, quite many model images of ETA, RUC, NGM, MRF, and NOGAPS
> > > > > > are missing from our data streaming. We received whole files of 
> > > > > > these
> > > > > > models, but we could not display all images of a type (eg. @500 
> > > > > > %pres)
> > > > > > in its entire time sequence: always missing some of them.
> > > > > > Our ldmd.log file shows that we could not connect
> > > > > > rtstats.unidata.ucar.edu. We could not figure it out. Was it the 
> > > > > > machine
> > > > > > down sometimes?
> > > > > > I gratefully appreciate it if you could provide your suggestions.
> > > > > >
> > > > > > Wen
> > > > > >
> > > > >
> > >
> >