[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[LDM #MIY-170840]: One of our two LDM servers is having problems

Hi Tom,

re:
> I am confused,

Join the group ;-:

re:
> latency seems to only be high to IDS|DDPLUS, but it’s fine to FNEXRAD and
> NIMAGE.

Hmm... this really is strange.  Our experience is that IDS|DDPLUS latencies
will be low when higher volume feed latencies are high when there is some
sort of artificial bandwidth limiting being imposed.  This "rule of thumb"
does not fit when the other feeds have low data volumes like FNEXRAD and
NIMAGE.

re:
> I have checked everything I can possibly think of on the server and can’t find
> anything wrong.
> 
> Any thoughts or ideas?

The only thing I can that is different is the number of products in the
IDS|DDPLUS data feed is MUCH higher than the number in either FNEXRAD or
NIMAGE:

http://rtstats.unidata.ucar.edu/cgi-bin/rtstats/rtstats_summary_volume?colaweb.gmu.edu

Data Volume Summary for colaweb.gmu.edu

Maximum hourly volume    280.377 M bytes/hour
Average hourly volume    169.992 M bytes/hour

Average products per hour      19735 prods/hour

Feed                           Average             Maximum     Products
                     (M byte/hour)            (M byte/hour)   number/hour
NIMAGE                  102.925    [ 60.547%]      182.564       37.000
IDS|DDPLUS               36.315    [ 21.363%]       65.906    19688.083
FNEXRAD                  30.751    [ 18.090%]       39.196        9.667

I imagine that some sort of "packet shaping" could be limiting the number
of products in a feed, but this is not how sites typically have their
systems setup.

Can you send us (as attachments, not as cut and pastes of listings):

~ldm/etc/ldmd.conf
~ldm/etc/registry.xml
~ldm/etc/pqact.conf

The reason I am asking for these is to see if the number of slots in the
LDM queue is way to small to handle the number of products in the IDS|DDPLUS
feed

Comment: this does _not_ feel like a realistic possibility, but...

Another thing that would help troubleshoot your LDM setup would be if
you would run the metrics gathering cron jobs:

# LDM metrics gathering
* * * * * bin/ldmadmin addmetrics > /dev/null
0 0 1 * * bin/ldmadmin newmetrics > /dev/null

These will create files named metrics.txt in the directory specified
in the <metrics></metrics> setting in the LDM registry, ~ldm/etc/registry.xml.
The information logged will show if one's LDM queue is product limited or
size limited.

re:
> All other network connectivity seems to be working as expected.

This really is weird!

Cheers,

Tom
--
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: MIY-170840
Department: Support LDM
Priority: Normal
Status: Closed
===================
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata 
inquiry tracking system and then made publicly available through the web.  If 
you do not want to have your interactions made available in this way, you must 
let us know in each email you send to us.