[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[LDM #SVU-966060]: LDM - LDM hangs up.



> Interesting results:
> Proto Recv-Q Send-Q Local Address               Foreign Address
> State       PID/Program name
> tcp        0      0 *:388                       *:*
> LISTEN      26370/rpc.ldmd
> tcp        0      0 fetch1.ops.nwsar.gov:41107  intranet1.arh.nwsar.gov:388
> ESTABLISHED 26448/rpc.ldmd
> tcp        0      0 fetch1.ops.nwsar.gov:388    jnuweb1.jnu.nwsar.gov:47180
> ESTABLISHED 5341/rpc.ldmd
> tcp        0      0 fetch1.ops.nwsar.gov:388    jnuweb1.jnu.nwsar.gov:47179
> ESTABLISHED 5260/rpc.ldmd
> tcp        0      0 fetch1.ops.nwsar.gov:388    jnuweb1.jnu.nwsar.gov:47178
> ESTABLISHED 5259/rpc.ldmd
> tcp        0      0 fetch1.ops.nwsar.gov:388    intranet1.arh.nwsar.g:55469
> ESTABLISHED 30777/rpc.ldmd
> tcp        0      0 fetch1.ops.nwsar.gov:388    intranet1.arh.nwsar.g:55468
> ESTABLISHED 30776/rpc.ldmd
> tcp        0      0 fetch1.ops.nwsar.gov:388    intranet1.arh.nwsar.g:55470
> ESTABLISHED 30778/rpc.ldmd
> tcp        0      0 fetch1.ops.nwsar.gov:388    arhdata.arh.nwsar.gov:2396
> ESTABLISHED 6047/rpc.ldmd
> tcp        0      0 fetch1.ops.nwsar.gov:388    intranet1.arh.nwsar.g:55484
> ESTABLISHED 1218/rpc.ldmd
> tcp        0      0 fetch1.ops.nwsar.gov:388    intranet1.arh.nwsar.g:55481
> ESTABLISHED 961/rpc.ldmd
> tcp        0      0 fetch1.ops.nwsar.gov:388    intranet1.arh.nwsar.g:55480
> ESTABLISHED 960/rpc.ldmd
> tcp        0      0 fetch1.ops.nwsar.gov:388    intranet1.arh.nwsar.g:55483
> ESTABLISHED 1035/rpc.ldmd
> tcp        0      0 fetch1.ops.nwsar.gov:388    intranet1.arh.nwsar.g:55482
> ESTABLISHED 962/rpc.ldmd
> tcp        0      0 fetch1.ops.nwsar.gov:388    intranet1.arh.nwsar.g:55479
> ESTABLISHED 959/rpc.ldmd
> tcp        0      0 fetch1.ops.nwsar.gov:388    intranet1.arh.nwsar.g:55478
> ESTABLISHED 958/rpc.ldmd
> tcp        0      0 fetch1.ops.nwsar.gov:388    intranet1.arh.nwsar.g:55502
> ESTABLISHED 3270/rpc.ldmd
> tcp        0      0 fetch1.ops.nwsar.gov:388    intranet1.arh.nwsar.g:55499
> ESTABLISHED 2770/rpc.ldmd
> tcp        0      0 fetch1.ops.nwsar.gov:388    intranet1.arh.nwsar.g:55489
> ESTABLISHED 1902/rpc.ldmd
> tcp        0      0 fetch1.ops.nwsar.gov:388    jnuweb1.jnu.nwsar.gov:47111
> ESTABLISHED 28596/rpc.ldmd
> tcp        0      0 fetch1.ops.nwsar.gov:41149  arhdata.arh.nwsar.gov:388
> ESTABLISHED 26446/rpc.ldmd
> tcp        0      0 fetch1.ops.nwsar.gov:41148  arhdata.arh.nwsar.gov:388
> ESTABLISHED 26445/rpc.ldmd
> tcp        0      0 fetch1.ops.nwsar.gov:41147  arhdata.arh.nwsar.gov:388
> ESTABLISHED 26450/rpc.ldmd
> tcp        0      0 fetch1.ops.nwsar.gov:41115  arhdata.arh.nwsar.gov:388
> ESTABLISHED 26449/rpc.ldmd
> tcp        0      0 fetch1.ops.nwsar.gov:41168  arhdata.arh.nwsar.gov:388
> ESTABLISHED 26467/rpc.ldmd
> tcp        0      0 fetch1.ops.nwsar.gov:41156  arhdata.arh.nwsar.gov:388
> ESTABLISHED 26468/rpc.ldmd
> tcp        0      0 fetch1.ops.nwsar.gov:41158  arhdata.arh.nwsar.gov:388
> ESTABLISHED 26451/rpc.ldmd
> tcp        0      0 fetch1.ops.nwsar.gov:41153  arhdata.arh.nwsar.gov:388
> ESTABLISHED 26464/rpc.ldmd
> tcp        0      0 fetch1.ops.nwsar.gov:41155  arhdata.arh.nwsar.gov:388
> ESTABLISHED 26487/rpc.ldmd
> tcp        0      0 fetch1.ops.nwsar.gov:41154  arhdata.arh.nwsar.gov:388
> ESTABLISHED 26462/rpc.ldmd
> tcp        0      0 fetch1.ops.nwsar.gov:41167  arhdata.arh.nwsar.gov:388
> ESTABLISHED 26486/rpc.ldmd
> tcp        0      0 fetch1.ops.nwsar.gov:41163  arhdata.arh.nwsar.gov:388
> ESTABLISHED 26466/rpc.ldmd
> tcp        0      0 fetch1.ops.nwsar.gov:41162  arhdata.arh.nwsar.gov:388
> ESTABLISHED 26465/rpc.ldmd
> tcp        0      0 fetch1.ops.nwsar.gov:388    jnuweb1.jnu.nwsar.gov:47116
> ESTABLISHED 28706/rpc.ldmd
> tcp        0      0 fetch1.ops.nwsar.gov:388    jnuweb1.jnu.nwsar.gov:47119
> ESTABLISHED 29106/rpc.ldmd
> tcp        0      0 fetch1.ops.nwsar.gov:388    jnuweb1.jnu.nwsar.gov:47118
> ESTABLISHED 28976/rpc.ldmd
> tcp        0      0 fetch1.ops.nwsar.gov:388    jnuweb1.jnu.nwsar.gov:47113
> ESTABLISHED 28598/rpc.ldmd
> tcp        0      0 fetch1.ops.nwsar.gov:388    jnuweb1.jnu.nwsar.gov:47112
> ESTABLISHED 28597/rpc.ldmd
> tcp        0      0 fetch1.ops.nwsar.gov:388    jnuweb1.jnu.nwsar.gov:47115
> ESTABLISHED 28783/rpc.ldmd
> tcp        0      0 fetch1.ops.nwsar.gov:388    jnuweb1.jnu.nwsar.gov:47114
> ESTABLISHED 28599/rpc.ldmd
> tcp        0      0 fetch1.ops.nwsar.gov:41117  jnuweb1.jnu.nwsar.gov:388
> ESTABLISHED 26454/rpc.ldmd
> tcp        0      0 fetch1.ops.nwsar.gov:41125  jnuweb1.jnu.nwsar.gov:388
> ESTABLISHED 26460/rpc.ldmd
> tcp        0      0 fetch1.ops.nwsar.gov:388    ls1-afc.anc.nwsar.gov:1518
> ESTABLISHED 18356/rpc.ldmd
> tcp        0      0 fetch1.ops.nwsar.gov:388    deimos.fai.nwsar.gov:39413
> ESTABLISHED 13555/rpc.ldmd
> tcp        0      0 fetch1.ops.nwsar.gov:388    deimos.fai.nwsar.gov:39415
> ESTABLISHED 13564/rpc.ldmd
> tcp        0      0 fetch1.ops.nwsar.gov:388    deimos.fai.nwsar.gov:39408
> ESTABLISHED 13517/rpc.ldmd
> tcp        0      0 fetch1.ops.nwsar.gov:388    deimos.fai.nwsar.gov:39409
> ESTABLISHED 13536/rpc.ldmd
> tcp        0      0 fetch1.ops.nwsar.gov:41138  galileo.wrh.noaa.gov:388
> ESTABLISHED 26484/rpc.ldmd
> tcp        0      0 fetch1.ops.nwsar.gov:41136  galileo.wrh.noaa.gov:388
> ESTABLISHED 26482/rpc.ldmd
> tcp        0      0 fetch1.ops.nwsar.gov:41137  galileo.wrh.noaa.gov:388
> ESTABLISHED 26483/rpc.ldmd
> tcp        0      0 fetch1.ops.nwsar.gov:388    deimos.fai.nwsar.gov:39404
> ESTABLISHED 13487/rpc.ldmd
> tcp        0      0 fetch1.ops.nwsar.gov:388    deimos.fai.nwsar.gov:39406
> ESTABLISHED 13501/rpc.ldmd
> tcp        0      0 fetch1.ops.nwsar.gov:388    deimos.fai.nwsar.gov:39401
> ESTABLISHED 13471/rpc.ldmd
> tcp        0      0 fetch1.ops.nwsar.gov:388    deimos.fai.nwsar.gov:39394
> ESTABLISHED 13385/rpc.ldmd
> tcp        0      0 fetch1.ops.nwsar.gov:388    deimos.fai.nwsar.gov:39395
> ESTABLISHED 13401/rpc.ldmd
> tcp        0      0 fetch1.ops.nwsar.gov:41405  deimos.fai.nwsar.gov:388
> ESTABLISHED 26461/rpc.ldmd
> tcp        0      0 fetch1.ops.nwsar.gov:388    prisma.fsl.noaa.gov:39596
> ESTABLISHED 6046/rpc.ldmd
> tcp        0      0 fetch1.ops.nwsar.gov:388    ls1-vrh.arh.nwsar.gov:3377
> ESTABLISHED 30042/rpc.ldmd
> tcp        0      0 fetch1.ops.nwsar.gov:388    ls1-ajk.jnu.nwsar.gov:2792
> ESTABLISHED 29976/rpc.ldmd
> tcp        0      0 fetch1.ops.nwsar.gov:41139  sr-srh-ridlin1.srh.noaa:388
> ESTABLISHED 26485/rpc.ldmd
> tcp        0      0 fetch1.ops.nwsar.gov:44100  eldm.fsl.noaa.gov:388
> ESTABLISHED 26488/rpc.ldmd
> tcp        0      0 fetch1.ops.nwsar.gov:44186  eldm.fsl.noaa.gov:388
> ESTABLISHED 26491/rpc.ldmd
> tcp        0      0 fetch1.ops.nwsar.gov:44187  eldm.fsl.noaa.gov:388
> ESTABLISHED 26492/rpc.ldmd
> tcp        0      0 fetch1.ops.nwsar.gov:44188  eldm.fsl.noaa.gov:388
> ESTABLISHED 26493/rpc.ldmd
> unix  2      [ ]         DGRAM                    1673234588 26370/rpc.ldmd

That's a lot of LDM connections, all right.

> -rw-r--r--  1 ldm users 0 Sep 25 16:10 ldmd.log
> It seems the logging is going to /var/log/messages for whatever reason.

LDM log messages going to the system log file isn't good.  Please send me the 
output from
the following commands:

    uname -a
    grep SELINUX /etc/selinux/config
    grep local /etc/syslog.conf
    ls -l $HOME/bin/hupsyslog
    grep LOG_LDM $HOME/src/config/ldmconfig.h

> grep -Fi 'Starting Up' `ls -tr /var/log/messages` | wc -l
> 214
> 
> Does this mean that every request from every server is treated as a separate
> ldmd child process?

Yes, every incoming request for data results in a child, upstream LDM process 
that's
resposible for servicing that request.

> I feed multiple versions of LDM.  AWIPS is a National Weather Service standard
> application that is currently running LDM-5.0.8.

AWIPS runs version 5.0.8 of the LDM?  Oh dear.  I'll have to think about this.

>  I've not had problems with my
> WS3 ldm-6.3.0 version talking to those servers.  Other servers in the field 
> are
> running various flavors ov LDM-6.* up to and including one that is running 
> 6.4.6.
> 
> These are my allow entries.. The request entries seemed to work fine on 
> ldm-6.4.6.
> ###############################################################################
> # Allow Entries
> ###############################################################################
> #
> # Under no circumstances comment out the next allow entry to localhost
> # The LDM will NOT start if the lines are commented out.
> allow   ANY
> ^((localhost|loopback)|(127\.0\.0\.1\.?$))
> allow   ANY     ^(.*\.nwsar\.gov\.?$)
> allow   ANY     ^(160\.0\.*\.?$)

Just so you know, the parentheses in the ALLOW entries aren't needed except in 
the
"localhost" one.  They don't hurt, they just don't do anything.

> allow   EXP     ^prisma\.fsl\.noaa\.gov
> allow   EXP     ^prismb\.fsl\.noaa\.gov
> allow   EXP     ^sr-srh-ridlin1\.srh\.noaa\.gov
> allow   EXP     ^cr-crh-ridlin1\.crh\.noaa\.gov

Which ALLOW entry is responsible for allowing the computers to connect that can 
with
version 6.3.0 but not with version 6.4.6?

Regards,
Steve Emmerson

Ticket Details
===================
Ticket ID: SVU-966060
Department: Support LDM
Priority: Normal
Status: Closed