[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: 20020419: RECLASS & h_clnt_call

Hi Kevin,

The RECLASS messages are showing us that the LDM is skipping a number
of products in order to catch up.

It opens an exchange between host and client to negotiate requests with
what is desired and what is available


The 'h_clnt_call:' notices are letting us know that it is taking an
exceedingly long time to send pieces of a large product.  It is likely
that the product that is being sent is large..


Both of these indicate that there may be poor connectivity between the two
machines.


The "runaway" rpc processes are a bit of a mystery, we have seen this
before, but each of the past instances do not seem germaine to your
situation.

1) Bad queue..deleting and remaking solved problem

2) Non-comaptible versions of ldm and not recoginizing feedtype

3) syslog and ldmd.log failing


comments also in body of text...


Any additional info you can provide may lead us to a solution, I look
forward to hopefully solving this issue..

Thank you,


-Jeff
____________________________                  _____________________
Jeff Weber                                    address@hidden
Unidata Support                               PH:303-497-8676 
NWS-COMET Case Study Library                  FX:303-497-8690
University Corp for Atmospheric Research      3300 Mitchell Ln
http://www.unidata.ucar.edu/staff/jweber      Boulder,Co 80307-3000
________________________________________      ______________________

On Fri, 19 Apr 2002, Unidata Support wrote:

> 
> ------- Forwarded Message
> 
> >To: <address@hidden>
> >cc: David Knight <address@hidden>
> >From: "Kevin R. Tyle" <address@hidden>
> >Subject: LDM log messages
> >Organization: UCAR/Unidata
> >Keywords: 200204191536.g3JFafa14575
> 
> Hi,
> 
> Can you explain the meaning of some of the following entries from
> our ldm log?
> 
> Particularly the repeated RECLASS messages and the h_clnt_call
> messages.
> 
> I am assuming they are due to slowdowns between redwood and
> the client LDM's, but am not sure.
> 
> We also run the IP Filter firewall package, and we keep state
> on incoming port 388 traffic to redwood.  A look at the state
> table right now shows a bunch of active states between us
> and wr.psi.ncc.edu, although only one rpc.ldmd is running between
> us and them.  Again, I suspect this may be due to some internet
> slowdown last night, since the states were all created near the
> same time, about 10 hours ago.  This may be outside your realm,
> but I am wondering why the state tables still show these connections
> as fully active--typically, with a protocol like FTP

Are these LDM owned processes?  I am curious because of the protocol, LDM
does not use FTP...

, once a
> connection breaks, the nature of the state changes to one that
> expires out of the state table much quicker.
> 
> Although this did not happen last night, we also have recently
> had instances of many rpc.ldmd processes being launched--the end
> result being a very high load on the server and no data until
> the LDM is restarted and all rpc.ldmd processes manually killed.
> We have set up monitoring to investigate this in more detail should
> it happen again, but it appears to occur when our .stats files
> show latencies > 3600 on DDPLUS and HDS.
> 
> Thanks for any info!
> 
> --Kevin
> 
> ______________________________________________________________________
> Kevin Tyle, Systems Administrator               **********************
> Dept. of Earth & Atmospheric Sciences           address@hidden
> University at Albany, ES-235                    518-442-4571 (voice)
> 1400 Washington Avenue                          518-442-5825 (fax)
> Albany, NY 12222                                **********************
> ______________________________________________________________________
> 
> Apr 19 04:24:50 redwood aspre(feed)[5749]: RECLASS:
> 20020419032353.835 TS_ENDT {{WMO,  ".*"}}
> Apr 19 04:25:51 redwood aspre(feed)[5749]: RECLASS: 20020419032455.332
> TS_ENDT {{WMO,  ".*"}}
> Apr 19 04:26:57 redwood aspre(feed)[5749]: RECLASS: 20020419032600.645
> TS_ENDT {{WMO,  ".*"}}
> Apr 19 04:28:07 redwood aspre(feed)[5749]: RECLASS: 20020419032710.578
> TS_ENDT {{WMO,  ".*"}}
> Apr 19 04:29:13 redwood aspre(feed)[5749]: RECLASS: 20020419032817.164
> TS_ENDT {{WMO,  ".*"}}
> Apr 19 04:30:29 redwood aspre(feed)[5749]: RECLASS: 20020419032933.031
> TS_ENDT {{WMO,  ".*"}}
> Apr 19 04:30:57 redwood aspre[7597]: Connection from aspre.gso.uri.edu
> Apr 19 04:30:57 redwood aspre[7597]: Connection reset by peer
> Apr 19 04:30:57 redwood aspre[7597]: Exiting
> Apr 19 04:31:57 redwood aspre(feed)[5749]: h_clnt_call: aspre.gso.uri.edu:
> BLKDATA: time elapsed  22.674846
> Apr 19 04:32:07 redwood aspre(feed)[5749]: RECLASS: 20020419033111.112
> TS_ENDT {{WMO,  ".*"}}
> Apr 19 04:33:14 redwood aspre(feed)[5749]: RECLASS: 20020419033218.555
> TS_ENDT {{WMO,  ".*"}}
> Apr 19 04:34:28 redwood aspre(feed)[5749]: RECLASS: 20020419033331.960
> TS_ENDT {{WMO,  ".*"}}
> Apr 19 04:35:15 redwood wr(feed)[26367]: h_clnt_call: wr.psi.ncc.edu:
> BLKDATA: time elapsed  21.547277
> Apr 19 04:35:40 redwood aspre(feed)[5749]: RECLASS: 20020419033443.938
> TS_ENDT {{WMO,  ".*"}}
> Apr 19 04:36:52 redwood aspre(feed)[5749]: RECLASS: 20020419033556.206
> TS_ENDT {{WMO,  ".*"}}
> Apr 19 04:37:54 redwood aspre(feed)[5749]: RECLASS: 20020419033657.879
> TS_ENDT {{WMO,  ".*"}}
> Apr 19 04:38:57 redwood aspre(feed)[5749]: RECLASS: 20020419033800.965
> TS_ENDT {{WMO,  ".*"}}
> Apr 19 04:40:24 redwood aspre(feed)[5749]: RECLASS: 20020419033927.848
> TS_ENDT {{WMO,  ".*"}}
> Apr 19 04:40:59 redwood aspre[7831]: Connection from aspre.gso.uri.edu
> Apr 19 04:40:59 redwood aspre[7831]: Connection reset by peer
> Apr 19 04:40:59 redwood aspre[7831]: Exiting
> Apr 19 04:41:32 redwood aspre(feed)[5749]: RECLASS: 20020419034035.825
> TS_ENDT {{WMO,  ".*"}}
> Apr 19 04:42:41 redwood aspre(feed)[5749]: RECLASS: 20020419034145.432
> TS_ENDT {{WMO,  ".*"}}
> Apr 19 04:43:52 redwood aspre(feed)[5749]: RECLASS: 20020419034255.858
> TS_ENDT {{WMO,  ".*"}}
> Apr 19 04:44:57 redwood aspre(feed)[5749]: RECLASS: 20020419034400.718
> TS_ENDT {{WMO,  ".*"}}
> Apr 19 04:46:38 redwood aspre(feed)[5749]: SAAU31 YBBN 190300: RPC: Timed
> out (5)
> Apr 19 04:46:38 redwood aspre(feed)[5749]: pq_sequence failed: I/O error
> (errno = 5)
> Apr 19 04:46:38 redwood aspre(feed)[5749]: Exiting
> Apr 19 04:46:45 redwood rpc.ldmd[21539]: child 5749 exited with status 1
> Apr 19 04:47:09 redwood aspre[8031]: Connection from aspre.gso.uri.edu
> Apr 19 04:47:09 redwood aspre(feed)[8031]: Starting Up: 20020419034643.989
> TS_ENDT {{WMO,  ".*"}}
> 
> 
> 
> ------- End of Forwarded Message
> 
>