[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Support #VOV-751174]: Re: [ldm-users] Problems getting data from idd.aos.wisc.edu

Hi Pete,

Quick note to let you know that we will be meeting tomorrow (Tuesday)
to discuss the situation on your machine.  It is also very likely that
we will be running tests of switching back and forth between LDM-6.11.2
and LDM-6.11.3.

More tomorrow...

Tom

> Ok, finally have a few minutes to devote to this.
> 
> First of all, why am I all of a sudden unable to get CONDUIT data from
> ncepldm1.woc.noaa.gov? This began without any changes on my end at 9:30
> AM CST on Friday morning. Can you ldmping that from unidata's end? Can
> someone check to make sure I didn't get firewalled out or disALLOWed
> some how?
> 
> Second -  Here is my extremely ignorant perception of what feels like is
> happening when starting ldm 6.11.2 vs 6.11.3
> 
> When running ldm 6.11.2 - if I stop the ldm to make a change to
> ldmd.conf or whatever, when I restart there is a usually a short period
> of time where not much data is flowing, I am making connections to my
> upstreams, and downstreams are making connections to me. The load on the
> individual ldm processes during this period is high - 50 to 80% of CPU -
> but usually within a minute or so, whatever is happening there ends,
> data starts blasting through, and the load on the individual CPUs goes down.
> 
> When I start ldm 6.11.3, the same thing seems to happen but does not
> seem to ever end. The load is high, data throughput is generally
> extremely slow, and eventually my latencies go up to the point where I
> give up and go back to 6.11.2.
> 
> I cannot say for certain that the latency problem is only with CONDUIT
> data, though with the data volume that seems to take the biggest hit.
> 
> I am at a loss as to how we go about investigating this. You are
> certainly welcome to poke around on our machine, and
> stop/recompile/restart to test, but since it is a production ldm server,
> we would be impacting my downstream sites as well as our ability to
> ingest data here, and I'd kind of like not to have to do that.
> 
> If it would be more efficient to bounce around ideas/troubleshooting
> strategies over the phone, give me a call on my cell - 608-217-8675.
> 
> Thanks,
> 
> Pete
> 
> On 02/18/2013 10:45 AM, Unidata LDM Support wrote:
> > Hi Gilbert,
> >
> > re:
> >> When you log into Pete's machine, do you see it happening after switching
> >> to 6.11.3?
> > We haven't tried that yet.
> >
> > re: I upgraded the real server backends of idd.unidata.ucar.edu
> >
> >> :-) I am glad to see this! The top level should always be running the
> >> latest and greatest, IMO. Otherwise, it sends the message that the
> >> update isn't important.
> > We disagree somewhat.  We do not like to take down/restart the nodes
> > in idd.unidata.ucar.edu.  As long as they are providing the service
> > they are designed to provide, we see no reason to touch them.  That
> > being said, however, I have been somewhat lax in installing current
> > LDMs on other Unidata machines.
> >
> > re:
> >> We do not get CONDUIT at NIU or at a company called AllisonHouse, but all
> >> are running 6.11.3 without CONDUIT...and without any issues.
> >>
> >> Pete, can you try just feeding from one upstream CONDUIT server?
> > I am working on a hypothesis that the problem that Pete is seeing
> > is intimately related to the NCEP CONDUIT toplevels.  Exactly what
> > may be going on is, however, unknown to me (i.e., I have no working
> > hypothesis that would explain the situation).
> >
> > Cheers,
> >
> > Tom
> > --
> > ****************************************************************************
> > Unidata User Support                                    UCAR Unidata Program
> > (303) 497-8642                                                 P.O. Box 3000
> > address@hidden                                   Boulder, CO 80307
> > ----------------------------------------------------------------------------
> > Unidata HomePage                       http://www.unidata.ucar.edu
> > ****************************************************************************
> >
> >
> > Ticket Details
> > ===================
> > Ticket ID: VOV-751174
> > Department: Support LDM
> > Priority: Normal
> > Status: Closed
> 
> 
> --
> Pete Pokrandt - Systems Programmer
> UW-Madison Dept of Atmospheric and Oceanic Sciences
> 608-262-3086  - address@hidden
> 
> 

Cheers,

Tom
--
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: VOV-751174
Department: Support LDM
Priority: Normal
Status: Open