Re: Question about possible bug in LDM

• To: Robb Kambic <rkambic@xxxxxxxxxxxxxxxx>
• Subject: Re: Question about possible bug in LDM
• From: Alan Steremberg <alans@xxxxxxxxxxxxxxxx>
• Date: Thu, 18 Jun 1998 18:24:35 -0400 (EDT)

The problem went away when we put a -close flag in the FILE command, and
we stopped running out of file descriptors.  We are running:

Alan

----------------------------
Alan Steremberg

On Thu, 18 Jun 1998, Robb Kambic wrote:

Hiya,

The problem is either an overloaded machine or a bug in the metar decoder.
No one else has reported a bug, but this is a different platform and a
different version of perl.  I would isolate the processes, do the EXP
pqact entries only and then the metar decoder actions only. If they work
fine seperately then the problem points to an overloaded machine.  Also,
it could be how Linux handles file descriptors. Do some testing to see the
results, also what is the machine type/os version?

Robb...


On Thu, 18 Jun 1998, Alan Steremberg wrote:

> ldm-5.0.5

> > ----------------------------

>  Alan Steremberg

> 650-462-9382 > http://www.wunderground.com > > On Thu, 18 Jun 1998, Robb Kambic wrote: > > > On Thu, 18 Jun 1998, Alan Steremberg wrote: > > > > > > > > Hi, > > > > > > We have been using LDM to mirror our web site to all of our web servers

> > > (Linux).  This has been working great, but I think we may have discovered

> > > a bug. > > > > > > Here is the scenario: > > > > > > We use a modified version of pqinsert to send all of the data files on the

> > > EXP feed to all of our web servers.  This is contains a lot of little text
> > > files.

> > > > > > We noticed that the metar converter (from the decoders) would end up

> > > running many times (3,4,10 copies of it).  This would eventually cripple
> > > our machine.
> > Hiya,

> > > > We had the problem of the ldm starting up multiple invocations of the

> > same decoder with early versions of the ldm, version ldm-5.0.5

> > fixed this problem. > > > > > > > > > > So, I turned on the debugging code inside of pqact, and I noticed that we

> > > forgot to do a -close on the FILE directive to save the EXP feed into the
> > > little files.  So what seemed to be happening is that we would run out of
> > > file descriptors, and then the unio_open routine would call on code to
> > > close two 'lru' files.  What happens if one of those 'lru' files is
> > > actually a PIPE to the metar decoder?  Does it kill the metar converter
> > > process?  Does it close down the pipe, and assume that a sigpipe will
> > > kill the converter?  Any ideas?  Is this a linux only bug?

> > > > > > > It does a sigpipe that send an eof to the decoders and they terminate.

> > Let's wait until the version of the ldm before classifying this as a bug.

> > > > Robb... > > > > > > > Thanks,

> > > Alan

> > > > > > ----------------------------

> > >  Alan Steremberg

> > > 650-462-9382 > > > http://www.wunderground.com > > > > > > > ==============================================================================

> > Robb Kambic                              Unidata Program Center
> > Software Engineer III                    Univ. Corp for Atmospheric Research
> > rkambic@xxxxxxxxxxxxxxxx                 WWW: http://www.unidata.ucar.edu/
> > 
==============================================================================

> > >

==============================================================================
Robb Kambic                                Unidata Program Center
Software Engineer III                      Univ. Corp for Atmospheric Research
rkambic@xxxxxxxxxxxxxxxx                   WWW: http://www.unidata.ucar.edu/
==============================================================================