Re: [ldm-users] my data seems to be blackhole-ing

While I'm not sure I understand Charles's hastiness to get this off list, I
am inclined to chalk this up to "freak accident" and just attempt
reinstalling LDM from the start on the server in question.

Clearly something isn't right with it...and it's worth a clean go.


On Mon, Jul 15, 2013 at 5:08 PM, Market, Patrick S. <MarketP@xxxxxxxxxxxx>wrote:

>  Seems like the support system is working well to me.  Why try to kill it?
> ****
>
> ** **
>
> Patrick****
>
> =====================================****
>
> Dr. Patrick S. Market****
>
> Professor of Synoptic Meteorology****
>
> Dept. of Soil, Env. & Atmospheric Sciences****
>
> University of Missouri****
>
> 302 ABNR, Columbia, MO  65211****
>
> Voice:  573-882-1496  /  Fax:  573-884-5070****
>
> ** **
>
> *From:* ldm-users-bounces@xxxxxxxxxxxxxxxx [mailto:
> ldm-users-bounces@xxxxxxxxxxxxxxxx] *On Behalf Of *Charles Watson
> *Sent:* Monday, July 15, 2013 5:04 PM
>
> *To:* Blair Trosper
> *Cc:* LDM Users
> *Subject:* Re: [ldm-users] my data seems to be blackhole-ing****
>
>  ** **
>
> Hai Guys!****
>
> ** **
>
> As fascinating as this discussion is, any chance you could take it "off
> list", since it doesn't seem to be a global issue?****
>
> ** **
>
> kthxbye!****
>
> ** **
>
> Chuck****
>
> ** **
>
> Chuck Watson****
>
> Watson Technical Consulting
>
> ****
>
> http://hurricane.methaz.org,  Tel 912.398.9753****
>
> http://satblog.methaz.org for commentary on hazard modeling and forecasts*
> ***
>
> ** **
>
> "In this house we obey the laws of thermodynamics!"****
>
> -- Homer Simpson****
>
> ** **
>
> On Jul 15, 2013, at 6:00 PM, Blair Trosper wrote:****
>
>
>
> ****
>
> It's definitely running:****
>
> ** **
>
> ldm@chicago-il-1:~$ ps aux | grep pqact****
>
> ldm      32737  0.3  0.3 8046468 50636 ?       S    16:53   0:01 pqact -v*
> ***
>
> ** **
>
> On Mon, Jul 15, 2013 at 4:59 PM, Karen Cooper - NOAA Affiliate <
> karen.cooper@xxxxxxxx> wrote:****
>
> Assuming that regular ldm logging is working it should not log anywhere
> else.  ****
>
> If you are seeing the data in your queue, but nothing new on disk (no
> changes to df -k) then I would question if pqact is running.****
>
> pqact in verbose mode should be loggin each item that goes through your
> queue -- even if it is ignoring it.  ****
>
> Another thing to check is to see is pqact is actually running and not
> crashing.  If you do a ps (or run top) can you see it?
>
> ****
>
> ** **
>
> On Mon, Jul 15, 2013 at 4:54 PM, Blair Trosper <
> blair.trosper@xxxxxxxxxxxxxxxxxxx> wrote:****
>
> Absolute paths were a no go, and putting verbose mode into:
> EXEC "pqact -v"****
>
> ** **
>
> ...seems to yield no extra info.  Or, if it is, it's putting it somewhere
> besides ldmd.log****
>
> ** **
>
> On Mon, Jul 15, 2013 at 4:52 PM, donna Cote <d-cote@xxxxxxxx> wrote:****
>
> Blair, from my experience, I've made explicit directory paths in my pqacts.
>
> I suggest looking for a /mnt/data or /nexrad2 directory
>
> Just a thought,
> Donna****
>
>
>
> On 7/15/13 4:45 PM, Blair Trosper wrote:****
>
>  My which I mean it only adds the contents of "ldmadmin watch" to the
> ldmd.log, which doesn't really help me.
>
> I used "ldmadmin restart -v", did you mean something else?  Perhaps I
> misunderstood.  :/
>
>
> On Mon, Jul 15, 2013 at 4:44 PM, Karen Cooper - NOAA Affiliate****
>
> <karen.cooper@xxxxxxxx <mailto:karen.cooper@xxxxxxxx>> wrote:
>
>     Try putting pqact into verbose mode.
>
>     exec "pqact -v"
>
>     and then restart ldm.  It should tell you everything pqact is doing.
>
>
>     On Mon, Jul 15, 2013 at 4:42 PM, Blair Trosper
>     <blair.trosper@xxxxxxxxxxxxxxxxxxx****
>
>     <mailto:blair.trosper@xxxxxxxxxxxxxxxxxxx>> wrote:
>
>         It's enabled and not commented.
>
>         Putting ldm into verbose mode didn't seem to yield anything
>         additional in the log files.  In fact, it's very UNdescriptive.
>
>
>         On Mon, Jul 15, 2013 at 4:41 PM, Karen Cooper - NOAA Affiliate****
>
>         <karen.cooper@xxxxxxxx <mailto:karen.cooper@xxxxxxxx>> wrote:
>
>             And of course what I really meant was the exec line in your
>             ldmd.conf...
>
>             exec    "pqact "
>
>
>             On Mon, Jul 15, 2013 at 4:40 PM, Karen Cooper - NOAA
>             Affiliate <karen.cooper@xxxxxxxx****
>
>             <mailto:karen.cooper@xxxxxxxx>> wrote:
>
>                 Blair,
>
>                 First a sanity check. -- Are you sure that pqact is****
>
>                 uncommented in your ldmadmin.pl <http://ldmadmin.pl>?****
>
>
>
>                 You can set pqact to verbose mode and then you should
>                 see what it's trying to do, as it will log to your
>                 ldmd.log files.  I often use this when I'm testing out
>                 new pqact entries that aren't working as I expect.
>
>
>                 On Mon, Jul 15, 2013 at 4:35 PM, Blair Trosper
>                 <blair.trosper@xxxxxxxxxxxxxxxxxxx****
>
>                 <mailto:blair.trosper@xxxxxxxxxxxxxxxxxxx>> wrote:
>
>                     I'm receiving several feeds as proven by ldmadmin
>                     watch...and the logs all look fine.  However, my
>                     data isn't being saved to disk.****
>
>                      From *registry.xml*:****
>
>
>
>                            <pqact>
>
>
> <config-path>/mnt/data/ldm/etc/pqact.conf</config-path>
>
>                         <datadir-path>/mnt/data/ldm/var/data</datadir-path>
>                            </pqact>
>
> ****
>
>                      From */mnt/data/ldm/etc/pqact.conf*:****
>
>
>
>                         # L2 data
>                         NEXRAD2
>
> ^L2-BZIP2/(....)/([0-9]{8})([0-9]{4})[0-9]{2}/([0-9]{1,4}).*/[SI]
>                                  FILE    nexrad2/\1/\1_\2-\3.part
>                         NEXRAD2
>
> ^L2-BZIP2/(....)/([0-9]{8})([0-9]{4})[0-9]{2}/([0-9]{1,4}).*/E
>                                  FILE    -close nexrad2/\1/\1_\2-\3.part
>                         (snip)
>
>
>                     However, following those two things, here's a****
>
>                     directory listing of */mnt/data/ldm/var/data*:****
>
>
>
>                         ldm@chicago-il-1:~/data$ ls -lsha
>                         total 8.0K
>                         4.0K drwxr-xr-x 2 ldm ldm 4.0K Jul 15 16:09 .
>                         4.0K drwxr-xr-x 5 ldm ldm 4.0K Jul 13 23:19 ..
>
>
>                     Nothing.  The same is true of other feeds which
>                     should be stashed in other subdirectories.
>
>                     The bandwidth is certainly registering, which I have
>                     verified at the router, so I'm definitely receiving
>                     the data...so...
>
>                     Where are my files going?  Any way to track this down?
>
>                     (I've also done "du -ksh" from the home directory of
>                     the ldm user over the last few hours, and it's not
>                     increasing.  Definitely nothing being written to
>                     hard disk.  And still no errors in the LDM or system
>                     logs.)
>
>                     --
>                     Blair Trosper
>                     Weather Data / Updraft Networks
>                     blair.trosper@xxxxxxxxxxxxxxxxxxx****
>
>                     <mailto:blair.trosper@xxxxxxxxxx>****
>
>
>                     NOC: 512-666-0536
>
>                     _______________________________________________
>                     ldm-users mailing list
>                     ldm-users@xxxxxxxxxxxxxxxx****
>
>                     <mailto:ldm-users@xxxxxxxxxxxxxxxx>****
>
>
>                     For list information or to unsubscribe,  visit:
>                     http://www.unidata.ucar.edu/mailing_lists/
>
>
>
>
>                 --
>                 “Climate is what you expect, weather is what you get."
>                   -- Robert A. Heinlein
>
>                 -------------------------------------------****
>
>                 Karen.Cooper@xxxxxxxx <mailto:Karen.Cooper@xxxxxxxx>
>
>                 Phone: 405-325-6982 <tel:405-325-6982>
>                 Cell: 405-834-8559 <tel:405-834-8559>****
>
>
>                 INDUS Corporation
>                 National Severe Storms Laboratory
>
>
>
>
>             --
>             “Climate is what you expect, weather is what you get."
>               -- Robert A. Heinlein
>
>             -------------------------------------------****
>
>             Karen.Cooper@xxxxxxxx <mailto:Karen.Cooper@xxxxxxxx>
>
>             Phone: 405-325-6982 <tel:405-325-6982>
>             Cell: 405-834-8559 <tel:405-834-8559>****
>
>
>             INDUS Corporation
>             National Severe Storms Laboratory
>
>
>
>
>         --
>         Blair Trosper
>         Weather Data / Updraft Networks****
>
>         blair.trosper@xxxxxxxxxxxxxxxxxxx <mailto:blair.trosper@xxxxxxxxxx
> >****
>
>
>         NOC: 512-666-0536
>
>
>
>
>     --
>     “Climate is what you expect, weather is what you get."
>       -- Robert A. Heinlein
>
>     -------------------------------------------****
>
>     Karen.Cooper@xxxxxxxx <mailto:Karen.Cooper@xxxxxxxx>
>
>     Phone: 405-325-6982 <tel:405-325-6982>
>     Cell: 405-834-8559 <tel:405-834-8559>****
>
>
>     INDUS Corporation
>     National Severe Storms Laboratory
>
>
>
>
> --
> Blair Trosper
> Weather Data / Updraft Networks****
>
> blair.trosper@xxxxxxxxxxxxxxxxxxx <mailto:blair.trosper@xxxxxxxxxx>****
>
>
> NOC: 512-666-0536
>
>
> _______________________________________________
> ldm-users mailing list
> ldm-users@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe,  visit:
> http://www.unidata.ucar.edu/mailing_lists/****
>
>
> _______________________________________________
> ldm-users mailing list
> ldm-users@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe,  visit:
> http://www.unidata.ucar.edu/mailing_lists/ ****
>
>
>
> ****
>
> ** **
>
> -- ****
>
> Blair Trosper****
>
> Weather Data / Updraft Networks****
>
> blair.trosper@xxxxxxxxxxxxxxxxxxx <blair.trosper@xxxxxxxxxx>****
>
> NOC:  512-666-0536****
>
>
> _______________________________________________
> ldm-users mailing list
> ldm-users@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe,  visit:
> http://www.unidata.ucar.edu/mailing_lists/ ****
>
>
>
>
> --
> “Climate is what you expect, weather is what you get."
>  -- Robert A. Heinlein
>
> -------------------------------------------
> Karen.Cooper@xxxxxxxx****
>
> Phone:  405-325-6982
> Cell:   405-834-8559****
>
>
> INDUS Corporation
> National Severe Storms Laboratory****
>
>
>
> ****
>
> ** **
>
> -- ****
>
> Blair Trosper****
>
> Weather Data / Updraft Networks****
>
> blair.trosper@xxxxxxxxxxxxxxxxxxx <blair.trosper@xxxxxxxxxx>****
>
> NOC:  512-666-0536****
>
> _______________________________________________
> ldm-users mailing list
> ldm-users@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe,  visit:
> http://www.unidata.ucar.edu/mailing_lists/ ****
>
> ** **
>



-- 
Blair Trosper
Weather Data / Updraft Networks
blair.trosper@xxxxxxxxxxxxxxxxxxx <blair.trosper@xxxxxxxxxx>
NOC:  512-666-0536
  • 2013 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the ldm-users archives: