Re: [ldm-users] my data seems to be blackhole-ing

Glad to hear its fixed. :)


Sent from me!  Please pardon any tpyos


On Jul 15, 2013, at 5:35 PM, Blair Trosper <
blair.trosper@xxxxxxxxxxxxxxxxxxx> wrote:

A reinstall magically healed it.  Let's chalk it up to it being a Monday.
 Thanks for trying to help, guys and gals.  Looks like we were doomed due
to...who knows why.  :)


On Mon, Jul 15, 2013 at 5:27 PM, George Trojan <george.trojan@xxxxxxxx>wrote:

>  Run pqact from the command line (as user ldm):
>
> [ldm@aawu64 pqact]$ pqact -vx -l -
>
> Then post the output.
>
> George
>
>
> On 07/15/2013 10:10 PM, Blair Trosper wrote:
>
> While I'm not sure I understand Charles's hastiness to get this off list,
> I am inclined to chalk this up to "freak accident" and just attempt
> reinstalling LDM from the start on the server in question.
>
>  Clearly something isn't right with it...and it's worth a clean go.
>
>
> On Mon, Jul 15, 2013 at 5:08 PM, Market, Patrick S. 
> <MarketP@xxxxxxxxxxxx>wrote:
>
>>  Seems like the support system is working well to me.  Why try to kill
>> it?
>>
>>
>>
>> Patrick
>>
>> =====================================
>>
>> Dr. Patrick S. Market
>>
>> Professor of Synoptic Meteorology
>>
>> Dept. of Soil, Env. & Atmospheric Sciences
>>
>> University of Missouri
>>
>> 302 ABNR, Columbia, MO  65211
>>
>> Voice:  573-882-1496  /  Fax:  573-884-5070
>>
>>
>>
>> *From:* ldm-users-bounces@xxxxxxxxxxxxxxxx [mailto:
>> ldm-users-bounces@xxxxxxxxxxxxxxxx] *On Behalf Of *Charles Watson
>> *Sent:* Monday, July 15, 2013 5:04 PM
>>
>> *To:* Blair Trosper
>> *Cc:* LDM Users
>>  *Subject:* Re: [ldm-users] my data seems to be blackhole-ing
>>
>>
>>
>> Hai Guys!
>>
>>
>>
>> As fascinating as this discussion is, any chance you could take it "off
>> list", since it doesn't seem to be a global issue?
>>
>>
>>
>> kthxbye!
>>
>>
>>
>> Chuck
>>
>>
>>
>> Chuck Watson
>>
>> Watson Technical Consulting
>>
>>   http://hurricane.methaz.org,  Tel 912.398.9753
>>
>> http://satblog.methaz.org for commentary on hazard modeling and forecasts
>>
>>
>>
>> "In this house we obey the laws of thermodynamics!"
>>
>> -- Homer Simpson
>>
>>
>>
>> On Jul 15, 2013, at 6:00 PM, Blair Trosper wrote:
>>
>>
>>
>>  It's definitely running:
>>
>>
>>
>> ldm@chicago-il-1:~$ ps aux | grep pqact
>>
>> ldm      32737  0.3  0.3 8046468 50636 ?       S    16:53   0:01 pqact -v
>>
>>
>>
>> On Mon, Jul 15, 2013 at 4:59 PM, Karen Cooper - NOAA Affiliate <
>> karen.cooper@xxxxxxxx> wrote:
>>
>> Assuming that regular ldm logging is working it should not log anywhere
>> else.
>>
>> If you are seeing the data in your queue, but nothing new on disk (no
>> changes to df -k) then I would question if pqact is running.
>>
>> pqact in verbose mode should be loggin each item that goes through your
>> queue -- even if it is ignoring it.
>>
>> Another thing to check is to see is pqact is actually running and not
>> crashing.  If you do a ps (or run top) can you see it?
>>
>>
>>
>> On Mon, Jul 15, 2013 at 4:54 PM, Blair Trosper <
>> blair.trosper@xxxxxxxxxxxxxxxxxxx> wrote:
>>
>> Absolute paths were a no go, and putting verbose mode into:
>> EXEC "pqact -v"
>>
>>
>>
>> ...seems to yield no extra info.  Or, if it is, it's putting it somewhere
>> besides ldmd.log
>>
>>
>>
>> On Mon, Jul 15, 2013 at 4:52 PM, donna Cote <d-cote@xxxxxxxx> wrote:
>>
>> Blair, from my experience, I've made explicit directory paths in my
>> pqacts.
>>
>> I suggest looking for a /mnt/data or /nexrad2 directory
>>
>> Just a thought,
>> Donna
>>
>>
>>
>> On 7/15/13 4:45 PM, Blair Trosper wrote:
>>
>>  My which I mean it only adds the contents of "ldmadmin watch" to the
>> ldmd.log, which doesn't really help me.
>>
>> I used "ldmadmin restart -v", did you mean something else?  Perhaps I
>> misunderstood.  :/
>>
>>
>> On Mon, Jul 15, 2013 at 4:44 PM, Karen Cooper - NOAA Affiliate
>>
>> <karen.cooper@xxxxxxxx <mailto:karen.cooper@xxxxxxxx>> wrote:
>>
>>     Try putting pqact into verbose mode.
>>
>>     exec "pqact -v"
>>
>>     and then restart ldm.  It should tell you everything pqact is doing.
>>
>>
>>     On Mon, Jul 15, 2013 at 4:42 PM, Blair Trosper
>>     <blair.trosper@xxxxxxxxxxxxxxxxxxx
>>
>>     <mailto:blair.trosper@xxxxxxxxxxxxxxxxxxx>> wrote:
>>
>>         It's enabled and not commented.
>>
>>         Putting ldm into verbose mode didn't seem to yield anything
>>         additional in the log files.  In fact, it's very UNdescriptive.
>>
>>
>>         On Mon, Jul 15, 2013 at 4:41 PM, Karen Cooper - NOAA Affiliate
>>
>>         <karen.cooper@xxxxxxxx <mailto:karen.cooper@xxxxxxxx>> wrote:
>>
>>             And of course what I really meant was the exec line in your
>>             ldmd.conf...
>>
>>             exec    "pqact "
>>
>>
>>             On Mon, Jul 15, 2013 at 4:40 PM, Karen Cooper - NOAA
>>             Affiliate <karen.cooper@xxxxxxxx
>>
>>             <mailto:karen.cooper@xxxxxxxx>> wrote:
>>
>>                 Blair,
>>
>>                 First a sanity check. -- Are you sure that pqact is
>>
>>                 uncommented in your ldmadmin.pl <http://ldmadmin.pl>?
>>
>>
>>
>>                 You can set pqact to verbose mode and then you should
>>                 see what it's trying to do, as it will log to your
>>                 ldmd.log files.  I often use this when I'm testing out
>>                 new pqact entries that aren't working as I expect.
>>
>>
>>                 On Mon, Jul 15, 2013 at 4:35 PM, Blair Trosper
>>                 <blair.trosper@xxxxxxxxxxxxxxxxxxx
>>
>>                 <mailto:blair.trosper@xxxxxxxxxxxxxxxxxxx>> wrote:
>>
>>                     I'm receiving several feeds as proven by ldmadmin
>>                     watch...and the logs all look fine.  However, my
>>                     data isn't being saved to disk.
>>
>>                      From *registry.xml*:
>>
>>
>>
>>                            <pqact>
>>
>>
>> <config-path>/mnt/data/ldm/etc/pqact.conf</config-path>
>>
>>
>> <datadir-path>/mnt/data/ldm/var/data</datadir-path>
>>                            </pqact>
>>
>>                       From */mnt/data/ldm/etc/pqact.conf*:
>>
>>
>>
>>                         # L2 data
>>                         NEXRAD2
>>
>> ^L2-BZIP2/(....)/([0-9]{8})([0-9]{4})[0-9]{2}/([0-9]{1,4}).*/[SI]
>>                                  FILE    nexrad2/\1/\1_\2-\3.part
>>                         NEXRAD2
>>
>> ^L2-BZIP2/(....)/([0-9]{8})([0-9]{4})[0-9]{2}/([0-9]{1,4}).*/E
>>                                  FILE    -close nexrad2/\1/\1_\2-\3.part
>>                         (snip)
>>
>>
>>                     However, following those two things, here's a
>>
>>                     directory listing of */mnt/data/ldm/var/data*:
>>
>>
>>
>>                         ldm@chicago-il-1:~/data$ ls -lsha
>>                         total 8.0K
>>                         4.0K drwxr-xr-x 2 ldm ldm 4.0K Jul 15 16:09 .
>>                         4.0K drwxr-xr-x 5 ldm ldm 4.0K Jul 13 23:19 ..
>>
>>
>>                     Nothing.  The same is true of other feeds which
>>                     should be stashed in other subdirectories.
>>
>>                     The bandwidth is certainly registering, which I have
>>                     verified at the router, so I'm definitely receiving
>>                     the data...so...
>>
>>                     Where are my files going?  Any way to track this down?
>>
>>                     (I've also done "du -ksh" from the home directory of
>>                     the ldm user over the last few hours, and it's not
>>                     increasing.  Definitely nothing being written to
>>                     hard disk.  And still no errors in the LDM or system
>>                     logs.)
>>
>>                     --
>>                     Blair Trosper
>>                     Weather Data / Updraft Networks
>>                     blair.trosper@xxxxxxxxxxxxxxxxxxx
>>
>>                     <mailto:blair.trosper@xxxxxxxxxx>
>>
>>
>>                     NOC: 512-666-0536
>>
>>                     _______________________________________________
>>                     ldm-users mailing list
>>                     ldm-users@xxxxxxxxxxxxxxxx
>>
>>                     <mailto:ldm-users@xxxxxxxxxxxxxxxx>
>>
>>
>>                     For list information or to unsubscribe,  visit:
>>                     http://www.unidata.ucar.edu/mailing_lists/
>>
>>
>>
>>
>>                 --
>>                 “Climate is what you expect, weather is what you get."
>>                   -- Robert A. Heinlein
>>
>>                 -------------------------------------------
>>
>>                 Karen.Cooper@xxxxxxxx <mailto:Karen.Cooper@xxxxxxxx>
>>
>>                 Phone: 405-325-6982 <tel:405-325-6982>
>>                 Cell: 405-834-8559 <tel:405-834-8559>
>>
>>
>>                 INDUS Corporation
>>                 National Severe Storms Laboratory
>>
>>
>>
>>
>>             --
>>             “Climate is what you expect, weather is what you get."
>>               -- Robert A. Heinlein
>>
>>             -------------------------------------------
>>
>>             Karen.Cooper@xxxxxxxx <mailto:Karen.Cooper@xxxxxxxx>
>>
>>             Phone: 405-325-6982 <tel:405-325-6982>
>>             Cell: 405-834-8559 <tel:405-834-8559>
>>
>>
>>             INDUS Corporation
>>             National Severe Storms Laboratory
>>
>>
>>
>>
>>         --
>>         Blair Trosper
>>         Weather Data / Updraft Networks
>>
>>         blair.trosper@xxxxxxxxxxxxxxxxxxx <mailto:
>> blair.trosper@xxxxxxxxxx>
>>
>>
>>         NOC: 512-666-0536
>>
>>
>>
>>
>>     --
>>     “Climate is what you expect, weather is what you get."
>>       -- Robert A. Heinlein
>>
>>     -------------------------------------------
>>
>>     Karen.Cooper@xxxxxxxx <mailto:Karen.Cooper@xxxxxxxx>
>>
>>     Phone: 405-325-6982 <tel:405-325-6982>
>>     Cell: 405-834-8559 <tel:405-834-8559>
>>
>>
>>     INDUS Corporation
>>     National Severe Storms Laboratory
>>
>>
>>
>>
>> --
>> Blair Trosper
>> Weather Data / Updraft Networks
>>
>> blair.trosper@xxxxxxxxxxxxxxxxxxx <mailto:blair.trosper@xxxxxxxxxx>
>>
>>
>> NOC: 512-666-0536
>>
>>
>> _______________________________________________
>> ldm-users mailing list
>> ldm-users@xxxxxxxxxxxxxxxx
>> For list information or to unsubscribe,  visit:
>> http://www.unidata.ucar.edu/mailing_lists/
>>
>>
>> _______________________________________________
>> ldm-users mailing list
>> ldm-users@xxxxxxxxxxxxxxxx
>> For list information or to unsubscribe,  visit:
>> http://www.unidata.ucar.edu/mailing_lists/
>>
>>
>>
>>
>>
>> --
>>
>> Blair Trosper
>>
>> Weather Data / Updraft Networks
>>
>> blair.trosper@xxxxxxxxxxxxxxxxxxx <blair.trosper@xxxxxxxxxx>
>>
>> NOC:  512-666-0536
>>
>>
>> _______________________________________________
>> ldm-users mailing list
>> ldm-users@xxxxxxxxxxxxxxxx
>> For list information or to unsubscribe,  visit:
>> http://www.unidata.ucar.edu/mailing_lists/
>>
>>
>>
>>
>> --
>> “Climate is what you expect, weather is what you get."
>>  -- Robert A. Heinlein
>>
>> -------------------------------------------
>> Karen.Cooper@xxxxxxxx
>>
>> Phone:  405-325-6982
>> Cell:   405-834-8559
>>
>>
>> INDUS Corporation
>> National Severe Storms Laboratory
>>
>>
>>
>>
>>
>> --
>>
>> Blair Trosper
>>
>> Weather Data / Updraft Networks
>>
>> blair.trosper@xxxxxxxxxxxxxxxxxxx <blair.trosper@xxxxxxxxxx>
>>
>> NOC:  512-666-0536
>>
>> _______________________________________________
>> ldm-users mailing list
>> ldm-users@xxxxxxxxxxxxxxxx
>> For list information or to unsubscribe,  visit:
>> http://www.unidata.ucar.edu/mailing_lists/
>>
>>
>>
>
>
>
>  --
> Blair Trosper
> Weather Data / Updraft Networks
> blair.trosper@xxxxxxxxxxxxxxxxxxx <blair.trosper@xxxxxxxxxx>
> NOC:  512-666-0536
>
>
> _______________________________________________
> ldm-users mailing listldm-users@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe,  visit: 
> http://www.unidata.ucar.edu/mailing_lists/
>
>
>
> _______________________________________________
> ldm-users mailing list
> ldm-users@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe,  visit:
> http://www.unidata.ucar.edu/mailing_lists/
>



-- 
Blair Trosper
Weather Data / Updraft Networks
blair.trosper@xxxxxxxxxxxxxxxxxxx <blair.trosper@xxxxxxxxxx>
NOC:  512-666-0536

_______________________________________________
ldm-users mailing list
ldm-users@xxxxxxxxxxxxxxxx
For list information or to unsubscribe,  visit:
http://www.unidata.ucar.edu/mailing_lists/
  • 2013 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the ldm-users archives: