Re: [ldm-users] my data seems to be blackhole-ing

Seems like the support system is working well to me.  Why try to kill it?

Patrick
=====================================
Dr. Patrick S. Market
Professor of Synoptic Meteorology
Dept. of Soil, Env. & Atmospheric Sciences
University of Missouri
302 ABNR, Columbia, MO  65211
Voice:  573-882-1496  /  Fax:  573-884-5070

From: ldm-users-bounces@xxxxxxxxxxxxxxxx 
[mailto:ldm-users-bounces@xxxxxxxxxxxxxxxx] On Behalf Of Charles Watson
Sent: Monday, July 15, 2013 5:04 PM
To: Blair Trosper
Cc: LDM Users
Subject: Re: [ldm-users] my data seems to be blackhole-ing

Hai Guys!

As fascinating as this discussion is, any chance you could take it "off list", 
since it doesn't seem to be a global issue?

kthxbye!

Chuck

Chuck Watson
Watson Technical Consulting

http://hurricane.methaz.org<http://hurricane.methaz.org/>,  Tel 912.398.9753
http://satblog.methaz.org for commentary on hazard modeling and forecasts

"In this house we obey the laws of thermodynamics!"
-- Homer Simpson

On Jul 15, 2013, at 6:00 PM, Blair Trosper wrote:


It's definitely running:

ldm@chicago-il-1:~$ ps aux | grep pqact
ldm      32737  0.3  0.3 8046468 50636 ?       S    16:53   0:01 pqact -v

On Mon, Jul 15, 2013 at 4:59 PM, Karen Cooper - NOAA Affiliate 
<karen.cooper@xxxxxxxx<mailto:karen.cooper@xxxxxxxx>> wrote:
Assuming that regular ldm logging is working it should not log anywhere else.
If you are seeing the data in your queue, but nothing new on disk (no changes 
to df -k) then I would question if pqact is running.
pqact in verbose mode should be loggin each item that goes through your queue 
-- even if it is ignoring it.
Another thing to check is to see is pqact is actually running and not crashing. 
 If you do a ps (or run top) can you see it?


On Mon, Jul 15, 2013 at 4:54 PM, Blair Trosper 
<blair.trosper@xxxxxxxxxxxxxxxxxxx<mailto:blair.trosper@xxxxxxxxxxxxxxxxxxx>> 
wrote:
Absolute paths were a no go, and putting verbose mode into:
EXEC "pqact -v"

...seems to yield no extra info.  Or, if it is, it's putting it somewhere 
besides ldmd.log

On Mon, Jul 15, 2013 at 4:52 PM, donna Cote 
<d-cote@xxxxxxxx<mailto:d-cote@xxxxxxxx>> wrote:
Blair, from my experience, I've made explicit directory paths in my pqacts.

I suggest looking for a /mnt/data or /nexrad2 directory

Just a thought,
Donna


On 7/15/13 4:45 PM, Blair Trosper wrote:
My which I mean it only adds the contents of "ldmadmin watch" to the
ldmd.log, which doesn't really help me.

I used "ldmadmin restart -v", did you mean something else?  Perhaps I
misunderstood.  :/


On Mon, Jul 15, 2013 at 4:44 PM, Karen Cooper - NOAA Affiliate
<karen.cooper@xxxxxxxx<mailto:karen.cooper@xxxxxxxx> 
<mailto:karen.cooper@xxxxxxxx<mailto:karen.cooper@xxxxxxxx>>> wrote:

    Try putting pqact into verbose mode.

    exec "pqact -v"

    and then restart ldm.  It should tell you everything pqact is doing.


    On Mon, Jul 15, 2013 at 4:42 PM, Blair Trosper
    <blair.trosper@xxxxxxxxxxxxxxxxxxx<mailto:blair.trosper@xxxxxxxxxxxxxxxxxxx>
    
<mailto:blair.trosper@xxxxxxxxxxxxxxxxxxx<mailto:blair.trosper@xxxxxxxxxxxxxxxxxxx>>>
 wrote:

        It's enabled and not commented.

        Putting ldm into verbose mode didn't seem to yield anything
        additional in the log files.  In fact, it's very UNdescriptive.


        On Mon, Jul 15, 2013 at 4:41 PM, Karen Cooper - NOAA Affiliate
        <karen.cooper@xxxxxxxx<mailto:karen.cooper@xxxxxxxx> 
<mailto:karen.cooper@xxxxxxxx<mailto:karen.cooper@xxxxxxxx>>> wrote:

            And of course what I really meant was the exec line in your
            ldmd.conf...

            exec    "pqact "


            On Mon, Jul 15, 2013 at 4:40 PM, Karen Cooper - NOAA
            Affiliate <karen.cooper@xxxxxxxx<mailto:karen.cooper@xxxxxxxx>
            <mailto:karen.cooper@xxxxxxxx<mailto:karen.cooper@xxxxxxxx>>> wrote:

                Blair,

                First a sanity check. -- Are you sure that pqact is
                uncommented in your ldmadmin.pl<http://ldmadmin.pl/> 
<http://ldmadmin.pl<http://ldmadmin.pl/>>?


                You can set pqact to verbose mode and then you should
                see what it's trying to do, as it will log to your
                ldmd.log files.  I often use this when I'm testing out
                new pqact entries that aren't working as I expect.


                On Mon, Jul 15, 2013 at 4:35 PM, Blair Trosper
                
<blair.trosper@xxxxxxxxxxxxxxxxxxx<mailto:blair.trosper@xxxxxxxxxxxxxxxxxxx>
                
<mailto:blair.trosper@xxxxxxxxxxxxxxxxxxx<mailto:blair.trosper@xxxxxxxxxxxxxxxxxxx>>>
 wrote:

                    I'm receiving several feeds as proven by ldmadmin
                    watch...and the logs all look fine.  However, my
                    data isn't being saved to disk.
                     From *registry.xml*:


                           <pqact>

                        <config-path>/mnt/data/ldm/etc/pqact.conf</config-path>

                        <datadir-path>/mnt/data/ldm/var/data</datadir-path>
                           </pqact>

                     From */mnt/data/ldm/etc/pqact.conf*:


                        # L2 data
                        NEXRAD2
                        
^L2-BZIP2/(....)/([0-9]{8})([0-9]{4})[0-9]{2}/([0-9]{1,4}).*/[SI]
                                 FILE    nexrad2/\1/\1_\2-\3.part
                        NEXRAD2
                        
^L2-BZIP2/(....)/([0-9]{8})([0-9]{4})[0-9]{2}/([0-9]{1,4}).*/E
                                 FILE    -close nexrad2/\1/\1_\2-\3.part
                        (snip)


                    However, following those two things, here's a
                    directory listing of */mnt/data/ldm/var/data*:


                        ldm@chicago-il-1:~/data$ ls -lsha
                        total 8.0K
                        4.0K drwxr-xr-x 2 ldm ldm 4.0K Jul 15 16:09 .
                        4.0K drwxr-xr-x 5 ldm ldm 4.0K Jul 13 23:19 ..


                    Nothing.  The same is true of other feeds which
                    should be stashed in other subdirectories.

                    The bandwidth is certainly registering, which I have
                    verified at the router, so I'm definitely receiving
                    the data...so...

                    Where are my files going?  Any way to track this down?

                    (I've also done "du -ksh" from the home directory of
                    the ldm user over the last few hours, and it's not
                    increasing.  Definitely nothing being written to
                    hard disk.  And still no errors in the LDM or system
                    logs.)

                    --
                    Blair Trosper
                    Weather Data / Updraft Networks
                    
blair.trosper@xxxxxxxxxxxxxxxxxxx<mailto:blair.trosper@xxxxxxxxxxxxxxxxxxx>
                    
<mailto:blair.trosper@xxxxxxxxxx<mailto:blair.trosper@xxxxxxxxxx>>

                    NOC: 512-666-0536<tel:512-666-0536>

                    _______________________________________________
                    ldm-users mailing list
                    
ldm-users@xxxxxxxxxxxxxxxx<mailto:ldm-users@xxxxxxxxxxxxxxxx>
                    
<mailto:ldm-users@xxxxxxxxxxxxxxxx<mailto:ldm-users@xxxxxxxxxxxxxxxx>>

                    For list information or to unsubscribe,  visit:
                    http://www.unidata.ucar.edu/mailing_lists/




                --
                "Climate is what you expect, weather is what you get."
                  -- Robert A. Heinlein

                -------------------------------------------
                Karen.Cooper@xxxxxxxx<mailto:Karen.Cooper@xxxxxxxx> 
<mailto:Karen.Cooper@xxxxxxxx<mailto:Karen.Cooper@xxxxxxxx>>

                Phone: 405-325-6982<tel:405-325-6982> 
<tel:405-325-6982<tel:405-325-6982>>
                Cell: 405-834-8559<tel:405-834-8559> 
<tel:405-834-8559<tel:405-834-8559>>

                INDUS Corporation
                National Severe Storms Laboratory




            --
            "Climate is what you expect, weather is what you get."
              -- Robert A. Heinlein

            -------------------------------------------
            Karen.Cooper@xxxxxxxx<mailto:Karen.Cooper@xxxxxxxx> 
<mailto:Karen.Cooper@xxxxxxxx<mailto:Karen.Cooper@xxxxxxxx>>

            Phone: 405-325-6982<tel:405-325-6982> 
<tel:405-325-6982<tel:405-325-6982>>
            Cell: 405-834-8559<tel:405-834-8559> 
<tel:405-834-8559<tel:405-834-8559>>

            INDUS Corporation
            National Severe Storms Laboratory




        --
        Blair Trosper
        Weather Data / Updraft Networks
        
blair.trosper@xxxxxxxxxxxxxxxxxxx<mailto:blair.trosper@xxxxxxxxxxxxxxxxxxx> 
<mailto:blair.trosper@xxxxxxxxxx<mailto:blair.trosper@xxxxxxxxxx>>

        NOC: 512-666-0536<tel:512-666-0536>




    --
    "Climate is what you expect, weather is what you get."
      -- Robert A. Heinlein

    -------------------------------------------
    Karen.Cooper@xxxxxxxx<mailto:Karen.Cooper@xxxxxxxx> 
<mailto:Karen.Cooper@xxxxxxxx<mailto:Karen.Cooper@xxxxxxxx>>

    Phone: 405-325-6982<tel:405-325-6982> <tel:405-325-6982<tel:405-325-6982>>
    Cell: 405-834-8559<tel:405-834-8559> <tel:405-834-8559<tel:405-834-8559>>

    INDUS Corporation
    National Severe Storms Laboratory




--
Blair Trosper
Weather Data / Updraft Networks
blair.trosper@xxxxxxxxxxxxxxxxxxx<mailto:blair.trosper@xxxxxxxxxxxxxxxxxxx> 
<mailto:blair.trosper@xxxxxxxxxx<mailto:blair.trosper@xxxxxxxxxx>>

NOC: 512-666-0536<tel:512-666-0536>


_______________________________________________
ldm-users mailing list
ldm-users@xxxxxxxxxxxxxxxx<mailto:ldm-users@xxxxxxxxxxxxxxxx>
For list information or to unsubscribe,  visit: 
http://www.unidata.ucar.edu/mailing_lists/

_______________________________________________
ldm-users mailing list
ldm-users@xxxxxxxxxxxxxxxx<mailto:ldm-users@xxxxxxxxxxxxxxxx>
For list information or to unsubscribe,  visit: 
http://www.unidata.ucar.edu/mailing_lists/



--
Blair Trosper
Weather Data / Updraft Networks
blair.trosper@xxxxxxxxxxxxxxxxxxx<mailto:blair.trosper@xxxxxxxxxx>
NOC:  512-666-0536

_______________________________________________
ldm-users mailing list
ldm-users@xxxxxxxxxxxxxxxx<mailto:ldm-users@xxxxxxxxxxxxxxxx>
For list information or to unsubscribe,  visit: 
http://www.unidata.ucar.edu/mailing_lists/



--
"Climate is what you expect, weather is what you get."
 -- Robert A. Heinlein

-------------------------------------------
Karen.Cooper@xxxxxxxx<mailto:Karen.Cooper@xxxxxxxx>
Phone:  405-325-6982<tel:405-325-6982>
Cell:   405-834-8559<tel:405-834-8559>

INDUS Corporation
National Severe Storms Laboratory



--
Blair Trosper
Weather Data / Updraft Networks
blair.trosper@xxxxxxxxxxxxxxxxxxx<mailto:blair.trosper@xxxxxxxxxx>
NOC:  512-666-0536
_______________________________________________
ldm-users mailing list
ldm-users@xxxxxxxxxxxxxxxx<mailto:ldm-users@xxxxxxxxxxxxxxxx>
For list information or to unsubscribe,  visit: 
http://www.unidata.ucar.edu/mailing_lists/

  • 2013 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the ldm-users archives: