Re: [ldm-users] my data seems to be blackhole-ing

Run pqact from the command line (as user ldm):

[ldm@aawu64 pqact]$ pqact -vx -l -

Then post the output.

George

On 07/15/2013 10:10 PM, Blair Trosper wrote:
While I'm not sure I understand Charles's hastiness to get this off list, I am inclined to chalk this up to "freak accident" and just attempt reinstalling LDM from the start on the server in question.

Clearly something isn't right with it...and it's worth a clean go.


On Mon, Jul 15, 2013 at 5:08 PM, Market, Patrick S. <MarketP@xxxxxxxxxxxx <mailto:MarketP@xxxxxxxxxxxx>> wrote:

    Seems like the support system is working well to me.  Why try to
    kill it?

    Patrick

    =====================================

    Dr. Patrick S. Market

    Professor of Synoptic Meteorology

    Dept. of Soil, Env. & Atmospheric Sciences

    University of Missouri

    302 ABNR, Columbia, MO  65211

    Voice: 573-882-1496 <tel:573-882-1496> /  Fax: 573-884-5070
    <tel:573-884-5070>

    *From:*ldm-users-bounces@xxxxxxxxxxxxxxxx
    <mailto:ldm-users-bounces@xxxxxxxxxxxxxxxx>
    [mailto:ldm-users-bounces@xxxxxxxxxxxxxxxx
    <mailto:ldm-users-bounces@xxxxxxxxxxxxxxxx>] *On Behalf Of
    *Charles Watson
    *Sent:* Monday, July 15, 2013 5:04 PM


    *To:* Blair Trosper
    *Cc:* LDM Users
    *Subject:* Re: [ldm-users] my data seems to be blackhole-ing

    Hai Guys!

    As fascinating as this discussion is, any chance you could take it
    "off list", since it doesn't seem to be a global issue?

    kthxbye!

    Chuck

    Chuck Watson

    Watson Technical Consulting

    http://hurricane.methaz.org <http://hurricane.methaz.org/>,  Tel
    912.398.9753 <tel:912.398.9753>

    http://satblog.methaz.org for commentary on hazard modeling and
    forecasts

    "In this house we obey the laws of thermodynamics!"

    -- Homer Simpson

    On Jul 15, 2013, at 6:00 PM, Blair Trosper wrote:



    It's definitely running:

    ldm@chicago-il-1:~$ ps aux | grep pqact

    ldm      32737  0.3  0.3 8046468 50636 ?       S    16:53 0:01
    pqact -v

    On Mon, Jul 15, 2013 at 4:59 PM, Karen Cooper - NOAA Affiliate
    <karen.cooper@xxxxxxxx <mailto:karen.cooper@xxxxxxxx>> wrote:

    Assuming that regular ldm logging is working it should not log
    anywhere else.

    If you are seeing the data in your queue, but nothing new on disk
    (no changes to df -k) then I would question if pqact is running.

    pqact in verbose mode should be loggin each item that goes through
    your queue -- even if it is ignoring it.

    Another thing to check is to see is pqact is actually running and
    not crashing.  If you do a ps (or run top) can you see it?

    On Mon, Jul 15, 2013 at 4:54 PM, Blair Trosper
    <blair.trosper@xxxxxxxxxxxxxxxxxxx
    <mailto:blair.trosper@xxxxxxxxxxxxxxxxxxx>> wrote:

    Absolute paths were a no go, and putting verbose mode into:
    EXEC "pqact -v"

    ...seems to yield no extra info.  Or, if it is, it's putting it
    somewhere besides ldmd.log

    On Mon, Jul 15, 2013 at 4:52 PM, donna Cote <d-cote@xxxxxxxx
    <mailto:d-cote@xxxxxxxx>> wrote:

    Blair, from my experience, I've made explicit directory paths in
    my pqacts.

    I suggest looking for a /mnt/data or /nexrad2 directory

    Just a thought,
    Donna



    On 7/15/13 4:45 PM, Blair Trosper wrote:

        My which I mean it only adds the contents of "ldmadmin watch"
        to the
        ldmd.log, which doesn't really help me.

        I used "ldmadmin restart -v", did you mean something else?
         Perhaps I
        misunderstood.  :/


        On Mon, Jul 15, 2013 at 4:44 PM, Karen Cooper - NOAA Affiliate

        <karen.cooper@xxxxxxxx <mailto:karen.cooper@xxxxxxxx>
        <mailto:karen.cooper@xxxxxxxx <mailto:karen.cooper@xxxxxxxx>>>
        wrote:

            Try putting pqact into verbose mode.

            exec "pqact -v"

            and then restart ldm.  It should tell you everything pqact
        is doing.


            On Mon, Jul 15, 2013 at 4:42 PM, Blair Trosper
            <blair.trosper@xxxxxxxxxxxxxxxxxxx
        <mailto:blair.trosper@xxxxxxxxxxxxxxxxxxx>

        <mailto:blair.trosper@xxxxxxxxxxxxxxxxxxx
        <mailto:blair.trosper@xxxxxxxxxxxxxxxxxxx>>> wrote:

                It's enabled and not commented.

                Putting ldm into verbose mode didn't seem to yield
        anything
                additional in the log files.  In fact, it's very
        UNdescriptive.


                On Mon, Jul 15, 2013 at 4:41 PM, Karen Cooper - NOAA
        Affiliate

            <karen.cooper@xxxxxxxx <mailto:karen.cooper@xxxxxxxx>
        <mailto:karen.cooper@xxxxxxxx <mailto:karen.cooper@xxxxxxxx>>>
        wrote:

                    And of course what I really meant was the exec
        line in your
        ldmd.conf...

                    exec  "pqact "


                    On Mon, Jul 15, 2013 at 4:40 PM, Karen Cooper - NOAA
                    Affiliate <karen.cooper@xxxxxxxx
        <mailto:karen.cooper@xxxxxxxx>

                <mailto:karen.cooper@xxxxxxxx
        <mailto:karen.cooper@xxxxxxxx>>> wrote:

                        Blair,

                        First a sanity check. -- Are you sure that
        pqact is

                  uncommented in your ldmadmin.pl
        <http://ldmadmin.pl/> <http://ldmadmin.pl <http://ldmadmin.pl/>>?



                        You can set pqact to verbose mode and then you
        should
                        see what it's trying to do, as it will log to your
        ldmd.log files.  I often use this when I'm testing out
                        new pqact entries that aren't working as I expect.


                        On Mon, Jul 15, 2013 at 4:35 PM, Blair Trosper
                        <blair.trosper@xxxxxxxxxxxxxxxxxxx
        <mailto:blair.trosper@xxxxxxxxxxxxxxxxxxx>

        <mailto:blair.trosper@xxxxxxxxxxxxxxxxxxx
        <mailto:blair.trosper@xxxxxxxxxxxxxxxxxxx>>> wrote:

        I'm receiving several feeds as proven by ldmadmin
        watch...and the logs all look fine.  However, my
        data isn't being saved to disk.

                       From *registry.xml*:



             <pqact>

        <config-path>/mnt/data/ldm/etc/pqact.conf</config-path>

        <datadir-path>/mnt/data/ldm/var/data</datadir-path>
             </pqact>

                       From */mnt/data/ldm/etc/pqact.conf*:



          # L2 data
          NEXRAD2
        ^L2-BZIP2/(....)/([0-9]{8})([0-9]{4})[0-9]{2}/([0-9]{1,4}).*/[SI]
                   FILE  nexrad2/\1/\1_\2-\3.part
          NEXRAD2
        ^L2-BZIP2/(....)/([0-9]{8})([0-9]{4})[0-9]{2}/([0-9]{1,4}).*/E
                   FILE  -close nexrad2/\1/\1_\2-\3.part
          (snip)


        However, following those two things, here's a

                      directory listing of */mnt/data/ldm/var/data*:



        ldm@chicago-il-1:~/data$ ls -lsha
          total 8.0K
          4.0K drwxr-xr-x 2 ldm ldm 4.0K Jul 15 16:09 .
          4.0K drwxr-xr-x 5 ldm ldm 4.0K Jul 13 23:19 ..


        Nothing.  The same is true of other feeds which
        should be stashed in other subdirectories.

        The bandwidth is certainly registering, which I have
        verified at the router, so I'm definitely receiving
        the data...so...

        Where are my files going?  Any way to track this down?

        (I've also done "du -ksh" from the home directory of
        the ldm user over the last few hours, and it's not
        increasing.  Definitely nothing being written to
        hard disk.  And still no errors in the LDM or system
        logs.)

                            --
        Blair Trosper
        Weather Data / Updraft Networks
        blair.trosper@xxxxxxxxxxxxxxxxxxx
        <mailto:blair.trosper@xxxxxxxxxxxxxxxxxxx>

        <mailto:blair.trosper@xxxxxxxxxx
        <mailto:blair.trosper@xxxxxxxxxx>>


        NOC: 512-666-0536 <tel:512-666-0536>

        _______________________________________________
        ldm-users mailing list
        ldm-users@xxxxxxxxxxxxxxxx <mailto:ldm-users@xxxxxxxxxxxxxxxx>

        <mailto:ldm-users@xxxxxxxxxxxxxxxx
        <mailto:ldm-users@xxxxxxxxxxxxxxxx>>


        For list information or to unsubscribe,  visit:
        http://www.unidata.ucar.edu/mailing_lists/




                        --
        "Climate is what you expect, weather is what you get."
                          -- Robert A. Heinlein

        -------------------------------------------

        Karen.Cooper@xxxxxxxx <mailto:Karen.Cooper@xxxxxxxx>
        <mailto:Karen.Cooper@xxxxxxxx <mailto:Karen.Cooper@xxxxxxxx>>

                        Phone: 405-325-6982 <tel:405-325-6982>
        <tel:405-325-6982 <tel:405-325-6982>>
                        Cell: 405-834-8559 <tel:405-834-8559>
        <tel:405-834-8559 <tel:405-834-8559>>


                        INDUS Corporation
        National Severe Storms Laboratory




                    --
                    "Climate is what you expect, weather is what you get."
                      -- Robert A. Heinlein

        -------------------------------------------

        Karen.Cooper@xxxxxxxx <mailto:Karen.Cooper@xxxxxxxx>
        <mailto:Karen.Cooper@xxxxxxxx <mailto:Karen.Cooper@xxxxxxxx>>

                    Phone: 405-325-6982 <tel:405-325-6982>
        <tel:405-325-6982 <tel:405-325-6982>>
                    Cell: 405-834-8559 <tel:405-834-8559>
        <tel:405-834-8559 <tel:405-834-8559>>


                    INDUS Corporation
                    National Severe Storms Laboratory




                --
                Blair Trosper
                Weather Data / Updraft Networks

        blair.trosper@xxxxxxxxxxxxxxxxxxx
        <mailto:blair.trosper@xxxxxxxxxxxxxxxxxxx>
        <mailto:blair.trosper@xxxxxxxxxx
        <mailto:blair.trosper@xxxxxxxxxx>>


                NOC: 512-666-0536 <tel:512-666-0536>




            --
            "Climate is what you expect, weather is what you get."
              -- Robert A. Heinlein

        -------------------------------------------

        Karen.Cooper@xxxxxxxx <mailto:Karen.Cooper@xxxxxxxx>
        <mailto:Karen.Cooper@xxxxxxxx <mailto:Karen.Cooper@xxxxxxxx>>

            Phone: 405-325-6982 <tel:405-325-6982> <tel:405-325-6982
        <tel:405-325-6982>>
            Cell: 405-834-8559 <tel:405-834-8559> <tel:405-834-8559
        <tel:405-834-8559>>


            INDUS Corporation
            National Severe Storms Laboratory




        --
        Blair Trosper
        Weather Data / Updraft Networks

        blair.trosper@xxxxxxxxxxxxxxxxxxx
        <mailto:blair.trosper@xxxxxxxxxxxxxxxxxxx>
        <mailto:blair.trosper@xxxxxxxxxx
        <mailto:blair.trosper@xxxxxxxxxx>>


        NOC: 512-666-0536 <tel:512-666-0536>


        _______________________________________________
        ldm-users mailing list
        ldm-users@xxxxxxxxxxxxxxxx <mailto:ldm-users@xxxxxxxxxxxxxxxx>
        For list information or to unsubscribe,  visit:
        http://www.unidata.ucar.edu/mailing_lists/


    _______________________________________________
    ldm-users mailing list
    ldm-users@xxxxxxxxxxxxxxxx <mailto:ldm-users@xxxxxxxxxxxxxxxx>
    For list information or to unsubscribe,  visit:
    http://www.unidata.ucar.edu/mailing_lists/



--
    Blair Trosper

    Weather Data / Updraft Networks

    blair.trosper@xxxxxxxxxxxxxxxxxxx <mailto:blair.trosper@xxxxxxxxxx>

    NOC: 512-666-0536 <tel:512-666-0536>


    _______________________________________________
    ldm-users mailing list
    ldm-users@xxxxxxxxxxxxxxxx <mailto:ldm-users@xxxxxxxxxxxxxxxx>
    For list information or to unsubscribe,  visit:
    http://www.unidata.ucar.edu/mailing_lists/




-- "Climate is what you expect, weather is what you get."
     -- Robert A. Heinlein

    -------------------------------------------
    Karen.Cooper@xxxxxxxx <mailto:Karen.Cooper@xxxxxxxx>

    Phone: 405-325-6982 <tel:405-325-6982>
    Cell: 405-834-8559 <tel:405-834-8559>


    INDUS Corporation
    National Severe Storms Laboratory



--
    Blair Trosper

    Weather Data / Updraft Networks

    blair.trosper@xxxxxxxxxxxxxxxxxxx <mailto:blair.trosper@xxxxxxxxxx>

    NOC: 512-666-0536 <tel:512-666-0536>

    _______________________________________________
    ldm-users mailing list
    ldm-users@xxxxxxxxxxxxxxxx <mailto:ldm-users@xxxxxxxxxxxxxxxx>
    For list information or to unsubscribe,  visit:
    http://www.unidata.ucar.edu/mailing_lists/




--
Blair Trosper
Weather Data / Updraft Networks
blair.trosper@xxxxxxxxxxxxxxxxxxx <mailto:blair.trosper@xxxxxxxxxx>
NOC: 512-666-0536


_______________________________________________
ldm-users mailing list
ldm-users@xxxxxxxxxxxxxxxx
For list information or to unsubscribe,  visit: 
http://www.unidata.ucar.edu/mailing_lists/

  • 2013 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the ldm-users archives: