[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

20050407: zlibg2gini decoding of NIMAGE GINI data (cont.)

>From: Celia Chen <address@hidden>
>Organization: NCAR/RAP
>Keywords: 200504061932.j36JWtv2010902 ldm-mcidas zlibg2gini

Hi Tom,

re:
>Here are ALL the files in /gini/20050406/EAST-CONUS/4km/IR from yesterday and 
>there is NOTHING since the last file being captured:
>
>total 64328
>drwxr-xr-x    2 ldm      ldm          4096 Apr  6 11:59 .
>drwxr-xr-x    6 ldm      ldm          4096 Apr  6 00:41 ..
>-rw-r--r--    1 ldm      ldm       1638933 Apr  5 18:25 IR_20050406_0015
>-rw-r--r--    1 ldm      ldm       1638933 Apr  5 18:58 IR_20050406_0045
 ...
>-rw-r--r--    1 ldm      ldm       1638933 Apr  6 00:11 IR_20050406_0600
>-rw-r--r--    1 ldm      ldm       1638933 Apr  6 00:41 IR_20050406_0630
>-rw-r--r--    1 ldm      ldm       1638933 Apr  6 01:08 IR_20050406_0701
>-rw-r--r--    1 ldm      ldm       1638933 Apr  6 01:24 IR_20050406_0715
>-rw-r--r--    1 ldm      ldm       1638933 Apr  6 01:37 IR_20050406_0731
>-rw-r--r--    1 ldm      ldm       1638933 Apr  6 01:54 IR_20050406_0745
>-rw-r--r--    1 ldm      ldm       1638933 Apr  6 02:10 IR_20050406_0801
>-rw-r--r--    1 ldm      ldm       1638933 Apr  6 02:29 IR_20050406_0815
>-rw-r--r--    1 ldm      ldm       1638933 Apr  6 02:58 IR_20050406_0845
>-rw-r--r--    1 ldm      ldm       1638933 Apr  6 03:29 IR_20050406_0915
>-rw-r--r--    1 ldm      ldm       1638933 Apr  6 03:58 IR_20050406_0945
>-rw-r--r--    1 ldm      ldm       1638933 Apr  6 04:30 IR_20050406_1015
>-rw-r--r--    1 ldm      ldm       1638933 Apr  6 04:57 IR_20050406_1045
>-rw-r--r--    1 ldm      ldm       1638933 Apr  6 05:11 IR_20050406_1101
>-rw-r--r--    1 ldm      ldm       1638933 Apr  6 05:28 IR_20050406_1115
>-rw-r--r--    1 ldm      ldm       1638933 Apr  6 11:59 IR_20050406_1745

For the same time period, do you see _any_ zlibg2gini entries in your
log file?  If there are _no_ zlibg2gini entries in the log file, I
would guess that either:

- the entry for NIMAGE GINI decoding got changed incorrectly

- the header for the NIMAGE GINI images changed such that your
  pqact.conf entry for zlibg2gini processing no longer matches

- a modification was made in your pqact.conf file _before_ the
  zlibg2gini entry AND that modification has a problem.

So, the first thing to check is if the zlibg2gini entry in pqact.conf
is OK.  If your action is in ~ldm/etc/pqact.conf, you can use
'ldmamdin' to check the file's inegrity:

ldmadmin pqactcheck

If this shows an error, I would venture to guess that it is either in
the zlibg2gini action or in a different action earlier it in the
pattern-action file.

If your pqact.conf file is not named 'pqact.conf', 'ldmadmin
pqactcheck' will need to be given the fully qualified pathname to the
pattern-action file.  Here is an example that illustrates this:

ldmadmin pqactcheck -p /usr/local/ldm/etc/GEMPAK/pqact.gempak_craft

re: what machine is the processing occurring on

>rasp is the machine that I have been running zlibg2gini on.

OK.  So you have NIMAGE data from two different sources feeding that
should be decoded.

re: specific information on your outage

>I have not seen any new data being decoded since 12:01 yesterday. (As shown in
>the log file.)

We are receiving the data with no problems and you indicate that the
same is true for you.  Since we FILE our products, not decode them, I
am able to test zlibg2gini on all of the files that we have received
since the 5th.  I had no problem running the decoder on 5 different
EAST-CONUS, 4km, IR images that we received today.  This tells me that
there is nothing wrong with the images being sent in the broadcast.

Also, since our FILE action has not changed, and since we use the
product header metadata to determine where to FILE the products, and
since the products are being filed in the correct place, I a led to
believe that there has not been any change to the product headers.

re: missing all or part of the data

>Nothing since yesterday at noon - missing all.

OK

re: still have the problem

>Yes.

Given what I wrote above, I am theorizing that your problem is likely
due to a pqact.conf action error _especially_ if you have no errors in
your log file.  Please run 'ldmadmin pqactcheck' to determine if your
pqact.conf file has problems.

By the way, I checked on zlibg2gini decoding on a Unidata community
member's machine and see that things are working there:

/home/ldm/logs% grep EAST-CONUS/4km/IR ldm-mcidas.log
Apr 07 05:41:52 zlibg2gini[21874]: output file pathname: 
/data/pub/raw/gini/EAST-CONUS/4km/IR/IR_20050407_0400
Apr 07 06:11:16 zlibg2gini[26558]: output file pathname: 
/data/pub/raw/gini/EAST-CONUS/4km/IR/IR_20050407_0430
Apr 07 06:32:37 zlibg2gini[1834]: output file pathname: 
/data/pub/raw/gini/EAST-CONUS/4km/IR/IR_20050407_0530
Apr 07 07:03:30 zlibg2gini[6202]: output file pathname: 
/data/pub/raw/gini/EAST-CONUS/4km/IR/IR_20050407_0600
Apr 07 07:29:19 zlibg2gini[12232]: output file pathname: 
/data/pub/raw/gini/EAST-CONUS/4km/IR/IR_20050407_0630
Apr 07 07:51:21 zlibg2gini[16366]: output file pathname: 
/data/pub/raw/gini/EAST-CONUS/4km/IR/IR_20050407_0701
Apr 07 08:01:32 zlibg2gini[17424]: output file pathname: 
/data/pub/raw/gini/EAST-CONUS/4km/IR/IR_20050407_0715
Apr 07 08:12:20 zlibg2gini[19428]: output file pathname: 
/data/pub/raw/gini/EAST-CONUS/4km/IR/IR_20050407_0731
Apr 07 08:30:58 zlibg2gini[23873]: output file pathname: 
/data/pub/raw/gini/EAST-CONUS/4km/IR/IR_20050407_0745
Apr 07 08:43:00 zlibg2gini[26115]: output file pathname: 
/data/pub/raw/gini/EAST-CONUS/4km/IR/IR_20050407_0801
Apr 07 08:55:24 zlibg2gini[28013]: output file pathname: 
/data/pub/raw/gini/EAST-CONUS/4km/IR/IR_20050407_0815
Apr 07 09:14:32 zlibg2gini[1337]: output file pathname: 
/data/pub/raw/gini/EAST-CONUS/4km/IR/IR_20050407_0845
Apr 07 09:43:19 zlibg2gini[7771]: output file pathname: 
/data/pub/raw/gini/EAST-CONUS/4km/IR/IR_20050407_0915
Apr 07 10:39:04 zlibg2gini[17513]: output file pathname: 
/data/pub/raw/gini/EAST-CONUS/4km/IR/IR_20050407_0945
Apr 07 11:39:54 zlibg2gini[27794]: output file pathname: 
/data/pub/raw/gini/EAST-CONUS/4km/IR/IR_20050407_1015
Apr 07 12:08:00 zlibg2gini[2328]: output file pathname: 
/data/pub/raw/gini/EAST-CONUS/4km/IR/IR_20050407_1045
Apr 07 12:09:52 zlibg2gini[2611]: output file pathname: 
/data/pub/raw/gini/EAST-CONUS/4km/IR/IR_20050407_1101
Apr 07 12:12:23 zlibg2gini[3264]: output file pathname: 
/data/pub/raw/gini/EAST-CONUS/4km/IR/IR_20050407_1115
Apr 07 12:17:27 zlibg2gini[5928]: output file pathname: 
/data/pub/raw/gini/EAST-CONUS/4km/IR/IR_20050407_1131
Apr 07 12:23:29 zlibg2gini[7498]: output file pathname: 
/data/pub/raw/gini/EAST-CONUS/4km/IR/IR_20050407_1145
Apr 07 12:27:48 zlibg2gini[8378]: output file pathname: 
/data/pub/raw/gini/EAST-CONUS/4km/IR/IR_20050407_1215
Apr 07 12:57:55 zlibg2gini[13062]: output file pathname: 
/data/pub/raw/gini/EAST-CONUS/4km/IR/IR_20050407_1245
Apr 07 13:12:13 zlibg2gini[15614]: output file pathname: 
/data/pub/raw/gini/EAST-CONUS/4km/IR/IR_20050407_1301
Apr 07 13:29:11 zlibg2gini[20087]: output file pathname: 
/data/pub/raw/gini/EAST-CONUS/4km/IR/IR_20050407_1315
Apr 07 13:41:56 zlibg2gini[22552]: output file pathname: 
/data/pub/raw/gini/EAST-CONUS/4km/IR/IR_20050407_1331
Apr 07 13:59:03 zlibg2gini[24968]: output file pathname: 
/data/pub/raw/gini/EAST-CONUS/4km/IR/IR_20050407_1345
Apr 07 14:12:24 zlibg2gini[27645]: output file pathname: 
/data/pub/raw/gini/EAST-CONUS/4km/IR/IR_20050407_1401
Apr 07 14:28:37 zlibg2gini[2007]: output file pathname: 
/data/pub/raw/gini/EAST-CONUS/4km/IR/IR_20050407_1415
Apr 07 14:43:07 zlibg2gini[4322]: output file pathname: 
/data/pub/raw/gini/EAST-CONUS/4km/IR/IR_20050407_1431
Apr 07 15:06:04 zlibg2gini[8118]: output file pathname: 
/data/pub/raw/gini/EAST-CONUS/4km/IR/IR_20050407_1445
Apr 07 15:37:44 zlibg2gini[14986]: output file pathname: 
/data/pub/raw/gini/EAST-CONUS/4km/IR/IR_20050407_1515
Apr 07 16:19:55 zlibg2gini[22292]: output file pathname: 
/data/pub/raw/gini/EAST-CONUS/4km/IR/IR_20050407_1531
Apr 07 16:37:19 zlibg2gini[25527]: output file pathname: 
/data/pub/raw/gini/EAST-CONUS/4km/IR/IR_20050407_1545
Apr 07 16:57:44 zlibg2gini[28138]: output file pathname: 
/data/pub/raw/gini/EAST-CONUS/4km/IR/IR_20050407_1601
Apr 07 17:30:10 zlibg2gini[5120]: output file pathname: 
/data/pub/raw/gini/EAST-CONUS/4km/IR/IR_20050407_1631
Apr 07 17:52:37 zlibg2gini[9125]: output file pathname: 
/data/pub/raw/gini/EAST-CONUS/4km/IR/IR_20050407_1645
Apr 07 17:54:57 zlibg2gini[9477]: output file pathname: 
/data/pub/raw/gini/EAST-CONUS/4km/IR/IR_20050407_1701
Apr 07 17:58:56 zlibg2gini[10192]: output file pathname: 
/data/pub/raw/gini/EAST-CONUS/4km/IR/IR_20050407_1715
Apr 07 18:06:01 zlibg2gini[12138]: output file pathname: 
/data/pub/raw/gini/EAST-CONUS/4km/IR/IR_20050407_1731

This machine is running Solaris SPARC, so the comparison may not be
totally valid if you are decoding on a Linux machine.

>Thanks.

No worries.

Tom
--
NOTE: All email exchanges with Unidata User Support are recorded in the
Unidata inquiry tracking system and then made publicly available
through the web.  If you do not want to have your interactions made
available in this way, you must let us know in each email you send to us.


*** Later ***

It turns out that the problem was that one pqact was being asked
to process (FILE) all NEXRAD Level II data, all CONDUIT data, 
all NIMAGE data, etc.  pqact was so far behind that it was not
processing much of any data, much less NIMAGE GINI images.

The solution was to work with Celia to run multiple instances of
pqact:

- one processes all data except NEXRAD2 and CONDUIT
- four process NEXRAD2 (split up by Nexrad)
- one processes all CONDUIT data

After the change in ldmd.conf to run 6 pqacts instead of 1, Celia's
machine is processing all of the data that it receives (over 3 GB per
hour) and keeping up.

Tom