We are spinning up a new Vad Wind Profiler dataflow and using the "ucnids"
program to decompress the data via LDM. Sometimes the output file is
decompressed, and other times it is NOT decompressed. I don't see any
orphaned ucnids processes or entries in the ldmd.log file until I turn on
verbose logging. The pqact entry is:
NEXRAD3 ^SDUS3. .... ([0-3][0-9])([0-2][0-9])([0-5][0-9]).*/p(NVW)(...)
PIPE -close /nfsops/ops_users/ldmopsng1/util/ucnids -n -
radar/nflowvad/\5/prod48/(\1:yy)(\1:mm)\1\2.\3
For the TLX 88D site in the past 24 hours there are about 151 files saved
(i.e. data is scrubbed after 24 hours). Of these 151 files, 73 files (i.e.
~half of the files) are not UNcompressed.
-rw-r--r--. 1 ldmopsng1 users 8598 Feb 25 20:53 21022520.43
-rw-r--r--. 1 ldmopsng1 users 8780 Feb 25 21:03 21022520.53
-rw-r--r--. 1 ldmopsng1 users 8968 Feb 25 21:12 21022521.03
-rw-r--r--. 1 ldmopsng1 users 3946 Feb 25 21:22 21022521.12 NOT
decompressed
-rw-r--r--. 1 ldmopsng1 users 3946 Feb 25 21:31 21022521.22 NOT
decompressed
The version of LDM is 6.13.11.
When I have set ldmd.log to be verbose for the pqact processing this data I
see entries like this, but the file was NOT decompressed (2131Z 21022521.22
file):
20210225T212217.975508Z pqact[11328]
palt.c:processProduct:1340 INFO 2650 20210225212217.885403
NEXRAD3 1145245 SDUS34 KOUN 252112 /pNVWTLX
20210225T212217.978774Z pqact[11328]
filel.c:fl_removeAndFree:425 INFO Deleting closed PIPE entry:
pid=6990, cmd="-close /nfsops/ops_users/ldmopsng1/util/ucnids -n -
radar/nflowvad/TLX/prod48/21022521.12"
20210225T212217.978880Z pqact[6990] filel.c:pipe_open:1798
INFO Executing decoder
"/nfsops/ops_users/ldmopsng1/util/ucnids"
It is odd the INFO for executing the decoder is in the log file after the
entry about LDM deleting the closed PIPE.
Below are the log entries for the 2113Z 21022521.03 file that WAS
decompressed:
20210225T211240.207544Z pqact[11328]
palt.c:processProduct:1340 INFO 2699 20210225211240.103390
NEXRAD3 1125246 SDUS34 KOUN 252103 /pNVWTLX
20210225T211240.209361Z pqact[17988] filel.c:pipe_open:1798
INFO Executing decoder
"/nfsops/ops_users/ldmopsng1/util/ucnids"
20210225T211240.211086Z pqact[11328]
filel.c:fl_removeAndFree:425 INFO Deleting closed PIPE entry:
pid=17989, cmd="-close /nfsops/ops_users/ldmopsng1/util/ucnids -n -
radar/nflowvad/TLX/prod48/21022521.03"
So both log entries look similar, and I'm not sure why one file is
decompressed and another is not. The only thing I see different is the
order of the log entries, the one out of order isn't DEcompressed, but I
suspect this has more to do the with system logging and is a coincidence.
The limits for the number of open files has been increased on this server.
The OPEN_MAX, ulimit -a, limit descriptors, was changed from 1021 for the
LDM user to 40,000 soft limit and 40960 for the hard limit.
Has anyone seen something like this with ucnids, or another type of
decoder, determined what was going on and have suggestions?
Thanks,
Gregg
--
**********************
*I'm primarily telecommuting. If you need to reach me, call my work cell
405-249-5310.*
*========================================================================Email
seems to be generating increasing inefficiencies in organizations. I
learned from a manager a Stanford Computer Science professor no longer uses
email for communication, but uses SNAIL mail, telephone calls, and person
to person visits. I'm considering the same. 405-325-2462*
*Storm Prediction Center*
*120 David L. Boren Blvd, Suite 2330Norman, OK 73072*
