>From: Arnold Hori <address@hidden> >Organization: University of Hawaii >Keywords: 200304282142.h3SLgA7U018448 Unidata-Wisconsin GOES-10 Hi Arnold, >Hi and Thanks Tom; >We have been watching the GOES-10 ingest times and noticed the times are >back to the pre-Apr18 times. OK. I will be working with SSEC to see if we can push the times up some more (i.e., not wait until 15 minutes after the nominal satellite time before creating the sector and inserting it into the LDM queue). >However, there is a problem in that some of >the images get ingested by 20minutes past the hour but our pnga2area >decoder does not start until more than 1hour after that time. So, the images are in your queue, but are not getting processed until later? If yes, then this typically indicates that your pqact is running behind on processing. >Attached >are two files which illustrate these events. the WVIR-???? have output >from a notifyme (-f MCIDAS -p GOES-10) plus the pnga2area output to the >ldm-mcidas.log file and the ls -l of the WV or IR mcidas file. WVIR-slow >show that the feed begin at about 15-20minutes after the hour, but the >pnga2area processing did not start until up to 60minutes after the ingest >time. WVIR-slow U.HAWAII vapor.soest.hawaii.edu output from 1) notifyme -h localhost -f MCIDAS -p GOES-10 output 2) pnga2area output to ldm-mcidas.log 3) ls -l of GOES-10/8km/WV and GOES-10/4km/IR files 4) Local time is 10hours after UTC; 0400z is 1800 Local 1) Apr 30 04:16:06 notifyme: 1407486 20030430041518.230 UNIWISC 000 pnga2area Q1 U9 120 GOES-10_IMG 0.65um 4km 20030430 0400 2) Apr 30 05:28:58 pnga2area: Starting Up Apr 30 05:28:58 pnga2area: output file pathname: data/gempak/images/sat/GOES-10/8km/WV/WV_20030430_0400 Apr 30 05:28:59 pnga2area: unPNG:: 158959 613456 3.8592 Apr 30 05:28:59 pnga2area: Exiting 3) -rw-r--r-- 1 ldm businger 613456 Apr 29 19:28 WV_20030430_0400 This does indeed show that pqact is running behind: - The notifyme shows that you received the product 48 seconds after it was first injected into the IDD (04:16:06 - 041518) which is not great, but it is not a show stopper. - the output from pnga2area shows that it was started by pqact at 05:28:58, which is 1:12:52 _after_ the image was received. You will notice that pnga2area finished at 05:28:29, so it took about a second for pnga2area to do the decoding. - the ls output agrees with the pnga2area log output >The second files show some of the data which were processed by >pnga2area in a timely fashion. WVIR-fast U.Hawaii vapor.soest.hawaii.edu output from notifyme; pnga2area; ls -la of WV & IR data which got processed soon after the data were ingested Local time is 10hours after UTC; 1500z is 1500Local 1) Apr 30 15:15:24 notifyme: 170694 20030430151518.737 UNIWISC 000 pnga2area Q1 UB 173 GOES-10_IMG 6.8um 8km 20030430 1500 2) Apr 30 15:15:24 pnga2area: Starting Up Apr 30 15:15:24 pnga2area: output file pathname: data/gempak/images/sat/GOES-10/8km/WV/WV_20030430_1500 Apr 30 15:15:26 pnga2area: unPNG:: 170694 613456 3.5939 Apr 30 15:15:26 pnga2area: Exiting 3) -rw-r--r-- 1 ldm businger 613456 Apr 30 05:15 WV_20030430_1500 This shows that pqact had caught up with the processing: - the image was received 6 seconds after it was first injected into the IDD (this is more like it) - pnga2area was started by pqact almost immediately after the product was received, and it took up to 2 seconds to finish decoding - the ls output agrees with the pnga2area log output >We are unsure if this is a problem at our end or if it is a transmission >problem. It does not appear to be a transmission problem since you received the images in a reasonable amount of time (although, a 48 second delay is a little high for the images when the client and upstream server are both running LDM-6). >Up until 2000z today/Apr30, we were using version 7.6.4 of the >pnga2area but have just downloaded the 7.8.0 version. We will be checking >if this version improves the processing. Is there anything else which we >should be checking? You should be using the v2002b version of the ldm-mcidas decoders, since v2002b is the latest version. >IF you are interested in the MKWC LAPS output you will find it at > http://hokukea.soest.hawaii.edu/current/laps/index.cgi >The cloud cover Parameter Group is the field which is most impacted by >missing WV and IR data. OK, thanks for the link. Back to your problem: Is your machine heavily loaded at the time of the "slow pnga2area" processing? Is pqact using an excessive amount of CPU during the times of "slow pnga2area" processing? Is your LDM queue large enough for the volume of data you are ingesting via the IDD? Do you have a LOT of actions in ~ldm/etc/pqact.conf (typical of a GEMPAK user)? Along these lines, do you have any problematic regular expressions in your ~ldm/etc/pqact.conf file (Steve Emmerson sent out a message some time ago about potentially bad regular expressions in both ldmd.conf requests and pqact.conf actions)? If you are using a single instance of pqact in your LDM setup, and if pqact is the culprit in the slow processing, and you do not have any problematic regular expressions in pqact.conf, then you might consider running a separate invocation of pqact to processes the Unidata-Wisconsin images. For instance, if your ~ldm/etc/ldmd.conf file fires up one pqact instance to process all ~ldm/etc/pqact.conf entries: exec "pqact" you could change this to two invocations, one for the UNIWISC images and the other for everything else: exec "pqact -f UNIWISC /usr/local/ldm/etc/pqact.uniwisc" exec "pqact -f ANY-UNIWISC /usr/local/ldm/etc/pqact.conf" Of course, you will need to move (not copy) all of your UNIWISC (aka MCIDAS) actions from ~ldm/etc/pqact.conf into ~ldm/etc/pqact.uniwisc and then stop and restart your LDM for this to have any effect. Tom >From address@hidden Thu May 1 15:51:44 2003 Hi Tom; Thanks for the analysis of yesterday. We do a lot of gempak decoding actions. Last week, we also had Steve Emmerson tuneup our LDM; he made some corrections to our pqact.conf file but said that they were mostly in good order so I have left pqact.conf unchanged. But I have increased the ldm.pq queue to 50MB (from 40MB) and installed the ldm-mcidas-2002b programs to our machine. And since that time, we have been getting GOES-10 files in a timely fashion. I have not yet checked on the LDM system load but suspect that you are correct about our cpu getting overloaded at some times. I have changed the time of our syscheck cron job to have it check our LDM machine at about the time the GOES-10 files are getting processed. The Mauna Kea Weather Center folks are now very happy with the situation and they also send their Mahalo Nui Loa (i.e., thanks-a-lot) to you folks for all the assistance. Aloha Nui Loa Arnold H/U.Hawaii/
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publicly available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us.