Hi Mike and Ryan,
I just took a closer look at the LDM log messages that were included
in the original message by Mike (to python-users? I never saw a post
to ldm-users). I think what is going on is a typo in the pattern-action
file action that is being run. Here is the log output I am referring
to:
20200721T120016.942335Z pqact[59355] filel.c:pipe_prodput:2178
ERROR Couldn't pipe product to decoder "-metadata
/home/scripts/ldm-alchemy/goes-restitch.py -q -d /home/data/goes -t 60
-e /home/scripts/sat/goes_mcidas/manager.py -p -l
/home/ldm/var/logs/goes-restitch/SU.log SU 211200"
The lack of the decoder name in the output strongly suggests that
there is not a tab between '-metadata' and the part immediately before
it. Please check your pattern-action file action and correct this
typo if, of course, that it exists and then do the following:
<as 'ldm'>
ldmadmin pqactcheck
If this reports no errors:
ldmadmin pqactHUP
Cheers,
Tom
On 7/21/20 12:49 PM, Ryan May wrote:
Hi Mike,
We're not seeing any issues with our feed or execution of goes-restitch.
I'm actually seeing TISU as Mesoscale-2 right now, with Mesoscale-1
under TISH. Regardless, no anomalous behavior with what we're running.
Are you seeing anything in the logs from the script itself? (i.e.
/home/ldm/var/logs/goes-restitch/SU.log)
For the Mesoscale sectors, goes-restitch isn't doing much, since the
sectors are generally only one tile (they are today). All that's
happening is removing WMO header/footer and adjusting a bit of netCDF
metadata.
The first place I'd look, then, is to see if something's going wrong
with /home/scripts/sat/goes_mcidas/manager.py.
Ryan
On Tue, Jul 21, 2020 at 10:34 AM Mike Zuranski <zuranski@xxxxxxxxxxxxxxx
<mailto:zuranski@xxxxxxxxxxxxxxx>> wrote:
Greetings,
I'm using Unidata's goes-restitch.py to merge the GOES-R tiles
coming from NOAAPort. It's been working well for years with nary an
issue, until this morning...
From what I can tell, GOES-16 Mesoscale-1 was moved over the
Atlantic (9.5N/41W, header of TISU.. ) at around 12Z, and almost
immediately my ldmd.log began filling up with what I'm pasting
below. After about an hour of that, it began to impact the rest of
our satellite processing operation. I've cut out G16 Meso1 from
processing at the pqact, remade the queue & restarted LDM, and
everything has been fine since. But if I try to re-enable that, my
LDM log starts filling up with those errors right away again. I
tried to set the goes-restitch.py logging to verbose, but no errors
were being reported in that log file, only the ldmd.log file.
If anyone else uses goes-restitch.py with NOAAPort tiles, are you
seeing the same thing? I don't store the tiles locally so I haven't
interrogated them yet to see if anything has changed, but it sure
seems like this script doesn't like something about these SU tiles
coming in.
Note: I am using a slightly modified version of goes-restitch.py to
execute another script once a full set of tiles has been stitched
together so it can be acted upon. If you're wondering about the "-e
/home/scripts/sat/goes_mcidas/manager.py -p" bit below, that's all
it's doing. I submitted a pull-request for this a while back, so
the code changes can be seen here:
https://github.com/Unidata/ldm-alchemy/pull/4. ; Again, this has
never been an issue for the several years I've been using it.
ldmd.log output example:
20200721T120016.942166Z pqact[59355]
pbuf.c:pbuf_flush:113 ERROR Broken pipe
20200721T120016.942279Z pqact[59355]
pbuf.c:pbuf_flush:113 ERROR Couldn't write to pipe:
fd=1021, len=4096
20200721T120016.942298Z pqact[59355]
filel.c:pipe_put:1930 ERROR Couldn't write
307601-byte product to pipe
20200721T120016.942317Z pqact[59355]
filel.c:pipe_out:2115 ERROR Couldn't write product
data to pipe
20200721T120016.942335Z pqact[59355]
filel.c:pipe_prodput:2178 ERROR Couldn't pipe product to
decoder "-metadata /home/scripts/ldm-alchemy/goes-restitch.py -q -d
/home/data/goes -t 60 -e /home/scripts/sat/goes_mcidas/manager.py -p
-l /home/ldm/var/logs/goes-restitch/SU.log SU 211200"
20200721T120016.942353Z pqact[59355]
filel.c:pipe_prodput:2187 ERROR Decoder terminated
prematurely
20200721T120016.942375Z pqact[59355]
filel.c:fl_removeAndFree:425 ERROR Deleting failed PIPE
entry: pid=66861, cmd="-metadata
/home/scripts/ldm-alchemy/goes-restitch.py -q -d /home/data/goes -t
60 -e /home/scripts/sat/goes_mcidas/manager.py -p -l
/home/ldm/var/logs/goes-restitch/SU.log SU 211200"
Thanks,
-Mike
======================
Mike Zuranski
Meteorology Support Analyst
College of DuPage - Nexlab
Weather.cod.edu <http://Weather.cod.edu>
======================
_______________________________________________
NOTE: All exchanges posted to Unidata maintained email lists are
recorded in the Unidata inquiry tracking system and made publicly
available through the web. Users who post to any of the lists we
maintain are reminded to remove any personal information that they
do not want to be made public.
python-users mailing list
python-users@xxxxxxxxxxxxxxxx <mailto:python-users@xxxxxxxxxxxxxxxx>
For list information, to unsubscribe, or change your membership
options, visit: https://www.unidata.ucar.edu/mailing_lists/
--
Ryan May, Ph.D.
Software Engineer
UCAR/Unidata
Boulder, CO
_______________________________________________
NOTE: All exchanges posted to Unidata maintained email lists are
recorded in the Unidata inquiry tracking system and made publicly
available through the web. Users who post to any of the lists we
maintain are reminded to remove any personal information that they
do not want to be made public.
ldm-users mailing list
ldm-users@xxxxxxxxxxxxxxxx
For list information or to unsubscribe, visit:
https://www.unidata.ucar.edu/mailing_lists/
--
+----------------------------------------------------------------------+
* Tom Yoksas UCAR Unidata Program *
* (303) 497-8642 (last resort) P.O. Box 3000 *
* yoksas@xxxxxxxx Boulder, CO 80307 *
* Unidata WWW Service http://www.unidata.ucar.edu/ *
+----------------------------------------------------------------------+
