[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[IDD #CIX-410192]: GOES R script

Hi Gerry,

re: questions?

> Plenty. I'm now trying to implement R99 (GLM), and grbfile.sh formats. And
> failing miserably. Still working, though.

The R99 products are NOT coming in on NOAAPort.  We have a feed to a machine
at Texas Tech University (TTU) that is getting a direct feed from somewhere
in NOAA.

re:
> First, thanks for the scripts. Is there a specific list of the various data 
> from GOES-R/S
> (16/17) in the [A-Z]99 format? I'm looking and not finding it.

I think that the PUG has what you are looking for:

https://www.goes-r.gov/users/docs/PUG-GRB-vol4.pdf

re:
> Looking at https://www.unidata.ucar.edu/data/rtgoesr_pqact.html, and 
> specifically at
> GLM  data, for which I had a request after I talked to you yesterday(!), I'm 
> doing the
> following in my pqact:
> 
> ### GLM GOES-16
> SATELLITE       (OR_(GLM)-L2-(....)_G(..)_s(....)(...).*)
> PIPE    -close
> util/grbfile.sh /work/LDM/GOES16/GLM/\2 \1 \2 \3 \5 \6 2160 logs/grb\4.log
> 
> ### GLM GOES-17
> SATELLITE       (OR_(GLM)-L2-(....)_G(..)_s(....)(...).*)
> PIPE    -close
> util/grbfile.sh /work/LDM/GOES16/GLM/\2 \1 \2 \3 \5 \6 2160 logs/grb\4.log
> 
> and in the log I'm seeing:
> 
> 20190418T210655.591255Z                pqact[18589]              
> filel.c:pipe_open() ERROR No such file or directory
> 20190418T210655.591332Z                pqact[18589]              
> filel.c:pipe_open() ERROR Couldn't execute decoder 
> "util/grbfile.sh";PATH=/usr/local/ldm/bin:/usr/local/ldm/bin:/usr/lib64/qt-3.3/bin:/usr/share/centrifydc/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin
> 20190418T210655.591359Z                pqact[18589]                
> pqact.c:cleanup() NOTE  Exiting
> 20190418T210655.593291Z                pqact[10395]              
> pbuf.c:pbuf_flush() ERROR Broken pipe
> 20190418T210655.593322Z                pqact[10395]              
> pbuf.c:pbuf_flush() ERROR Couldn't write to pipe: fd=1022, len=4096
> 20190418T210655.593335Z                pqact[10395]               
> filel.c:pipe_put() ERROR Couldn't write 158219-byte product to pipe
> 20190418T210655.593347Z                pqact[10395]               
> filel.c:pipe_out() ERROR Couldn't write product data to pipe
> 20190418T210655.593359Z                pqact[10395]           
> filel.c:pipe_prodput() ERROR Couldn't pipe product to decoder "-close 
> util/grbfile.sh /work/LDM/GOES16/GLM/GLM 
> OR_GLM-L2-LCFA_G17_s20191082106200_e20191082106400_c20191082106416.nc GLM 
> LCFA 2019 108 2160 logs/grb17.log"
> 20190418T210655.593371Z                pqact[10395]       
> filel.c:fl_removeAndFree() ERROR Deleting failed PIPE entry: pid=18589, 
> cmd="-close util/grbfile.sh /work/LDM/GOES16/GLM/GLM 
> OR_GLM-L2-LCFA_G17_s20191082106200_e20191082106400_c20191082106416.nc GLM 
> LCFA 2019 108 2160 logs/grb17.log"
> 20190418T210655.593402Z                pqact[10395]              
> palt.c:prodAction() ERROR Couldn't process product: feedtype=SATELLITE, 
> pattern="(OR_(GLM)-L2-(....)_G(..)_s(....)(...).*)", action=pipe, 
> args="-close    util/grbfile.sh    /work/LDM/GOES16/GLM/\2 \1 \2 \3 \5 \6 
> 2160 logs/grb\4.log"
> 20190418T210655.594010Z                pqact[18590]              
> filel.c:pipe_open() ERROR No such file or directory
> 20190418T210655.594068Z                pqact[18590]              
> filel.c:pipe_open() ERROR Couldn't execute decoder 
> "util/grbfile.sh";PATH=/usr/local/ldm/bin:/usr/local/ldm/bin:/usr/lib64/qt-3.3/bin:/usr/share/centrifydc/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin
> 20190418T210655.594092Z                pqact[18590]                
> pqact.c:cleanup() NOTE  Exiting
> 20190418T210655.594992Z                pqact[10395]              
> pbuf.c:pbuf_flush() ERROR Broken pipe
> 20190418T210655.595021Z                pqact[10395]              
> pbuf.c:pbuf_flush() ERROR Couldn't write to pipe: fd=1022, len=4096
> 20190418T210655.595033Z                pqact[10395]               
> filel.c:pipe_put() ERROR Couldn't write 158219-byte product to pipe
> 20190418T210655.595045Z                pqact[10395]               
> filel.c:pipe_out() ERROR Couldn't write product data to pipe
> 20190418T210655.595057Z                pqact[10395]           
> filel.c:pipe_prodput() ERROR Couldn't pipe product to decoder "-close 
> util/grbfile.sh /work/LDM/GOES16/GLM/GLM 
> OR_GLM-L2-LCFA_G17_s20191082106200_e20191082106400_c20191082106416.nc GLM 
> LCFA 2019 108 2160 logs/grb17.log"
> 20190418T210655.595069Z                pqact[10395]       
> filel.c:fl_removeAndFree() ERROR Deleting failed PIPE entry: pid=18590, 
> cmd="-close util/grbfile.sh /work/LDM/GOES16/GLM/GLM 
> OR_GLM-L2-LCFA_G17_s20191082106200_e20191082106400_c20191082106416.nc GLM 
> LCFA 2019 108 2160 logs/grb17.log"
> 20190418T210655.595097Z                pqact[10395]              
> palt.c:prodAction() ERROR Couldn't process product: feedtype=SATELLITE, 
> pattern="(OR_(GLM)-L2-(....)_G(..)_s(....)(...).*)", action=pipe, 
> args="-close    util/grbfile.sh    /work/LDM/GOES16/GLM/\2 \1 \2 \3 \5 \6 
> 2160 logs/grb\4.log"
> 20190418T210655.595167Z                pqact[10395]                   
> filel.c:reap() ERROR Child 18589 exited with status 1
> 20190418T210655.595206Z                pqact[10395]                   
> filel.c:reap() ERROR Child 18590 exited with status 1
> 
> Trying to understand where my problem might lie. And, yes, my util
> scripts reside in ~ldm/util.

It looks like your action(s) is(are) not finding util/grbfile.sh.  The
relative location we use is based on our changing our LDM registry 
(~ldm/etc/registry.xml) datadir-path value to look like old LDM
installations:

  <pqact>
    <config-path>/opt/ldm/etc/pqact.conf</config-path>
    <datadir-path>/opt/ldm</datadir-path>
  </pqact>

This makes the current working directory for our pattern-action files
actions ~ldm, not ~ldm/var.  You can change your relative reference to
grbfile.sh to an absolute one.

re:
> AND now I'm logging the GLM data but not seeing R99 L2 prods. Still playing.

Again, the R99 products are not being distributed in NOAAPort but they
may be at some point.  I have no idea if/when this might be.

Also, we are receiving value added GLM imagery/grid products from TTU
that are being created at TTU.  These products are being added to a
revamp of the NIMAGE datasetream that has not been announced yet since
we are currently getting some sites to test the feed to make sure
that all bugs have been worked out.

So, what is the NIMAGE revamp?

NIMAGE has historically been populated with GINI imagery distributed
in NOAAPort (NIMAGE stands for NOAAPort IMAGE).  When GOES-15 gets turned
off in the June/July time frame, there should/will be no more GINI
images generated and sent in NOAAPort.

We are going to populate NIMAGE with things that are currently being
distributed in the NOTHER and HDS IDD datastreams that originate from
NOAAPort AND value added L2 products that are originate from somewhere
else (e.g., TTU).  One thing to keep in the back of one's mind is:
all GOES-R/S imagery and level 2 products are actually ALL level 2
products.

Here is what the content of NIMAGE will look like:

https://www.unidata.ucar.edu/data/nimage.html

NB: I do not yet have any examples of the value added GLM imagery/grids
from TTU in this page, but I will soon.

The content will be:

- full ABI scenes created by stitching together the tiles delivered
  in NOAAPort

- all of the other L2 products delivered in NOAAPort, but the products
  will have the NOAAPort header and footer stripped off

- value added L2 products from non-NOAAPort source(s)

  The first of these is the GLM imagery/grids from TTU

If you are willing to test the revamped NIMAGE datastream, I can give
you access to the machine on which it is being created.  When the
revamped NIMAGE datastream goes live (i.e., becomes part of the IDD),
we will expect test users to switch over to an IDD feed, and stop
feeding from the test feed.


Cheers,

Tom
--
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: CIX-410192
Department: Support IDD
Priority: Normal
Status: Closed
===================
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata 
inquiry tracking system and then made publicly available through the web.  If 
you do not want to have your interactions made available in this way, you must 
let us know in each email you send to us.