[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Support #JLK-672206]: GOES full disk imagery in NOAAPort or IDD



Hi Matt,

re:
> Thanks Tom,

> Is there a simple way to ingest ADDE served data into LDM and create a product
> queue that way? (I gather this is the opposite problem from the normal 
> request).

"Simple" is more-or-less in the eyes of the person doing the procedure :-)

The UNIWISC content is created by:

- cron-initiated McIDAS processes accessing real-time GOES dataservers
  and creating (via the McIDAS IMGCOPY command) local sector files
  (in McIDAS AREA format)

- the local sector files are then PNG compressed by the ldm-mcidas routine
  'area2png'

- the PNG compressed versions of the sectors are inserted into an LDM
  queue while specifying a useful LDM/IDD Product ID

So you can see that McIDAS-X is at the heart of the creation of the
UNIWISC process, and the original data source is real-time GOES ingest
machines which, in turn, are powered by McIDAS Satellite Data Ingest
(SDI) software.

By the way, NOAA maintains several freely accessible ADDE servers that
have GOES data and more (e.g., MTSAT, FY2, METEOSAT (for the synoptic
hours only), etc.).

re: UNIWISC contains East sectors that contain South American coverage

> That may be a good start.  I was looking here
> http://www.unidata.ucar.edu/data/datastream.html
> and did not find that described.

I looked around for the document that I wrote when I proposed the addition
of South American coverage to the East sectors to the Unidata Users Committee.
Since I was not finding the document quickly, I decided to send off the
reply that had some of the information in it.

re:
> Do you know what the "product code" for the composite scan including SH is
> and what pqact lines I need to ingest that data?

The LDM/IDD Product IDs for the UNIWISC datastream can be viewed using the
LDM utility 'notifyme'.  Example:

<as 'ldm'>
notifyme -vl- -f UNIWISC -h idd.unidata.ucar.edu -o 3600

Dec 29 15:13:47 notifyme[15118] NOTE: Starting Up: idd.unidata.ucar.edu: 
20111229141347.358 TS_ENDT {{UNIWISC,  ".*"}}
Dec 29 15:13:47 notifyme[15118] NOTE: LDM-5 desired product-class: 
20111229141347.358 TS_ENDT {{UNIWISC,  ".*"}}
Dec 29 15:13:47 notifyme[15118] INFO: Resolving idd.unidata.ucar.edu to 
128.117.140.3 took 0.000514 seconds
Dec 29 15:13:47 notifyme[15118] NOTE: NOTIFYME(idd.unidata.ucar.edu): OK
Dec 29 15:13:48 notifyme[15118] INFO:  1345060 20111229141717.196 UNIWISC 000  
pnga2area Q0 CC 1120 COMPOSITE UNKBAND 1km 20111229 0000
Dec 29 15:13:48 notifyme[15118] INFO:    93489 20111229141904.959 UNIWISC 000  
pnga2area Q0 CE 1140 GOES-15_SND UNKBAND 1km 20111229 1300
Dec 29 15:13:48 notifyme[15118] INFO:   822868 20111229143222.088 UNIWISC 000  
pnga2area Q1 UE 1232 GOES-13_IMG 13.3um 4km 20111229 1415
Dec 29 15:13:48 notifyme[15118] INFO:  1912293 20111229143222.235 UNIWISC 000  
pnga2area Q1 UI 152 GOES-13_IMG 10.7um 4km 20111229 1415
Dec 29 15:13:48 notifyme[15118] INFO:  1906372 20111229143224.219 UNIWISC 000  
pnga2area Q1 UD 1222 GOES-13_IMG 3.9um 4km 20111229 1415
Dec 29 15:13:48 notifyme[15118] INFO:  1034540 20111229143229.732 UNIWISC 000  
pnga2area Q1 UW 212 GOES-13_IMG 6.5um 4km 20111229 1415
Dec 29 15:13:48 notifyme[15118] INFO:    93489 20111229143409.222 UNIWISC 000  
pnga2area Q0 CE 1140 GOES-15_SND UNKBAND 1km 20111229 1300
Dec 29 15:13:48 notifyme[15118] INFO:  2867031 20111229143614.692 UNIWISC 000  
pnga2area Q1 UV 142 GOES-13_IMG 0.65um 4km 20111229 1415
Dec 29 15:13:48 notifyme[15118] INFO:   998732 20111229144134.604 UNIWISC 000  
pnga2area Q3 U7 1242 GOES-15_IMG 3.9um 4km 20111229 1430
Dec 29 15:13:48 notifyme[15118] INFO:   539890 20111229144145.386 UNIWISC 000  
pnga2area Q3 UB 172 GOES-15_IMG 6.5um 4km 20111229 1430
Dec 29 15:13:48 notifyme[15118] INFO:  1172364 20111229144146.381 UNIWISC 000  
pnga2area Q3 U9 122 GOES-15_IMG 0.65um 4km 20111229 1430
Dec 29 15:13:48 notifyme[15118] INFO:   954201 20111229144146.918 UNIWISC 000  
pnga2area Q3 U5 132 GOES-15_IMG 10.7um 4km 20111229 1430
Dec 29 15:13:48 notifyme[15118] INFO:   800724 20111229144210.011 UNIWISC 000  
pnga2area Q3 U8 1252 GOES-15_IMG 13.3um 4km 20111229 1430
Dec 29 15:13:49 notifyme[15118] INFO:   120822 20111229144507.491 UNIWISC 000  
pnga2area Q0 CA 1100 GOES-15_SND UNKBAND 1km 20111229 1400
Dec 29 15:13:49 notifyme[15118] INFO:   114323 20111229144608.706 UNIWISC 000  
pnga2area Q0 CB 1110 GOES-15_SND UNKBAND 1km 20111229 1400
Dec 29 15:13:49 notifyme[15118] INFO:  1713917 20111229144655.469 UNIWISC 000  
pnga2area Q1 U1 190 GOES-9_IMG UNKBAND 1km 20111229 1200
Dec 29 15:13:49 notifyme[15118] INFO:   119335 20111229144807.376 UNIWISC 000  
pnga2area Q0 CD 1130 GOES-15_SND UNKBAND 1km 20111229 1400
Dec 29 15:13:49 notifyme[15118] INFO:    90069 20111229144907.367 UNIWISC 000  
pnga2area Q0 CE 1140 GOES-15_SND UNKBAND 1km 20111229 1400
Dec 29 15:13:49 notifyme[15118] INFO:    90069 20111229150405.614 UNIWISC 000  
pnga2area Q0 CE 1140 GOES-15_SND UNKBAND 1km 20111229 1400
Dec 29 15:13:49 notifyme[15118] INFO:     6162 20111229150514.270 UNIWISC 000  
pnga2area Q1 U3 202 GRAPHICS 0.65um 5km 20111229 1459
Dec 29 15:13:49 notifyme[15118] INFO:  1940126 20111229151215.078 UNIWISC 000  
pnga2area Q3 UI 152 GOES-13_IMG 10.7um 4km 20111229 1445
Dec 29 15:13:49 notifyme[15118] INFO:   778467 20111229151220.298 UNIWISC 000  
pnga2area Q3 UE 1232 GOES-13_IMG 13.3um 4km 20111229 1445
Dec 29 15:13:49 notifyme[15118] INFO:  1048312 20111229151225.547 UNIWISC 000  
pnga2area Q3 UW 212 GOES-13_IMG 6.5um 4km 20111229 1445
Dec 29 15:13:49 notifyme[15118] INFO:  2013481 20111229151228.214 UNIWISC 000  
pnga2area Q3 UD 1222 GOES-13_IMG 3.9um 4km 20111229 1445
Dec 29 15:13:49 notifyme[15118] INFO:  3024868 20111229151241.881 UNIWISC 000  
pnga2area Q3 UV 142 GOES-13_IMG 0.65um 4km 20111229 1445

This listing shows that the Product IDs for East sectors look like:

pnga2area Q3 UV 142 GOES-13_IMG 0.65um 4km 20111229 1445
pnga2area Q3 UD 1222 GOES-13_IMG 3.9um 4km 20111229 1445
pnga2area Q3 UW 212 GOES-13_IMG 6.5um 4km 20111229 1445
pnga2area Q3 UI 152 GOES-13_IMG 10.7um 4km 20111229 1445
pnga2area Q3 UE 1232 GOES-13_IMG 13.3um 4km 20111229 1445

One could key on a couple of different items in the Product ID:

- McIDAS product code - UV, UD, UW, UI, UE
- satellite - GOES-13_IMG

Keying on the satellite seems like the easiest thing to do, but
the satellite indicator changes when the satellite acting as
GOES-East changes (meaning that the indicator used to be
GOES-12_IMG when GOES-12 was GOES-East).  We use the McIDAS
product code instead of the satellite indicator since it does
not change when the GOES-East platform changes.

Here are the pattern-action file actions that we use to
FILE the UNIWISC products:

# Vis (0.65 um)
UNIWISC ^pnga2area Q. (U[^ACXY13]) (.*) (.*)_IMG (0.65)um (.*) (........) (....)
        PIPE    -close  util/ldmfile.sh
        /machine/data/ldm/mcidas/images/sat/\3/\5/VIS/VIS_\6_\7

# Short IR (3.9 um)
UNIWISC ^pnga2area Q. (U[^ACXY13]) (.*) (.*)_IMG (3.9)um (.*) (........) (....)
        PIPE    -close  util/ldmfile.sh
        /machine/data/ldm/mcidas/images/sat/\3/\5/\4/\4_\6_\7

# WV (6.[58] um)
UNIWISC ^pnga2area Q. (U[^ACXY13]) (.*) (.*)_IMG (6..)um (.*) (........) (....)
        PIPE    -close  util/ldmfile.sh
        /machine/data/ldm/mcidas/images/sat/\3/\5/WV/WV_\6_\7

# IR (10.7 um)
UNIWISC ^pnga2area Q. (U[^ACXY13]) (.*) (.*)_IMG (10.7)um (.*) (........) (....)
        PIPE    -close  util/ldmfile.sh
        /machine/data/ldm/mcidas/images/sat/\3/\5/IR/IR_\6_\7

# Long IR (GOES-East: 13.3 um; GOES-West: 12.0 um)
UNIWISC ^pnga2area Q. (U[8E]) (.*) (.*)_IMG (.*)um (.*) (........) (....)
        PIPE    -close  util/ldmfile.sh
        /machine/data/ldm/mcidas/images/sat/\3/\5/\4/\4_\6_\7

# Educational floater 1 (typically GOES-East sector/band)
UNIWISC ^pnga2area Q. (UA) (.*) (.*)_IMG (.*)um (.*) (........) (....)
        PIPE    -close  util/ldmfile.sh
        /machine/data/ldm/mcidas/images/sat/\3_fl1/\5/\4/\4_\6_\7

NB: these actions are likely more complicated than what you need/want:

- we use the script 'ldmfile.sh' to do the filing

  'ldmfile.sh' logs the receipt of a product and, on machines like
  adde.ucar.edu, creates a directory of the most recently received
  products; it currently keeps 96 images in the 'current' set.

- these actions also process both East and West sectors

  This is done by including the McIDAS product codes for both East
  and West sector products in the extended regular expressions.

re:
> I think the above questions are a great start, thanks for your help!

No worries.

re:
> I think a larger scale and more long term solution for me is to set up
> a ADDE server node here, perhaps add it to the
> publicly accessible mcidas adde server pool as a public good....

OK, this would be a very nice service for the community.  Please let me
know if you would like help in the installation and configuration of McIDAS
(you need McIDAS because ADDE is part of McIDAS).

Cheers,

Tom
--
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: JLK-672206
Department: Support Datastream
Priority: Normal
Status: Closed