[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
20030616: importing and contouring data on GOES imagery (cont.)
- Subject: 20030616: importing and contouring data on GOES imagery (cont.)
- Date: Wed, 18 Jun 2003 12:05:44 -0600
>From: Owen Cooper <address@hidden>
>Organization: Aeronomy Laboratory/NOAA
>Keywords: 200306112200.h5BM0fLd024027 McIDAS TXT2MD PTCON
Hi Owen,
>Thank you for the advice on reading ASCII text into MCIDAS for
>contouring onto GOES imagery.
No worries.
>I tried registering a new SCHEMA file but ran into a problem.
>Please see the attached text file which should explain all.
>Please let me know if you have any ideas as to what the problem may be,
>and let me know if you need any more info.
As I said on the phone yesterday, the missing piece is knowing what
a schema definition file should look like. A description of an
existing schema will probably help:
-- DCNLDN --
" NAME VSN DATE ID TEXTID
" ---- --- ---- -- ------
SCHEMA NLDN 2 98337 0 "NLDN DATA OBS UNIDATA FORMAT
"KEY SCALE UNITS DESCRIPTION
"--- ----- ----- -----------
ROWS 1000 "1000 ROWS EACH WITH 1000 COLS
DAY CYD "YEAR AND JULIAN DAY
TIME HMS "TIME OF FIRST OBSERVATION IN ROW
CMAX "LAST COLUMN CONTAINING DATA
COLUMNS 1000 "DEFAULT # OF COLS
DATA "START OF DATA SECTION
HMS HMS "ACTUAL TIME OF OBSERVATION
LAT 4 DEG "LATITUDE (DECIMAL DEG)
LON 4 DEG "LONGITUDE (DECIMAL DEG)
SGNL 1 "SIGNAL STRENGTH (150 measures ~ 30 kamps)
MULT "MULTIPLICITY
ENDSCHEMA
Each schema description:
- starts with a line that begins with SCHEMA
- the SCHEMA line defines the 4-character name of the schema; the version
of the schema (schema definitions can evolve); the date the version
was created; an ID for the schema (not quite sure what this is!);
and a textual description as a quote field
- definitions of #ROWS, #COLUMNS, and DATA keys (e.g. DAY, TIME, ..., MULT
in the listing above). MD file keys are each 4 characters or less in
length
- specification of how many significant digits of a key value is to
be kept in the MD file (all MD file values are scaled integers)
- units of the keys (recognized units are defined in the file FLDWDTH)
- comments about the keys (these comments are not used for anything,
but are useful for the reader)
Your first attempt of defining a schema was an attempt to replicate
the output format that you would see when listing a schema that
is registered (i.e., has been added to SCHEMA using SCHE) using LSCHE.
This was a good guess, but will not work:
############################################################################
NAME: FLEXP VERSION: 1 DATE: 2003167 TEXTID: "FLEXPART CO CONCENTRATIONS
---- -------
DEFAULT NUMBER OF ROWS: 33000 INTEGER ID: 0
COLS: 3 MISSING DATA VALUE: -999
REPEAT GROUP: NUMBER OF REPETITIONS: 0
STARTING POSITION: 1
SIZE: 3
NUMBER OF KEYS IN ROW HEADER: 3
COL HEADER: 0 STARTING AT POSITION 1
DATA RECORD: 3 STARTING AT POSITION 1
----
6 TOTAL
KEY SCALE UNIT
---- ----- ----
LON 6 DEG
LAT 5 DEG
CO 7 CON
################################################################################
>I'm not sure what the following parameters are for:
>
>NUMBER OF REPETITIONS (seeing as I only want to read each column once, I
>assume the # of repititions is 0)
The number of repetitions is used for schema like for soundings where the
same set of data are available for multiple levels.
>STARTING POSITION ??
>SIZE ???
>INTEGER ID ????
>COL HEADER: STARTING AT POSITION ???
>DATA RECORD: STARTING AT POSITION ???
>Can you take a look at this SCHEMA and let me know where I'm going wrong?
These are artifacts of the LSCHE output. The schema example above (DCNLDN)
should go a long way towards explaining what needs to be done. The
schema I came up with below should galvanize some of the concepts.
I played around with the data file you gave me access to yesterday, and
came up with a schema that, while pretty simple minded, works for what
you want:
" NAME VSN DATE ID TEXTID
" ---- --- ---- -- ------
SCHEMA FLXP 1 01168 0 "FLEXPART CO CONCENTRATIONS
"KEY SCALE UNITS DESCRIPTION
"--- ----- ----- -----------
ROWS 90 "90 ROWS
LAT 4 DEG "LATITUDE (DECIMAL DEG)
COLUMNS 360 "DEFAULT # OF COLS
LON 4 DEG "LONGITUDE (DECIMAL DEG)
DATA "START OF DATA SECTION
CO 2 CON "COD CONCENTRATION
ENDSCHEMA
This schema defines an MD file that is 90 rows long; each row has
360 columns; and each column has one data point, the CO concentration.
Notice that I chose the LAT as a ROW header; the LON as a COLumn header,
and CO as the data key. The reason for this was mostly so that the
TXT2MD conversion utility would work without complaint.
What you end up with is a MD file where each ROW represents all data at
a single Latitude, each COLumn represents all data at a single a single
Longitude, and there is exactly one datum for each ROW,COL cell.
I then used this schema to create an output MD file and contour its values
on top of a world map:
- create DCFLXP with the contents above in the ~mcidas/workdata directory.
If you are doing this as a user other than 'mcidas', then you would
create this file in your ~/mcidas/data directory
- since I didn't want to change the SCHEMA file that I send out in the
distribution, I made a copy of it into my McIDAS working directory:
cd ~mcidas/workdata
cp ~mcidas/data/SCHEMA .
redirect.k ADD SCHEMA \".
If you are doing this as a user other than 'mcidas', do the same steps
in your McIDAS working directory:
cd ~/mcidas/data
cp ~mcidas/data/SCHEMA .
redirect.k ADD SCHEMA \".
- next, I registered the schema in the copy of SCHEMA that I can modify:
sche.k DCFLXP
I verified the schema registry using LSCHE:
lsche.k FLXP
NAME: FLXP VERSION: 1 DATE: 2001168 TEXTID: "FLEXPART CO CONCENTRATIONS
---- -------
DEFAULT NUMBER OF ROWS: 90 INTEGER ID: 0
COLS: 360 MISSING DATA VALUE:
-2139062144
REPEAT GROUP: NUMBER OF REPETITIONS: 1
STARTING POSITION: 3
SIZE: 1
NUMBER OF KEYS IN ROW HEADER: 1
COL HEADER: 1 STARTING AT POSITION 2
DATA RECORD: 1 STARTING AT POSITION 3
----
3 TOTAL
KEY SCALE UNIT KEY SCALE UNIT KEY SCALE UNIT
---- ----- ---- ---- ----- ---- ---- ----- ----
LAT 4 DEG LON 4 DEG CO 2 CON
LSCHE: DONE
- the file you gave me FTP access to looked like one long line in Unix:
there were carriage returns at the end of each logical line, not
new lines. TXT2MD expects the textual data file to be in ASCII
format with each line ending in a new line. Given this, I translated
your file into a format I could use:
tr '\015' '\012' < flexpart_20030604030000 > flexprt.txt
The 'tr' invocation converts CR characters (octal 015) into NL characters
(octal 012).
I wrote the output into a file whose name follows the old DOS 8.3
name format since that is what TXT2MD demands (this should be changed,
but...).
- after converting your file, I edited it to add one additional line:
change:
LON LAT CO
-178.5 0.5 1.63
...
to:
LON LAT CO
UNIT=*-1 X X
-178.5 0.5 1.63
...
The reason for this addition is that I _assumed_ that your LON values
were using the West negative convention, while McIDAS uses a West
positive convention. The *-1 syntax for the first positional value
for UNIT tells TXT2MD to multiply each value by -1 thus converting
its sign. BTW the ability to convert units in the UNIT= line was an
addition I made to TXT2MD a number of years ago; it does not exist in
the SSEC version (just in case you find yourself using an SSEC
distribution of McIDAS and wonder why you can't do unit conversions).
- finally, I was able to run TXT2MD and convert your file:
<I was still in ~mcidas/workdata, you may well be in ~/mcidas/data>
mdu.k DEL 1000 <- just to make sure that the output MD file
doesn't exist yet
txt2md.k flexprt.txt 1000 FLXP
All data values were read in from your text file and stored in MDXX1000.
The next step was to verify that I could contour the data on top of
a navigated display. In order to do this, I first made sure that
I had the MYDATA datasets defined:
batch.k MYDATA.BAT
Then I contoured the data and saved the output into a GIF (tm) file for you
to take a look at:
mcenv
erase.k
map.k WORLD
ptcon.k MYDATA/PTSRCS.1000 PAR=CO CINT=15 NAV=C LSIZE=5
frmsave.k 1 flexprt.gif
exit
Have a look at the result in:
http://www.unidata.ucar.edu/staff/tom/gifs/owen/flexprt.gif
I guess it is now an exercise for the reader (you ;-) to verify that
the display of the data is correct.
>Thanks!
Please let me know if there is anything that I did that does not make
sense.
>Thanks!
No worries.
Tom
>From address@hidden Wed Jun 18 12:14:43 2003
Hi Tom,
This looks awesome, and I will give it a try very soon.
Next week I will post near real-time GOES-east and Meteosat IR images on
the web and will overlay the carbon monoxide data from the FLEXPART
model. As far as I know this will be the first realtime display of
pollution plumes travelling across the Atlatntic and superimposed onto
geostationary satellite data. Once everything is working I'll send you
the URL so you can see the animations.
Thanks!
Owen
Owen R. Cooper
Cooperative Institute for Research in Environmental Sciences (CIRES)
University of Colorado/NOAA Aeronomy Laboratory
NOAA Aeronomy Laboratory, R/AL4
325 Broadway
Boulder CO 80305
ph: (303) 497-3599
fax: (303) 497-5686
