>From: "John Hobbie" <address@hidden>
>Organization: NMSU/NSBF
>Keywords: 200503181935.j2IJZWv2008699 ldm-mcidas decoder
Hi John,
>I am beginning to think that I am dumber than the average bear. Tom, I
>tried your fixes and that fixed that problem, but I still had others. I
>therefore figured that I had not followed the instructions carefully,
>specifically about putting the workdata in MCDATA and in the path. I
>went back over the instructions, and couldn't find where I messed up,
>so I went back and got the users_guide.pdf file for 2004 and printed
>out the installation pages. I then cleaned everything up once again
>and started from scratch.
In looking down toward the end of your email, I see that the problem
you are seeing is related to the execution of the ldm-mcidas decoder
pnga2area, not with McIDAS per se.
>(An aside here. We are trying to set up all the machines to be
>"standard Unidata format". That is, data directories are where
>Unidata suggests them, config files & *.BAT files etc, don't have to be
>modified, etc. The idea that upgrades can be done by anyone, even
>without going to school and with only minimal Unix/Linux knowlege;
>system maintenance can be done by anyone with no or minimal tinkering
>to the files.
I think that this is an excellent idea!
>Another objective is to make the machines totally
>interchangable so that if one goes down in flight season, a spare can
>be put in place with virtually no effort, save changing networking
>configurations.
This is a very good approach. 'Standard" installations will make
troubleshooting so much easier on the next person that has to figure
out what is going wrong, but had nothing to do with an installation.
>With that in mind, I have been setting up the last
>machine -- a new machine -- with all the latest software from Unidata
>and going straight by the installation instructions, taking notes as I
>go, so those who follow can easily do an upgrade. And once I have
>identified any gotchas, have one of the other guys set up a machine.
>Anyway, I think I am getting senile and am missing something--a lot--
>because I seem to be having no end of problems with the mcidas
>installation.)
Again, as I look through the problem you include below, it is not with
McIDAS, but, rather with the ldm-mcidas decoder, pnga2area.
>Anyway, the problem is with the image data. We have NOAAport. All the
>image data from there, NEXRAD and GOES, are being saved just fine via
>ldm and are accessible.
Very good.
>None of the Wisconsin image data coming over
>the IDD are being stored. The ldm logs have similar broken pipe
>entries to the following.
>
>Mar 18 15:42:43 psnldm pqact[20178]: pbuf_flush (22) write: Broken pipe
>Mar 18 15:42:43 psnldm pqact[20178]: pipe_put: -closepnga2area-vl/usr/local/ld
> m/logs/ldm-mcidas.logdata/gempak/images/sat/GOES-10/4km/12.0/12.0_20050318_15
> 30 write error
>Mar 18 15:42:43 psnldm pqact[20178]: pipe_prodput: trying again
>Mar 18 15:42:43 psnldm pqact[20178]: pbuf_flush (22) write: Broken pipe
>Mar 18 15:42:43 psnldm pqact[20178]: pipe_put: -closepnga2area-vl/usr/local/ld
> m/logs/ldm-mcidas.logdata/gempak/images/sat/GOES-10/4km/12.0/12.0_20050318_15
> 30 write error
...
>umask is set to 002 in both ldm and mcidas in the .cshrc files.
>(However, I didn't erase all the data files during the set up and they
>may have beein established before I set umask in the .cshrc files--does
>that make a difference?)
It might _if_ the group of the user running the LDM changed. I have
seen several instances of sites setting up the 'ldm' and 'mcidas' users
in different groups, doing software installations, and then going back
and changing the groups for either or both, and then changing umask.
What happens in this case is that directories have been created by a
user that is not identical to the current definitions -- the group has
changed. In cases like this, write attempts can fail in seeming
unexplainable ways.
>I went back and recursively from the top
>(/data/ldm/gempak) did a chmod 775 on all the files and directories
>below in response to the write error, thinking I have permission
>problems.
This was a good move. The next move I would recommend is recursively
changing the owner of the files and directories. for instance, if the
group of the 'ldm' user is 'users', change the owner of the directories
and all of their contents as 'root' using:
<as 'root'>
cd /data/ldm
chown -R ldm:users *
>The owners are ldm:users and everybody -- mcidas, gempak
>and the user account, weather, are all of the group, users.
Yes, but were they in the 'users' group when the directories and first
set of files were created?
>I also am
>noting that no log file, ldm-mcidas.log, exists. The log file,
>XCD_START.LOG, has a steady stream of DMSYN: SYSKEY.TAB not
>found
You have just provided what is probably the most important piece of
information.
>There were similar decoder errors of SYSKEY.TAB not found for
>other decoders, eg. DMSFC, for a few lines at start up, but then
>nothing. Only the synoptic decoder continues to have problems.
Here is what is most likely happening. The ldm-mcidas decder pnga2area
needs to be able to read AND write three files in the directory in
which it will create its output: SYSKEY.TAB, ROUTE.SYS, and SCHEMA.
McIDAS' approach to files is that it will create a file that it needs
to read if it doesn't already exist. It could well be the case that
you did not copy SYSKEY.TAB, ROUTE.SYS and SCHEMA from the McIDAS
distribution to the directory in which pnga2area will create its output
before starting the LDM. If this is the reason for the decode
failures, the fix is easy:
<as 'ldm'>
ldmadmin stop <- stop the LDM (for a big)
<as 'mcidas'>
cd ~mcidas/data
cp SYSKEY.TAB SCHEMA /data/ldm/mcidas <- if this is the output directory
cd ~mcidas/workdata
cp ROUTE.SYS /datat/ldm/mcidas
cd /data/ldm/mcidas
chmod 664 SSYSKEY.TAB SCHEMA ROUTE.SYS
<as 'ldm'>
ldmadmin start <- start the LDM back up
Please note that I offer this _if_ it is your only problem. I do not
think that it is. The reason I say this is that the errors you
included above show that you are have some ~ldm/etc/pqact.conf entries
that run pnga2area telling it how to name output files and where to put
them -- i.e., those actions are NOT using the McIDAS routing table,
ROUTE.SYS, and will not try to update the McIDAS system key table,
SYSKEY.TAB. At the same time, you might well have an action that
_does_ need ROUTE.SYS and SYSKEY.TAB. This action would look like:
# CIMSS and UW Products decoded into AREA files using McIDAS routing table
MCIDAS ^pnga2area Q. (..) (.*) (.*) (.*) (.*) (........) (....)
PIPE -close
decoders/pnga2area -vl logs/ldm-mcidas.log
-d data/mcidas -r \1,\2
(mind where white space should be tabs!)
Here is what I would do to further investigate the problem you report
above:
1) make sure that the user running your LDM can read/write to the
the directories that pnga2area is trying to write to:
<as 'ldm'>
data/gempak/images/sat/GOES-10/4km/12.0/12.0_20050318_1530
cd ~ldm
touch data/gempak/images/sat/xxx
rm data/gempak/images/sat/xxx
touch data/gempak/images/sat/GOES-10/xxx
rm data/gempak/images/sat/GOES-10/xxx
touch data/gempak/images/sat/GOES-10/4km/xxx
rm data/gempak/images/sat/GOES-10/4km/xxx
touch data/gempak/images/sat/GOES-10/4km/12.0/xxx
rm data/gempak/images/sat/GOES-10/4km/12.0/xxx
If all of these work, 'ldm' can read/write the directories. If any
fail, investigate why it failed -- it is probably due to a permission
problem.
2) make sure that 'pnga2area' can be found in the PATH for 'ldm':
<as 'ldm'>
which pnga2area
If this does not return back the directory containing pnga2area, correct
your PATH setting in your .cshrc file, soure the file, and stop/restart
your LDM.
3) the 'which pnga2area' command in 2) should tell you that the decoder
can be found and is marked as being executable. It does not tell you if
it is executable on your system. Verify that the copy of 'pnga2area' that
was found is executable on your system. For instance, if 'pnga2area' was
found in the ~ldm/decoders directory (the recommended location), then
run:
ldd ~ldm/decoders/pnga2area
This will list back all of the shared libraries that 'pnga2area' needs
and where it finds them. If it can't find a needed library, it will
tell you. In this unexpected case, you may have the wrong version
of 'pnga2area' installed on your system. I am _not_ expecting this
case.
>I know this is a long laundry list of problems, but I am hoping they
>are symptomatic of just one dumb error like missing a line in the
>instructions, but I fear, they indicated multiple problems.
I think that I have touched the necessary bases:
- 'ldm' being able to create and write into a directory
- the decoder being able to run
If this does not solve your problem, would it be possible for me to login
to your system to look at your setup? This would be _much_ quicker than
trying to guess what could be wrong.
>Thanks in advance for all your help,
No worries. I am sure that the problem you are seeing is something minor.
The ideas I listed above were my best shots at fixing things without
being able to see the setup. If they don't work, giving me a login would
get you going faster than anything.
Cheers,
Tom
--
NOTE: All email exchanges with Unidata User Support are recorded in the
Unidata inquiry tracking system and then made publicly available
through the web. If you do not want to have your interactions made
available in this way, you must let us know in each email you send to us.
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publicly available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us.