[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

20040324: using a proxy server for ADDE transactions (cont.)

>From: Paul Neeley <address@hidden>
>Organization: University of Nevada-Reno/DRI
>Keywords: 200402202326.i1KNQKrV017862 McIDAS ADDE proxy server

Hi Paul,

Well, I found the problem, and it was totally trivial:

The failures are all due to MCCOMPRESS being set (MCCOMPRESS=TRUE), but
there neither 'compress' or 'uncompress' are installed on feather!
I was so consumed by thinking that using a proxy server was causing a
problem that I didn't think of the most obvious possibility on a Linux
machine:  compress/uncompress were not installed when the OS was
installed.

FYI, I found the problem by using the 'strace' utility:

<login as 'mcidas'>
cd workdata

dataloc.k LIST RTIAMGES
Group Name                    Server IP Address
--------------------         ----------------------------------------
RTIMAGES                     PROXY-ADDE

<LOCAL-DATA> indicates that data will be accessed from the local data directory.
DATALOC -- done

strace -f -ff -o dsinfo dsinfo.k I RTIMAGES

One of the strace files created in this test, dsinfo.8185 (dsinfo.<pid>)
told me where the _real_ problem was to be found:

 ...
write(4, "\300\250\310\v\0\0\1\367LWPR", 12) = 12
rt_sigaction(SIGPIPE, {SIG_DFL}, NULL, 8) = 0
fstat64(0, {st_mode=S_IFIFO|0600, st_size=160, ...}) = 0
read(0, "\300\250\310\v\0\0\1\364\0\0\0\0USER\0\0\0\0\0\0\0\0\0"..., 160) = 160
fstat64(4, {st_mode=S_IFSOCK|0777, st_size=0, ...}) = 0
fcntl64(4, F_GETFL)                     = 0x802 (flags O_RDWR|O_NONBLOCK)
select(5, NULL, [4], NULL, {120, 0})    = 1 (out [4], left {120, 0})
rt_sigaction(SIGPIPE, {SIG_IGN}, {SIG_DFL}, 8) = 0
write(4, "\300\250\310\v\0\0\1\364\0\0\0\0USER\0\0\0\0\0\0\0\0\0"..., 160) = 160
rt_sigaction(SIGPIPE, {SIG_DFL}, NULL, 8) = 0
pipe([5, 6])                            = 0
fork()                                  = 8186
close(0)                                = 0
dup(5)    = 0
close(6)                                = 0
execve("/bin/uncompress", ["uncompress", "-cf"], [/* 41 vars */]) = -1 ENOENT 
(No such file or directory)
execve("/usr/bsd/uncompress", ["uncompress", "-cf"], [/* 41 vars */]) = -1 
ENOENT (No such file or directory)
execve("/usr/bin/uncompress", ["uncompress", "-cf"], [/* 41 vars */]) = -1 
ENOENT (No such file or directory)
execve("/usr/contrib/uncompress", ["uncompress", "-cf"], [/* 41 vars */]) = -1 
ENOENT (No such file or directory)
_exit(1)                                = ?

The last execve lines show that uncompress was looked for in /bin,
/usr/bsd, /usr/bin, and then /usr/contrib, and not found.  Since
uncompress was not the process exited with a non-zero status
(exit(1)).  This caused the McIDAS layer doing the read to get a
zero length read when it expected 4 bytes.  That is the meaning of
the message that is seen when McIDAS debug output is turned on with
the DEV=CCC keyword:

dsinfo.k I RTIMAGES DEV=CCC

dsinfo.k* ***** REQ. CFILE-->ALA.RTIMAGES<--
dsinfo.k* Read of server has failed. Wanted  4
dsinfo.k*                               Did
    No Datasets found of Type: IMAGE in Group: RTIMAGES
DSINFO -- done


So, the solution is to install compress/uncompress from the appropriate
RPM, and things will work!


I do, however, have an observation:  It appears to me that your setup
is such that the proxy server always forwards ADDE requests to one
particular server.  Am I correct in this?  If yes, that really defeats
the purpose of ADDE.  ADDE is designed so that the client side (user)
can specify whatever server s/he wants to get data from using the
DATALOC command.  This way, if a server for a particular dataset is
unavailable for whatever reason (network is unreachable; the machine is
down; etc.), the user can simply "point" at a different machine for the
dataset s/he wants to access.  Also, it is the case that not every ADDE
server has every type of ADDE dataset to serve.  A good example of this
is that motherlode.ucar.edu (aka adde.ucar.edu) has a wide variety of
data that it serves through ADDE, but MODIS data is not one of them.
nanuk.eosdis.nasa.gov, on the other hand, _only_ serves global,
realtime MODIS data.  Another difference between these two servers is
that motherlode supports both compressed and uncompressed transfers
while nanuk only supports uncompressed transfers.

The typical way that users get around a firewall blocking access to one
of the ports being used by ADDE is for the firewall to open up access
to the port(s) that ADDE uses.  Those ports are:

500 - uncompressed transfers
503 - 'compress' compressed transfers
112 - 'gunzip' compressed transfers

McIDAS is moving towards only using port 112 since it is registered as
a McIDAS data transmission port (registered by Unidata back in the
early 90s), but it will not be there for at least a couple of years.

We do not feel that opening these ports is a security threat since
inetd/xinetd is configured to only run the McIDAS mcserv process and if
it dies, the connection is terminated (i.e., one cannot exploit a hole
and be left in a shell with 'root' privilege).

So, _if_ the proxy server setup at DRI is such that requests are always
forwarded to a particular ADDE server, Melanie's use of ADDE will be
severely limited.  In that case, I would suggest that it would be
better to bore a hole in your firewall for ports 500 and 503 now, and
for port 112 in the future, or to set her up with a machine that is
outside of the DRI firewall.

I will be happy to discuss the above issues in more detail with you and
Melanie if you like.

I am off of feather now, so you can shut down access.

Cheers,

Tom
--
NOTE: All email exchanges with Unidata User Support are recorded in the
Unidata inquiry tracking system and then made publically available
through the web.  If you do not want to have your interactions made
available in this way, you must let us know in each email you send to us.

>From address@hidden  Wed Mar 24 17:51:50 2004

thanks tom...

I will gladly get you an adult beverage if I ever get the chance!!!!

paul

Paul S. Neeley
Unix Systems Administrator
Desert Research Institute
2215 Raggio Parkway
Reno, NV  89512
voice: 775.673.7426
cell:  775.691.2337
email: address@hidden