netCDF Authorization Support

Table of Contents

1. Introduction
2. URL-Based Authentication
3. RC File Authentication
4. Redirection-Based Authentication
5. URL Constrained RC File Entries
6. Client-Side Certificates
7. Appendix A. All RC-File Keys
8. Appendix B. ESG Access in Detail

Introduction

With some exceptions (e.g. see the section on redirection) The libcurl authorization mechanisms can be accessed in two ways

1. Inserting the username and password into the url, or
2. Accessing information from a so-called rc file named either .daprc or .dodsrc

URL-Based Authentication

    http://username:password@host/...

RC File Authentication

The file must be called one of the following names: ".daprc" or ".dodsrc" If both .daprc and .dodsrc exist, then the .daprc file will take precedence.

The rc file is searched for first in the current directory and then in the home directory (as defined by the HOME environment variable).

The rc file format is a series of lines of the general form:

[<host:port>]<key>=<value>

The currently defined set of authorization-related keys are as follows. The second column is the affected curl_easy_setopt option(s), if any.

Key	Affected curl_easy_setopt Options	Notes
HTTP.COOKIEJAR	CURLOPT_COOKIEJAR
HTTP.COOKIEFILE	CURLOPT_COOKIEJAR	Alias for CURLOPT_COOKIEJAR
HTTP.PROXY_SERVER	CURLOPT_PROXY, CURLOPT_PROXYPORT, CURLOPT_PROXYUSERPWD
HTTP.SSL.CERTIFICATE	CURLOPT_SSLCERT
HTTP.SSL.KEY	CURLOPT_SSLKEY
HTTP.SSL.KEYPASSWORD	CURLOPT_KEYPASSWORD
HTTP.SSL.CAINFO	CURLOPT_SSLCAINFO
HTTP.SSL.CAPATH	CURLOPT_SSLCAPATH
HTTP.SSL.VERIFYPEER	CURLOPT_SSL_VERIFYPEER
HTTP.SSL.VALIDATE	CURLOPT_SSL_VERIFYPEER, CURLOPT_SSL_VERIFYHOST
HTTP.CREDENTIALS.USERPASSWORD	CURLOPT_USERPASSWORD
HTTP.NETRC	N.A.	Specify path of the .netrc file

Password Authentication

Cookie Jar

Certificate Authentication

HTTP.SSL.KEY is essentially the same as HTTP.SSL.CERTIFICATE and should always have the same value.

HTTP.SSL.KEYPASSWORD specifies the password for accessing the HTTP.SSL.CERTIFICAT/HTTP.SSL.key file.

HTTP.SSL.CAPATH specifies the path to a directory containing trusted certificates for validating server certificates.

HTTP.SSL.VALIDATE is a boolean (1/0) value that if true (1) specifies that the client should verify the server's presented certificate.

HTTP.PROXY_SERVER specifies the url for accessing the proxy: e.g. http://[username:password@]host[:port]

HTTP.NETRC specifies the absolute path of the .netrc file. See redirection authorization for information about using .netrc.

Redirection-Based Authentication

The process is usually as follows.

1. The client contacts the server of interest (SOI), the actual data provider using, typically http protocol.
2. The SOI sends a redirect to the client to connect to the e.g. URS system using the 'https' protocol (note the use of https instead of http).
3. The client authenticates with URS.
4. URS sends a redirect (with authorization information) to send the client back to the SOI to actually obtain the data.

It turns out that libcurl uses the password in the .daprc file — or from the url — only for the initial connection. This causes problems because the redirected connection is the one that actually requires the password. This is where .netrc comes in. Libcurl will use .netrc for the redirected connection. It is possible to cause libcurl to use the .daprc password always, but this introduces a security hole because it may send the initial user+pwd to the redirection site. In summary, if you are using redirection, then you must create a .netrc file to hold the password for the site to which the redirection is sent.

The format of this .netrc file will contain content that typically look like this.

machine mmmmmm login xxxxxx password yyyyyy

The .netrc file can be specified by putting the following line in your .daprc/.dodsrc file.

HTTP.NETRC=<path to netrc file>

One final note. In using this, it is almost certain that you will need to specify a real cookie jar file (HTTP.COOKIEJAR) so that the redirect site can pass back authorization information.

URL Constrained RC File Entries

Examples.

[remotetest.unidata.ucar.edu]HTTP.VERBOSE=1
or
[fake.ucar.edu:9090]HTTP.VERBOSE=0

For example, the URL

http://remotetest.unidata.ucar.edu/thredds/dodsC/testdata/testData.nc

Similarly,

http://fake.ucar.edu:9090/dts/test.01

Client-Side Certificates

• HTTP.COOKIEJAR — a file path for storing cookies across re-direction.
• HTTP.NETRC — the path to the netrc file.
• HTTP.SSL.CERTIFICATE — the file path for the client side certificate file.
• HTTP.SSL.KEY — this should have the same value as HTTP.SSL.CERTIFICATE.
• HTTP.SSL.CAPATH — the path to a "certificates" directory.
• HTTP.SSL.VALIDATE — force validation of the server certificate.

Appendix A. All RC-File Keys

Appendix B. URS Access in Detail

Appendix C. ESG Access in Detail

In order to access ESG datasets, however, it is necessary to register as a user with ESG and to setup your environment so that proper authentication is established between an netcdf client program and the ESG data server. Specifically, it is necessary to use what is called "client-side keys" to enable this authentication. Normally, when a client accesses a server in a secure fashion (using "https"), the server provides an authentication certificate to the client. With client-side keys, the client must also provide a certificate to the server so that the server can know with whom it is communicating.

The netcdf library uses the curl library and it is that underlying library that must be properly configured.

Terminology

• Keystore - a repository to hold the client side key.
• Truststore - a repository to hold a chain of certificates that can be used to validate the certificate sent by the server to the client.

Initial Steps

1. Register at http://badc.nerc.ac.uk/register to obtain access to badc and to obtain an openid, which will looks something like:

   https://ceda.ac.uk/openid/Firstname.Lastname

2. Ask BADC for access to whatever datasets are of interest.

3. Obtain short term credentials at http://grid.ncsa.illinois.edu/myproxy/MyProxyLogon/ You will need to download and run the MyProxyLogon program. This will create a keyfile in, typically, the directory ".globus". The keyfile will have a name similar to this: "x509up_u13615" The other elements in ".globus" are certificates to use in validating the certificate your client gets from the server.

4. Obtain the program source ImportKey.java from this location: http://www.agentbob.info/agentbob/79-AB.html (read the whole page, it will help you understand the remaining steps).

Building the KeyStore

    openssl pkcs8 -topk8 -nocrypt -in x509up_u13615 -inform PEM -out key.der -outform DER

    openssl x509 -in x509up_u13615 -inform PEM -out cert.der -outform DER

    java -classpath  -Dkeypassword="" -Dkeystore=./ key.der cert.der

Building the TrustStore

  keytool -trustcacerts -storepass "password" -v -keystore "truststore"  -importcert -file "${c}"

Running the C Client

• HTTP.COOKIEJAR=~/.dods_cookies
• HTTP.NETRC=~/.netrc
• HTTP.SSL.CERTIFICATE=~/esgkeystore
• HTTP.SSL.KEY=~/esgkeystore
• HTTP.SSL.CAPATH=~/.globus
• HTTP.SSL.VALIDATE=1

As noted, also uses re-direction based authentication. So, when it receives an initial connection from a client, it redirects to a separate authentication server. When that server has authenticated the client, it redirects back to the original url to complete the request.

Script for creating Stores

#!/bin/sh -x
KEYSTORE="esgkeystore"
TRUSTSTORE="esgtruststore"
GLOBUS="globus"
TRUSTROOT="certificates"
CERT="x509up_u13615"
TRUSTROOTPATH="$GLOBUS/$TRUSTROOT"
CERTFILE="$GLOBUS/$CERT"
PWD="password"

D="-Dglobus=$GLOBUS"
CCP="bcprov-jdk16-145.jar" 
CP="./build:${CCP}" 
JAR="myproxy.jar"

# Initialize needed directories
rm -fr build
mkdir build
rm -fr $GLOBUS
mkdir $GLOBUS
rm -f $KEYSTORE
rm -f $TRUSTSTORE

# Compile MyProxyCmd and ImportKey
javac -d ./build -classpath "$CCP" *.java
javac -d ./build ImportKey.java

# Execute MyProxyCmd
java -cp "$CP myproxy.MyProxyCmd

# Build the keystore
openssl pkcs8 -topk8 -nocrypt -in $CERTFILE -inform PEM -out key.der -outform DER
openssl x509 -in $CERTFILE -inform PEM -out cert.der -outform DER
java -Dkeypassword=$PWD -Dkeystore=./${KEYSTORE} -cp ./build ImportKey key.der cert.der

# Clean up the certificates in the globus directory
for c in ${TRUSTROOTPATH}/*.0 ; do
    alias=`basename $c .0`
    sed -e '0,/---/d' <$c >/tmp/${alias}
    echo "-----BEGIN CERTIFICATE-----" >$c       
    cat /tmp/${alias} >>$c
done
 
# Build the truststore
for c in ${TRUSTROOTPATH}/*.0 ; do
    alias=`basename $c .0`
    echo "adding: $TRUSTROOTPATH/${c}"
    echo "alias: $alias"
    yes | keytool -trustcacerts -storepass "$PWD" -v -keystore ./$TRUSTSTORE -alias $alias -importcert -file "${c}"
done
exit