Annotated TDS Configuration file

These are parameters that control the TDS that can be set by the TDS administrator in ${tomcat_home}/content/thredds/threddsConfig.xml.

In general, you do not have to change any of these parameters, since the TDS will use default settings. If you do change any of these, you must restart the thredds webapp (eg through the tomcat manager application) for them to take effect. The exception is that the catalogRoot elements will be reread by doing a Debug/Reinit, if you have remote management enabled.

Here is an annotated example threddsConfig.xml:

Elements in threddsConfig.xml

Catalog Roots

  <catalogRoot>tempCatalog.xml</catalogRoot>
  <catalogRoot>idd/catalog.xml</catalogRoot>
  <catalogRoot>catgen/subdir/enhancedCatalog-1.0.xml</catalogRoot>

These elements name your root catalogs that are not referenced from your main catalog ( ${tomcat_home}/content/thredds/catalog.xml). On startup, the TDS reads the main catalog and any root catalogs you list here, plus any catalogs that are referenced by them in a catalogRef. Data roots and other needed information are found and cached. All the catalogs found in this way are called static catalogs, and all static catalogs must live under the ${tomcat_home}/content/thredds directory. When you do a Debug/Reinit, the static catalogs are reread.

CDM library Disk cache 

  <DiskCache>
    <alwaysUse>false</alwaysUse>
    <dir>/temp/cache/</dir>
    <scour>1 hour</scour>
    <maxSize>10 Gb</maxSize>
  </DiskCache>

These elements control where the CDM/NetCDF-Java library writes temporary files, for example when it needs to unzip files, or write GRIB index files, etc. If alwaysUse is true, these temporary files will always be written to the cache directory specified by dir (choosing a cache directory). If alwaysUse is false, TDS will try to write them to the same directory as the original file, and if the TDS doesnt have write permission it will then write the files to the cache directory. Write permission will be determined by what rights the Tomcat user has (the user that starts up Tomcat). For security reasons, you want to carefully limit the file permissions of the Tomcat user.

When opening a file, if alwaysUse is true, TDS looks only in the cache directory for the temporary file. If alwaysUse is false, TDS will first look for the temporary file in the same directory as the original file, and if not found, then will look in the cache.

Every scour amount of time, the largest items in the cache will be deleted, until the directory has less than maxSize bytes. Note that the directory will sometimes exceed maxSize, and will only be knocked back to maxSize when the scour thread runs.

If not otherwise set, the TDS will use the ${tomcat_home}/content/thredds/cache/ directory.

THREDDS File Handle Caching 

  <NetcdfFileCache>
    <minFiles>200</minFiles>
    <maxFiles>400</maxFiles>
    <scour>10 min</scour>
  </NetcdfFileCache>
  <NetcdfDatasetCache>
    <minFiles>100</minFiles>
    <maxFiles>200</maxFiles>
    <scour>10 min</scour>
  </NetcdfDatasetCache>
  <HTTPFileCache>
    <minFiles>25</minFiles>
    <maxFiles>40</maxFiles>
    <scour>10 min</scour>
  </HTTPFileCache>

These elements control the size of the TDS cache for objects/open file handles for NetcdfFile, NetcdfDataset, and files served through HTTP Range-Byte requests, respectively. Up to maxFiles objects will be cached, and every scour amount of time, older items in the cache will be released, until only minFiles objects are left. The amount of time specified by scour uses any valid udunits time string, such as sec, min, hour, day.

GRIB indexing 

  <GribIndexing>
    <setExtendIndex>false</setExtendIndex>
    <alwaysUseCache>false</alwaysUseCache>
  </GribIndexing>

This is for servers that have dynamically growing GRIB files (such as TDS/IDD servers). New GRIB records may be appended to a GRIB file. Normally the library will detect this and auomaticallly extend the GRIB index. When GRIB indexing is being done by an external process, its best to turn off automatic index extension, by setting setExtendIndex to false, so there is no possibility of a collision by the GRIB indexing process and the TDS. If you have setExtendIndex true, you should also set alwaysUseCache to true, to avoid collisions with an external indexer.

Aggregation Cache 

  <AggregationCache>
    <dir>/temp/acache/</dir>
    <scour>24 hours</scour>
    <maxAge>90 days</maxAge>
  </AggregationCache>

If you have joinExisting Aggregations, coordinate information will be written to a cache directory specified by dir (choosing a cache directory). Every scour amount of time, any item older than maxAge will be deleted. Set scour to -1 to not scour at all.

If not otherwise set, the TDS will use the ${tomcat_home}/content/thredds/cacheAged/ directory.

Aggregation

  <Aggregation>
    <typicalDataset>latest</typicalDataset>
  </Aggregation>

You can control how NcML Aggregation chooses its typical / template dataset, the one it uses to populate the metadata for the resulting aggregated dataset. Valid values are random (default), latest and penultimate (latest but one).

CdmValidatorService

  <CdmValidatorService>
    <allow>false</allow>
    <dir>/data/tmp/thredds/cdmValidateCache/</dir>
    <maxFileUploadSize>1 Gb</maxFileUploadSize>
    <scour>10 min</scour>
    <maxAge>-1 days</maxAge>
  </CdmValidatorService>

This controls the CDM Validation Service, which allows users to send URLs and to upload files to the TDS, in order to analyze the coordinate systems (see motherlode CDM validation on motherlode ). By default, this service is off.

  1. allow : if true, allow this service
  2. dir : the working directory to put uploaded files in (choosing a cache directory)
  3. maxFileUploadSize : maximum size of uploaded file, can use units of Kb, Mb, Gb, Tb
  4. scour : how often to scour the working directory
  5. maxAge: how long to leave the files in the working directory. If this value is <=0 (or you dont include this element), the files will be deleted immediately.

If CdmValidatorService is allowed but the directory is not set, the TDS will use the ${tomcat_home}/content/thredds/cdmValidate/ directory for uploaded files.

NetcdfSubsetService

  <NetcdfSubsetService>
    <allow>true</allow>
    <dir>/temp/ncache/</dir>
    <scour>15 min</scour>
    <maxAge>30 min</maxAge>
  </NetcdfSubsetService>

This controls the NetCDF Subset Service, which allows users to specify a subset of a dataset and download equivilent netCDF files. By default, this service is off.

  1. allow : if true, allow this service. If you don't use a NetcdfSubset service in your catalogs, there will be no valid URLs for this service.
  2. dir : the working directory for files to be downloaded (choosing a cache directory)
  3. scour : how often to scour the working directory, to delete files that were not successfully downloaded.
  4. maxAge: how long to leave the files in the working directory while the download is occurring. The files are deleted after a successful download.

If NetcdfSubsetService is allowed but the directory is not set, the TDS will use the ${tomcat_home}/content/thredds/ncss/grid/cache/ directory for temporary files.

OPeNDAP Service 

  <Opendap>
    <ascLimit>50</ascLimit>
    <binLimit>500</binLimit>
    <serverVersion>opendap/3.7</serverVersion>
  </Opendap>

This controls the OPeNDAP data service.

  1. ascLimit: maximum size of an ascii data request , in Megabytes.
  2. binLimit: maximum size of a binary data request , in Megabytes.
  3. serverVersion: this is the String thats returned by the OPeNDAP getVersion request, and also placed into the XDOS-Server HTTP Header on all OPeNDAP responses.

WCS Service 

  <WCS>
    <allow>false</allow>
    <dir>/temp/ncache/</dir>
    <scour>15 min</scour>
    <maxAge>30 min</maxAge>
  </WCS>

This controls the Web Coverage Service (WCS) , which allows WCS clients to specify a subset of a dataset and download GeoTIFF or netCDF files. By default, this service is off.

  1. allow : to allow, change this to true. You must also add a WCS service in your catalogs, for the gridded datasets you want to server through WCS.
  2. dir : the working directory for files to be downloaded (choosing a cache directory)
  3. scour : how often to scour the working directory, to delete files that were not successfully downloaded.
  4. maxAge: how long to leave the files in the working directory while the download is occurring. The files are deleted after a successful download.

If WCS is allowed but the directory is not set, the TDS will use the ${tomcat_home}/content/thredds/wcs/wcache/ directory for temporary files.

WMS Service (version 4.0)

  <WMS>
    <allow>false</allow>
    <paletteLocationDir>/data/my/pallettes/</paletteLocationDir>
    <ogcMetaXML>OGCMeta.xml</ogcMetaXML>  
    <dir>/temp/ncache/</dir>
    <scour>15 min</scour>
    <maxAge>30 min</maxAge>
  </WMS>

This controls the Web Map Service (WMS) , which allows WMS clients to get images of gridded datasets. By default, this service is off.

  1. allow : to enable the WMS service, change this to true. You must also add a WMS service element in your catalogs for the gridded datasets you want to serve through WMS.
  2. paletteLocationDir: optionally specify your own pallette files, by specifying the directory where they are contained. If the directory location starts with a "/", the path is absolute, otherwise it is relative to ${tomcat_home}/content/thredds/. If you dont specify it, or specify it incorrectly, the default pallettes will be used, which are in the war file under WEB-INF/pallettes.
  3. ogcMetaXML: optionally specify your own OGC metadata file. Use an absolute file path, or a path reletive to ${tomcat_home}/content/thredds/. Otherwise the default file WEB-INF/OGCMeta.xml. is used.
  4. dir : the working directory for files to be downloaded (choosing a cache directory)
  5. scour : how often to scour the working directory, to delete files that were not successfully downloaded.
  6. maxAge: how long to leave the files in the working directory while the download is occurring. The files are deleted after a successful download.

Loadable Viewers

   <Viewer>my.package.MyViewer</Viewer>

You can place a link to your own Viewer on the TDS HTML page, by loading a viewer at runtime.

Loadable Dataset Source 

   <DatsetSource>my.package.DatsetSourceImpl</DatsetSource>

You can add a DataSource - essentially an IOSP with access to Servlet request parameters, by loading a dataset source at runtime.

FmrcInventory

  <FmrcInventory>
    <openType>XML_ONLY</openType>
  </FmrcInventory>

This controls how the Forecast Model Run Collection Inventory Servlet opens FMRC inventory files. XML_ONLY means to only use the XML if it already exists, assuming new files are generated externally.

NetCDF-Java runtime Loading 

  <nj22Config>
    <ioServiceProvider class="edu.univ.ny.stuff.FooFiles"/>
    <coordSysBuilder convention="foo" class="test.Foo"/>
    <coordTransBuilder name="atmos_ln_sigma_coordinates" type="vertical" class="my.stuff.atmosSigmaLog"/>
    <typedDatasetFactory datatype="Point" class="gov.noaa.obscure.file.Flabulate"/>
    <table type="GRIB1" filename="/home/rkambic/grib/tables/userlookup.lst"/>
    <table type="GRIB2" filename="/home/rkambic/grib/tables/grib2userparameters"/>
  </nj22Config>

These elements allow you to specify Runtime parameters for the Netcdf-Java library from the threddsConfig file. See the Netcdf-Java tutorial for an overview.

Cache Locations

The various cache locations at times will contain large amounts of data. You should choose a location that will not fill up (especially if that location affects other important locations like /opt, /home, etc). If you have a large disk for your data, that may be a good location for the cache directories. On unix-like machines, you can run 'df' to get a listing of disks on your machine. The listing includes size and mount location.

This document is maintained by John Caron and was last updated June, 2009