TDS Configuration fileTDS Configuration fileThe TDS configuration file ${tomcat_home}/content/thredds/threddsConfig.xml allows the TDS administrator to control the behavior of the TDS. This is the complete reference document for TDS version 4.2.
In general, you do not have to change any of these parameters, since the TDS will use default settings. If you do change any of these, you must restart the thredds webapp (e.g., through the tomcat manager application) for them to take effect. The exception is that the catalogRoot elements will be reread by doing a Debug/Reinit, if you have remote management enabled.
Here is an annotated example threddsConfig.xml.
Contents
In the serverInformation element, you can provide basic
information about your server including contact information, and
information about the group that hosts the server.
<serverInformation>
<name>Initial TDS Installation</name>
<logoUrl>threddsIcon.gif</logoUrl>
<logoAltText>Initial TDS Installation</logoAltText>
<abstract>Scientific Data</abstract>
<keywords>meteorology, atmosphere, climate, ocean, earth science</keywords>
<contact>
<name>Support</name>
<organization>My Group</organization>
<email>support@my.group</email>
<phone></phone>
</contact>
<hostInstitution>
<name>My Group</name>
<webSite>http://www.my.group/</webSite>
<logoUrl>myGroup.gif</logoUrl>
<logoAltText>My Group</logoAltText>
</hostInstitution>
</serverInformation>
The information provided in the serverInformation element is used in:
Coming soon, it will also be used in:
Some of the information provided in the serverInformation element
supersedes information in the htmlSetup element (described below).
The superseded information is listed as deprecated in the description of the
htmlSetup element below.
NOTE: URLs may be absolute or relative. Relative URLs must be relative to the webapp URL, i.e., http://server:port/thredds/.
In the htmlSetup element, you can configure which CSS
documents are used in all the HTML pages generated by the TDS as well
as a number of basic icons.
The following shows the default values used if the htmlSetup
element is not configured:
<htmlSetup>
<cssPage>tds.css</cssPage>
<cssCatalog>tdsCat.css</cssCatalog>
<folderIconUrl>folder.gif</folderIconUrl>
<folderIconAlt>Folder</folderIconAlt>
<datasetIconUrl>dataset.gif</datasetIconUrl> <!-- Not currently used. -->
<datasetIconAlt>Dataset</datasetIconAlt> <!-- Not currently used. -->
<!-- DEPRECATED: use serverInformation element instead. -->
<installName>Motherlode TDS</installName>
<installLogoUrl>unidataLogo.gif</installLogoUrl>
<installLogoAlt>Motherlode TDS</installLogoAlt>
<hostInstName>Unidata</hostInstName>
<hostInstUrl>http://www.unidata.ucar.edu/</hostInstUrl>
<hostInstLogoUrl>unidataLogo.gif</hostInstLogoUrl>
<hostInstLogoAlt>Unidata</hostInstLogoAlt>
<!-- END DEPRECATED -->
</htmlSetup>
NOTE: URLs may be absolute or relative, relative URLs must be relative to the webapp URL, i.e., http://server:port/thredds/.
since version 4..1
<catalogWriting>
<useBytesForDataSize>false<useBytesForDataSize/>
</catalogWriting>
<catalogRoot>tempCatalog.xml</catalogRoot>
<catalogRoot>idd/catalog.xml</catalogRoot>
<catalogRoot>catgen/subdir/enhancedCatalog-1.0.xml</catalogRoot>
These elements name your root catalogs that are not referenced from your main catalog ( ${tomcat_home}/content/thredds/catalog.xml). On startup, the TDS reads the main catalog and any root catalogs you list here, plus any catalogs that are referenced by them in a catalogRef. Data roots and other needed information are found and cached. All the catalogs found in this way are called static catalogs, and all static catalogs must live under the ${tomcat_home}/content/thredds directory. When you do a Debug/Reinit, the static catalogs are reread.
<Viewer>my.package.MyViewer</Viewer>
You can place a link to your own Viewer on the TDS HTML page, by loading a viewer at runtime. This line is needed in the config file only if you are writing your own Java class.
<DatasetSource>my.package.DatasetSourceImpl</DatasetSource>
You can add a DataSource - essentially an IOSP with access to Servlet request parameters, by loading a dataset source at runtime.
Catalog services are available by default for catalogs served by the local TDS. But for remote catalogs these services must be explicitly enabled in threddsConfig.xml:
<CatalogServices>
<allowRemote>true</allowRemote>
</CatalogServices>
<Opendap>
<ascLimit>50</ascLimit>
<binLimit>500</binLimit>
<serverVersion>opendap/3.7</serverVersion>
</Opendap>
This controls the OPeNDAP data service. Because its easy for a user to inadvertantly request very large amounts of data, the TDS limits the size of the data response. In our experience legitimate requests ask for subset sizes that are well below the defaults.
The OGC WCS service provided as part of the TDS is described in more detail here.
By default this service is disabled in the TDS. It can be enabled by including the following in the
threddsConfig.xml file:
<WCS> <allow>true</allow> </WCS>
The following shows all the configuration options available in the WCS section of the
threddsConfig.xml file with the default values shown:
<WCS> <allow>false</allow> <dir>(see the note below)</dir> <scour>15 min</scour> <maxAge>30 min</maxAge> </WCS>
We recommend that you include in the threddsConfig.xml file only the options you want to change. Here is the description of the
various options:
Note: If WCS is allowed but the directory is not set, the TDS will use
the ${tomcat_home}/content/thredds/wcs/wcache/ directory for
temporary files.
The OGC WMS service provided as part of the TDS is described in more detail here.
By default this service is disabled in the TDS. It can be enabled by including the following in the
threddsConfig.xml file:
<WMS> <allow>true</allow> </WMS>
The following shows all the configuration options available in the WMS section of the
threddsConfig.xml file with the default values shown:
<WMS> <allow>false</allow> <allowRemote>false</allowRemote> <paletteLocationDir>/WEB-INF/palettes</paletteLocationDir> <maxImageWidth>2048</maxImageWidth> <maxImageHeight>2048</maxImageHeight> </WMS>
We recommend that you include in the threddsConfig.xml file only the options you
want to change. Here is the description of the various options:
This service is an experimental prototype. As such it may be subject to change.
<NetcdfSubsetService> <allow>true</allow> <dir>/tmp/thredds/ncSubsetCache/</dir> <scour>15 min</scour> <maxAge>30 min</maxAge> </NetcdfSubsetService>
This controls the NetcdfSubset service (URL: "/thredds/ncss/grid/...") which supports subset requests in terms of latitude/longitude bounding boxes and date ranges rather than array index ranges. More information is available on the NetcdfSubset Service Reference page.
By default, this service is not enabled. It can be enabled by setting the value of the allow element to "true".
By default the ncISO services are disabled in the TDS. They can be enabled by including the following in the
threddsConfig.xml file:
<NCISO> <ncmlAllow>true</ncmlAllow> <uddcAllow>true</uddcAllow> <isoAllow>true</isoAllow> </NCISO>
Each of the allow elements above enables the corresponding ncISO service (NCML, UDDC, and ISO). The ncISO services are described in more detail on the "ncISO" page.
There are no further ncISO options available in the threddsConfig.xml.
<Aggregation>
<typicalDataset>latest</typicalDataset>
</Aggregation>
You can control how NcML Aggregation chooses its typical / template dataset, the one it uses to populate the metadata for the resulting aggregated dataset. Valid values are first, random, latest and penultimate (latest but one). The default for versin 4.2 is penultimate.
<GribIndexing>
<setExtendIndex>false</setExtendIndex>
<alwaysUseCache>false</alwaysUseCache>
</GribIndexing>
This is for servers that have dynamically growing GRIB files (such as TDS/IDD servers). New GRIB records may be appended to a GRIB file. Normally the library will detect this and auomaticallly extend the GRIB index. When GRIB indexing is being done by an external process, its best to turn off automatic index extension, by setting setExtendIndex to false, so there is no possibility of a collision by the GRIB indexing process and the TDS. If you have setExtendIndex true, you should also set alwaysUseCache to true, to avoid collisions with an external indexer.
<nj22Config>
<ioServiceProvider class="edu.univ.ny.stuff.FooFiles"/>
<coordSysBuilder convention="foo" class="test.Foo"/>
<coordTransBuilder name="atmos_ln_sigma_coordinates" type="vertical" class="my.stuff.atmosSigmaLog"/>
<typedDatasetFactory datatype="Point" class="gov.noaa.obscure.file.Flabulate"/>
<table type="GRIB1" filename="/home/rkambic/grib/tables/userlookup.lst"/>
<table type="GRIB2" filename="/home/rkambic/grib/tables/grib2userparameters"/>
</nj22Config>
These elements allow you to specify Runtime parameters for the Netcdf-Java library from the threddsConfig file. See the Netcdf-Java tutorial for an overview.
<DiskCache>
<alwaysUse>false</alwaysUse>
<dir>/temp/cache/</dir>
<scour>1 hour</scour>
<maxSize>10 Gb</maxSize>
</DiskCache>
These elements control where the CDM/NetCDF-Java library writes temporary files, for example when it needs to unzip files, or write GRIB index files, etc. If alwaysUse is true, these temporary files will always be written to the cache directory specified by dir (choosing a cache directory). If alwaysUse is false, TDS will try to write them to the same directory as the original file, and if the TDS doesnt have write permission it will then write the files to the cache directory. Write permission will be determined by what rights the Tomcat user has (the user that starts up Tomcat). For security reasons, you want to carefully limit the file permissions of the Tomcat user.
When opening a file, if alwaysUse is true, TDS looks only in the cache directory for the temporary file. If alwaysUse is false, TDS will first look for the temporary file in the same directory as the original file, and if not found, then will look in the cache.
Every scour amount of time, the largest items in the cache will be deleted, until the directory has less than maxSize bytes. Note that the directory will sometimes exceed maxSize, and will only be knocked back to maxSize when the scour thread runs.
If not otherwise set, the TDS will use the ${tomcat_home}/content/thredds/cache/cdm directory. We recommend that you use this default, by not specifying the DiskCache.dir.
since version 4.1
<AggregationCache>
<dir>/temp/acache/</dir>
<scour>24 hours</scour>
<maxAge>90 days</maxAge>
</AggregationCache>
If you have joinExisting Aggregations, coordinate information will be written to a cache directory specified by dir (choosing a cache directory). If not otherwise set, the TDS will use the ${tomcat_home}/content/thredds/cache/agg/ directory. We recommend that you use this default, by not specifying a AggregationCache.dir element.
Every scour amount of time, any item that hasnt been changed since maxAge time will be deleted. Set scour to -1 to not scour if you have aggregations that never change. Otherwise, make maxAge longer than the longest time between changes. Basically, you dont want to remove active aggregations.
This cache information is intended to be permanent, it stores coordinate information from each file in the aggregation, so that the file does not have to be opened each time the dataset is opened. If you have large joinExisting aggregations, there will be a very pronounced difference with and without this cache.
The cache information is updated based on the recheckEvery field in the joinExisting aggregation element.
since version 4.2
This is where persistent information is kept about feature collections. If you dont have any feature collections, there should be no memory use.
<FeatureCollection>
<dir>/tomcat_home/content/thredds/cache/collection/</dir> <maxSize>20 Mb</maxSize> <jvmPercent>2</jvmPercent>
</FeatureCollection>
<NetcdfFileCache>
<minFiles>50</minFiles>
<maxFiles>100</maxFiles>
<scour>10 min</scour>
</NetcdfFileCache>
<HTTPFileCache>
<minFiles>5</minFiles>
<maxFiles>10</maxFiles>
<scour>12 min</scour>
</HTTPFileCache>
<GribCollection>
<minFiles>100</minFiles>
<maxFiles>200</maxFiles>
<scour>15 min</scour>
</GribCollection
These elements control the size of the TDS cache for objects/open file handles for 1) NetcdfFile objects, 2) files served through HTTP Range-Byte requests,and 3) GribCollection files, respectively. Up to maxFiles objects will be cached, and every scour amount of time, older items in the cache will be released, until only minFiles objects are left. The scour element uses any valid udunits time string, such as sec, min, hour, day. To disable the cache, set maxFiles to 0.
You may also set cache=false on a datasetRoot element. This controls whether a file is put in the NetcdfFile object cache. All files that are found through that datasetRoot are affected. Example:
<catalog xmlns="http://www.unidata.ucar.edu/namespaces/thredds/InvCatalog/v1.0"
xmlns:xlink="http://www.w3.org/1999/xlink" name="THREDDS Catalog for NetCDF Files" version="1.0.3">
<service name="ncdods" serviceType="OpenDAP" base="/thredds/dodsC/"/>
<datasetRoot path="FVCOM" location="/http/www/CODFISH/Data/COM/" cache="false"/>
<datasetRoot path="ZZTOP" location="/http/www/CODFISH/Data/ZCOM/" />
<dataset name="GOM2 Forecast" ID="gom2_nocache" serviceName="ncdods" urlPath="FVCOM/NECOFS/Forecasts/NECOFS_GOM2_FORECAST.nc" dataType="Grid"/>
<dataset name="GOM3 Forecast" ID="gom3_nocache" serviceName="ncdods" urlPath="FVCOM/NECOFS/Forecasts/NECOFS_GOM3_FORECAST.nc" dataType="Grid"/>
<dataset name="MASSBAY Forecast" ID="massbay_nocache" serviceName="ncdods" urlPath="ZZTOP/NECOFS/Forecasts/NECOFS_FVCOM_OCEAN_MASSBAY_FORECAST.nc" dataType="Grid"/>
</catalog>
In the above example, the files
will not be cached, but this one will be cached:
since version 4.1
The ehcache object cache is backed up in this directory. Currently, only OS directory scans are cached here.
<ehcache>
<configFile>/tomcat_home/webapp/WEB-INF/ehcache.xml</configFile>
<dir>/tomcat_home/content/thredds/ehcache/</dir>
</ehcache>
since version 4.2.8
Static catalogs are cached for performance, under the assumption there are a small number of them. If you have a large number of static catalogs, you may turn caching off to save memory, at the cost of some performace slowdown. Root catalogs are always cached, as are catalogs with datasetFmrc and featureCollections in them.
<Catalog>
<cache>false</cache>
</Catalog>
As of TDS 4.2 (stable release), the various cache directory locations are now by default are all under {tomcat_home}/content/thredds/ :
cache location description AggregationCache.dir cache/agg/ for joinExisting aggregations only: write XML files here. CdmValidatorService.dir cache/cdmValidate/ temporary files for cdmvalidator (seperate war) DiskCache.dir cache/cdm/ only used when non-writeable data directory or alwaysUse = true, puts CDM indexes, decompressed files, etc. into this directory
ehcache.dir cache/ehcache/ object cache, put ehcache backup file in this directory FeatureCollection.dir cache/collection/ when we read GridDataset for FMRC, write an XML summary, store in BDB in this directory NetcdfSubsetService.dir cache/ncss/ temporary files for NCSS WCS.dir cache/wcs/ temporary files for WCS
We recommend that you use these defaults, by not specifying them in the threddsConfig.xml file. If you need to move the cache location, move all of them by using a link in {tomcat_home}/content/thredds/.
These various caches at times may contain large amounts of data. You should choose a location that will not fill up (especially if that location affects other important locations like /opt, /home, etc). If you have a large disk for your data, that may be a good location for the cache directories. On unix-like machines, you can run 'df' to get a listing of disks on your machine. The listing includes size and mount location.
contentRoot ElementDO NOT USE
Intended to chain together multiple "content" directories. For instance, the following:
<contentRoots>
<contentRoot>motherlode</contentRoot>
<contentRoot>idd</contentRoot>
</contentRoots>
chains together:
dataRoot ElementNOT YET IMPLEMENTED
Eventually will provide macro-replacment in datasetScan location attributes. For instance:
<DataRoots>
<idd>/data/ldm/pub</idd>
</DataRoots>
Could be used with:
<datasetScan ... location="@iddDataRoot@/fsl">...</datasetScan>
This document is maintained by elves and was last updated May 2011