Hi,
We have a large dataset which in total is around 27Gb for a single
variable across the complete temporal and spatial domain. We have found
that if we request the complete data set through WCS we simply get a
HTTP 400 error with a serviceException message of 'Unknown problem'
(full output below). When we looked in the Thredds logs we find an
error stating that variable cannot be over 4294967292 bytes. Is this
size configurable? if not is there any sensible way to detect this error
in the client? We would like to be able to catch this error and then
split the get coverage request up into more manageable chunks.
Thanks for an help
Olly Clements
CURL output :
curl -i
"http://earthserver.pml.ac.uk/thredds/wcs/CCI-v095-M?crs=OGC%3ACRS84&service=WCS&format=NetCDF3&request=GetCoverage&version=1.0.0&coverage=chlor_a&time=1997-12-01%2F2013-12-31";
HTTP/1.1 400 Bad Request
Date: Mon, 16 Dec 2013 10:34:34 GMT
Content-Type: application/vnd.ogc.se_xml;charset=ISO-8859-1
Connection: close
Transfer-Encoding: chunked
<?xml version="1.0" encoding="UTF-8"?>
<ServiceExceptionReport xmlns="http://www.opengis.net/ogc"; version="1.2.0">
<ServiceException>Unknown problem</ServiceException>
</ServiceExceptionReport>
Thredds.log :
2013-12-16T09:43:52.021 +0000 [ 248161287][ 90503] INFO -
thredds.server.wcs.WCSServlet - Remote host: 192.171.164.51 - Request:
"GET
/thredds/wcs/CCI-v095-M?crs=OGC%3ACRS84&service=WCS&format=NetCDF3&request=GetCoverage&version=1.0.0&coverage=chlor_a&time=1997-12-01%2F2013-12-31
HTTP/1.1"
2013-12-16T09:43:52.021 +0000 [ 248161287][ 90503] DEBUG -
thredds.servlet.DatasetHandler - DatasetHandler wants /CCI-v095-M
2013-12-16T09:43:52.021 +0000 [ 248161287][ 90503] DEBUG -
thredds.servlet.DatasetHandler - -- DatasetHandler found NcmlDataset=
CCI-v095-M
2013-12-16T09:43:52.437 +0000 [ 248161703][ 90503] DEBUG -
thredds.wcs.v1_0_0_1.WcsCoverage - writeCoverageDataToFile():
ncFile=/opt/tomcat/apache-tomcat-7.0.29/content/thredds/cache/wcs/WCS4099895692493199293.nc
2013-12-16T09:43:52.438 +0000 [ 248161704][ 90503] INFO -
ucar.nc2.dt.grid.NetcdfCFWriter - Request size = 27620 Mbytes
2013-12-16T09:43:52.438 +0000 [ 248161704][ 90503] ERROR -
thredds.server.wcs.v1_0_0_1.WcsHandler - Unknown problem.
java.lang.IllegalArgumentException: Variable size in bytes 27620352000
may not exceed 4294967292
at
ucar.nc2.NetcdfFileWriteable.addVariable(NetcdfFileWriteable.java:431)
at
ucar.nc2.NetcdfFileWriteable.addVariable(NetcdfFileWriteable.java:372)
at ucar.nc2.FileWriter.writeVariable(FileWriter.java:581)
at ucar.nc2.FileWriter.writeVariables(FileWriter.java:600)
at ucar.nc2.dt.grid.NetcdfCFWriter.makeFile(NetcdfCFWriter.java:189)
at
thredds.wcs.v1_0_0_1.WcsCoverage.writeCoverageDataToFile(WcsCoverage.java:280)
at
thredds.wcs.v1_0_0_1.GetCoverage.writeCoverageDataToFile(GetCoverage.java:169)
at thredds.server.wcs.v1_0_0_1.WcsHandler.handleKVP(WcsHandler.java:125)
at thredds.server.wcs.WCSServlet.doGet(WCSServlet.java:251)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:621)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at
thredds.servlet.filter.RequestQueryFilter.doFilter(RequestQueryFilter.java:122)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at
thredds.servlet.filter.RequestPathFilter.doFilter(RequestPathFilter.java:105)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:225)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:168)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98)
at
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:927)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:200)
at
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:585)
at
org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
2013-12-16T09:43:52.439 +0000 [ 248161705][ 90503] INFO -
thredds.server.wcs.v1_0_0_1.WcsHandler - Request Completed - 400 - -1 - 418
Latest news: www.pml.ac.uk and @PlymouthMarine
Plymouth Marine Laboratory (PML) is a company limited by guarantee registered
in England & Wales, company number 4178503. Registered Charity No. 1091222.
Registered Office: Prospect Place, The Hoe, Plymouth PL1 3DH, UK.
This message is private and confidential. If you have received this message in
error, please notify the sender and remove it from your system. You are
reminded that e-mail communications are not secure and may contain viruses; PML
accepts no liability for any loss or damage which may be caused by viruses.
