[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: opendap headers



Hi Bob:

Whats happening is that the opendap call is failing, so NJ sees if it can be served through HTTP with byte ranges.

So the question is, why does it fail with the opendap call? I just checked the URL http://coastwatch.pfel.noaa.gov/erddap/griddap/erdMHchla8day.dds and it seems ok:

Dataset {
  Float64 time[time = 1516];
  Float64 altitude[altitude = 1];
  Float64 latitude[latitude = 3001];
  Float64 longitude[longitude = 7201];
  GRID {
    ARRAY:
Float32 chlorophyll[time = 1516][altitude = 1][latitude = 3001][longitude = 7201];
    MAPS:
      Float64 time[time = 1516];
      Float64 altitude[altitude = 1];
      Float64 latitude[latitude = 3001];
      Float64 longitude[longitude = 7201];
  } chlorophyll;
} erdMHchla8day;

however when NJ tries to read coordinate variables:

http://coastwatch.pfel.noaa.gov/erddap/griddap/erdMHchla8day.dds?time,altitude,latitude

an error comes up:

Error {
  code = 3 ;
message = "java.lang.Exception: You must request just one dimension variable at a time.
at gov.noaa.pfel.erddap.dataset.EDDGrid.parseAxisDapQuery(EDDGrid.java:772)
at 
gov.noaa.pfel.erddap.dataset.AxisDataAccessor.<init>(AxisDataAccessor.java:82)
at gov.noaa.pfel.erddap.dataset.EDDGrid.saveAsDDS(EDDGrid.java:2541)
at gov.noaa.pfel.erddap.dataset.EDDGrid.respondToDapQuery(EDDGrid.java:1307)
at gov.noaa.pfel.erddap.Erddap.doDap(Erddap.java:765)
at gov.noaa.pfel.erddap.Erddap.doGet(Erddap.java:288)

" ;
} ;

which is a bug in the server really. NJ is compounding it by not giving useful error messages. It turns out to be tricky to extract the error message from the dods library, but ill give it another try.

some possible hints:

1) if you replace http with dods, you can avoid the check for HTTP which can complicate things, eg use

dods://coastwatch.pfel.noaa.gov/erddap/griddap/erdMHchla8day

in your NJ client

2) turn on NJ debugging by calling

  DODSNetcdfFile.setDebugFlags( new DebugFlagsImpl("DODS/serverCall"));

before opening file

other dods flags are:

    debugCE = debugFlag.isSet("DODS/constraintExpression");
    debugServerCall = debugFlag.isSet("DODS/serverCall");
    debugOpenResult = debugFlag.isSet("DODS/debugOpenResult");
    debugDataResult = debugFlag.isSet("DODS/debugDataResult");
    debugCharArray = debugFlag.isSet("DODS/charArray");
    debugConstruct = debugFlag.isSet("DODS/constructNetcdf");
    debugPreload = debugFlag.isSet("DODS/preload");
    debugTime = debugFlag.isSet("DODS/timeCalls");
    showNCfile = debugFlag.isSet("DODS/showNCfile");
    debugAttributes = debugFlag.isSet("DODS/attributes");
    debugCached = debugFlag.isSet("DODS/cache");

3) i often take the opendap calls NJ is doing (as dumped by the serverCall debug flag)and paste them into a browser where any error messages are shown.

Bob Simons wrote:
It seems that in order for NetcdfFile.open to work with a (OPeN)DAP source, the DAP server has to include "Accept-Ranges" and "Content-Length" in the http header.

A program that I wrote, ERDDAP, tries to follow the DAP standard and act as an DAP server. But when I try to access it with (for example) NetcdfFile nc = NetcdfFile.open("http://coastwatch.pfel.noaa.gov/erddap/griddap/erdMHchla8day";);

I get the following error:

Exception in thread "main" java.io.IOException: Server does not support byte Ranges at ucar.unidata.io.http.HTTPRandomAccessFile.<init>(HTTPRandomAccessFile.java:99) at ucar.unidata.io.http.HTTPRandomAccessFile.<init>(HTTPRandomAccessFile.java:78)
        at ucar.nc2.NetcdfFile.getRaf(NetcdfFile.java:277)
        at ucar.nc2.NetcdfFile.open(NetcdfFile.java:258)
        at ucar.nc2.NetcdfFile.open(NetcdfFile.java:231)
        at ucar.nc2.NetcdfFile.open(NetcdfFile.java:219)
        at ucar.nc2.NetcdfFile.open(NetcdfFile.java:207)
        at gov.noaa.pfel.coastwatch.TestAll.main(TestAll.java:273)

I looked at the code for HTTPRandomAccessFile and saw the requirements for "Accept-Ranges" and "Content-Length".

In the DAP standard, section 7.1 "Response Headers", I don't see any mention of "Accept-Ranges" and "Content-Length".

Am I missing something?

Comments? Suggestions?

Is there a chance you could remove the reliance on those headers?

(To respond to the obvious question: No, I would prefer not to include either of these headers. They imply that the content length is known at the start of the response. My program (among other features) streams opendap sequence data from various sources. It repeatedly gets a chunk of data from the source and sends it to the client, all as one sequence. So the total content length is not known ahead of time.)

Thank you for looking into this.

Sincerely,

Bob Simons
Satellite Data Product Manager
Environmental Research Division
NOAA Southwest Fisheries Science Center
1352 Lighthouse Ave
Pacific Grove, CA 93950-2079
(831)658-3205
address@hidden

The contents of this message are mine personally and
do not necessarily reflect any position of the
Government or the National Oceanic and Atmospheric
Administration.
<>< <>< <>< <>< <>< <>< <>< <>< <><