Hi Sean,
Historically (as in Apache1.x) it did, but was abandoned in Apache-2.x.
https://en.wikipedia.org/wiki/Apache_JServ_Protocol
N
> On Jan 18, 2019, at 9:59 AM, Sean Arms <sarms@xxxxxxxx> wrote:
>
> Let me bring in our Tomcat wrangler, Jen. I thought mod_jk used AJP?
>
> On Fri, Jan 18, 2019 at 10:48 AM Nathan Potter <ndp@xxxxxxxxxxx> wrote:
> Sean,
>
> Interesting.
>
> I thought that mod_jk had been abandoned/superseded for/by the Apache JServ
> Protocol (AJP) protocol with the introduction of Apache 2.x
>
>
> N
>
>
> > On Jan 18, 2019, at 9:21 AM, Sean Arms <sarms@xxxxxxxx> wrote:
> >
> > Hi Nathan, all,
> >
> > netCDF-Java does proper encoding as of 4.6.12 and 5.0.0-beta6 (runtime
> > configurable), and in the past it encoded as well - there was a period of
> > time in the 4.5 and 4.6 line of development in which the encoding was
> > skipped, but that's no longer the case.
> >
> > Interestingly enough, if you run Apache with mod_jk, non-encoded urls will
> > be accepted and passed along to tomcat in a usable form without issue, but
> > I have no idea how long that will last.
> >
> > Sean
> >
> > On Thu, Jan 17, 2019 at 4:25 PM Nathan Potter <ndp@xxxxxxxxxxx> wrote:
> > Hi Thomas,
> >
> > I believe that I worked with Robert Schmunk (the author of Panoply) on this
> > very problem and I think the reason he is using netcdfAll-5.0.0-SNAPSHOT is
> > that it solves the outgoing encoding issue on the client side. Is that
> > right Robert? (We may not hear back from him as I suspect he's furloughed
> > and unavailable a.t.m.)
> >
> > If the client cannot be cajoled into producing the correct URL encoding
> > then I think the work falls back to relaxing the server's relaxedQueryChars
> > configuration.
> >
> > Which is certainly less than ideal.
> >
> > And to end on a cheery note - once the netcdf-java library is release it
> > can take some time (up to a year) before it gets rolled into a Matlab
> > release.
> >
> > Sorry I can't be more helpful.
> >
> >
> > Sincerely,
> >
> > Nathan
> >
> >
> > > On Jan 17, 2019, at 1:39 PM, Thomas Cook <tmcook@xxxxxxxx> wrote:
> > >
> > > OK Given Nathan and Christian's comments, I'm focussing my
> > > troubleshooting on client side.
> > > What I've learned
> > > 1) Matlab's native netCDF toolbox depends on netCDF 4.3.3.1. I
> > > couldn't find an easy way to update the netCDF version inside my
> > > Matlab install.
> > > 2) I updated the netcdf4-python module to use netCDF 4.6.12. This did
> > > not solve the issue.
> > > 3) Panoply (which works) is using netcdfAll-5.0.0-SNAPSHOT. I couldn't
> > > find an older version to download, nor was I able to run panoply with
> > > an older netcdfAll.jar
> > > 4) Another user has reported success using the SNTOOLS/MEXNC package
> > > to read the files. This also uses a version of netcdfAll.jar for its
> > > decoding. I can't verify this yet.
> > >
> > > So not sure there's much more I can do with on the client side. I'll
> > > mess with the Dockerfile like Christian suggested, and try using some
> > > non FMRC collections to test with the clients. If anyone has any
> > > ideas, I'm open!
> > > Thanks to all who have replied and I'll update if I make any headway.
> > > Tom
> > >
> > > On Thu, Jan 17, 2019 at 5:53 AM Nathan Potter <ndp@xxxxxxxxxxx> wrote:
> > >>
> > >>
> > >>
> > >>> On Jan 17, 2019, at 5:37 AM, Christian Skarby <christians@xxxxxx> wrote:
> > >>>
> > >>> For what it's worth; from what I've seen encoding errors would result
> > >>> in a reject from tomcat before the request is passed to the TDS
> > >>> application, and it would then probably not show up in the
> > >>> threddsServlet.log.
> > >>
> > >> This is the behavior that we have seen.
> > >>
> > >> Tomcat's rejection is an empty document and response headers ala:
> > >>
> > >> HTTP/1.1 400 Bad Request
> > >> Server: Apache-Coyote/1.1
> > >> Transfer-Encoding: chunked
> > >> Date: Thu, 17 Jan 2019 13:47:48 GMT
> > >> Connection: close
> > >>
> > >>>
> > >>> Perhaphs Tom, you could try to just change the application code in your
> > >>> docker image (extract war or extracted thredds servlet from one version
> > >>> and then inject it in the other). I guess this should be possible,
> > >>> although I have not looked into the official docker images for thredds.
> > >>> If you make this fly it would be possible to say if it is the servlet
> > >>> or the other libraries in the docker image that causes you problems.
> > >>>
> > >>> Regarding the possible encoding errors: Tomcat introduced the more
> > >>> strict behavior in v9.0.8 (and also v8.5.31, v8.0.52 and v7.0.87).
> > >>>
> > >>> See the corresponding tomcat changelogs:
> > >>> • 62273: Implement configuration options to work-around
> > >>> specification non-compliant user agents (including all the major
> > >>> browsers) that do not correctly %nn encode URI paths and query strings
> > >>> as required by RFC 7230 and RFC 3986. (markt)
> > >>> See this thread in the google group for opendap.org:
> > >>> https://groups.google.com/a/opendap.org/forum/#!msg/support/ixTqhDXoLZQ/IT0lvZQ7CAAJ
> > >>>
> > >>> Tomcat-documentation:
> > >>> https://tomcat.apache.org/tomcat-9.0-doc/config/http.html
> > >>>
> > >>> relaxedPathChars
> > >>> The HTTP/1.1 specification requires that certain characters are %nn
> > >>> encoded when used in URI paths. Unfortunately, many user agents
> > >>> including all the major browsers are not compliant with this
> > >>> specification and use these characters in unencoded form. To prevent
> > >>> Tomcat rejecting such requests, this attribute may be used to specify
> > >>> the additional characters to allow. If not specified, no addtional
> > >>> characters will be allowed. The value may be any combination of the
> > >>> following characters: " < > [ \ ] ^ ` { | } . Any other characters
> > >>> present in the value will be ignored.
> > >>>
> > >>> relaxedQueryChars
> > >>> The HTTP/1.1 specification requires that certain characters are %nn
> > >>> encoded when used in URI query strings. Unfortunately, many user agents
> > >>> including all the major browsers are not compliant with this
> > >>> specification and use these characters in unencoded form. To prevent
> > >>> Tomcat rejecting such requests, this attribute may be used to specify
> > >>> the additional characters to allow. If not specified, no addtional
> > >>> characters will be allowed. The value may be any combination of the
> > >>> following characters: " < > [ \ ] ^ ` { | } . Any other characters
> > >>> present in the value will be ignored.
> > >>>
> > >>>
> > >>> As the OPeNDAP spesification (see section 6.1.1.2 on page 22) requires
> > >>> the use of [ and ] in the query string, and several clients doesn't
> > >>> quote these characters, we need to tweak the relaxedQueryChars
> > >>> configuration for tomcat.
> > >>
> > >> For completeness I will add that the DAP protocol also utilizes the <
> > >> and > characters for formulating relational constraints, however these
> > >> are limited to constraining Sequence data types. I don't know that the
> > >> TDS supports this data type so they may not be required in the
> > >> relaxedQueryChars list.
> > >>
> > >>
> > >>> Regards,
> > >>> Christian Skarby
> > >>>
> > >>> Den tor. 17. jan. 2019 kl. 05:42 skrev Thomas Cook <tmcook@xxxxxxxx>:
> > >>> I do not know how to debug the matlab or python clients, but this is
> > >>> what shows up in the threddsServlet.log
> > >>>
> > >>> Request: "GET
> > >>> /thredds/dodsC/HFR/USWC/6km/hourly/RTV/HFRADAR_US_West_Coast_6km_Resolution_Hourly_RTV_best.ncd.dods?lat,lon,site%5flat,site%5flon,site%5fcode,site%5fnetCode,procParams
> > >>> HTTP/1.0"
> > >>>
> > >>> Thanks
> > >>> Tom
> > >>>
> > >>> On Wed, Jan 16, 2019 at 7:47 PM Dennis Heimbigner <dmh@xxxxxxxx> wrote:
> > >>>>
> > >>>> Is there anyway you can see the exact URL being sent to the server?
> > >>>>
> > >>>> On 1/16/2019 8:24 PM, Thomas Cook wrote:
> > >>>>> Thanks Sean.
> > >>>>> The server is http://hfrnet-tds.ucsd.edu/thredds (although I just
> > >>>>> downgraded back to 4.6.12-SNAPSHOT). I will see about making my test
> > >>>>> server accessible to you, but that will take a little time to
> > >>>>> configure on my end.
> > >>>>> Let me know if you think I should open an issue on the github page and
> > >>>>> move the discussion there.
> > >>>>> Tom
> > >>>>>
> > >>>>> On Wed, Jan 16, 2019 at 6:32 PM Sean Arms <sarms@xxxxxxxx> wrote:
> > >>>>>> Hi Tom,
> > >>>>>>
> > >>>>>> So you are hitting up an opendap endpoint using all three libraries
> > >>>>>> / tools (python, matlab, and panoply)? Is this particular server
> > >>>>>> publicly accessible?
> > >>>>>>
> > >>>>>> I have a bad feeling that this is due to URL encoding (or lack
> > >>>>>> thereof) on the client side and changes made in tomcat 8.5, which it
> > >>>>>> looks like the thredds-docker repository bumped up to with the
> > >>>>>> 4.6.12 release.
> > >>>>>>
> > >>>>>> Cheers,
> > >>>>>>
> > >>>>>> Sean
> > >>>>>>
> > >>>>>>
> > >>>>>> On Wed, Jan 16, 2019 at 7:06 PM Thomas Cook <tmcook@xxxxxxxx> wrote:
> > >>>>>>> HI I just wanted to add some details.
> > >>>>>>>
> > >>>>>>> The functionality works when I downgraded to the docker container
> > >>>>>>> 4.6.12-SNAPSHOT
> > >>>>>>>
> > >>>>>>> My data is part of a FMRC collection.
> > >>>>>>>
> > >>>>>>> Thanks,
> > >>>>>>> Tom
> > >>>>>>>
> > >>>>>>> On Wed, Jan 16, 2019 at 4:51 PM Thomas Cook <tmcook@xxxxxxxx> wrote:
> > >>>>>>>> HI
> > >>>>>>>> I just yesterday updated my thredds-docker to 4.6.12 from
> > >>>>>>>> 4.6.12-snapshot. It seems to be running fine, except I'm getting
> > >>>>>>>> errors using matlab ncread and python netcdf4.dataset. When I try
> > >>>>>>>> to
> > >>>>>>>> access any of the multi-dimensional (time, lat, lon) variables I
> > >>>>>>>> get
> > >>>>>>>> the following errors
> > >>>>>>>> with matlab:
> > >>>>>>>> Error using netcdflib
> > >>>>>>>> The NetCDF library encountered an error during execution of
> > >>>>>>>> 'getVarsShort' function - 'Access failure (-77)'.
> > >>>>>>>>
> > >>>>>>>> Error in netcdf.getVar (line 136)
> > >>>>>>>> data = netcdflib(funcstr,ncid,varid,varargin{:});
> > >>>>>>>>
> > >>>>>>>> Error in internal.matlab.imagesci.nc/read (line 635)
> > >>>>>>>> data = netcdf.getVar(gid, varid, ...
> > >>>>>>>>
> > >>>>>>>> Error in ncread (line 58)
> > >>>>>>>> vardata = ncObj.read(varName, varargin{:});
> > >>>>>>>>
> > >>>>>>>> Error in readUVfromTDS (line 56)
> > >>>>>>>> u=ncread(url, 'u_mean', [1 1 1], [2 2 2]);
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> and with python:
> > >>>>>>>> netCDF4/_netCDF4.pyx in netCDF4._netCDF4.Variable.__getitem__()
> > >>>>>>>>
> > >>>>>>>> netCDF4/_netCDF4.pyx in netCDF4._netCDF4.Variable._get()
> > >>>>>>>>
> > >>>>>>>> netCDF4/_netCDF4.pyx in netCDF4._netCDF4._ensure_nc_success()
> > >>>>>>>>
> > >>>>>>>> RuntimeError: NetCDF: Access failure
> > >>>>>>>>
> > >>>>>>>> I know that the latest docker contains netcdf 4.6, where as the
> > >>>>>>>> snapshot had 4.4. Could it be a version problem with the netCD4
> > >>>>>>>> python
> > >>>>>>>> and matlab libraries? I should be running the latest netcdf python
> > >>>>>>>> version and not too far behind with matlab versions.
> > >>>>>>>>
> > >>>>>>>> The weird things are that the index variables are accessible (time,
> > >>>>>>>> lat, lon etc), and the multiple-dimension data is accessible just
> > >>>>>>>> fine using the OpenDap web interface and Panoply. So I'm a little
> > >>>>>>>> confused. I'm addressing this list since I think there might be
> > >>>>>>>> some
> > >>>>>>>> matlab and python expertise here and I think there were some
> > >>>>>>>> messages
> > >>>>>>>> about the Access failure error in the past.
> > >>>>>>>>
> > >>>>>>>> Thanks,
> > >>>>>>>> Tom
> > >>>>>>> _______________________________________________
> > >>>>>>> NOTE: All exchanges posted to Unidata maintained email lists are
> > >>>>>>> recorded in the Unidata inquiry tracking system and made publicly
> > >>>>>>> available through the web. Users who post to any of the lists we
> > >>>>>>> maintain are reminded to remove any personal information that they
> > >>>>>>> do not want to be made public.
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> thredds mailing list
> > >>>>>>> thredds@xxxxxxxxxxxxxxxx
> > >>>>>>> For list information or to unsubscribe, visit:
> > >>>>>>> http://www.unidata.ucar.edu/mailing_lists/
> > >>>>> _______________________________________________
> > >>>>> NOTE: All exchanges posted to Unidata maintained email lists are
> > >>>>> recorded in the Unidata inquiry tracking system and made publicly
> > >>>>> available through the web. Users who post to any of the lists we
> > >>>>> maintain are reminded to remove any personal information that they
> > >>>>> do not want to be made public.
> > >>>>>
> > >>>>>
> > >>>>> thredds mailing list
> > >>>>> thredds@xxxxxxxxxxxxxxxx
> > >>>>> For list information or to unsubscribe, visit:
> > >>>>> http://www.unidata.ucar.edu/mailing_lists/
> > >>>>
> > >>>> _______________________________________________
> > >>>> NOTE: All exchanges posted to Unidata maintained email lists are
> > >>>> recorded in the Unidata inquiry tracking system and made publicly
> > >>>> available through the web. Users who post to any of the lists we
> > >>>> maintain are reminded to remove any personal information that they
> > >>>> do not want to be made public.
> > >>>>
> > >>>>
> > >>>> thredds mailing list
> > >>>> thredds@xxxxxxxxxxxxxxxx
> > >>>> For list information or to unsubscribe, visit:
> > >>>> http://www.unidata.ucar.edu/mailing_lists/
> > >>>
> > >>> _______________________________________________
> > >>> NOTE: All exchanges posted to Unidata maintained email lists are
> > >>> recorded in the Unidata inquiry tracking system and made publicly
> > >>> available through the web. Users who post to any of the lists we
> > >>> maintain are reminded to remove any personal information that they
> > >>> do not want to be made public.
> > >>>
> > >>>
> > >>> thredds mailing list
> > >>> thredds@xxxxxxxxxxxxxxxx
> > >>> For list information or to unsubscribe, visit:
> > >>> http://www.unidata.ucar.edu/mailing_lists/
> > >>> _______________________________________________
> > >>> NOTE: All exchanges posted to Unidata maintained email lists are
> > >>> recorded in the Unidata inquiry tracking system and made publicly
> > >>> available through the web. Users who post to any of the lists we
> > >>> maintain are reminded to remove any personal information that they
> > >>> do not want to be made public.
> > >>>
> > >>>
> > >>> thredds mailing list
> > >>> thredds@xxxxxxxxxxxxxxxx
> > >>> For list information or to unsubscribe, visit:
> > >>> http://www.unidata.ucar.edu/mailing_lists/
> > >>
> > >> = = =
> > >> Nathan Potter ndp at opendap.org
> > >> OPeNDAP, Inc. +1.541.231.3317
> > >>
> >
> > = = =
> > Nathan Potter ndp at opendap.org
> > OPeNDAP, Inc. +1.541.231.3317
> >
> > _______________________________________________
> > NOTE: All exchanges posted to Unidata maintained email lists are
> > recorded in the Unidata inquiry tracking system and made publicly
> > available through the web. Users who post to any of the lists we
> > maintain are reminded to remove any personal information that they
> > do not want to be made public.
> >
> >
> > thredds mailing list
> > thredds@xxxxxxxxxxxxxxxx
> > For list information or to unsubscribe, visit:
> > http://www.unidata.ucar.edu/mailing_lists/
>
> = = =
> Nathan Potter ndp at opendap.org
> OPeNDAP, Inc. +1.541.231.3317
>
= = =
Nathan Potter ndp at opendap.org
OPeNDAP, Inc. +1.541.231.3317
