Hello,
Yes, mod_jk does use AJP and is very much alive and well. It provides a
lot of configuration options (JkOptions) to fine tune the connection
between the httpd server and tomcat (if needed).
http://tomcat.apache.org/connectors-doc/webserver_howto/apache.html
Cheers,
Jen
On Fri, Jan 18, 2019 at 10:59 AM Sean Arms <sarms@xxxxxxxx> wrote:
> Let me bring in our Tomcat wrangler, Jen. I thought mod_jk used AJP?
>
> On Fri, Jan 18, 2019 at 10:48 AM Nathan Potter <ndp@xxxxxxxxxxx> wrote:
>
>> Sean,
>>
>> Interesting.
>>
>> I thought that mod_jk had been abandoned/superseded for/by the Apache
>> JServ Protocol (AJP) protocol with the introduction of Apache 2.x
>>
>>
>> N
>>
>>
>> > On Jan 18, 2019, at 9:21 AM, Sean Arms <sarms@xxxxxxxx> wrote:
>> >
>> > Hi Nathan, all,
>> >
>> > netCDF-Java does proper encoding as of 4.6.12 and 5.0.0-beta6 (runtime
>> configurable), and in the past it encoded as well - there was a period of
>> time in the 4.5 and 4.6 line of development in which the encoding was
>> skipped, but that's no longer the case.
>> >
>> > Interestingly enough, if you run Apache with mod_jk, non-encoded urls
>> will be accepted and passed along to tomcat in a usable form without issue,
>> but I have no idea how long that will last.
>> >
>> > Sean
>> >
>> > On Thu, Jan 17, 2019 at 4:25 PM Nathan Potter <ndp@xxxxxxxxxxx> wrote:
>> > Hi Thomas,
>> >
>> > I believe that I worked with Robert Schmunk (the author of Panoply) on
>> this very problem and I think the reason he is using
>> netcdfAll-5.0.0-SNAPSHOT is that it solves the outgoing encoding issue on
>> the client side. Is that right Robert? (We may not hear back from him as I
>> suspect he's furloughed and unavailable a.t.m.)
>> >
>> > If the client cannot be cajoled into producing the correct URL encoding
>> then I think the work falls back to relaxing the server's relaxedQueryChars
>> configuration.
>> >
>> > Which is certainly less than ideal.
>> >
>> > And to end on a cheery note - once the netcdf-java library is release
>> it can take some time (up to a year) before it gets rolled into a Matlab
>> release.
>> >
>> > Sorry I can't be more helpful.
>> >
>> >
>> > Sincerely,
>> >
>> > Nathan
>> >
>> >
>> > > On Jan 17, 2019, at 1:39 PM, Thomas Cook <tmcook@xxxxxxxx> wrote:
>> > >
>> > > OK Given Nathan and Christian's comments, I'm focussing my
>> > > troubleshooting on client side.
>> > > What I've learned
>> > > 1) Matlab's native netCDF toolbox depends on netCDF 4.3.3.1. I
>> > > couldn't find an easy way to update the netCDF version inside my
>> > > Matlab install.
>> > > 2) I updated the netcdf4-python module to use netCDF 4.6.12. This did
>> > > not solve the issue.
>> > > 3) Panoply (which works) is using netcdfAll-5.0.0-SNAPSHOT. I couldn't
>> > > find an older version to download, nor was I able to run panoply with
>> > > an older netcdfAll.jar
>> > > 4) Another user has reported success using the SNTOOLS/MEXNC package
>> > > to read the files. This also uses a version of netcdfAll.jar for its
>> > > decoding. I can't verify this yet.
>> > >
>> > > So not sure there's much more I can do with on the client side. I'll
>> > > mess with the Dockerfile like Christian suggested, and try using some
>> > > non FMRC collections to test with the clients. If anyone has any
>> > > ideas, I'm open!
>> > > Thanks to all who have replied and I'll update if I make any headway .
>> > > Tom
>> > >
>> > > On Thu, Jan 17, 2019 at 5:53 AM Nathan Potter <ndp@xxxxxxxxxxx>
>> wrote:
>> > >>
>> > >>
>> > >>
>> > >>> On Jan 17, 2019, at 5:37 AM, Christian Skarby <christians@xxxxxx>
>> wrote:
>> > >>>
>> > >>> For what it's worth; from what I've seen encoding errors would
>> result in a reject from tomcat before the request is passed to the TDS
>> application, and it would then probably not show up in the
>> threddsServlet.log.
>> > >>
>> > >> This is the behavior that we have seen.
>> > >>
>> > >> Tomcat's rejection is an empty document and response headers ala:
>> > >>
>> > >> HTTP/1.1 400 Bad Request
>> > >> Server: Apache-Coyote/1.1
>> > >> Transfer-Encoding: chunked
>> > >> Date: Thu, 17 Jan 2019 13:47:48 GMT
>> > >> Connection: close
>> > >>
>> > >>>
>> > >>> Perhaphs Tom, you could try to just change the application code in
>> your docker image (extract war or extracted thredds servlet from one
>> version and then inject it in the other). I guess this should be possible,
>> although I have not looked into the official docker images for thredds. If
>> you make this fly it would be possible to say if it is the servlet or the
>> other libraries in the docker image that causes you problems.
>> > >>>
>> > >>> Regarding the possible encoding errors: Tomcat introduced the more
>> strict behavior in v9.0.8 (and also v8.5.31, v8.0.52 and v7.0.87).
>> > >>>
>> > >>> See the corresponding tomcat changelogs:
>> > >>> • 62273: Implement configuration options to work-around
>> specification non-compliant user agents (including all the major browsers)
>> that do not correctly %nn encode URI paths and query strings as required by
>> RFC 7230 and RFC 3986. (markt)
>> > >>> See this thread in the google group for opendap.org:
>> > >>>
>> https://groups.google.com/a/opendap.org/forum/#!msg/support/ixTqhDXoLZQ/IT0lvZQ7CAAJ
>> > >>>
>> > >>> Tomcat-documentation:
>> > >>> https://tomcat.apache.org/tomcat-9.0-doc/config/http.html
>> > >>>
>> > >>> relaxedPathChars
>> > >>> The HTTP/1.1 specification requires that certain characters are %nn
>> encoded when used in URI paths. Unfortunately, many user agents including
>> all the major browsers are not compliant with this specification and use
>> these characters in unencoded form. To prevent Tomcat rejecting such
>> requests, this attribute may be used to specify the additional characters
>> to allow. If not specified, no addtional characters will be allowed. The
>> value may be any combination of the following characters: " < > [ \ ] ^ ` {
>> | } . Any other characters present in the value will be ignored.
>> > >>>
>> > >>> relaxedQueryChars
>> > >>> The HTTP/1.1 specification requires that certain characters are %nn
>> encoded when used in URI query strings. Unfortunately, many user agents
>> including all the major browsers are not compliant with this specification
>> and use these characters in unencoded form. To prevent Tomcat rejecting
>> such requests, this attribute may be used to specify the additional
>> characters to allow. If not specified, no addtional characters will be
>> allowed. The value may be any combination of the following characters: " <
>> > [ \ ] ^ ` { | } . Any other characters present in the value will be
>> ignored.
>> > >>>
>> > >>>
>> > >>> As the OPeNDAP spesification (see section 6.1.1.2 on page 22)
>> requires the use of [ and ] in the query string, and several clients
>> doesn't quote these characters, we need to tweak the relaxedQueryChars
>> configuration for tomcat.
>> > >>
>> > >> For completeness I will add that the DAP protocol also utilizes the
>> < and > characters for formulating relational constraints, however these
>> are limited to constraining Sequence data types. I don't know that the TDS
>> supports this data type so they may not be required in the
>> relaxedQueryChars list.
>> > >>
>> > >>
>> > >>> Regards,
>> > >>> Christian Skarby
>> > >>>
>> > >>> Den tor. 17. jan. 2019 kl. 05:42 skrev Thomas Cook <tmcook@xxxxxxxx
>> >:
>> > >>> I do not know how to debug the matlab or python clients, but this is
>> > >>> what shows up in the threddsServlet.log
>> > >>>
>> > >>> Request: "GET
>> /thredds/dodsC/HFR/USWC/6km/hourly/RTV/HFRADAR_US_West_Coast_6km_Resolution_Hourly_RTV_best.ncd.dods?lat,lon,site%5flat,site%5flon,site%5fcode,site%5fnetCode,procParams
>> > >>> HTTP/1.0"
>> > >>>
>> > >>> Thanks
>> > >>> Tom
>> > >>>
>> > >>> On Wed, Jan 16, 2019 at 7:47 PM Dennis Heimbigner <dmh@xxxxxxxx>
>> wrote:
>> > >>>>
>> > >>>> Is there anyway you can see the exact URL being sent to the server?
>> > >>>>
>> > >>>> On 1/16/2019 8:24 PM, Thomas Cook wrote:
>> > >>>>> Thanks Sean.
>> > >>>>> The server is http://hfrnet-tds.ucsd.edu/thredds (although I just
>> > >>>>> downgraded back to 4.6.12-SNAPSHOT). I will see about making my
>> test
>> > >>>>> server accessible to you, but that will take a little time to
>> > >>>>> configure on my end.
>> > >>>>> Let me know if you think I should open an issue on the github
>> page and
>> > >>>>> move the discussion there.
>> > >>>>> Tom
>> > >>>>>
>> > >>>>> On Wed, Jan 16, 2019 at 6:32 PM Sean Arms <sarms@xxxxxxxx> wrote:
>> > >>>>>> Hi Tom,
>> > >>>>>>
>> > >>>>>> So you are hitting up an opendap endpoint using all three
>> libraries / tools (python, matlab, and panoply)? Is this particular server
>> publicly accessible?
>> > >>>>>>
>> > >>>>>> I have a bad feeling that this is due to URL encoding (or lack
>> thereof) on the client side and changes made in tomcat 8.5, which it looks
>> like the thredds-docker repository bumped up to with the 4.6.12 release.
>> > >>>>>>
>> > >>>>>> Cheers,
>> > >>>>>>
>> > >>>>>> Sean
>> > >>>>>>
>> > >>>>>>
>> > >>>>>> On Wed, Jan 16, 2019 at 7:06 PM Thomas Cook <tmcook@xxxxxxxx>
>> wrote:
>> > >>>>>>> HI I just wanted to add some details.
>> > >>>>>>>
>> > >>>>>>> The functionality works when I downgraded to the docker
>> container
>> > >>>>>>> 4.6.12-SNAPSHOT
>> > >>>>>>>
>> > >>>>>>> My data is part of a FMRC collection.
>> > >>>>>>>
>> > >>>>>>> Thanks,
>> > >>>>>>> Tom
>> > >>>>>>>
>> > >>>>>>> On Wed, Jan 16, 2019 at 4:51 PM Thomas Cook <tmcook@xxxxxxxx>
>> wrote:
>> > >>>>>>>> HI
>> > >>>>>>>> I just yesterday updated my thredds-docker to 4.6.12 from
>> > >>>>>>>> 4.6.12-snapshot. It seems to be running fine, except I'm
>> getting
>> > >>>>>>>> errors using matlab ncread and python netcdf4.dataset. When I
>> try to
>> > >>>>>>>> access any of the multi-dimensional (time, lat, lon) variables
>> I get
>> > >>>>>>>> the following errors
>> > >>>>>>>> with matlab:
>> > >>>>>>>> Error using netcdflib
>> > >>>>>>>> The NetCDF library encountered an error during execution of
>> > >>>>>>>> 'getVarsShort' function - 'Access failure (-77)'.
>> > >>>>>>>>
>> > >>>>>>>> Error in netcdf.getVar (line 136)
>> > >>>>>>>> data = netcdflib(funcstr,ncid,varid,varargin{:});
>> > >>>>>>>>
>> > >>>>>>>> Error in internal.matlab.imagesci.nc/read (line 635)
>> > >>>>>>>> data = netcdf.getVar(gid, varid, ...
>> > >>>>>>>>
>> > >>>>>>>> Error in ncread (line 58)
>> > >>>>>>>> vardata = ncObj.read(varName, varargin{:});
>> > >>>>>>>>
>> > >>>>>>>> Error in readUVfromTDS (line 56)
>> > >>>>>>>> u=ncread(url, 'u_mean', [1 1 1], [2 2 2]);
>> > >>>>>>>>
>> > >>>>>>>>
>> > >>>>>>>> and with python:
>> > >>>>>>>> netCDF4/_netCDF4.pyx in netCDF4._netCDF4.Variable.__getitem__()
>> > >>>>>>>>
>> > >>>>>>>> netCDF4/_netCDF4.pyx in netCDF4._netCDF4.Variable._get()
>> > >>>>>>>>
>> > >>>>>>>> netCDF4/_netCDF4.pyx in netCDF4._netCDF4._ensure_nc_success()
>> > >>>>>>>>
>> > >>>>>>>> RuntimeError: NetCDF: Access failure
>> > >>>>>>>>
>> > >>>>>>>> I know that the latest docker contains netcdf 4.6, where as the
>> > >>>>>>>> snapshot had 4.4. Could it be a version problem with the
>> netCD4 python
>> > >>>>>>>> and matlab libraries? I should be running the latest netcdf
>> python
>> > >>>>>>>> version and not too far behind with matlab versions.
>> > >>>>>>>>
>> > >>>>>>>> The weird things are that the index variables are accessible
>> (time,
>> > >>>>>>>> lat, lon etc), and the multiple-dimension data is accessible
>> just
>> > >>>>>>>> fine using the OpenDap web interface and Panoply. So I'm a
>> little
>> > >>>>>>>> confused. I'm addressing this list since I think there might
>> be some
>> > >>>>>>>> matlab and python expertise here and I think there were some
>> messages
>> > >>>>>>>> about the Access failure error in the past.
>> > >>>>>>>>
>> > >>>>>>>> Thanks,
>> > >>>>>>>> Tom
>> > >>>>>>> _______________________________________________
>> > >>>>>>> NOTE: All exchanges posted to Unidata maintained email lists are
>> > >>>>>>> recorded in the Unidata inquiry tracking system and made
>> publicly
>> > >>>>>>> available through the web. Users who post to any of the lists
>> we
>> > >>>>>>> maintain are reminded to remove any personal information that
>> they
>> > >>>>>>> do not want to be made public.
>> > >>>>>>>
>> > >>>>>>>
>> > >>>>>>> thredds mailing list
>> > >>>>>>> thredds@xxxxxxxxxxxxxxxx
>> > >>>>>>> For list information or to unsubscribe, visit:
>> http://www.unidata.ucar.edu/mailing_lists/
>> > >>>>> _______________________________________________
>> > >>>>> NOTE: All exchanges posted to Unidata maintained email lists are
>> > >>>>> recorded in the Unidata inquiry tracking system and made publicly
>> > >>>>> available through the web. Users who post to any of the lists we
>> > >>>>> maintain are reminded to remove any personal information that they
>> > >>>>> do not want to be made public.
>> > >>>>>
>> > >>>>>
>> > >>>>> thredds mailing list
>> > >>>>> thredds@xxxxxxxxxxxxxxxx
>> > >>>>> For list information or to unsubscribe, visit:
>> http://www.unidata.ucar.edu/mailing_lists/
>> > >>>>
>> > >>>> _______________________________________________
>> > >>>> NOTE: All exchanges posted to Unidata maintained email lists are
>> > >>>> recorded in the Unidata inquiry tracking system and made publicly
>> > >>>> available through the web. Users who post to any of the lists we
>> > >>>> maintain are reminded to remove any personal information that they
>> > >>>> do not want to be made public.
>> > >>>>
>> > >>>>
>> > >>>> thredds mailing list
>> > >>>> thredds@xxxxxxxxxxxxxxxx
>> > >>>> For list information or to unsubscribe, visit:
>> http://www.unidata.ucar.edu/mailing_lists/
>> > >>>
>> > >>> _______________________________________________
>> > >>> NOTE: All exchanges posted to Unidata maintained email lists are
>> > >>> recorded in the Unidata inquiry tracking system and made publicly
>> > >>> available through the web. Users who post to any of the lists we
>> > >>> maintain are reminded to remove any personal information that they
>> > >>> do not want to be made public.
>> > >>>
>> > >>>
>> > >>> thredds mailing list
>> > >>> thredds@xxxxxxxxxxxxxxxx
>> > >>> For list information or to unsubscribe, visit:
>> http://www.unidata.ucar.edu/mailing_lists/
>> > >>> _______________________________________________
>> > >>> NOTE: All exchanges posted to Unidata maintained email lists are
>> > >>> recorded in the Unidata inquiry tracking system and made publicly
>> > >>> available through the web. Users who post to any of the lists we
>> > >>> maintain are reminded to remove any personal information that they
>> > >>> do not want to be made public.
>> > >>>
>> > >>>
>> > >>> thredds mailing list
>> > >>> thredds@xxxxxxxxxxxxxxxx
>> > >>> For list information or to unsubscribe, visit:
>> http://www.unidata.ucar.edu/mailing_lists/
>> > >>
>> > >> = = =
>> > >> Nathan Potter ndp at opendap.org
>> > >> OPeNDAP, Inc. +1.541.231.3317
>> > >>
>> >
>> > = = =
>> > Nathan Potter ndp at opendap.org
>> > OPeNDAP, Inc. +1.541.231.3317
>> >
>> > _______________________________________________
>> > NOTE: All exchanges posted to Unidata maintained email lists are
>> > recorded in the Unidata inquiry tracking system and made publicly
>> > available through the web. Users who post to any of the lists we
>> > maintain are reminded to remove any personal information that they
>> > do not want to be made public.
>> >
>> >
>> > thredds mailing list
>> > thredds@xxxxxxxxxxxxxxxx
>> > For list information or to unsubscribe, visit:
>> http://www.unidata.ucar.edu/mailing_lists/
>>
>> = = =
>> Nathan Potter ndp at opendap.org
>> OPeNDAP, Inc. +1.541.231.3317
>>
>>
