[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[THREDDS #JRP-282661]: Fwd: WMS in THREDDS 5

Hi Alan,

Interesting. Do you see any stack traces in the thredds or tomcat log files?
I am wondering if there is an error in the TDS or netCDF-Java when it is trying
to assemble the aggregation.

Sean

> Sean,
> 
> Thanks for the response.
> 
> I've raised your main point with others in ESGF, so we'll see what they say.
> 
> Regarding the 404, the ESGF THREDDS server doesn't have any particular
> special access to the data files.  The files are group-readable by a
> group that the tomcat user is a member of in /etc/group (as a
> supplementary group), and jsvc with the "-user" flag seems to honour
> this in setting the process group list.  Anyway, I copied some files
> to the tomcat home directory (which didn't yet exist, so I created it
> and its contents, all with owner tomcat).  I then changed the path in
> the XML.  And, using the vanilla THREDDS and after restarting tomcat,
> I am still getting the 404 here.  I attach the current state of
> catalog.xml.  Not sure what I am doing wrong.  I attached an "strace
> -f" to the process, and see that it is making no attempt to access the
> files when I click the opendap link.  The stuff under the
> <datasetScan> is all fine.
> 
> Thanks again.
> 
> Regards,
> Alan
> 
> On 13 October 2017 at 18:59, Unidata THREDDS Support
> <address@hidden> wrote:
> > Greetings Alan,
> >
> > So the main reason the WMS service isn't working in ESGF is because
> > the web xml is missing all of the necessary information to map the
> > wms  request urls to the wms code:
> >
> > <!-- edal-wms servlet -->
> >
> >   <servlet>
> >     <servlet-name>wms</servlet-name>
> >     <servlet-class>thredds.server.wms.ThreddsWmsServlet</servlet-class>
> >     <load-on-startup>2</load-on-startup>
> >   </servlet>
> >
> >   <servlet-mapping>
> >     <servlet-name>wms</servlet-name>
> >     <url-pattern>/wms/*</url-pattern>
> >   </servlet-mapping>
> >
> >   <filter>
> >     <filter-name>RequestQueryFilter</filter-name>
> >     <filter-class>thredds.servlet.filter.RequestQueryFilter</filter-class>
> >   </filter>
> >
> >   <filter-mapping>
> >     <filter-name>RequestQueryFilter</filter-name>
> >     <servlet-name>wms</servlet-name>
> >   </filter-mapping>
> >
> >   <!-- end edal-wms servlet -->
> >
> > You can try adding the xml snippet above to the web.xml file shipped
> > with the ESGF release, but it may take more work than that. I'd
> > highly recommend contacting them to see why they removed it and
> > to see what it would take to add it back (there are likely security
> > issues related to this).
> >
> > As for the 404, this one is a bit more tricky. Is it possible that the ESGF
> > TDS has special credentials to access the data off of disk? One way to test
> > would be to take three or so files from the directory:
> >
> > /neodc/esacci/cloud/data/phase-2/L3C/AVHRR-PM/v2.0
> >
> > and put them in a temporary directory in the tomcat users home directory.
> > Then, update catalog.xml and use the new path in the location attribute of 
> > the
> > scan element in the ncml aggregation. Be sure to use the vanilla TDS.
> >
> > If that does not work, is there a way I could get three or so of the 
> > AVHRR-PM/v2.0
> > files you are using so that I can debug locally?
> >
> > Cheers,
> >
> > Sean
> >
> >> Sean,
> >>
> >> Okay, I decided to look at the AggTest now.  I am getting a 404 with
> >> the opendap.  The WMS gives a java.io.FileNotFoundException.  But the
> >> scan location does exist (and I can access it okay when I "su" to the
> >> tomcat user).  What should I try now please?
> >>
> >> Thanks,
> >> Alan
> >>
> >> On 13 October 2017 at 18:31, Alan Iwi (STFC) <address@hidden> wrote:
> >> > Sean,
> >> >
> >> > Here's the web.xml to quickly answer your point 2.  I'll look at your
> >> > point 1 also, but it might be next week.
> >> >
> >> > Regards,
> >> > Alan
> >> >
> >> > On 13 October 2017 at 18:12, Unidata THREDDS Support
> >> > <address@hidden> wrote:
> >> >> Oops...here is the attachment.
> >> >>
> >> >> Sean
> >> >>
> >> >>> Greetings Alan,
> >> >>>
> >> >>> The reason I had you try with the vanilla TDS war and and simple 
> >> >>> example
> >> >>> was to see if we could even access WMS on your server, or if there was
> >> >>> a potential issue outside of the TDS (maybe Tomcat, Apache, etc). Given
> >> >>> that the simple test works, it looks like the issue is related to a 
> >> >>> difference
> >> >>> in the esgf bundled TDS.
> >> >>>
> >> >>> There are two things I'd like to try now.
> >> >>>
> >> >>> 1) Try the aggregation with the vanilla TDS by adding the following
> >> >>> to the main catalog.xml (the one with the simple datasetScan that 
> >> >>> works):
> >> >>>
> >> >>> <dataset ID="Aggtest"
> >> >>> name="AggTest"
> >> >>> urlPath="AggTest">
> >> >>>
> >> >>> <metadata>
> >> >>> <dataType>GRID</dataType>
> >> >>> </metadata>
> >> >>>
> >> >>> <netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2";>
> >> >>> <aggregation dimName="time" type="joinExisting">
> >> >>> <variableAgg name="cee"/>
> >> >>> <variableAgg name="cee_corr_unc"/>
> >> >>> <scan location="/neodc/esacci/cloud/data/phase-2/L3C/AVHRR-PM/v2.0" 
> >> >>> suffix=".nc"/>
> >> >>> </aggregation>
> >> >>> </netcdf>
> >> >>> </dataset>
> >> >>>
> >> >>> between the </datasetScan> and </catalog> entries at the end of the 
> >> >>> catalog.xml file.
> >> >>> Note I've attached an updated copy of catalog.xml with the changes in 
> >> >>> case the xml
> >> >>> snippets above do not show up in this message. Check to see if things 
> >> >>> still work in the first
> >> >>> dataset, and then see if they work with your aggregation. Try hitting 
> >> >>> the opendap link
> >> >>> of your aggregation dataset first to make sure it does not return a 
> >> >>> 404 or 500.
> >> >>>
> >> >>> 2) Can you send me the thredds/WEB-INF/web.xml file from the webapps 
> >> >>> directory
> >> >>> included with the ESGF distribution?
> >> >>>
> >> >>> Thanks!
> >> >>>
> >> >>> Sean
> >> >>>
> >> >>> > Changing the address to the support address as Sean requested.]
> >> >>> >
> >> >>> > I have tried with the a top-level catalogue containing both the "test
> >> >>> > all files in a directory" <datasetScan> element and also the "Earth
> >> >>> > System Root" <catalogRef> element containing files published by the
> >> >>> > ESGF publisher.  I have tried this with both the vanilla thredds.war
> >> >>> > file you supplied and also with the thredds directory from ESGF.
> >> >>> >
> >> >>> > The situation now seems to be:
> >> >>> >
> >> >>> > 1) if I try with the vanilla thredds.war file you supplied, then the
> >> >>> > "test all files in a directory" link in your test top-level catalogue
> >> >>> > seem to work, including the WMS links, but I have two problems:
> >> >>> >
> >> >>> > (a) it is a very different example from the one which I was trying to
> >> >>> > get working, and I do not know how to adapt it -- namely, it has a
> >> >>> > <datasetScan> which finds a bunch of datasets consisting of one file
> >> >>> > each, whereas I want to serve a single dataset but that dataset is to
> >> >>> > consist of an aggregation of netCDF files
> >> >>> >
> >> >>> > (b) with the vanilla thredds.war file, I do not have the changes made
> >> >>> > by ESGF, and if I try to use a catalogue produced by the ESGF
> >> >>> > publisher then it does not work, for example with the HTTPServer 
> >> >>> > links
> >> >>> > it redirects to /thredds/restrictedAccess/esg-user and then Chrome
> >> >>> > reports "ERR_INVALID_HTTP_RESPONSE", and if I insert WMS links then I
> >> >>> > get a NullPointerException.
> >> >>> >
> >> >>> > 2) If I reinstate the thredds/ directory supplied by ESGF (removing
> >> >>> > the vanilla thredds directory and thredds.war), then we are back 
> >> >>> > where
> >> >>> > we were before, namely that the datasets published by the ESGF
> >> >>> > publisher work as regards access methods other than wms (e.g.
> >> >>> > HTTPServer), but /thredds/wms and any URLs below it get a 404 error
> 
> 


Ticket Details
===================
Ticket ID: JRP-282661
Department: Support THREDDS
Priority: Normal
Status: Open
===================
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata 
inquiry tracking system and then made publicly available through the web.  If 
you do not want to have your interactions made available in this way, you must 
let us know in each email you send to us.