Use of http proxy?

[John Caron]

Hi Nick:

DConnect2 is my (NetCDF-Java library) version of the OPenDAP Java client HTTP handler, that uses http commons library, so we should work together to make this work.

It sounds like we need to add proxy support by configuring the http commons library. I havent used an http proxy before, so I will need your help. If you have a clear idea of how to proceed, let me know.

John

Nick Bower wrote:
> Bummer.  Found the real exception and the source - DODS libraries are 
> using the commons http client, which requires explicit proxy setting at 
> an application level (ie proxies must be a supported feature of DAP).  
> But even when skirting DNS availability using hosts file, we go from this,
> 
> java.net.UnknownHostException: www.marine.csiro.au
>        at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:177)
>        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
>        at java.net.Socket.connect(Socket.java:519)
>        at java.net.Socket.connect(Socket.java:469)
>        at java.net.Socket.<init>(Socket.java:366)
>        at java.net.Socket.<init>(Socket.java:239)
>        at 
> org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:79) 
> 
>        at 
> org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:121) 
> 
>        at 
> org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:706)
>        at 
> org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1321) 
> 
>        at 
> org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:386) 
> 
>        at 
> org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:170) 
> 
>        at 
> org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:396)
>        at 
> org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:324)
>        at opendap.dap.DConnect2.openConnection(DConnect2.java:218)
> 
> to this,
> 
> C:\Program Files\IDV_2.2>jre\bin\java -Dhttp.proxyHost=proxyhost 
> -Dhttp.proxyPor
> t=3128 -Xmx512m -jar idv.jar
> 25/05/2007 14:52:15 org.apache.commons.httpclient.HttpMethodDirector 
> executeWith
> Retry
> INFO: I/O exception (java.net.ConnectException) caught when processing 
> request:
> Connection timed out: connect
> 25/05/2007 14:52:15 org.apache.commons.httpclient.HttpMethodDirector 
> executeWith
> Retry
> INFO: Retrying request
> 25/05/2007 14:53:00 org.apache.commons.httpclient.HttpMethodDirector 
> executeWith
> Retry
> INFO: I/O exception (java.net.ConnectException) caught when processing 
> request:
> Connection timed out: connect
> 
> Looking at the source below, it would indeed seem DAP does not support 
> proxies.  This is hard to believe - it's more effective than any EULA 
> for curtailing commercial use.  I'll take my grumbles to the DAP group I 
> guess... :(
> 
> opendap.dap.DConnect2.java:
> 
>  private void openConnection(String urlString, Command command) throws 
> IOException, DAP2Exception, ParseException {
> 
>    initHttpClient();
>    GetMethod method = new GetMethod(urlString);
>    method.setFollowRedirects(true);
> 
>    if (acceptCompress)
>      method.setRequestHeader(new Header("Accept-Encoding", 
> "deflate,gzip"));
> 
>    // enable sessions
>    if (allowSessions)
>      method.setRequestHeader(new Header("X-Accept-Session", "true"));
> 
>    InputStream is = null;
>    try {
>      // Execute the method.
> 
>      int statusCode = _httpClient.executeMethod(method);
>      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> 
> 
> 
> 
> 
> Nick Bower wrote:
>> IDV seems to be having trouble with our http proxy (squid).  I've set 
>> the following global parameters in runIDV.bat in the usual way;
>>
>> -Dhttp.proxyHost=proxyhost -Dhttp.proxyPort=3128
>>
>> however IDV fails to retrieve any data in the Data Chooser.  See 
>> traceback below.  Could it have something to do with IDV insisting 
>> DODS URI's begin with dods:// perhaps?  What is the recommended 
>> solution for using a http proxy?
>>
>> By the way, I can navigate the server in the "Display" / web browsing 
>> tab.  It's only the Data Chooser that fails.
>>
>> Thanks, Nick
>>
>> An error has occurred:
>> There was an error loading the data:
>> Error creating data source:dods.grid with: 
>> http://www.marine.csiro.au/dods/nph-dods/dods-data/bl/SPINUP4/temp/
>> Grid data source failed making data set: 
>> dods://www.marine.csiro.au/dods/nph-dods/dods-data/bl/SPINUP4/temp/
>> Cant open 
>> dods://www.marine.csiro.au/dods/nph-dods/dods-data/bl/SPINUP4/temp/
>> or as DODS 
>> dods://www.marine.csiro.au/dods/nph-dods/dods-data/bl/SPINUP4/temp/
>> www.marine.csiro.au
>>
>> java.io.FileNotFoundException: Cant open 
>> dods://www.marine.csiro.au/dods/nph-dods/dods-data/bl/SPINUP4/temp/ or 
>> as DODS 
>> dods://www.marine.csiro.au/dods/nph-dods/dods-data/bl/SPINUP4/temp/
>> www.marine.csiro.au
>>    at ucar.nc2.dataset.NetcdfDataset.openDODS(NetcdfDataset.java:380)
>>    at ucar.nc2.dataset.NetcdfDataset.openFile(NetcdfDataset.java:316)
>>    at ucar.nc2.dataset.NetcdfDataset.openDataset(NetcdfDataset.java:175)
>>    at 
>> ucar.nc2.dataset.NetcdfDatasetCache.acquire(NetcdfDatasetCache.java:203)
>>    at 
>> ucar.nc2.dataset.NetcdfDatasetCache.acquire(NetcdfDatasetCache.java:183)
>>    at 
>> ucar.nc2.dataset.NetcdfDatasetCache.acquire(NetcdfDatasetCache.java:171)
>>    at ucar.nc2.dt.grid.GridDataset.open(GridDataset.java:71)
>>    at 
>> ucar.unidata.data.grid.GeoGridDataSource.doMakeDataSet(GeoGridDataSource.java:726) 
>>
>>    at 
>> ucar.unidata.data.grid.GeoGridDataSource.getDataset(GeoGridDataSource.java:743) 
>>
>>    at 
>> ucar.unidata.data.grid.GeoGridDataSource.initAfterCreation(GeoGridDataSource.java:335) 
>>
>>    at 
>> ucar.unidata.data.DataManager.createDataSource(DataManager.java:1186)
>>    at 
>> ucar.unidata.idv.IntegratedDataViewer.createDataSource(IntegratedDataViewer.java:1698) 
>>
>>    at 
>> ucar.unidata.idv.IntegratedDataViewer.makeDataSource(IntegratedDataViewer.java:1617) 
>>
>>    at 
>> ucar.unidata.idv.IntegratedDataViewer.makeDataSource(IntegratedDataViewer.java:1595) 
>>
>>    at 
>> ucar.unidata.idv.IntegratedDataViewer.handleAction(IntegratedDataViewer.java:1446) 
>>
>>    at ucar.unidata.idv.DefaultIdv.handleAction(DefaultIdv.java:112)
>>    at 
>> ucar.unidata.idv.IntegratedDataViewer.handleAction(IntegratedDataViewer.java:1378) 
>>
>>    at 
>> ucar.unidata.idv.chooser.UrlChooser.loadURLInner(UrlChooser.java:250)
>>    at ucar.unidata.idv.chooser.UrlChooser.loadURL(UrlChooser.java:222)
>>    at 
>> ucar.unidata.idv.chooser.UrlChooser.doLoadInThread(UrlChooser.java:269)
>>    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>    at 
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) 
>>
>>    at 
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) 
>>
>>    at java.lang.reflect.Method.invoke(Method.java:585)
>>    at ucar.unidata.util.Misc$2.run(Misc.java:919)
>>    at ucar.unidata.util.Misc$3.run(Misc.java:947)
>>
>>
>> And another example from your own servers...
>>
>> java.io.FileNotFoundException: Cant open 
>> dods://motherlode.ucar.edu:8080/thredds/dodsC/modelsNc/NCEP/NAM/CONUS_80km/NAM_CONUS_80km_20070524_1200.nc 
>> or as DODS 
>> dods://motherlode.ucar.edu:8080/thredds/dodsC/modelsNc/NCEP/NAM/CONUS_80km/NAM_CONUS_80km_20070524_1200.nc 
>>
>> motherlode.ucar.edu
>>    at ucar.nc2.dataset.NetcdfDataset.openDODS(NetcdfDataset.java:380)
>>    at ucar.nc2.dataset.NetcdfDataset.openFile(NetcdfDataset.java:316)
>>    at ucar.nc2.dataset.NetcdfDataset.openDataset(NetcdfDataset.java:175)
>>    at 
>> ucar.nc2.dataset.NetcdfDatasetCache.acquire(NetcdfDatasetCache.java:203)
>>    at 
>> ucar.nc2.dataset.NetcdfDatasetCache.acquire(NetcdfDatasetCache.java:183)
>>    at 
>> ucar.nc2.dataset.NetcdfDatasetCache.acquire(NetcdfDatasetCache.java:171)
>>    at ucar.nc2.dt.grid.GridDataset.open(GridDataset.java:71)
>>    at 
>> ucar.unidata.data.grid.GeoGridDataSource.doMakeDataSet(GeoGridDataSource.java:726) 
>>
>>    at 
>> ucar.unidata.data.grid.GeoGridDataSource.getDataset(GeoGridDataSource.java:743) 
>>
>>    at 
>> ucar.unidata.data.grid.GeoGridDataSource.initAfterCreation(GeoGridDataSource.java:335) 
>>
>>    at 
>> ucar.unidata.data.DataManager.createDataSource(DataManager.java:1186)
>>    at 
>> ucar.unidata.idv.IntegratedDataViewer.createDataSource(IntegratedDataViewer.java:1698) 
>>
>>    at 
>> ucar.unidata.idv.IntegratedDataViewer.makeDataSource(IntegratedDataViewer.java:1617) 
>>
>>    at 
>> ucar.unidata.idv.IntegratedDataViewer.makeDataSource(IntegratedDataViewer.java:1595) 
>>
>>    at 
>> ucar.unidata.idv.IntegratedDataViewer.makeDataSource(IntegratedDataViewer.java:1576) 
>>
>>    at 
>> ucar.unidata.idv.chooser.IdvChooser.makeDataSource(IdvChooser.java:610)
>>    at 
>> ucar.unidata.idv.chooser.ThreddsHandler.processNodes(ThreddsHandler.java:842) 
>>
>>    at 
>> ucar.unidata.idv.chooser.ThreddsHandler.processNode(ThreddsHandler.java:782) 
>>
>>    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>    at 
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) 
>>
>>    at 
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) 
>>
>>    at java.lang.reflect.Method.invoke(Method.java:585)
>>    at ucar.unidata.util.Misc$2.run(Misc.java:921)
>>    at ucar.unidata.util.Misc$3.run(Misc.java:947)
>>
>>
>>
>>
> 
> =============================================================================== 
> 
> To unsubscribe idvusers, visit:
> http://www.unidata.ucar.edu/mailing-list-delete-form.html
> =============================================================================== 
>