Hi fedor:
Generally the caching is helpful, and sometimes necessary. For some reason the
call is very slow, you might check with the server admin to see why. We are
looking into some optimisations that will help.
thanks for reporting this problem.
fedor baart wrote:
> Hi,
>
> I looked a bit further into this. I found that the DODSNetcdfFile upon
> construction also started to cache some string and coordinate variables.
> That's why the GET
> /opendap/rijkswaterstaat/jarkus/transect.nc.dods?areaname,year HTTP/1.1
> request is done.
> The toolsui library is used in the matlab snctools library. This matlab
> library constructs the DODSNetcdfFile object at each variable or
> metadata request.
> I solved this temporary by making the setPreload method public and using
> this from matlab, but as it states in the code this is not intended for
> use other then debugging.
>
> There are possible ways to improve performance when using the
> DODSNetcdfFile and snctools:
>
> - remove preloading from constructor in DODSNetcdfFile
> This will decrease performance on applications which assume the
> DODSNetcdfFile to be able to cache.
> - option 1 and make preloading a decorator (comparable to bufferedReader
> in java.io <http://java.io>)
> new PreloadingFile(new DODSNetcdfFile(url))
> - use a caching or loading strategy, perhaps static:
> DODSNetcdfFile.setLoadingStrategy(new PreloadStrategy())
> - make preloading optional through an argument
> new DODSNetcdfFile(url, null, false)
> - make preloading setable
> DODSNetcdfFile.setPreloading(false)
> - cancel at the preloading step. This can now be done with a canceltask
> but I don't quite see how to use this.
> - make snctools use connection pooling so it wont open a new
> DODSNetcdfFile at each function call.
>
> Personaly I think caching should be optional to allow uses like snctools
> uses it now. The first three options may decrease performance for
> existing applications but seem to me most elegant. The fourth and fifth
> option don't require any change of existing code but this doesn't realy
> fit a "java style" of programming. The 6th I do not understand and the
> 7th leaves it up to the snctools.
>
> How do you look at this?
>
> Kind regards,
>
> Fedor Baart
>
>
>
>
>
>
>
> On Tue, Nov 11, 2008 at 9:42 PM, fedor baart <f.baart@xxxxxxxxx
> <mailto:f.baart@xxxxxxxxx>> wrote:
>
> Hi,
>
> I am working on a European project Micore for which I setup an
> opendap server. I am experiencing some performance issues.
>
> I tried using the toolsUI-2.2.22.jar with mexnc and matlab to access
> data and noticed that for every request it takes about 10 seconds to
> call a function.
>
> Also when I used the toolsUI as a ui and tried using the ncdump
> function it took about 10 seconds for it to return.
>
> Using wireshark I stored the connection information which contains
> the following information:
>
> after 0.2 seconds:
> GET /opendap/rijkswaterstaat/jarkus/transect.nc.dds HTTP/1.1
> returns after 0.21 seconds with HTTP OK and dimensions
> after 0.22 seconds:
> GET /opendap/rijkswaterstaat/jarkus/transect.nc.das HTTP/1.1
> returns after 0.23 seconds with HTTP OK and attributes
> after 0.24 seconds:
> GET /opendap/rijkswaterstaat/jarkus/transect.nc.dods?areaname,year
> HTTP/1.1
> after 10 seconds returns with HTTP OK and a lot of data
>
> So what happens:
> 0.25 seconds is spend enquiring the information needed for the
> ncdump output.
> 10 seconds is spend on reading some variables, information which is
> not needed
>
> The site I tried this with is
>
> http://micore.wldelft.nl/opendap/rijkswaterstaat/jarkus/transect.nc.html
>
> I also found the same behaviour though with different variables on
> another site I tried:
>
> http://penguin.sfos.uaf.edu/opendap/FNMOC/sst/nc/200810/sst_2008102406.nc.bz2.html
>
> toolsUI-4.0 gives the same results
>
> Is anyone experiencing similar behaviour?
> Can I assume this to be an error?
> If it is an error in which source code (version 2.2.22 or 4.0 or
> svn?) should it be fixed?
> Can anyone suggest a way to speed up my queries to the opendap
> server if this is expected behaviour?
>
> Thanks,
>
> Fedor Baart
>
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> netcdf-java mailing list
> netcdf-java@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe, visit:
> http://www.unidata.ucar.edu/mailing_lists/
