Due to the current gap in continued funding from the U.S. National Science Foundation (NSF), the NSF Unidata Program Center has temporarily paused most operations. See NSF Unidata Pause in Most Operations for details.
Due to the current gap in continued funding from the U.S. National Science Foundation (NSF), the NSF Unidata Program Center has temporarily paused most operations. See NSF Unidata Pause in Most Operations for details.
Hi Phoebe: > > Hi, I am working on a distributed data system using your netcdf Java/rmi > API and find myself unclear about the efficiency issue involved. I > understand for local netcdf files a buffer is used in > ucar.netcdf.RandomAccessFile to speed IO operations and probably the same > mechanism used on the RMI server side. Is there any buffering on the > client side or each getXXX/putXXX call has to involve a round trip over > the network? Would copyIn/Out be more efficient for remote access in this > scenario? Any information would be very appreciated. > > --Best regards > > --Phoebe Zhang The getXXX/putXXX calls are horrendously slow for both remote and local access; you _must_ use copyin/copyout for efficient access. We have a different, prototype netcdf API that we are experimenting with that makes this clearer, since it is a common mistake to use the getXXX/setXXX methods. However we have not yet incorporated the RMI part into the new API. Note that you must be running a server to use RemoteNetcdf. We are also considering other ways to provide remote netcdf files, eg using a web server instead of an RMI server. There is currently no client-side buffering. For large files, it can be hard to decide how to do that: in memory or construct local file caches? Keep the local file cache or delete when the remote file is closed? _From your questions, I assume you are interested in efficiency. The overhead of the RMI is probably small compared to network latency for small amounts of data transfered (using copyin), not so good for large data transfer. We will also be evaluating new, more efficient Java I/O methods that are currently under development at Sun. We expect that these will eliminate the current I/O efficiencies and that Java will be as fast as C for local access. Those are a year or two off, however. We'd like to hear how you want to use remote netcdf files, how big they are, average data transfer size, latency needs, etc. We are considering possible new development and your input would be useful to us.
netcdf-java archives: