[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[netCDFJava #MQO-415619]: Efficiently serializing NetCDF in memory objects

It wasn't really intended as advice. I honestly do not know
what is the right way to expose the inmemory stuff in netcdf-Java.
BTW, the Nc4Iosp can be turned on in ways other than "useForReading".
It is possible to directly insert Nc4Iosp into the Iosp search list.
But since it is a list, just because it is in the IOSP search list
does not mean it will be used if there is some overriding IOSP
earlier in the search list.


> 
> Thanks for your remarks and advices.
> 
> First, I would like to remind something. Thredds API uses a configuration 
> which can be overwriting by using class "RuntimeConfigParser" component. In 
> this configuration, there is a property specific section for 
> "Netcdf4Clibrary". Among these properties, there is "useForReading".
> 
> useForReading: By default, the native library is only used for writing 
> NetCDF-4 files; a pure-Java layer is
> responsible for reading them. However, if this property is set to true, then 
> it will be used for reading
> NetCDF-4 (and HDF5) files as well.
> 
> The second point is that existing NetcdfFile memory-based open based on using 
> an inmemory RAF only works with pure-Java layer.
> Because if you use Nc4Iosp, the inmemory RAF is closed in open function of 
> service.
> 
> Currently, Nc4Iosp is only used for netcdf4 file or if you force service 
> version in open function.
> 
> Thus, my proposal was :
> 1/ To make current open inmemory function working with Nc4Iosp service. If 
> you do not use Nc4Iosp service, the file is opened as classical open inmemory.
> 2/ Add openDiskless functions to expose Diskless capability. If you do not 
> use Nc4Iosp service, the file is opened as classical open.
> 3/ Add openExistingInMemory functions to expose InMemory capability in write 
> mode. If you do not use Nc4Iosp service, the file is opened with classical 
> open.
> 4/ Add createNewInMemory functions to expose InMemory capability in write 
> mode. If you do not use Nc4Iosp service, the file is created with classical 
> create.
> 5/ Add createNewDiskless functions to expose Diskless capability in write 
> mode. If you do not use Nc4Iosp service, the file is created with classical 
> create.
> 
> I am not really sure to understand what you really mean when you say :
> " Exporting a similar API specifically for netcdf-3/4 files seems to me like 
> a poor design decision."
> Because it is not what I tried to do. What let you think that?
> 
> I had class diagram to let you see functions changes, and the mark down.
> 
> Do you have news about the issue on netcdf-c rise in my last mail?
> 
> If you have any questions or remarks, do not hesitate.
> 
> Thanks
> Regards
> 
> LANDIECH Matthieu
> 
> 
> 
> -----Message d'origine-----
> De : Unidata netCDF Java Support <address@hidden>
> Envoyé : mardi 19 juin 2018 22:07
> À : address@hidden
> Cc : address@hidden; address@hidden; address@hidden; address@hidden; 
> address@hidden
> Objet : [netCDFJava #MQO-415619]: Efficiently serializing NetCDF in memory 
> objects
> 
> I had a chance to revisit this issue and have been thinking about how to 
> expose it in netdf-Java.
> 
> As you did, the obvious place to expose the inmemory functionality thru 
> NetcdfFile.java. The problems is that NetcdfFile already as a memory-based 
> open based on using an inmemory RAF file.
> Exporting a similar API specifically for netcdf-3/4 files seems to me like a 
> poor design decision.
> 
> One possibility is to have the NetcdfFile inmemory API check to see if the 
> file is being opened as a netcdf-3/4 file and then try to use the Nc4Iosp 
> class to bypass the inmemory RAF and use the netcdf-c functionality directly. 
> This seems a bit of a hack.
> 
> Your given solution may turn out to be the best, even tho IMO it kind of 
> compilicates the NetcdfFile interface.
> 
> It might help if you would provide some design notes on why you chose the 
> solution you did.
> 
> 
> 
> 
> 
> 
> > I push all my modification on "inmemory_2" branch on
> >
> > https://github.com/mlandiech/thredds.git
> >
> >
> >
> > You can find new capabilities test in
> >
> > cdm-test/src/test/java/ucar/nc2/jni/netcdf/TestNc4IospInMemoryDiskless
> > .java
> >
> >
> >
> > Regards,
> >
> >
> >
> > -------------------------------------------
> >
> > LANDIECH Matthieu
> >
> > Tel : +33(0)5 62 88 75 64
> >
> >
> >
> >
> >
> >
> >
> 
> =Dennis Heimbigner
> Unidata
> 
> 
> Ticket Details
> ===================
> Ticket ID: MQO-415619
> Department: Support netCDF Java
> Priority: Critical
> Status: Open
> ===================
> NOTE: All email exchanges with Unidata User Support are recorded in the 
> Unidata inquiry tracking system and then made publicly available through the 
> web.  If you do not want to have your interactions made available in this 
> way, you must let us know in each email you send to us.
> 
> 
> 
> 

=Dennis Heimbigner
  Unidata


Ticket Details
===================
Ticket ID: MQO-415619
Department: Support netCDF Java
Priority: Critical
Status: Open
===================
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata 
inquiry tracking system and then made publicly available through the web.  If 
you do not want to have your interactions made available in this way, you must 
let us know in each email you send to us.