Re: [netcdf-java] NcML and "values" support clarification?


John Caron wrote:
Hi Nick:

Im not sure what your use case is. Are you generating the data values yourself, or do they come out of a netcdf file?

Hi John.  3 cases (for when you're back);
1) Build NcML representations on the fly myself in a data ingest (producer), preferably using a NetCDF/NcML API. This NcML would be serialized to String and distributed via JMS to consumers. 2) Flexibly parse NcML in data consumers (eg databases, monitoring) for pulling out attributes/values etc, again using an API. 3) Dumping NcML messages directly into NetCDF files that will form a master NetCDF store.

and full values support all the way through.

At the moment my messages comprise of Java Serializable simple bean-like classes, but this is not a good approach for cross-platform integration (ie working with .NET producers/consumers) and they're of course not self describing either.


BTW, I will be on vacations until next week, You might want to look at NcMLWriter.java in the meanwhile.

Ah yes I see NcMLWriter.writeXML / NcMLWriter.writeValues now - I had been looking at NetcdfFile itself. 2 questions;

NcML.writeXML, being the method that generates NcML as I understand, seems to want to read it's data via a NetcdfFile however I'd be wanting to create containers in memory. Are temporary files essential?

NCMLReader looks more promising - you seem to be able to return an in-memory NetcdfDataset via readNcML(InputStream...). Maybe this NetcdfDataset is the common API - will writeXML work if passing it a NetcdfDataset?

Hopefully I'm getting across that "files" are really just one of many consumer end-points in my scenario and not something I'm trying to focus my data around. More important is the in-memory container classes and the ability to conveniently convert these to/from NcML (with values support).




Nick Bower wrote:
I want to verify the api support for variable values in NcML.

Some background - we're looking at NcML as a potential cross-platform format to exchange realtime data via Java/.Net messaging. We already use NetCDF extensively in our organisation where we've conventionally collected long periods of data at a single place, but the idea is that we want to explore a more modern distributed realtime messaging architecture where data volume in any single message is low, but delivery frequency and schema variability from different sources is high. In this way NcML could be used to represent measurement datums, say one NcML message specifying a measurement timestamp from a remote logging station contain the measurements of several instruments. NcML also has the benefit of 1) facilitating downstream aggregation by message consumers directly to NetCDF, 2) GML extension. We're using Mule ESB (Java) to integrate this btw.

My question regards the support for variable "values". Support exists in the xsd, but appears totally lacking in both the java api and ncdump/ncgen utilities. Both the api javadoc and utilities man pages explicity say that any generated NcML will not include values.

Can someone please clarify what is the status of supporting variable values in NcML, and what is the roadmap for api inclusion?

Thanks, Nick