[netcdf-java] NcML and "values" support clarification?

[Nick Bower]

John Caron wrote:
> Hi Nick:
>
> Im not sure what your use case is. Are you generating the data values 
> yourself, or do they come out of a netcdf file?

Hi John.  3 cases (for when you're back);
1) Build NcML representations on the fly myself in a data ingest 
(producer), preferably using a NetCDF/NcML API.  This NcML would be 
serialized to String and distributed via JMS to consumers.
2) Flexibly parse NcML in data consumers (eg databases, monitoring) for 
pulling out attributes/values etc, again using an API.
3) Dumping NcML messages directly into NetCDF files that will form a 
master NetCDF store.

and full values support all the way through.

At the moment my messages comprise of Java Serializable simple bean-like 
classes, but this is not a good approach for cross-platform integration 
(ie working with .NET producers/consumers) and they're of course not 
self describing either.

>
> BTW, I will be on vacations until next week, You might want to look at 
> NcMLWriter.java in the meanwhile.

Ah yes I see NcMLWriter.writeXML / NcMLWriter.writeValues now - I had 
been looking at NetcdfFile itself.  2 questions;

NcML.writeXML, being the method that generates NcML as I understand, 
seems to want to read it's data via a NetcdfFile however I'd be wanting 
to create containers in memory.  Are temporary files essential?

NCMLReader looks more promising - you seem to be able to return an 
in-memory NetcdfDataset via readNcML(InputStream...).  Maybe this 
NetcdfDataset is the common API - will writeXML work if passing it a 
NetcdfDataset?

Hopefully I'm getting across that "files" are really just one of many 
consumer end-points in my scenario and not something I'm trying to focus 
my data around.  More important is the in-memory container classes and 
the ability to conveniently convert these to/from NcML (with values 
support).


>
>
> Nick Bower wrote:
>> I want to verify the api support for variable values in NcML.
>>
>> Some background - we're looking at NcML as a potential cross-platform 
>> format to exchange realtime data via Java/.Net messaging.  We already 
>> use NetCDF extensively in our organisation where we've conventionally 
>> collected long periods of data at a single place, but the idea is 
>> that we want to explore a more modern distributed realtime messaging 
>> architecture where data volume in any single message is low, but 
>> delivery frequency and schema variability from different sources is 
>> high.  In this way NcML could be used to represent measurement 
>> datums, say one NcML message specifying a measurement timestamp from 
>> a remote logging station contain the measurements of several 
>> instruments.  NcML also has the benefit of 1) facilitating downstream 
>> aggregation by message consumers directly to NetCDF, 2) GML 
>> extension.  We're using Mule ESB (Java) to integrate this btw.
>>
>> My question regards the support for variable "values".  Support 
>> exists in the xsd, but appears totally lacking in both the java api 
>> and ncdump/ncgen utilities.  Both the api javadoc and utilities man 
>> pages explicity say that any generated NcML will not include values.
>>
>> Can someone please clarify what is the status of supporting variable 
>> values in NcML, and what is the roadmap for api inclusion?
>>
>> Thanks, Nick
>>