Re: [netcdf-java] NcML joinNew bug

• To: Christian Ward-Garrison <cwardgar@xxxxxxxx>, Java NetCDF <netcdf-java@xxxxxxxxxxxxxxxx>
• Subject: Re: [netcdf-java] NcML joinNew bug
• From: John Caron <caron@xxxxxxxxxxxxxxxx>
• Date: Wed, 31 Mar 2010 08:00:44 -0600

Hi Christian:

Actually I prefer to answer questions on this email group, as it allows others 
to comment also. The support-netcdf-java (aka esupport) is better to make sure 
questions dont get lost, however.

some responses below

Christian Ward-Garrison wrote:

Hi John,

The following is valid NcML according to ncml-2.2.xsd, but causes an exception in toolsUI

<netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2";>
<aggregation dimName="outer" type="joinNew">
<netcdf>
<dimension name="inner" length="4" />
<variable name="inner" shape="inner" type="int">
<values>1 2 3 4</values>
</variable>
</netcdf>
</aggregation>
</netcdf>

java.lang.NullPointerException

at ucar.nc2.ncml.AggregationOuterDimension$DatasetOuterDimension.<init>(AggregationOuterDimension.java:609) at ucar.nc2.ncml.AggregationOuterDimension.makeDataset(AggregationOuterDimension.java:555)

    at ucar.nc2.ncml.Aggregation.addExplicitDataset(Aggregation.java:194)
    at ucar.nc2.ncml.NcMLReader.readAgg(NcMLReader.java:1313)
    at ucar.nc2.ncml.NcMLReader.readNetcdf(NcMLReader.java:466)
    at ucar.nc2.ncml.NcMLReader.readNcML(NcMLReader.java:414)
    at ucar.nc2.ncml.NcMLReader.readNcML(NcMLReader.java:235)
    at ucar.nc2.ncml.NcMLReader.readNcML(NcMLReader.java:185)
    at ucar.nc2.dataset.NetcdfDataset.acquireNcml(NetcdfDataset.java:797)

at ucar.nc2.dataset.NetcdfDataset.openOrAcquireFile(NetcdfDataset.java:673)

    at ucar.nc2.dataset.NetcdfDataset.openFile(NetcdfDataset.java:567)
    at ucar.nc2.ui.ToolsUI$NCdumpPanel.run(ToolsUI.java:1378)
    at ucar.nc2.ui.ToolsUI$GetDataTask.run(ToolsUI.java:4264)
    at java.lang.Thread.run(Unknown Source)

NJ is trying to come up with coordinate values for the new dimension. There is no "coordValue" attribute for the nested dataset, so it looks for a "location" attribute, which also doesn't exist. Unfortunately, NJ doesn't anticipate this possibility, and a NullPointerException gets thrown as a result.

The resolution for this problem is to add a "coordValue" attribute, but the schema doesn't enforce that and the exception doesn't suggest that. Maybe it's not possible to modify the schema such that it requires a "coordValue" or "location" attribute for each dataset in a joinNew aggregation (I don't know a ton about what's possible with XSD), but it at least seems worthwhile to throw an exception with a more useful error message.

Yes, XML Schema is inadequate for complex constraints like that. For various 
reasons its difficult to get good error detection in parsing NcML, but ill try 
your example and see if I can improve it.

As you note, JoinNew aggregation requires a coordinate variable. One needs to 
read the docs at

 http://www.unidata.ucar.edu/software/netcdf/ncml/v2.2/Aggregation.html

to learn about these subtleties. It appears however, that the doc is not all 
that clear on this point of needing a coordinate variable. Ill try to improve 
that also.