Hi,
I'm aggregating some gridded data with an ncml entry in our THREDDS
catalog.xml file. The entry looks like this:
<?xml version="1.0" encoding="UTF-8"?>
<catalog name="NGDC THREDDS Server Default Catalog"
xmlns="http://www.unidata.ucar.edu/namespaces/thredds/InvCatalog/v1.0";
xmlns:xlink="http://www.w3.org/1999/xlink";
version="1.0.2">
<service name="aggServices" serviceType="compound" base="">
<service name="dapService2" serviceType="OpenDAP"
base="/thredds/dodsC/" />
<service name="wcsService2" serviceType="WCS" base="/thredds/wcs/" />
<service name="wms2" serviceType="WMS" base="/thredds/wms/" />
</service>
<dataset name="USTEC STEC Aggregation" ID="STEC-Agg"
urlPath="stec-aggregation">
<metadata inherited="true">
<serviceName>aggServices</serviceName>
<dataType>Grid</dataType>
</metadata>
<netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2";>
<attribute name="Comment" value="Aggregation for USTEC STEC" />
<aggregation dimName="Time" type="joinExisting" recheckEvery="15 min">
<scan location="/nfs/ustec_netcdf/stec/latest7days"
suffix=".nc" />
</aggregation>
</netcdf>
</dataset>
</catalog>
Inside the netCDF files are grids from satellites:
time 1: satellite_01, satellite_02, satellite_03
time 2: satellite_01, satellite_02, satellite_03
time 3: satellite_01, satellite_03
time 4: satellite_03, satellite_02, satellite_01
and so on.
Sometimes data from a satellite is not present, as satellite_02 is
missing at time 3 at in the example above. This appears as an all white
image in the time sequence when viewing the satellite_02 result with
IDV. What is the best way to handle this. Is there a missing grid flag,
or should there be an empty image inserted, so that there is always a
satellite_02 grid? We are also getting an error from IDV
java.io.IOException: Inconsistent array length read: 1165128303 !=
1914731274
at ucar.nc2.dt.grid.GeoGrid.readDataSlice(GeoGrid.java:616)
at ucar.nc2.dt.grid.GeoGrid.readDataSlice(GeoGrid.java:524)
at ucar.nc2.dt.grid.GeoGrid.readVolumeData(GeoGrid.java:445)
at ucar.visad.data.GeoGridFlatField.readData(GeoGridFlatField.java:220)
at ucar.visad.data.CachedFlatField.getMyValues(CachedFlatField.java:479)
at ucar.visad.data.CachedFlatField.getMyValues(CachedFlatField.java:467)
at
ucar.visad.data.CachedFlatField.unpackFloats(CachedFlatField.java:614)
at visad.FlatField.getFloats(FlatField.java:1567)
at
visad.ShadowFunctionOrSetType.doTransform(ShadowFunctionOrSetType.java:1404)
at
visad.java3d.ShadowFunctionOrSetTypeJ3D.doTransform(ShadowFunctionOrSetTypeJ3D.java:195)
at
visad.java3d.ShadowFunctionOrSetTypeJ3D.recurseRange(ShadowFunctionOrSetTypeJ3D.java:1209)
at
visad.java3d.ShadowFunctionOrSetTypeJ3D$1.run(ShadowFunctionOrSetTypeJ3D.java:181)
at visad.util.ThreadManager$1.run(ThreadManager.java:292)
at visad.util.ThreadPool$ThreadMinnow.run(ThreadPool.java:98)
Do you think this error is due to the missing satellite_02 grids?
One more question: Also, as can be seen at time 4, the satellite grids
are not always in the same order in the netCDF file. Is this a problem?
I can have my colleague regenerate the data with grids in a consistent
order if needed.
If I have answers to the above, I can probably figure out the following
on my own.
If all satellites must be present, maybe we need to start a new
aggregation if a new satellite is added to the set or one is retired
from service?
Maybe we should be splitting out each satellite into their own netCDF files?
Thanks,
Ken
