[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Error with re-hosted FMRC best time series



Hi Kevin:

Heres a summary of the situation so far:

1. I still have the missing values attribute issue on my TODO list for 4.2 Fmrc. 2. The "inconsistent array length read" on the client always indicates an error on the server. If you own that server, can you look in the threddsServlet.log files and find the corresponding entry and send to me? alternately, send me the entire log file for the time it happened. 3. it appears that you are wrapping an opendap dataset with the ncml. Is that the same TDS or a different server? 4. weve delayed the TDS release to get some WMS fixes in, ill check that you have the latest bug fixes. 5. I may have to get some files from you and reproduce the problem. Are those files available on the server?

Im out of town till Weds, so i probably cant do much more till Im back in the office. Ill check in then after ive had time to check stuff.

Kevin O'Brien wrote:
Hi John -

Sure -here's what it looks like. Sorry, the cut and past will likely wrap some of the lines...

Is the belw what you want?

thanks -
Kevin




<?xml version="1.0" encoding="UTF-8"?>
<catalog name="kob test server"
xmlns="http://www.unidata.ucar.edu/namespaces/thredds/InvCatalog/v1.0";
       xmlns:xlink="http://www.w3.org/1999/xlink";>

 <service name="all" base="" serviceType="compound">
   <service name="odap" serviceType="OpenDAP" base="/thredds/dodsC/" />
<service name="http" serviceType="HTTPServer" base="/thredds/fileServer/" />
   <!--service name="wcs" serviceType="WCS" base="/thredds/wcs/" /-->
   <service name="wms" serviceType="WMS" base="/thredds/wms/" />
<!--service name="ncss" serviceType="NetcdfSubset" base="/thredds/ncss/" /-->
 </service>

             <dataset name="NCOM Region 2 Aggregation">
<dataset name="NCOM Region 2 Best Time Series" ID="NCOM_reg_2_best_time_series" urlPath="NCOM_Region_2_Aggregation">
                 <serviceName>all</serviceName>
<netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2"; location="http://edac-dap.northerngulfinstitute.org/thredds/dodsC/ncom_reg2_agg/NCOM_Region_
2_Aggregation_best.ncd">
                  <variable name="water_temp">
<attribute name="_FillValue" type="float" value="NaN" /> <attribute name="missing_value" type="float" value="NaN" />
                  </variable>
                  <variable name="salinity">
<attribute name="_FillValue" type="float" value="NaN" /> <attribute name="missing_value" type="float" value="NaN" />
                  </variable>
                  <variable name="water_u">
<attribute name="_FillValue" type="float" value="NaN" /> <attribute name="missing_value" type="float" value="NaN" />
                  </variable>
                  <variable name="water_v">
<attribute name="_FillValue" type="float" value="NaN" /> <attribute name="missing_value" type="float" value="NaN" />
                  </variable>
                  <variable name="surf_el">
<attribute name="missing_value" type="float" value="NaN" /> <attribute name="_FillValue" type="float" value="NaN" />
                  </variable>
                 </netcdf>
               </dataset>
             </dataset>

<service name="remoteOPeNDAP_00A" serviceType="OPENDAP" base="http://edac-dap.northerngulfinstitute.org/thredds/dodsC/"; />

             <dataset name="NCOM Region 2 Aggregation remote">
<dataset name="NCOM Region 2 Best Time Series remote" ID="NCOM_reg_2_best_time_series_remote" urlPath="ncom_reg2_agg/NCOM_Region_2_Aggregation_best.ncd">
                 <serviceName>remoteOPeNDAP_00A</serviceName>
                 <dataType>GRID</dataType>
               </dataset>
             </dataset>



john caron wrote:
Kevin O'Brien wrote:
Hi John -

The TDS version at
   http://ferret.pmel.noaa.gov/geoide/geoIDECleanCatalog.xml

is at 4.1.0.

I just upgraded my development version that is serving the same dataset to 4.1.20100428.1437, which is the most recent download version, and am seeing the same thing. Would it be easier for you if I upgraded the public version to the latest?

thanks
Kevin



john caron wrote:
Hi Kevin:

What version of the TDS/

Kevin O'Brien wrote:
Hi John -

I'm seeing another interesting problem in terms of looking at forecast model collections. One of the problems we had with the FMRC putting in NAN's without changing the attributes to indicate such was that certain clients (notably ferret), kinda freaked out when dealing with the NaN's it found. One way I can fix this without a change in the FMRC (that you've been discussing with Steve) is to of course wrap some NcML around the data and serve it myself. Something like:

              <dataset name="NCOM Region 2 Aggregation">
<dataset name="NCOM Region 2 Best Time Series" ID="NCOM_reg_2_best_time_series" urlPath="NCOM_Region_2_Aggregation">
                  <serviceName>all</serviceName>
<netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2"; location="http://edac-dap.northerngulfinstitute.org/thredds/dodsC/ncom_reg2_agg/NCOM_Region_2_Aggregation_best.ncd";>
                   <variable name="water_temp">
<attribute name="_FillValue" type="float" value="NaN" /> <attribute name="missing_value" type="float" value="NaN" />
                   </variable>
                   <variable name="salinity">
<attribute name="_FillValue" type="float" value="NaN" /> <attribute name="missing_value" type="float" value="NaN" />
                   </variable>
                   <variable name="water_u">
<attribute name="_FillValue" type="float" value="NaN" /> <attribute name="missing_value" type="float" value="NaN" />
                   </variable>
                   <variable name="water_v">
<attribute name="_FillValue" type="float" value="NaN" /> <attribute name="missing_value" type="float" value="NaN" />
                   </variable>
                   <variable name="surf_el">
<attribute name="missing_value" type="float" value="NaN" /> <attribute name="_FillValue" type="float" value="NaN" />
                   </variable>
                  </netcdf>
                </dataset>
              </dataset>


So, I've done that, and in doing so have come across an issue where, for some reason, Java netcdf is not able to read that particular file/aggregation. I'll use IDV as an example of what I see.

If I tell IDV to add the following catalog:

    http://ferret.pmel.noaa.gov/geoide/geoIDECleanCatalog.xml

and pick the appropriate dataset, like so:




And then ask IDV to load the data, here's the error message that I get:





I actually see the same error when trying to get LAS configuration data for the same dataset using our addXML Java utility, but thought showing it in IDV would probably make more sense to you. If I just point right directly to the best time series in the FMRC, I don't have this problem...

Any ideas? Is there any other information I can supply that might help...

Thanks
Kevin

p.s  the full error dump from above is:

java.io.IOException: Inconsistent array length read: 1165128303 != 1914731274 at ucar.nc2.dods.DODSNetcdfFile.readArrays(DODSNetcdfFile.java:1216)
    at ucar.nc2.dods.DODSNetcdfFile.<init>(DODSNetcdfFile.java:318)
at ucar.nc2.dataset.NetcdfDataset.acquireDODS(NetcdfDataset.java:783) at ucar.nc2.dataset.NetcdfDataset.openOrAcquireFile(NetcdfDataset.java:657) at ucar.nc2.dataset.NetcdfDataset.openDataset(NetcdfDataset.java:434) at ucar.nc2.dataset.NetcdfDataset.acquireDataset(NetcdfDataset.java:523)
    at ucar.nc2.dt.grid.GridDataset.open(GridDataset.java:103)
    at ucar.nc2.dt.grid.GridDataset.open(GridDataset.java:89)
at ucar.unidata.data.grid.GeoGridDataSource.doMakeDataSet(GeoGridDataSource.java:1084) at ucar.unidata.data.grid.GeoGridDataSource.getDataset(GeoGridDataSource.java:1106) at ucar.unidata.data.grid.GeoGridDataSource.initAfterCreation(GeoGridDataSource.java:471) at ucar.unidata.data.DataManager.createDataSource(DataManager.java:1565) at ucar.unidata.idv.IntegratedDataViewer.createDataSource(IntegratedDataViewer.java:1992) at ucar.unidata.idv.IntegratedDataViewer.makeDataSource(IntegratedDataViewer.java:1909) at ucar.unidata.idv.IntegratedDataViewer.makeDataSource(IntegratedDataViewer.java:1843) at ucar.unidata.idv.IntegratedDataViewer.makeDataSource(IntegratedDataViewer.java:1816) at ucar.unidata.idv.chooser.IdvChooser.makeDataSource(IdvChooser.java:943) at ucar.unidata.idv.chooser.XmlChooser.makeDataSource(XmlChooser.java:280) at ucar.unidata.idv.chooser.ThreddsHandler.processNodes(ThreddsHandler.java:819) at ucar.unidata.idv.chooser.ThreddsHandler.doLoad(ThreddsHandler.java:677) at ucar.unidata.idv.chooser.XmlChooser.doLoadInThread(XmlChooser.java:715)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at ucar.unidata.util.Misc$2.run(Misc.java:1068)
    at ucar.unidata.util.Misc$3.run(Misc.java:1096)


--
Kevin O'Brien UW/JISAO Research Scientist NOAA/PMEL/TMAP
206-526-6751                    http://www.pmel.noaa.gov

"The contents of this message are mine personally and do
not necessarily reflect any position of the Government
or the  National Oceanic and Atmospheric Administration."



Hey Kevin , what Id like to try is the latest 4.2 with some new configuration. Can you send me the TDS config catalog for that particular dataset? thanks