[thredds] Aggregated variables showing up as zeros
John Caron
caron at unidata.ucar.edu
Fri Feb 8 10:29:24 MST 2008
Hi Dave:
Can you send me a file with the problem dimension? Its probably a simple bug that can be fixed.
David Robertson wrote:
> Hi,
>
> I have encountered a thredds aggregation problem. I have the very latest
> thredds.war (3.16.30), java 1.6 and tomcat 6.0.14. The typicalDataset is
> set to penultimate for the aggregation. We are aggregating on the
> unlimited variable ocean_time. Our aggregation block in the catalog.xml
> file looks like this:
>
> <dataset name="Averages"
> ID="roms_mabgom_v4_averages"
> urlPath="roms/mabgom/v4/avg" >
>
> <netcdf
> xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2">
> <aggregation dimName="ocean_time" type="joinExisting">
> <scan
> location="/home/om/dods-data/thredds/roms/mabgom/v4/avg/"
> suffix=".nc" />
> </aggregation>
> </netcdf>
>
> </dataset>
>
> I also looked in the aggregation cache file and all the ncoords and
> coordValue fields look good.
>
> The error occurs when you ask for an ocean_time record range that does
> not start at the beginning of a file. For example, our nc files have 10
> records each so if I ask for ocean_times 0 through 15 or 10 through 25
> everything is correct and fine. However, if I ask for 1 through 15 or
> even 9 through 15 I get the values for records 10 through 15 then a
> bunch of zeros.
>
> http://tashtego.marine.rutgers.edu:8080/thredds/dodsC/roms/mabgom/v4/avg.ascii?ocean_time[0:1:15]
>
> Dataset {
> Float64 ocean_time[ocean_time = 16];
> } roms/mabgom/v4/avg;
> ---------------------------------------------
> ocean_time[16]
> 4.56116478E9, 4.56120954E9, 4.5612543E9, 4.56129906E9, 4.56134382E9,
> 4.56138858E9, 4.56143334E9, 4.5614781E9, 4.56152286E9, 4.56156762E9,
> 4.56161238E9, 4.56165714E9, 4.5617019E9, 4.56174666E9, 4.56179142E9,
> 4.56183618E9
>
>
> http://tashtego.marine.rutgers.edu:8080/thredds/dodsC/roms/mabgom/v4/avg.ascii?ocean_time[1:1:15]
>
> Dataset {
> Float64 ocean_time[ocean_time = 15];
> } roms/mabgom/v4/avg;
> ---------------------------------------------
> ocean_time[15]
> 4.56161238E9, 4.56165714E9, 4.5617019E9, 4.56174666E9, 4.56179142E9,
> 4.56183618E9, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0
>
>
> http://tashtego.marine.rutgers.edu:8080/thredds/dodsC/roms/mabgom/v4/avg.ascii?ocean_time[9:1:15]
>
> Dataset {
> Float64 ocean_time[ocean_time = 7];
> } roms/mabgom/v4/avg;
> ---------------------------------------------
> ocean_time[7]
> 4.56161238E9, 4.56165714E9, 4.5617019E9, 4.56174666E9, 4.56179142E9,
> 4.56183618E9, 0.0
>
> As you can see it's only getting records from the second file. The even
> stranger thing is that you get the right data (ubar in this example)
> using time records not starting at the beginning of a file.
>
> http://tashtego.marine.rutgers.edu:8080/thredds/dodsC/roms/mabgom/v4/avg.ascii?ubar[9:1:15][0:1:0][0:1:0]
>
> Dataset {
> Float32 ubar[ocean_time = 7][eta_u = 1][xi_u = 1];
> } roms/mabgom/v4/avg;
> ---------------------------------------------
> ubar[7][1][1]
> [0][0], 0.02386481
> [1][0], 0.011283709
> [2][0], -0.0020005547
> [3][0], -0.02091563
> [4][0], -0.036607344
> [5][0], -0.04679432
> [6][0], -0.05165326
>
> That data for ubar is correct. It's the last record in one file and the
> first 6 in another. If you tried to get the time values to line up with
> this data it would be wrong:
>
> http://tashtego.marine.rutgers.edu:8080/thredds/dodsC/roms/mabgom/v4/avg.ascii?ocean_time[9:1:15],ubar[9:1:15][0:1:0][0:1:0]
>
> Dataset {
> Float64 ocean_time[ocean_time = 7];
> Float32 ubar[ocean_time = 7][eta_u = 1][xi_u = 1];
> } roms/mabgom/v4/avg;
> ---------------------------------------------
> ocean_time[7]
> 4.56161238E9, 4.56165714E9, 4.5617019E9, 4.56174666E9, 4.56179142E9,
> 4.56183618E9, 0.0
>
> ubar[7][1][1]
> [0][0], 0.02386481
> [1][0], 0.011283709
> [2][0], -0.0020005547
> [3][0], -0.02091563
> [4][0], -0.036607344
> [5][0], -0.04679432
> [6][0], -0.05165326
>
> That first record of ocean time, you will notice, is the value for time
> record 10 (first record in second file) whereas the first entry under
> ubar is the value for time record 9 (last record in the first file).
> This problem occurs with any aggregation set we have that has files with
> more than one record in them. The only reason the problem doesn't occur
> with files that have one record is because no matter what record you
> start with it's at the beginning of a file.
>
> In logs/catalina.out I get errors like:
>
> 2008-02-06T10:33:40.835 -0600 [ 1010687][ 47] ERROR -
> ucar.nc2.ncml.Aggregation - readAggCoord incorrect number of coordinates
> dataset=/home/om/dods-data/thredds/roms/mabgom/v4/avg/mabgom_avg_0001.nc
>
> If I don't start at the beginning of a file (first time record ending in
> 0) and/or end at the end of a file. For Example, if I ask for records 5-15:
>
> http://tashtego.marine.rutgers.edu:8080/thredds/dodsC/roms/mabgom/v4/avg.ascii?ocean_time[5:1:15]
>
> produces the catalina.out errors:
>
> 2008-02-06T10:50:53.315 -0600 [ 2043167][ 712] ERROR -
> ucar.nc2.ncml.Aggregation - readAggCoord incorrect number of coordinates
> dataset=/home/om/dods-data/thredds/roms/mabgom/v4/avg/mabgom_avg_0001.nc
> 2008-02-06T10:50:53.316 -0600 [ 2043168][ 712] ERROR -
> ucar.nc2.ncml.Aggregation - readAggCoord incorrect number of coordinates
> dataset=/home/om/dods-data/thredds/roms/mabgom/v4/avg/mabgom_avg_0002.nc
>
> As you can see I get one error for not starting that the beginning of a
> file and another error for not ending at the end of a file.
>
> However, if I use the "NCDump Variable Data" tool from toolsUI.jar to
> access the same range (5-15) no errors are produced in catalina.out and
> the correct data values are displayed (no zeros).
>
> Any assistance would be greatly appreciated.
>
> Thanks,
> David Robertson
> _______________________________________________
> thredds mailing list
> thredds at unidata.ucar.edu
> For list information or to unsubscribe, visit: http://www.unidata.ucar.edu/mailing_lists/
More information about the thredds
mailing list