Hi Bob,
Sorry for the delay in responding. I would have expected an exception as
well, but apparently this is working as intended. Specifically, we're
trying to match the behavior of the netcdf C library, which allows
truncated files [1]. I verified this with ncdump, which printed all data
values without error. For truncated values, ncdump printed '0', which is
the default for floats in C. NetCDF-Java, on the other hand, used whatever
values were left in the RandomAccessFile buffer after the last successful
read for truncated values. Maybe that's a problem.
If you like, it's possible to disallow truncated files by setting
ucar.nc2.iosp.netcdf3.N3header.disallowFileTruncation to true, in which
case you'll get the exception you're expecting.
Cheers,
Christian
[1]
http://www.unidata.ucar.edu/software/netcdf/netcdf-4/newdocs/netcdf-c/Fill-Values.html#Fill-Values
On Tue, Oct 18, 2016 at 11:56 AM, Bob Simons - NOAA Federal <
bob.simons@xxxxxxxx> wrote:
> In our system, and I suspect elsewhere, sometimes .nc files don't get
> completely copied from one place to another. Normally, this is caught and
> dealt with (e.g., by the script doing the copying), but sometimes it isn't
> (e.g., a computer is restarted while a script is running), leaving a
> shortened file in place.
>
> I wanted to understand how netcdf-java (v4.6.6) reacted to these shorted
> files. So I made the attached file by copying only the first 1/6th of a
> valid file. Then I tried to read it with this code
>
> NetcdfFile ncFile = NetcdfFile.open("/git/erddapTest/nc/
> invalidShortened.nc");
> try {
> Variable var = ncFile.findVariable("MWcdom");
> //size=[1,1,2321,4001]
> Array array = var.read();
> System.out.println("shape=" + Arrays.toString(array.
> getShape()));
>
> array = var.read("0,0,0:2000:1000,0:4000:1000");
> System.out.println("shape=" + Arrays.toString(array.
> getShape()));
>
> //System.out.println(array.toString());
> System.out.println("Shouldn't get here!");
> ncFile.close();
> } catch (Throwable t) {
> try {
> ncFile.close();
> } catch (Throwable t2) {
> }
> System.out.println("caught: " + t.toString());
> }
>
> Note that Variable.read() documentation says that the method throws an
> IOException.
>
> To my surprise, the above code doesn't throw any exception.
> netcdf-java says it can read the entire var and read the specified section
> (which gets values from throughout the variable).
>
> Is that what you expected?
> Am I doing something wrong in my test?
> Doesn't netcdf-java check the validity of the file?
> Why don't the read()s cause an IOException?
>
> Thank you for looking into this.
>
> --
> Sincerely,
>
> Bob Simons
> IT Specialist
> Environmental Research Division
> NOAA Southwest Fisheries Science Center
> 99 Pacific St., Suite 255A (New!)
> Monterey, CA 93940 (New!)
> Phone: (831)333-9878 (New!)
> Fax: (831)648-8440
> Email: bob.simons@xxxxxxxx
>
> The contents of this message are mine personally and
> do not necessarily reflect any position of the
> Government or the National Oceanic and Atmospheric Administration.
> <>< <>< <>< <>< <>< <>< <>< <>< <><
>
>
> _______________________________________________
> NOTE: All exchanges posted to Unidata maintained email lists are
> recorded in the Unidata inquiry tracking system and made publicly
> available through the web. Users who post to any of the lists we
> maintain are reminded to remove any personal information that they
> do not want to be made public.
>
>
> netcdf-java mailing list
> netcdf-java@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe, visit:
> http://www.unidata.ucar.edu/mailing_lists/
>
