Due to the current gap in continued funding from the U.S. National Science Foundation (NSF), the NSF Unidata Program Center has temporarily paused most operations. See NSF Unidata Pause in Most Operations for details.
Due to the current gap in continued funding from the U.S. National Science Foundation (NSF), the NSF Unidata Program Center has temporarily paused most operations. See NSF Unidata Pause in Most Operations for details.
Thank you. That is very useful information. I am sad to hear that the conscious default is for the library not to throw an error when the file is truncated (but I understand), but happy that I can use ucar.nc2.iosp.netcdf3.N3header. disallowFileTruncation to change this behavior for my work. On Wed, Nov 2, 2016 at 7:16 AM, Christian Ward-Garrison <cwardgar@xxxxxxxx> wrote: > Hi Bob, > > Sorry for the delay in responding. I would have expected an exception as > well, but apparently this is working as intended. Specifically, we're > trying to match the behavior of the netcdf C library, which allows > truncated files [1]. I verified this with ncdump, which printed all data > values without error. For truncated values, ncdump printed '0', which is > the default for floats in C. NetCDF-Java, on the other hand, used whatever > values were left in the RandomAccessFile buffer after the last successful > read for truncated values. Maybe that's a problem. > > If you like, it's possible to disallow truncated files by setting > ucar.nc2.iosp.netcdf3.N3header.disallowFileTruncation to true, in which > case you'll get the exception you're expecting. > > Cheers, > Christian > > [1] http://www.unidata.ucar.edu/software/netcdf/netcdf-4/ > newdocs/netcdf-c/Fill-Values.html#Fill-Values > > On Tue, Oct 18, 2016 at 11:56 AM, Bob Simons - NOAA Federal < > bob.simons@xxxxxxxx> wrote: > >> In our system, and I suspect elsewhere, sometimes .nc files don't get >> completely copied from one place to another. Normally, this is caught and >> dealt with (e.g., by the script doing the copying), but sometimes it isn't >> (e.g., a computer is restarted while a script is running), leaving a >> shortened file in place. >> >> I wanted to understand how netcdf-java (v4.6.6) reacted to these shorted >> files. So I made the attached file by copying only the first 1/6th of a >> valid file. Then I tried to read it with this code >> >> NetcdfFile ncFile = NetcdfFile.open("/git/erddapTe >> st/nc/invalidShortened.nc"); >> try { >> Variable var = ncFile.findVariable("MWcdom"); >> //size=[1,1,2321,4001] >> Array array = var.read(); >> System.out.println("shape=" + >> Arrays.toString(array.getShape())); >> >> array = var.read("0,0,0:2000:1000,0:4000:1000"); >> System.out.println("shape=" + >> Arrays.toString(array.getShape())); >> >> //System.out.println(array.toString()); >> System.out.println("Shouldn't get here!"); >> ncFile.close(); >> } catch (Throwable t) { >> try { >> ncFile.close(); >> } catch (Throwable t2) { >> } >> System.out.println("caught: " + t.toString()); >> } >> >> Note that Variable.read() documentation says that the method throws an >> IOException. >> >> To my surprise, the above code doesn't throw any exception. >> netcdf-java says it can read the entire var and read the specified >> section (which gets values from throughout the variable). >> >> Is that what you expected? >> Am I doing something wrong in my test? >> Doesn't netcdf-java check the validity of the file? >> Why don't the read()s cause an IOException? >> >> Thank you for looking into this. >> >> -- >> Sincerely, >> >> Bob Simons >> IT Specialist >> Environmental Research Division >> NOAA Southwest Fisheries Science Center >> 99 Pacific St., Suite 255A (New!) >> Monterey, CA 93940 (New!) >> Phone: (831)333-9878 (New!) >> Fax: (831)648-8440 >> Email: bob.simons@xxxxxxxx >> >> The contents of this message are mine personally and >> do not necessarily reflect any position of the >> Government or the National Oceanic and Atmospheric Administration. >> <>< <>< <>< <>< <>< <>< <>< <>< <>< >> >> >> _______________________________________________ >> NOTE: All exchanges posted to Unidata maintained email lists are >> recorded in the Unidata inquiry tracking system and made publicly >> available through the web. Users who post to any of the lists we >> maintain are reminded to remove any personal information that they >> do not want to be made public. >> >> >> netcdf-java mailing list >> netcdf-java@xxxxxxxxxxxxxxxx >> For list information or to unsubscribe, visit: >> http://www.unidata.ucar.edu/mailing_lists/ >> > > -- Sincerely, Bob Simons IT Specialist Environmental Research Division NOAA Southwest Fisheries Science Center 99 Pacific St., Suite 255A (New!) Monterey, CA 93940 (New!) Phone: (831)333-9878 (New!) Fax: (831)648-8440 Email: bob.simons@xxxxxxxx The contents of this message are mine personally and do not necessarily reflect any position of the Government or the National Oceanic and Atmospheric Administration. <>< <>< <>< <>< <>< <>< <>< <>< <><
netcdf-java archives: