Hi Clifford, > So lets say I have NetCDF file that I am trying to read with a Section that > looks like this: > > 0:0, 0:0, 0:719, 0:1439 > > When I read this entire section at once, I get back 1036800 elements > clocking in at 4147200 bytes. So I says to myself, I says 'what if I split > the read up into multiple sections, and process the partial Array results > one at a time, so that I'm not holding a reference to ~4mb of memory". > > So I split the read up such that it looks like the following (this is the > actual toString() result of each Section immediately before the read takes > place) > > 0:0, 0:0, 0:359, 0:719 > > 0:0, 0:0, 360:719, 720:1439 > > BUT, the results from each of these arrays combined totals only > 518400 elements weighing in at 2073600 bytes - I'm basically missing half > the data here! It appears that you have only requested half the data. To get all the data, you either need the four sections specified by 0:0, 0:0, 0:359, 0:719 0:0, 0:0, 0:359, 720:1439 0:0, 0:0, 360:719, 0:719 0:0, 0:0, 360:719, 720:1439 or you need to make each of the two sections request half the data: 0:0, 0:0, 0:359, 0:1439 0:0, 0:0, 360:719, 0:1439 --Russ > Whats more, If I make the Sections smaller and increase the number of > variable reads in an attempt to reduce memory usage even further, I lose > even more data! > > What am I doing wrong here? If Sections are not appropriate for 'chunking' > data reads, is there a mechanism in the library that is more suited to this > task? > -- > Clifford M. Harms > > Russ Rew UCAR Unidata Program address@hidden http://www.unidata.ucar.edu Ticket Details =================== Ticket ID: YRD-168110 Department: Support netCDF Priority: Normal Status: Closed
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publicly available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us.