[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[netCDFJava #SSK-311219]: Slicing in multiple dimensions



> so the recommended workaround for 2.22 is to use:
> 
> Array data = var.read("1,3,:,:");
> Array result = data.reduce();
> 
> instead of:
> 
> Variable tmp = var;
> tmp = tmp.slice(0, 1);
> tmp = tmp.slice(0, 3);
> Array result = tmp.read();

yes, for sure

> 
> is that correct? Is this workaround less performant or memory intensive
> than the original code (I presume that the data has to be read and then
> rearranged by the reduce())? 

will be equal if not slightly better
no data is moved by reduce()


We won't always slice on the leading
> dimensions and performance/memory requirements are fairly critical for
> us.
> 
> Constructing the segment string will be a little bit awkward as the
> required input information is a bit scattered; collecting this
> information in the sliced variables (originally MultiArrays) was a lot
> more convenient but I'm sure I can find a solution if there's no other
> way.

its slightly more efficient to construct the index (2.2) or Section (4.0) 
object directly, rather than using the "sectionSpec" string, but for large 
reads the IO will dominate.

the point of slice is to create "first class" variables that can be manipulated 
independently, so I understand that dealing with the original is more awkward. 
Obviously I didnt get the composition correct in 2.2, which is one reason i 
rewrote it in 4.0.

> 
> It's a also a shame about the variable names having to start with a
> character. We aren't really worried about portability and building in a
> workround (adding an extra character just to meet the naming convention
> and then removing it again as necessary) will add clutter to otherwise
> fairly elegant code. It looks like this will only be an issue when we
> move to NetCDF 4, though.

its something we've debated, but thats the way it is for now. you can write to 
russ rew (address@hidden) to present your use case if you want. 

anyway, I would recommend that you move to 4.0 sooner than later. as you see, 
there are a number of fixes that wont get backported, and all new features and 
speedups will be only in 4.0. But I understand your concerns for production 
code. right now 4.0 code is still stabilizing, but it will move to beta in a 
month or two.


> 
> Many thanks for your help,


you're welcome.


Ticket Details
===================
Ticket ID: SSK-311219
Department: Support netCDF Java
Priority: Normal
Status: Open