Hi Fei and group --
In my experience the biggest bottleneck in situations like these is usually
disk IO. So if we can tolerate having vectors of weights -- w(i) for all i,
w(j) for all j, etc -- in memory, then we could use a recursive algorithm.
Signature something like:
void compute_weights(
std::vector<int> cursor,
const std::vector<const NcVar *> & inputWeights,
std::vector<std::vector<float> *> inputWeightData,
std::vector<std::vector<float> *> & outputWeightData);
The idea is that initially, the cursor specifies "invalid" (-1) indices for
each of the inputWeights. The function has logic:
* when the number of specified dims is < (N-1), give the left-most unspecified
index a value of 0. Read the inputWeights corresponding to this new index into
memory (inputWeightData).
* else only the last dim is unspecified.
* Read the inputWeights for it into memory (inputWeightData).
* Compute and store the output weights for each index of the last dimension.
* Find the "next" suite of dimensions to the left of the last. (Increment
the 2nd to last, or if its at the end, then increment the 3rd to last and set
the 2nd to last to 0, etc.)
* Call the function recursively.
...a rough sketch.
If one can't tolerate a full set of input weight vectors in memory, then
presumably this algorithm can be generalized such that for the M (0<M<N)
left-most dimensions, the full weight vectors should not be read in, and
instead, results are computed one index at a time.
Thoughts?
Kurt Hanson
-----Original Message-----
From: owner-netcdfgroup@xxxxxxxxxxxxxxxx
[mailto:owner-netcdfgroup@xxxxxxxxxxxxxxxx]On Behalf Of Fei Liu
Sent: Tuesday, March 21, 2006 11:24 AM
To: netcdfgroup@xxxxxxxxxxxxxxxx
Subject: what's the best method to compute weight value for arbitrary
data format?
Hi group, sometimes there is a need to compute a weight array when
averaging netCDF result. For example, for data(z,y,x)
weight(i,j,k) = wx(i)*wy(j)*wz(k)
but for data(z,x), it's
weight(i,k) = wx(i)*wz(k)
This variable weight has to be constructed during run time without any
assumption of the underlying data organization (data(t,x), data(t,z)
etc.). I have an algorithm to compute this variable dynamicsally but it's
not very efficient. What suggestions do you have?
--
Fei Liu, PhD. phone: 609-4526551
RSIS/GFDL/NOAA fax: 609-9875063
P.O. Box 308, 201 Forrestal Rd. http://www.gfdl.noaa.gov/~fil
Princeton, NJ 08542-0308 email: Fei.Liu@xxxxxxxx
