Konrad,
>Date: Tue, 11 Sep 2001 12:13:45 +0200
>From: Konrad Hinsen <hinsen@xxxxxxxxxxxxxxx>
>To: noon@xxxxxxxxxxxxxxxxxxxx
>Subject: Re: performance degrades with filesize
The above message contained the following:
> I think I found what slows it down. In my Python interface, every
> write operation to an array with an unlimited dimension is followed by
> a call to nc_inq_dimlen() in order to keep the internal dimension
> count up to date. I had assumed that this call would be cheap and
> independent of the file size, since it ought to be no more than an
> access to some data structure that the netCDF library should keep in
> memory.
I ported a previously-posted Python script to C and tested it. There was
no degradation in performance. I included calls to nc_inq_dimlen(). The
program is enclosed.
I suspect the problem lies in the Python interface.
Regards,
Steve Emmerson <http://www.unidata.ucar.edu>
#include <netcdf.h>
#include <time.h>
#include <stdio.h>
#define X0_SIZE 10
#define X1_SIZE 50
main()
{
int ncId;
int x0Id, x1Id, x2Id, x3Id, x4Id, timeId;
nc_create("garbage.nc", NC_CLOBBER, &ncId);
nc_def_dim(ncId, "x0", X0_SIZE, &x0Id);
nc_def_dim(ncId, "x1", X1_SIZE, &x1Id);
nc_def_dim(ncId, "x2", 23, &x2Id);
nc_def_dim(ncId, "x3", 15, &x3Id);
nc_def_dim(ncId, "x4", 125, &x4Id);
nc_def_dim(ncId, "time", NC_UNLIMITED, &timeId);
{
int y0Id, y1Id;
int dimIds[3];
dimIds[0] = timeId;
dimIds[1] = x0Id;
dimIds[2] = x1Id;
nc_def_var(ncId, "y0", NC_FLOAT, 3, dimIds, &y0Id);
nc_def_var(ncId, "y1", NC_FLOAT, 1, dimIds, &y1Id);
nc_enddef(ncId);
{
float y0[X1_SIZE*X0_SIZE];
float y1[1];
size_t y0Start[] = {0, 0, 0};
size_t y0Count[] = {1, X0_SIZE, X1_SIZE};
size_t y1Start[] = {0};
size_t y1Count[] = {1};
clock_t oldClock = clock();
int i, j, time = 0;
for (i = 0; i < 10; i++)
{
for (j = 0; j < 100; j++)
{
size_t x0Len, x1Len, timeLen;
y0Start[0] = y1Start[0] = time++;
nc_put_vara_float(ncId, y0Id, y0Start, y0Count, y0);
{
nc_inq_dimlen(ncId, x0Id, &x0Len);
nc_inq_dimlen(ncId, x1Id, &x1Len);
nc_inq_dimlen(ncId, timeId, &timeLen);
}
nc_put_vara_float(ncId, y1Id, y1Start, y1Count, y1);
{
nc_inq_dimlen(ncId, x0Id, &x0Len);
nc_inq_dimlen(ncId, x1Id, &x1Len);
nc_inq_dimlen(ncId, timeId, &timeLen);
}
}
{
clock_t newClock = clock();
printf(
"%f\n", (newClock - oldClock)/(double)CLOCKS_PER_SEC);
oldClock = newClock;
}
}
nc_close(ncId);
}
}
}
