[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: 980722: netCDF problem

>To: address@hidden
>From: address@hidden (Carrie Gonzalez)
>Subject: netCDF problem
>Organization: .
>Keywords: 199807222032.OAA10741

Carrie,

>    Thank you for your help.  Now, I see what you are saying about the core
>    problem.  I am still having problems trying to get done what I need.
>    In all actuality, what I need is a data "triplet", that is, three points
>    that are associated with one another.  That is why I allocated 3 * 128
>    points, resulting in 128 triplets.  However, using the 128, 128, 128 was
>    wrong.  How can I accomplish this task?  I tried backing off the 128 issue
>    and tried writing one data triplet at a time, so I set defined the new
>    variable "Data_Triplet" to consist of 3 dimensions, all being 1 in length.
>    However, when I use ncvarput(), I end up with a single value in the data
>    file.  I debugged and found many calls to ncvarput() and then read in the
>    netCDF manual that if you want to write a single value, set count to 
>    (0, 0, ....).  That explains my single value in the file BUT how do I
>    accomplish writing the triplet?  The data_matrix is an array of 3 floats,
>    so I just need a way of specifying 1 value to each of the 3 dimensions.

You have defined your variable using 3 dimensions:

    dimensions:
            nx = 128 ;
            ny = 128 ;
            nz = 128 ;
    variables:
            float Data_Triplet(nx, ny, nz) ;

which gives you a rank-3 (3 dimensional) variable that can hold
128*128*128 = 2097152 float values.

If you really only want 128 triplets of floats, you should instead
define it as something like:

    dimensions:
            nx = 128 ;
            n  = 3   ;
    variables:
            float Data_Triplet(nx, n) ;

which is a rank-2 variable that will hold 3*128 values.  So, for
example, the 10th data triple will be 

    Data_Triplet(9,0), Data_Triplet(9,1), Data_Triplet(9,2)

You could also define the data triplets variable the other way

            float Data_Triplet(n, nx) ;

but then the nx dimension would be varying fastest, and I don't think
you would call the data "triplets", it would be more like three separate
lists of 128 data values.

Looking at your program, I think what you may really want is a set of
Data_Triplets for every record in the input file, in which case you would
want to define your Data_Triplet variable something like

    dimensions:
            numrecs = unlimited;
            n  = 3   ;
            nx = 128 ;
    variables:
            float Data_Triplet(numrecs, nx, n) ;

(I may be wrong about this assumption; perhaps you want to average or
accumulate values over the record dimension ...).

Now to write 128*3 values at once to this variable, you need to use a
start of (0,0) and a count of (128,3), and you need to make sure the
internal array of 128*3 values from which you are writing has the values
in the right order, namely the (0,0) value, then the (0,1) value, then
the (0,2) value, then the (1,0), (1,1), (1,2), ..., (127,0), (127,1),
and (127,2) values.  Your internal "data_matrix" variable appears to be
the transpose of this, with its values defined in the order (0,0),
(1,0), (2,0), ..., (127,0), (0,1), (1,1), ..., (127,1), (0,2), (1,2),
(127,2), as I interpret it from this code:

   for (loop = 0; loop < num_records; ++loop)
    {
 ...
      index_data = 0 * num_steps;
      index_scan = 1 * num_steps;
      index_az = 2 * num_steps;

      for (t1 = 0; t1 < num_steps; ++t1)
       {
         data_matrix[index_data + t1] = data_values[t1];
         data_matrix[index_scan + t1] = scan_values[t1];
         data_matrix[index_az + t1] = az_values[t1];
       }
    }

Also, the above shows you are not writing the values of data_matrix
after you have gathered them up, but instead overwriting the values of
data_matrix from the 0th record with the values from the first record,
then the second record, and so on, so that when you get out of the 

   for (loop = 0; loop < num_records; ++loop)
    {
 ...
    }

loop, you have only the values for the last record stored in the
data_matrix array.  Hence, what you subsequently write to the
Data_Triplet netCDF variable will only contain the 3*128 values from the
last record of the input file.

If you instead use something like:

   for (loop = 0; loop < num_records; ++loop)
    {
 ...

      for (t1 = 0; t1 < num_steps; ++t1)
       {
         data_matrix[3 * t1 + 0] = data_values[t1];
         data_matrix[3 * t1 + 1] = scan_values[t1];
         data_matrix[3 * t1 + 2] = az_values[t1];
       }

you will have the data_matrix values in the right order to write them
all with one call to the netCDF variable Data_Triplet, as follows:

   three_start[0] = loop;  // which record to write
   three_start[1] = 0;
   three_start[2] = 0;

   three_count[0] = 1;  // write 1 record's worth of data
   three_count[1] = 3;
   three_count[2] = 128;

 ...
   status = ncvarput (id_new, data_id, three_start, three_count, data_matrix);
 ...
    }

where the call to ncvarput should be in the for loop that goes through
the records if you want Data_Triplet to contain the data for all the
records in the input file.

This means you will have to reorder your code to create the dimension
and variables for the new output file before the loop that reads the
records from the input file, so in each iteration of the loop you will
be reading a record's worth of data from the three input variables and
writing a record's worth of data to the Data_Triplet output variables.

We usually can't provide such detailed answers to support questions, but
I made an exception here in hopes this will get you off on the right
foot so you can make good use of the netCDF software ...

--Russ

_____________________________________________________________________

Russ Rew                                         UCAR Unidata Program
address@hidden                     http://www.unidata.ucar.edu

> #include <netcdf.h>
> #include <string.h>
> #include <stdlib.h>
> #include <iostream.h>
> #include <stdio.h>
> 
> int main (int argc, char **argv)
> {
>    char netcdf_filename[180], new_filename[180];
>    float *scan_values, *az_values, *data_values, *data_matrix;
>    void *tmp_ptr;
>    size_t bytes;
> 
>    //  These variable types are dictated by netCDF types.
> 
>    nc_type dtype;
>    int id_old, id_new, status, NumDims, dims[3], data_id;
>    int sensor_id, az_id, scan_id, num_id, time_id;
>    char var_name[80];
>    register long loop, t1, index_data, index_scan, index_az;
>    long num_steps, num_records, one_start[1], one_count[1];
>    long two_start[2], two_count[2], three_start[3], three_count[3];
> 
>    if (argc != 2)
>     {
>       cout << "Calling sequence is either filter_netCDF filename.nc\n";
>       exit (-1);
>     }
> 
>    //  Get the name of the netCDF file to be filtered.
> 
>    memset (netcdf_filename, 0, sizeof (netcdf_filename));
>    strcpy (netcdf_filename, *++argv);
>    ncopts = 0;
> 
>    //  Valid netCDF data file?
> 
>    id_old = ncopen ((const char *) netcdf_filename, NC_NOWRITE);
>    if (id_old == -1)
>     {
>       cout << "Can't open the netCDF file " << netcdf_filename << '\n';
>       exit (-1);
>     }
> 
>    //  Determine the number of records and the number of scan values per 
> record.
> 
>    time_id = ncdimid (id_old, "Time");
>    if (time_id == -1)
>     {
>       cout << "Can't retrieve the dimension id for Time." << '\n';
>       exit (-1);
>     }
> 
>    status = ncdiminq (id_old, time_id, var_name, &num_records);
>    if (time_id == -1)
>     {
>       cout << "Can't retrieve the number of records in the file." << '\n';
>       exit (-1);
>     }
> 
>    num_id = ncdimid (id_old, "numValues");
>    if (num_id == -1)
>     {
>       cout << "Can't determine the dimension size of the data." << '\n';
>       exit (-1);
>     }
> 
>    status = ncdiminq (id_old, num_id, var_name, &num_steps);
>    if (time_id == -1)
>     {
>       cout << "Can't retrieve the number of steps defined." << '\n';
>       exit (-1);
>     }
> 
>    //  Retrieve the ids for the variables of interest.
> 
>    sensor_id = ncvarid (id_old, "SENSOR1");
>    if (sensor_id == -1)
>     {
>       cout << "Can't retrieve the id for the data variable." << '\n';
>       exit (-1);
>     }
> 
>    scan_id = ncvarid (id_old, "Center_Scan");
>    if (scan_id == -1)
>     {
>       cout << "Can't retrieve the id for the scan variable." << '\n';
>       exit (-1);
>     }
> 
>    az_id = ncvarid (id_old, "START_AZIMUTHAL1");
>    if (scan_id == -1)
>     {
>       cout << "Can't retrieve the id for the azimuthal variable." << '\n';
>       exit (-1);
>     }
> 
>    //  Allocate space to hold the scan values to be read.
> 
>    bytes = sizeof (float) * num_steps;
>    if ((tmp_ptr = malloc (bytes)) == NULL)
>     {
>       cout << "Error allocating memory to hold scan values.\n";
>       exit (-1);
>     }
>    scan_values = (float *) tmp_ptr;
> 
>    //  Retrieve the center scan values.
> 
>    one_start[0] = 0;
>    one_count[0] = num_steps;
>    status = ncvarget (id_old, scan_id, one_start, one_count, scan_values);
>    if (status == -1)
>     {
>       cout << "Can't read scan values from " << netcdf_filename << '\n';
>       exit (-1);
>     }
> 
>    //  Allocate space to hold the data values read per record.
> 
>    bytes = sizeof (float) * num_steps;
>    if ((tmp_ptr = malloc (bytes)) == NULL)
>     {
>       cout << "Error allocating memory to hold data values.\n";
>       exit (-1);
>     }
>    data_values = (float *) tmp_ptr;
> 
>    //  Allocate space to hold the azimuthal angles read per record.
> 
>    bytes = sizeof (float) * num_steps;
>    if ((tmp_ptr = malloc (bytes)) == NULL)
>     {
>       cout << "Error allocating memory to hold azimuthal values.\n";
>       exit (-1);
>     }
>    az_values = (float *) tmp_ptr;
> 
>    //  Allocate space to hold the "triplet" of data, per scan step.
>    //  One record, not all records.
> 
>    NumDims = 3;
>    bytes = NumDims * sizeof (float) * num_steps;
>    if ((tmp_ptr = malloc (bytes)) == NULL)
>     {
>       cout << "Error allocating memory to hold data triplets.\n";
>       exit (-1);
>     }
>    data_matrix = (float *) tmp_ptr;
> 
>    //  Gather the individual data values, record by record, and create
>    //  a triplet to be written to the new file.
> 
>    two_start[1] = 0;
>    two_count[1] = num_steps;
>    for (loop = 0; loop < num_records; ++loop)
>     {
>       two_start[0] = loop;
>       two_count[0] = 1;
>       status = ncvarget (id_old, sensor_id, two_start, two_count, 
> data_values);
>       if (status == -1)
>        {
>          cout << "Can't read data values from " << netcdf_filename << 
>                  " for record " << loop << '\n';
>          exit (-1);
>        }
> 
>       status = ncvarget (id_old, az_id, two_start, two_count, az_values);
>       if (status == -1)
>        {
>          cout << "Can't read azimuthal values from " << netcdf_filename << 
>                  " for record " << loop << '\n';
>          exit (-1);
>        }
> 
>       index_data = 0 * num_steps;
>       index_scan = 1 * num_steps;
>       index_az = 2 * num_steps;
> 
>       for (t1 = 0; t1 < num_steps; ++t1)
>        {
>          data_matrix[index_data + t1] = data_values[t1];
>          data_matrix[index_scan + t1] = scan_values[t1];
>          data_matrix[index_az + t1] = az_values[t1];
>        }
>     }
> 
>    status = ncclose (id_old);
>    if (status == -1)
>     {
>       cout << "Can't closing the netCDF file " << netcdf_filename << '\n';
>       exit (-1);
>     }
> 
>    //  Need to create the new netCDF file.
> 
>    sprintf (new_filename, "New_%s", netcdf_filename);
>    id_new = nccreate ((const char *) new_filename, NC_CLOBBER);
>    if (id_new == -1)
>     {
>       cout << "Can't open the new netCDF file " << new_filename << '\n';
>       exit (-1);
>     }
> 
>    three_start[0] = 0;
>    three_start[1] = 0;
>    three_start[2] = 0;
>    three_count[0] = three_count[1] = three_count[2] = num_steps;
>    dims[0] = ncdimdef (id_new, "nx", three_count[0]);
>    if (dims[0] == -1)
>     {
>       cout << "Error defining nx dimension for new netCDF file." << '\n';
>       exit (-1);
>     }
>    dims[1] = ncdimdef (id_new, "ny", three_count[1]);
>    if (dims[1] == -1)
>     {
>       cout << "Error defining ny dimension for new netCDF file." << '\n';
>       exit (-1);
>     }
>    dims[2] = ncdimdef (id_new, "nz", three_count[2]);
>    if (dims[2] == -1)
>     {
>       cout << "Error defining nz dimension for new netCDF file." << '\n';
>       exit (-1);
>     }
>    dtype = NC_FLOAT;
> 
>    data_id = ncvardef (id_new, "Data_Triplet", dtype, NumDims, dims);
>    if (data_id == -1)
>     {
>       cout << "Error defining variable for new netCDF file." << '\n';
>       exit (-1);
>     }
>    ncendef (id_new);
> 
>    status = ncvarput (id_new, data_id, three_start, three_count, data_matrix);
>    if (status == -1)
>     {
>       cout << "Error writing data matrix to new netCDF file." << '\n';
>       exit (-1);
>     }
>    ncclose (id_new);
> 
>    return(0);
> }