Re: [netcdfgroup] bug mixing dap reads and netcdf4 writes

Russ Rew

March 14, 2013 9:01 AM

Hi Jeff,

The C program you included gets an "Invalid argument" error due to the
call

status = nc_set_default_format(NC_NETCDF4, NULL);

because NC_NETCDF4 is not a valid argument to that function. what you
intended most likely was

status = nc_set_default_format(NC_FORMAT_NETCDF4, NULL);

which doesn't return an error status.

It's also not clear what effect you intended this function call to
achieve. That function is to change the format of netCDF files to be
created by future calls to nc_create (or nc__create), but has no
relevance to calls to nc_open. In particular, it won't make it possible
to read netCDF-4 enhanced data model features using the existing DAP
protocol, because the DAP4 protocol is still being implemented as part
of the OPULS project. No clients or servers exist yet for DAP4.

Nevertheless, there's apparently an unintended interaction between the
call to nc_set_default with NC_FORMAT_NETCDF4 and a subsequent attempt
to open and read data from a DAP server. I've created a Jira ticket for
this, if you want to follow progress on it:

https://bugtracking.unidata.ucar.edu/browse/NCF-243

--Russ

Russ:  Right - my c example was flawed (and not consistent with the python example).  The python interface uses set_default_format to set the output format of a new file created by nc_create.  Here's an updated C example:

#include <stdio.h>
#include <stdlib.h>
#include <netcdf.h>
#define handle_error(e) {printf("Error: %s\n", nc_strerror(e)); exit(2);}

int main(int *argc, char **argv)
{
    char flnm[] = "http://www.smast.umassd.edu:8080/thredds/dodsC/fvcom/hindcasts/wave_gom3";
    char flnm2[] = "http://www.smast.umassd.edu:8080/thredds/dodsC/fvcom/hindcasts/30yr_gom3";
    char flnmo[] = "dummy.nc";
    int status;                        /* error status */
    int ncid, ncid2, ncido;            /* netCDF ID */

    status = nc_open(flnm, NC_NOWRITE, &ncid);
    if (status != NC_NOERR) handle_error(status);

    status = nc_set_default_format(NC_FORMAT_NETCDF4, NULL);
    if (status != NC_NOERR) handle_error(status);

    status = nc_create(flnmo, NC_WRITE | NC_CLOBBER, &ncid);
    if (status != NC_NOERR) handle_error(status);

    status = nc_open(flnm2, NC_NOWRITE, &ncid);
    if (status != NC_NOERR) handle_error(status);

    return ( 0 );

}

This still produces

Error: NetCDF: Invalid argument

and this error message is (apparently) being emitted from the last nc_open call (after the nc_create).  The error goes away if the nc_set_default format code sets the file format to anything other than NC_FORMAT_NETCDF.  I understand that the DAP protocol does not yet support the NETCDF4 data model, but since the last nc_open call is NC_NOWRITE, I wouldn't expect the nc_set_default call to affect it at all.

Does that make sense?

-Jeff

Jeff Whitaker

March 13, 2013 9:19 AM

Here's a python program that triggers the error:

>> import netCDF4
>> nc=netCDF4.Dataset('http://www.smast.umassd.edu:8080/thredds/dodsC/fvcom/hindcasts/wave_gom3')
>> nco=netCDF4.Dataset('dummy.nc', 'w', format='NETCDF4')
>> nc2=netCDF4.Dataset('http://www.smast.umassd.edu:8080/thredds/dodsC/fvcom/hindcasts/30yr_gom3')

Traceback (most recent call last):
  File "issue170.py", line 4, in <module>
    nc2=netCDF4.Dataset('http://www.smast.umassd.edu:8080/thredds/dodsC/fvcom/hindcasts/30yr_gom3')
  File "netCDF4.pyx", line 1386, in netCDF4.Dataset.__init__ (netCDF4.c:19186)
    raise RuntimeError((<char *>nc_strerror(ierr)).decode('ascii'))
RuntimeError: NetCDF: Invalid argument

and here's the C counterpart

#include <stdlib.h>
#include <netcdf.h>
#define handle_error(e) {printf("Error: %s\n", nc_strerror(e)); exit(2);}

int main(int *argc, char **argv)
{
    char flnm[] = "http://www.smast.umassd.edu:8080/thredds/dodsC/fvcom/hindcasts/wave_gom3";
    char flnm2[] = "http://www.smast.umassd.edu:8080/thredds/dodsC/fvcom/hindcasts/30yr_gom3";
    char flnmo[] = "dummy.nc";
    int status;                        /* error status */
    int ncid, ncid2, ncido;            /* netCDF ID */

    status = nc_set_default_format(NC_NETCDF4, NULL);
    if (status != NC_NOERR) handle_error(status);

    status = nc_open(flnm, NC_NOWRITE, &ncid);
    if (status != NC_NOERR) handle_error(status);

    status = nc_open(flnmo, NC_WRITE | NC_CLOBBER, &ncid);
    if (status != NC_NOERR) handle_error(status);

    status = nc_open(flnm2, NC_NOWRITE, &ncid);
    if (status != NC_NOERR) handle_error(status);

    return ( 0 );
}

which produces

Error: NetCDF: Invalid argument

Note that the file write sandwiched between the DAP reads must be NETCDF4 format to trigger the error.  Tested with 4.2.1.1 and 4.3.0rc2.

Regards, Jeff

---