Unit Testing in the netCDF-c library using cmake

It appears to be the case that all current tests in the netcdf-c source tree are integration tests. That is they only access the library using calls to the externally visible API.

The DAP4 code testing is different in that it includes (for the first time) unit tests. These tests reference code that is considered internal to the library. For the library generated under autoconf, this is not a problem because those symbols are visible in the library: nothing is hidden.

However, when using cmake and Visual Studio, the occurrence of _declspec(dllimport) and _declspec(dllexport) tags causes all symbols to be hidden except those explicitly exported via _declspec(dllexport). This declaration is hidden by the EXTERNL macro (see e.g. include/netcdf.h and netcdf_mem.h).

This means that unit tests will not work when loaded with the netcdf library using the targetlinklibrary() function in cmake.

In any case, and in order to get around this, I1. hypothesized the following options.

  1. Build two versions of the library: one for integration tests and installation and one for unit tests. One possible way to do this is to use a Visual Studio .dep file to export the otherwise hidden symbols. Frankly, I have no idea how do to this under cmake.
  2. Use the ADD_LIBRARY(X OBJECT ...) + $<TARGET_OBJECTS...> mechanism to load the individual object files containing the symbols needed for unit testing.

Neither solution is appealing.

I attempted to use the #2 approach, but it failed. It appears the the ADD_LIBRARY(... OBJECT ...) command does not operate as I expected. Perhaps someone else can figure out how to make this work.

So, for now, I have expunged the unit tests when building using cmake.

Notes:

  1. The top-level CMakeLists.txt defines what I call a "shift point" where _declspec(dllimport) is used instead of _declspec(dllexport). This is defined by the command REMOVEDEFINITIONS(-DDLLEXPORT).

  2. We could suppress all internal symbols from the netcdf library generated by autoconf. We would do so by using a program like strip to only expose the exact netcdf library API and hide all other symbols. The proper time to do this would be as part of an install-hook that hid the symbols at the point when the library was being installed.

  3. Note that any .h file that includes _declspec (vid EXTERNL) must be referenced by code compiled before the shift point (see above) in the CMakeLists.txt file. But watch out that no API function in the .h file is duplicated int some other .h file in order to prevent complaints by cmake about linkage errors.

  4. EXTERNL should not be used except in include/*.h files that are intended to define an external API.

Implementing Multi-threaded IO for the NetCDF-3 format

The issue of multi-threaded read and write of NetCDF files has repeatedly arisen in the netcdf mailing list (netcdfgroup@unidata.ucar.edu). Read on for a discussion of some options.

[Read More]

High Performance Netcdf-4 Proposal

This documents outlines a proposal to create an alternate Netcdf-4 file format targeted to high-performance, READ-ONLY, access.

[Read More]

NetCDF Compression

The steady state of disks is full. --Ken Thompson

Introduction

From our support questions, it appears that the major feature of netCDF-4 attracting users to upgrade their libraries from netCDF-3 is compression. The netCDF-4 libraries inherit the capability for data compression from the HDF5 storage layer underneath the netCDF-4 interface. Linking a program that uses netCDF to a netCDF-4 library allows the program to read compressed data without changing a single line of the program source code. Writing netCDF compressed data only requires a few extra statements. And the nccopy utility program supports converting classic netCDF format data to or from compressed data without any programming.

[Read More]

Accessing netCDF Data by Coordinates

Library software like netCDF or HDF5 provides access to multidimensional data by array indices, but we would often rather access data by coordinates, such as points on the Earth's surface or space-time bounding boxes.  This blog, with an accompanying iPython notebook, explores some issues with correctness and efficiency in accessing data by coordinates instead of array indices in a real-world example.

[Read More]
Unidata Developer's Blog
A weblog about software development by Unidata developers*
Unidata Developer's Blog
A weblog about software development by Unidata developers*

Welcome

FAQs

News@Unidata blog

Take a poll!

What if we had an ongoing user poll in here?

Browse By Topic
  • feed AWIPS II (13)
Browse by Topic
« February 2017
SunMonTueWedThuFriSat
   
1
2
3
4
5
6
7
8
9
10
11
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
    
       
Today