Implementing Thread-safe Access to the netCDF-C Library
Thread-Safe Access to the netcdf-c API
Initial Draft: 2017-2-21
Last Revised: 2017-5-30
Author: Dennis Heimbigner, Unidata
Table of Contents
- Introduction
- Architectural Considerations
- Locking Regime
- Problem 1: Mostly Auxiliary Functions
- Problem 2: Internal Functions calling Core Functions
- Steps to Implementing Proposed Architecture
- Conclusion
Introduction
This document proposes an architecture for implementing thread-safe access to the netcdf-c library. Here, the term "thread-safe" means that multiple threads can access the netcdf-c library safely (i.e. without interference or deadlock or race conditions). This does not mean that the library is itself multi-threaded. Rather, access to the library is serialized so that only one thread at a time is executing the library code.
It is proposed that thread-safe operation is to be implemented such that all calls to the netcdf-c API are protected by a binary semaphore using a lock-unlock protocol. This means that all calls to the API are "serialized" in the sense that each API call is completed before any other call to the API can be executed. This means that in a multi-threaded environment, it is possible for all threads to safely access the netcdf-c library.
This approach comes with some caveats.
- If two different threads attempt to access the same file, then interference is still possible.
- Using thread-safe access simultaneously with MPI parallelism may not be safe. This is still unresolved
Architectural Considerations
At the moment, the implementation of the netcdf-c API resides in files in the libdispatch directory. Basically, all the code in libdispatch falls into the following categories.
Dispatch functions -- These functions directly invoke methods in the dispatch table and typically have this form.
int nc_xxx(...) { NC* ncp; int stat = NC_check_id(ncid,&ncp); if(stat != NC_NOERR) return stat; return ncp-> dispatch-> XXX(...); }Auxiliary functions -- These functions just invoke some other function in the API, but possibly with some special values for the arguments of the called function. Here is an example.
int nc_inq_varname(int ncid, int varid, char *name) { return nc_inq_var(ncid, varid, name, NULL, NULL, NULL, NULL); }- Complex functions -- These functions do complex computation including calling a variety of internal functions.
- Internal functions -- All other code in libdispatch is considered internal.
Functions in classes 1 and 3 are considered to be part of the API core. The followig Figure shows the notional relationship between the function classes.
<svg version="1.1" viewBox="0.0 0.0 500.0 300.0" fill="none" stroke="none" stroke-linecap="square" stroke-miterlimit="10" xmlns="https://www.w3.org/2000/svg" xmlns:xlink="https://www.w3.org/1999/xlink">
<clippath id="p.0">
<path d="m0 0l372.9895 0l0 260.06824l-372.9895 0l0 -260.06824z" clip-rule="nonzero"></path>
</clippath>
<g clip-path="url(#p.0)">
<path fill="#000000" fill-opacity="0.0" d="m0 0l372.9895 0l0 260.06824l-372.9895 0z" fill-rule="evenodd"></path>
<path fill="#000000" fill-opacity="0.0" d="m12.656168 48.842518l347.3701 0l0 196.56694l-347.3701 0z" fill-rule="evenodd"></path>
<path stroke="#000000" stroke-width="4.0" stroke-linejoin="round" stroke-linecap="butt" d="m12.656168 48.842518l347.3701 0l0 196.56694l-347.3701 0z" fill-rule="evenodd"></path>
<path fill="#000000" fill-opacity="0.0" d="m32.27034 62.745407l138.4567 0l0 43.874016l-138.4567 0z" fill-rule="evenodd"></path>
<path fill="#000000" d="m44.70572 93.22241l0 -17.1875l12.421875 0l0 2.03125l-10.15625 0l0 5.265625l9.5 0l0 2.015625l-9.5 0l0 5.84375l10.546875 0l0 2.03125l-12.8125 0zm14.2734375 0l4.546875 -6.46875l-4.203125 -5.984375l2.640625 0l1.90625 2.921875q0.546875 0.828125 0.875 1.390625q0.515625 -0.765625 0.9375 -1.359375l2.109375 -2.953125l2.515625 0l-4.296875 5.859375l4.625 6.59375l-2.59375 0l-2.546875 -3.875l-0.6875 -1.03125l-3.265625 4.90625l-2.5625 0zm18.015625 -1.890625l0.3125 1.859375q-0.890625 0.203125 -1.59375 0.203125q-1.15625 0 -1.796875 -0.359375q-0.625 -0.375 -0.890625 -0.96875q-0.25 -0.59375 -0.25 -2.484375l0 -7.171875l-1.546875 0l0 -1.640625l1.546875 0l0 -3.078125l2.09375 -1.265625l0 4.34375l2.125 0l0 1.640625l-2.125 0l0 7.28125q0 0.90625 0.109375 1.171875q0.125 0.25 0.375 0.40625q0.25 0.140625 0.71875 0.140625q0.34375 0 0.921875 -0.078125zm10.589844 -2.125l2.171875 0.28125q-0.515625 1.90625 -1.90625 2.96875q-1.390625 1.046875 -3.5625 1.046875q-2.734375 0 -4.34375 -1.671875q-1.59375 -1.6875 -1.59375 -4.734375q0 -3.140625 1.609375 -4.875q1.625 -1.734375 4.203125 -1.734375q2.5 0 4.078125 1.703125q1.59375 1.703125 1.59375 4.78125q0 0.1875 -0.015625 0.5625l-9.28125 0q0.109375 2.046875 1.15625 3.140625q1.046875 1.09375 2.609375 1.09375q1.15625 0 1.96875 -0.609375q0.828125 -0.609375 1.3125 -1.953125zm-6.9375 -3.40625l6.953125 0q-0.140625 -1.5625 -0.796875 -2.359375q-1.0 -1.21875 -2.609375 -1.21875q-1.453125 0 -2.453125 0.984375q-0.984375 0.96875 -1.09375 2.59375zm11.753906 7.421875l0 -12.453125l1.90625 0l0 1.78125q1.375 -2.0625 3.953125 -2.0625q1.125 0 2.0625 0.40625q0.953125 0.40625 1.421875 1.0625q0.46875 0.65625 0.65625 1.5625q0.125 0.578125 0.125 2.046875l0 7.65625l-2.109375 0l0 -7.578125q0 -1.28125 -0.25 -1.921875q-0.25 -0.640625 -0.875 -1.015625q-0.625 -0.390625 -1.46875 -0.390625q-1.34375 0 -2.328125 0.859375q-0.984375 0.859375 -0.984375 3.25l0 6.796875l-2.109375 0zm12.503906 -3.71875l2.09375 -0.328125q0.171875 1.25 0.96875 1.921875q0.8125 0.671875 2.25 0.671875q1.453125 0 2.15625 -0.59375q0.703125 -0.59375 0.703125 -1.390625q0 -0.71875 -0.625 -1.125q-0.421875 -0.28125 -2.15625 -0.71875q-2.3125 -0.578125 -3.21875 -1.0q-0.890625 -0.4375 -1.359375 -1.1875q-0.453125 -0.765625 -0.453125 -1.671875q0 -0.828125 0.375 -1.53125q0.390625 -0.71875 1.046875 -1.1875q0.484375 -0.359375 1.328125 -0.609375q0.859375 -0.265625 1.828125 -0.265625q1.46875 0 2.578125 0.421875q1.109375 0.421875 1.625 1.15625q0.53125 0.71875 0.734375 1.921875l-2.0625 0.28125q-0.140625 -0.96875 -0.8125 -1.5q-0.671875 -0.546875 -1.90625 -0.546875q-1.453125 0 -2.078125 0.484375q-0.625 0.484375 -0.625 1.125q0 0.40625 0.265625 0.734375q0.25 0.34375 0.8125 0.5625q0.3125 0.125 1.859375 0.546875q2.234375 0.59375 3.109375 0.984375q0.890625 0.375 1.390625 1.109375q0.515625 0.71875 0.515625 1.796875q0 1.046875 -0.625 1.984375q-0.609375 0.9375 -1.765625 1.453125q-1.15625 0.5 -2.625 0.5q-2.421875 0 -3.703125 -1.0q-1.265625 -1.015625 -1.625 -3.0zm12.859375 -11.03125l0 -2.4375l2.109375 0l0 2.4375l-2.109375 0zm0 14.75l0 -12.453125l2.109375 0l0 12.453125l-2.109375 0zm4.5351562 -6.21875q0 -3.46875 1.921875 -5.125q1.609375 -1.390625 3.921875 -1.390625q2.5625 0 4.1875 1.6875q1.625 1.6875 1.625 4.640625q0 2.40625 -0.71875 3.78125q-0.71875 1.375 -2.09375 2.140625q-1.375 0.765625 -3.0 0.765625q-2.625 0 -4.234375 -1.671875q-1.609375 -1.6875 -1.609375 -4.828125zm2.171875 0q0 2.390625 1.03125 3.578125q1.046875 1.1875 2.640625 1.1875q1.5625 0 2.609375 -1.1875q1.046875 -1.203125 1.046875 -3.65625q0 -2.3125 -1.0625 -3.5q-1.046875 -1.1875 -2.59375 -1.1875q-1.59375 0 -2.640625 1.1875q-1.03125 1.1875 -1.03125 3.578125zm11.957031 6.21875l0 -12.453125l1.90625 0l0 1.78125q1.375 -2.0625 3.953125 -2.0625q1.125 0 2.0625 0.40625q0.953125 0.40625 1.421875 1.0625q0.46875 0.65625 0.65625 1.5625q0.125 0.578125 0.125 2.046875l0 7.65625l-2.109375 0l0 -7.578125q0 -1.28125 -0.25 -1.921875q-0.25 -0.640625 -0.875 -1.015625q-0.625 -0.390625 -1.46875 -0.390625q-1.34375 0 -2.328125 0.859375q-0.984375 0.859375 -0.984375 3.25l0 6.796875l-2.109375 0zm12.503906 -3.71875l2.09375 -0.328125q0.171875 1.25 0.96875 1.921875q0.8125 0.671875 2.25 0.671875q1.453125 0 2.15625 -0.59375q0.703125 -0.59375 0.703125 -1.390625q0 -0.71875 -0.625 -1.125q-0.421875 -0.28125 -2.15625 -0.71875q-2.3125 -0.578125 -3.21875 -1.0q-0.890625 -0.4375 -1.359375 -1.1875q-0.453125 -0.765625 -0.453125 -1.671875q0 -0.828125 0.375 -1.53125q0.390625 -0.71875 1.046875 -1.1875q0.484375 -0.359375 1.328125 -0.609375q0.859375 -0.265625 1.828125 -0.265625q1.46875 0 2.578125 0.421875q1.109375 0.421875 1.625 1.15625q0.53125 0.71875 0.734375 1.921875l-2.0625 0.28125q-0.140625 -0.96875 -0.8125 -1.5q-0.671875 -0.546875 -1.90625 -0.546875q-1.453125 0 -2.078125 0.484375q-0.625 0.484375 -0.625 1.125q0 0.40625 0.265625 0.734375q0.25 0.34375 0.8125 0.5625q0.3125 0.125 1.859375 0.546875q2.234375 0.59375 3.109375 0.984375q0.890625 0.375 1.390625 1.109375q0.515625 0.71875 0.515625 1.796875q0 1.046875 -0.625 1.984375q-0.609375 0.9375 -1.765625 1.453125q-1.15625 0.5 -2.625 0.5q-2.421875 0 -3.703125 -1.0q-1.265625 -1.015625 -1.625 -3.0z" fill-rule="nonzero"></path>
<path fill="#000000" fill-opacity="0.0" d="m204.09186 70.74541l138.4567 0l0 43.874016l-138.4567 0z" fill-rule="evenodd"></path>
<path fill="#000000" d="m253.94716 86.72241l6.5937653 -17.1875l2.453125 0l7.03125 17.1875l-2.59375 0l-2.0 -5.203125l-7.1875 0l-1.890625 5.203125l-2.4062653 0zm4.9531403 -7.0625l5.828125 0l-1.796875 -4.75q-0.8125 -2.171875 -1.21875 -3.5625q-0.328125 1.65625 -0.921875 3.28125l-1.890625 5.03125zm12.9453125 7.0625l0 -17.1875l6.46875 0q1.71875 0 2.625 0.171875q1.265625 0.203125 2.109375 0.796875q0.859375 0.59375 1.375 1.671875q0.53125 1.0625 0.53125 2.328125q0 2.203125 -1.40625 3.71875q-1.390625 1.515625 -5.03125 1.515625l-4.40625 0l0 6.984375l-2.265625 0zm2.265625 -9.015625l4.4375 0q2.203125 0 3.125 -0.8125q0.9375 -0.828125 0.9375 -2.3125q0 -1.078125 -0.546875 -1.84375q-0.546875 -0.765625 -1.4375 -1.015625q-0.578125 -0.15625 -2.125 -0.15625l-4.390625 0l0 6.140625zm14.1171875 9.015625l0 -17.1875l2.28125 0l0 17.1875l-2.28125 0z" fill-rule="nonzero"></path>
<path fill="#000000" d="m261.41983 109.69116l2.28125 0.578125q-0.71875 2.796875 -2.578125 4.28125q-1.859375 1.46875 -4.546875 1.46875q-2.7656403 0 -4.5156403 -1.125q-1.734375 -1.140625 -2.640625 -3.28125q-0.90625 -2.15625 -0.90625 -4.609375q0 -2.6875 1.015625 -4.6875q1.03125 -2.0 2.921875 -3.03125q1.890625 -1.03125 4.1718903 -1.03125q2.578125 0 4.328125 1.3125q1.765625 1.3125 2.453125 3.6875l-2.234375 0.53125q-0.609375 -1.875 -1.75 -2.734375q-1.125 -0.859375 -2.84375 -0.859375q-1.9843903 0 -3.3125153 0.953125q-1.328125 0.953125 -1.875 2.546875q-0.53125 1.59375 -0.53125 3.296875q0 2.1875 0.625 3.828125q0.640625 1.640625 1.984375 2.453125q1.359375 0.796875 2.9218903 0.796875q1.921875 0 3.234375 -1.09375q1.328125 -1.109375 1.796875 -3.28125zm4.0195312 -0.1875q0 -3.46875 1.921875 -5.125q1.609375 -1.390625 3.921875 -1.390625q2.5625 0 4.1875 1.6875q1.625 1.6875 1.625 4.640625q0 2.40625 -0.71875 3.78125q-0.71875 1.375 -2.09375 2.140625q-1.375 0.765625 -3.0 0.765625q-2.625 0 -4.234375 -1.671875q-1.609375 -1.6875 -1.609375 -4.828125zm2.171875 0q0 2.390625 1.03125 3.578125q1.046875 1.1875 2.640625 1.1875q1.5625 0 2.609375 -1.1875q1.046875 -1.203125 1.046875 -3.65625q0 -2.3125 -1.0625 -3.5q-1.046875 -1.1875 -2.59375 -1.1875q-1.59375 0 -2.640625 1.1875q-1.03125 1.1875 -1.03125 3.578125zm11.941406 6.21875l0 -12.453125l1.890625 0l0 1.890625q0.734375 -1.328125 1.34375 -1.75q0.625 -0.421875 1.359375 -0.421875q1.0625 0 2.171875 0.6875l-0.734375 1.953125q-0.765625 -0.453125 -1.546875 -0.453125q-0.6875 0 -1.25 0.421875q-0.546875 0.40625 -0.78125 1.140625q-0.34375 1.125 -0.34375 2.46875l0 6.515625l-2.109375 0zm16.539062 -4.015625l2.171875 0.28125q-0.515625 1.90625 -1.90625 2.96875q-1.390625 1.046875 -3.5625 1.046875q-2.734375 0 -4.34375 -1.671875q-1.59375 -1.6875 -1.59375 -4.734375q0 -3.140625 1.609375 -4.875q1.625 -1.734375 4.203125 -1.734375q2.5 0 4.078125 1.703125q1.59375 1.703125 1.59375 4.78125q0 0.1875 -0.015625 0.5625l-9.28125 0q0.109375 2.046875 1.15625 3.140625q1.046875 1.09375 2.609375 1.09375q1.15625 0 1.96875 -0.609375q0.828125 -0.609375 1.3125 -1.953125zm-6.9375 -3.40625l6.953125 0q-0.140625 -1.5625 -0.796875 -2.359375q-1.0 -1.21875 -2.609375 -1.21875q-1.453125 0 -2.453125 0.984375q-0.984375 0.96875 -1.09375 2.59375z" fill-rule="nonzero"></path>
<path fill="#000000" fill-opacity="0.0" d="m117.11286 171.63255l138.4567 0l0 43.87401l-138.4567 0z" fill-rule="evenodd"></path><path fill="#000000" d="m148.55019 202.10956l0 -17.1875l2.28125 0l0 17.1875l-2.28125 0zm6.0117188 0l0 -12.453125l1.90625 0l0 1.78125q1.375 -2.0625 3.953125 -2.0625q1.125 0 2.0625 0.40625q0.953125 0.40625 1.421875 1.0625q0.46875 0.65625 0.65625 1.5625q0.125 0.578125 0.125 2.046875l0 7.65625l-2.109375 0l0 -7.578125q0 -1.28125 -0.25 -1.921875q-0.25 -0.640625 -0.875 -1.015625q-0.625 -0.390625 -1.46875 -0.390625q-1.34375 0 -2.328125 0.859375q-0.984375 0.859375 -0.984375 3.25l0 6.796875l-2.109375 0zm17.957031 -1.890625l0.3125 1.859375q-0.890625 0.203125 -1.59375 0.203125q-1.15625 0 -1.796875 -0.359375q-0.625 -0.375 -0.890625 -0.96875q-0.25 -0.59375 -0.25 -2.484375l0 -7.171875l-1.546875 0l0 -1.640625l1.546875 0l0 -3.078125l2.09375 -1.265625l0 4.34375l2.125 0l0 1.640625l-2.125 0l0 7.28125q0 0.90625 0.109375 1.171875q0.125 0.25 0.375 0.40625q0.25 0.140625 0.71875 0.140625q0.34375 0 0.921875 -0.078125zm10.589844 -2.125l2.171875 0.28125q-0.515625 1.90625 -1.90625 2.96875q-1.390625 1.046875 -3.5625 1.046875q-2.734375 0 -4.34375 -1.671875q-1.59375 -1.6875 -1.59375 -4.734375q0 -3.140625 1.609375 -4.875q1.625 -1.734375 4.203125 -1.734375q2.5 0 4.078125 1.703125q1.59375 1.703125 1.59375 4.78125q0 0.1875 -0.015625 0.5625l-9.28125 0q0.109375 2.046875 1.15625 3.140625q1.046875 1.09375 2.609375 1.09375q1.15625 0 1.96875 -0.609375q0.828125 -0.609375 1.3125 -1.953125zm-6.9375 -3.40625l6.953125 0q-0.140625 -1.5625 -0.796875 -2.359375q-1.0 -1.21875 -2.609375 -1.21875q-1.453125 0 -2.453125 0.984375q-0.984375 0.96875 -1.09375 2.59375zm11.738281 7.421875l0 -12.453125l1.890625 0l0 1.890625q0.734375 -1.328125 1.34375 -1.75q0.625 -0.421875 1.359375 -0.421875q1.0625 0 2.171875 0.6875l-0.734375 1.953125q-0.765625 -0.453125 -1.546875 -0.453125q-0.6875 0 -1.25 0.421875q-0.546875 0.40625 -0.78125 1.140625q-0.34375 1.125 -0.34375 2.46875l0 6.515625l-2.109375 0zm8.0078125 0l0 -12.453125l1.90625 0l0 1.78125q1.375 -2.0625 3.953125 -2.0625q1.125 0 2.0625 0.40625q0.953125 0.40625 1.421875 1.0625q0.46875 0.65625 0.65625 1.5625q0.125 0.578125 0.125 2.046875l0 7.65625l-2.109375 0l0 -7.578125q0 -1.28125 -0.25 -1.921875q-0.25 -0.640625 -0.875 -1.015625q-0.625 -0.390625 -1.46875 -0.390625q-1.34375 0 -2.328125 0.859375q-0.984375 0.859375 -0.984375 3.25l0 6.796875l-2.109375 0zm21.472656 -1.53125q-1.171875 0.984375 -2.265625 1.40625q-1.078125 0.40625 -2.3125 0.40625q-2.046875 0 -3.15625 -1.0q-1.09375 -1.0 -1.09375 -2.5625q0 -0.921875 0.40625 -1.671875q0.421875 -0.75 1.09375 -1.203125q0.671875 -0.46875 1.515625 -0.703125q0.625 -0.15625 1.875 -0.3125q2.5625 -0.3125 3.765625 -0.734375q0.015625 -0.421875 0.015625 -0.546875q0 -1.28125 -0.609375 -1.8125q-0.796875 -0.71875 -2.390625 -0.71875q-1.5 0 -2.203125 0.53125q-0.703125 0.515625 -1.046875 1.84375l-2.0625 -0.28125q0.28125 -1.328125 0.921875 -2.140625q0.640625 -0.8125 1.859375 -1.25q1.21875 -0.453125 2.828125 -0.453125q1.59375 0 2.59375 0.375q1.0 0.375 1.46875 0.953125q0.46875 0.5625 0.65625 1.4375q0.09375 0.53125 0.09375 1.9375l0 2.8125q0 2.9375 0.140625 3.71875q0.140625 0.78125 0.53125 1.5l-2.203125 0q-0.328125 -0.65625 -0.421875 -1.53125zm-0.171875 -4.71875q-1.15625 0.46875 -3.453125 0.796875q-1.296875 0.1875 -1.84375 0.421875q-0.53125 0.234375 -0.828125 0.6875q-0.28125 0.453125 -0.28125 1.0q0 0.84375 0.625 1.40625q0.640625 0.5625 1.875 0.5625q1.21875 0 2.171875 -0.53125q0.953125 -0.53125 1.390625 -1.453125q0.34375 -0.71875 0.34375 -2.109375l0 -0.78125zm5.3476562 6.25l0 -17.1875l2.109375 0l0 17.1875l-2.109375 0z" fill-rule="nonzero"></path>
<path fill="#000000" fill-opacity="0.0" d="m113.314964 5.141732l138.4567 0l0 43.874016l-138.4567 0z" fill-rule="evenodd"></path>
<path fill="#000000" d="m142.50034 35.61874l0 -17.1875l11.59375 0l0 2.03125l-9.3125 0l0 5.328125l8.0625 0l0 2.015625l-8.0625 0l0 7.8125l-2.28125 0zm22.425781 0l0 -1.828125q-1.453125 2.109375 -3.9375 2.109375q-1.109375 0 -2.0625 -0.421875q-0.953125 -0.421875 -1.421875 -1.0625q-0.453125 -0.640625 -0.640625 -1.5625q-0.140625 -0.625 -0.140625 -1.96875l0 -7.71875l2.109375 0l0 6.90625q0 1.65625 0.140625 2.234375q0.1875 0.828125 0.828125 1.3125q0.65625 0.46875 1.609375 0.46875q0.9375 0 1.765625 -0.484375q0.84375 -0.5 1.1875 -1.328125q0.34375 -0.84375 0.34375 -2.4375l0 -6.671875l2.109375 0l0 12.453125l-1.890625 0zm5.1445312 0l0 -17.1875l2.109375 0l0 17.1875l-2.109375 0zm5.3320312 0l0 -17.1875l2.109375 0l0 17.1875l-2.109375 0zm10.4375 0l6.59375 -17.1875l2.453125 0l7.03125 17.1875l-2.59375 0l-2.0 -5.203125l-7.1875 0l-1.890625 5.203125l-2.40625 0zm4.953125 -7.0625l5.828125 0l-1.796875 -4.75q-0.8125 -2.171875 -1.21875 -3.5625q-0.328125 1.65625 -0.921875 3.28125l-1.890625 5.03125zm12.9453125 7.0625l0 -17.1875l6.46875 0q1.71875 0 2.625 0.171875q1.265625 0.203125 2.109375 0.796875q0.859375 0.59375 1.375 1.671875q0.53125 1.0625 0.53125 2.328125q0 2.203125 -1.40625 3.71875q-1.390625 1.515625 -5.03125 1.515625l-4.40625 0l0 6.984375l-2.265625 0zm2.265625 -9.015625l4.4375 0q2.203125 0 3.125 -0.8125q0.9375 -0.828125 0.9375 -2.3125q0 -1.078125 -0.546875 -1.84375q-0.546875 -0.765625 -1.4375 -1.015625q-0.578125 -0.15625 -2.125 -0.15625l-4.390625 0l0 6.140625zm14.1171875 9.015625l0 -17.1875l2.28125 0l0 17.1875l-2.28125 0z" fill-rule="nonzero"></path>
<path fill="#000000" fill-opacity="0.0" d="m360.02625 147.12599l-347.3701 0" fill-rule="evenodd"></path>
<path stroke="#000000" stroke-width="8.0" stroke-linejoin="round" stroke-linecap="butt" d="m360.02625 147.12599l-347.3701 0" fill-rule="evenodd"></path>
<path fill="#000000" fill-opacity="0.0" d="m12.656168 50.304462l175.52756 0l0 68.755905l-175.52756 0z" fill-rule="evenodd"></path>
<path stroke="#000000" stroke-width="2.0" stroke-linejoin="round" stroke-linecap="butt" d="m12.656168 50.304462l175.52756 0l0 68.755905l-175.52756 0z" fill-rule="evenodd"></path>Locking Regime
The simplest approach to thread-safety is to surround all calls to API functions with a LOCK/UNLOCK protocol. This is how the HDF5 library operates, for example.
Our proposal is to implement locking using a single, global binary semaphore. This is extremely simple and is well-supported under all versions of ~nix~ (using libpthreads) as well as Windows (built-in).
One consequence of this decision is that there must be no recursive calls to locked functions. If it happens, it will cause a deadlock. This means specifically that core functions and internal functions cannot invoke core functions (directly or transitively).
An example of adding locking to a core function is shown in this example.
int nc_xxx(...) {
NC* ncp;
int stat = NC_NOERR;
LOCK();
if((stat=NC_check_id(ncid,&ncp)) != NC_NOERR) goto done;
stat = ncp->
dispatch->XXX(...);
done:
UNLOCK();
return stat;
} The done label is used to provide a single exit to ensure that UNLOCK is always invoked before exiting the function.
Note that we do not need to add locking to our class 1 (Auxiliary) functions since they just invoke a core function (class 2 or 3) that does the actual locking. Because of this, it will pay to try to convert as many API calls as possible to be auxiliary functions. Currently, there are a number of class 2/3 functions that could be converted with small effort by revising the set of core functions.
Note also that we assume that all internal functions will be invoked either by other internal functions or by core API functions that use a locking protocol. Hence these internal functions do not need to use a locking protocol. In fact, if they did, it could cause a deadlock.
Problem 1: Mostly Auxiliary Functions
It turns out that there are a few functions that are mostly auxiliary functions except that they invoke some internal functions to get information not available through the standard netcdfd-c API. One example is the _NCDEFAULT_getvars function. It invokes two internal functions:
- NC_is_recvar
- NC_getshape
The solution is to "expose" these internal functions in the core API by providing wrappers for them that use the locking regime. Using this approach, it should be possible to increase the number of auxiliary functions that do not need to directly use locking.
Note, that exposing these functions does not mean that they are part of the public netcdf-c library API; only that they are accessible to our external functions.
Problem 2: Internal Functions calling Core Functions
This is the big problem is implementing thread-safety. It turns out that some internal code invokes core API functions. This mostly occurs inside the libdap2 and libdap4 code. This is a problem because it violates the no recursive call rule and will lead to deadlock.
The simplest solution to this problem is to change all recursive calls from the internal code to the core API code to no longer call the core API. Instead, the direct calls can, in most cases, be changed to call directly into the dispatch layer. The cost is increased complexity in the internal code. To some degree, this complexity can be mitigated by using macros to hide the complexity. In a few cases, some extra internal functions may have to be introduced into the libdispatch code to make this change possible or to simplify the required changes.
Steps to Implementing Proposed Architecture
The key to implementing the proposed architecture is to slowly refactor the code in libdispatch to properly segregate the auxiliary functions from the core API from the internal code.
The following sequence of actions is proposed.
- Create two new files: libdispatch/daux.c and libdispatch/dapi.c.
- Move auxiliary functions into daux.c and the core api functions into dapi.c.
- Add extra functions in dapi.c to expose functions like _NCgetshape (see above).
- Move, where possible, code from dapi.c to daux.c using the exposed functions in #3.
- Identify the recursive calls in internal code. This can be accomplished by temporarily renaming the functions in dapi.c and dextend.c and then recompiling. That should flush out all such recursive calls.
- Convert the calls identified in #5 to call through the dispatcher instead.
- Add locking to dapi.c.
- Test and fix the resulting code to look for missed recursive calls.
Conclusion
Assuming the above approach is correct, then we should be able to make the netcdf-c library thread-safe with a straightforward, if tedious, sequences of changes.
Add new comment