cmultithreadinglinux-kernelatomicvolatile

READ_ONCE and WRITE_ONCE in Parallel programming


In the book "Is Parallel Programming Hard, And, If So,What Can You Do About It?", the author uses several macros that I don't understand what they actually do.

#define ACCESS_ONCE(x) (*(volatile typeof(x) *)&(x))

#define READ_ONCE(x) \
({ typeof(x) ___x = ACCESS_ONCE(x); ___x; })

#define WRITE_ONCE(x, val) \
do { ACCESS_ONCE(x) = (val); } while (0)

I don't understand what ACCESS_ONCE macro does and why it needs to cast and de-reference from and to an object of type volatile pointer.

and what is the usage of __x at the end of READ_ONCE macro?

In the following also there are some usages of these macros that (again) i don't understand.

Here is a list of situations allowing plain loads and stores for some accesses to a given variable, while requiring markings (such as READ_ONCE() and WRITE_ONCE()) for other accesses to that same variable:

  1. A shared variable is only modified by a given owning CPU or thread, but is read by other CPUs or threads. All stores must use WRITE_ONCE(). The owning CPU or thread may use plain loads. Everything else must use READ_ONCE() for loads.
  2. A shared variable is only modified while holding a given lock, but is read by code not holding that lock. All stores must use WRITE_ONCE(). CPUs or threads holding the lock may use plain loads. Everything else must use READ_ONCE() for loads.
  3. A shared variable is only modified while holding a given lock by a given owning CPU or thread, but is read by other CPUs or threads or by code not holding that lock. All stores must use WRITE_ONCE(). The owning CPU or thread may use plain loads, as may any CPU or thread holding the lock. Everything else must use READ_ONCE() for loads.
  4. A shared variable is only accessed by a given CPU or thread and by a signal or interrupt handler running in that CPU’s or thread’s context. The handler can use plain loads and stores, as can any code that has prevented the handler from being invoked, that is, code that has blocked signals and/or interrupts. All other code must use READ_ONCE() and WRITE_ONCE().
  5. A shared variable is only accessed by a given CPU or thread and by a signal or interrupt handler running in that CPU’s or thread’s context, and the handler always restores the values of any variables that it has written before return. The handler can use plain loads and stores, as can any code that has prevented the handler from being invoked, that is, code that has blocked signals and/or interrupts. All other code can use plain loads, but must use WRITE_ONCE() to prevent store tearing, store fusing, and invented stores.

First of all how can we use these macros to have simultaneous access to memory? AFAIK the volatile keyword is not safe for concurrent memory access.

In the item number 1, how can we use READ_ONCE and WRITE_ONCE to access a shared variable without data race?

And in item number 2, why does he use WRITE_ONCE macro when write is only allowed by holding lock. and why doesn't read need to hold the lock?


Solution

  • These macros are ways to enforce some level of atomicy (but no synchronization) on supporting compilers (GCC, maybe some others). They are used heavily inside Linux as it predates C11 by a huge margin.

    In GCC semantics, volatile results in emitting exactly one instruction accessing the pointed-to value (at least if that value is word-sized). On all architectures Linux supports, aligned word-sized accesses are atomic so the overall construction results in one single atomic access. (I mean machine word ofc, not the WORD type on some well-known platform).

    To my knowledge, that is equivalent to using C++ atomic with memory_order_relaxed (as was pointed in comments) with the exception that those require every access to be atomic while that’s not actually required in some patterns (like, if only one thread writes to a variable, its own reads don’t need to be atomic, and atomic read-write operations are definitely unnecessary in such case).