2 .\" Copyright 2015-2017 Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 .\" SPDX-License-Identifier: Linux-man-pages-copyleft
6 .TH membarrier 2 (date) "Linux man-pages (unreleased)"
8 membarrier \- issue memory barriers on a set of threads
11 .RI ( libc ", " \-lc )
15 .BR "#include <linux/membarrier.h>" \
16 " /* Definition of " MEMBARRIER_* " constants */"
17 .BR "#include <sys/syscall.h>" " /* Definition of " SYS_* " constants */"
18 .B #include <unistd.h>
20 .BI "int syscall(SYS_membarrier, int " cmd ", unsigned int " flags \
25 glibc provides no wrapper for
27 necessitating the use of
32 system call helps reducing the overhead of the memory barrier
33 instructions required to order memory accesses on multi-core systems.
34 However, this system call is heavier than a memory barrier, so using it
37 as simple as replacing memory barriers with this
38 system call, but requires understanding of the details below.
40 Use of memory barriers needs to be done taking into account that a
41 memory barrier always needs to be either matched with its memory barrier
42 counterparts, or that the architecture's memory model doesn't require the
45 There are cases where one side of the matching barriers (which we will
46 refer to as "fast side") is executed much more often than the other
47 (which we will refer to as "slow side").
48 This is a prime target for the use of
50 The key idea is to replace, for these matching
51 barriers, the fast-side memory barriers by simple compiler barriers,
56 asm volatile ("" : : : "memory")
60 and replace the slow-side memory barriers by calls to
63 This will add overhead to the slow side, and remove overhead from the
64 fast side, thus resulting in an overall performance increase as long as
65 the slow side is infrequent enough that the overhead of the
67 calls does not outweigh the performance gain on the fast side.
71 argument is one of the following:
73 .BR MEMBARRIER_CMD_QUERY " (since Linux 4.3)"
74 Query the set of supported commands.
75 The return value of the call is a bit mask of supported
77 .BR MEMBARRIER_CMD_QUERY ,
78 which has the value 0,
79 is not itself included in this bit mask.
80 This command is always supported (on kernels where
84 .BR MEMBARRIER_CMD_GLOBAL " (since Linux 4.16)"
85 Ensure that all threads from all processes on the system pass through a
86 state where all memory accesses to user-space addresses match program
87 order between entry to and return from the
90 All threads on the system are targeted by this command.
92 .BR MEMBARRIER_CMD_GLOBAL_EXPEDITED " (since Linux 4.16)"
93 Execute a memory barrier on all running threads of all processes that
94 previously registered with
95 .BR MEMBARRIER_CMD_REGISTER_GLOBAL_EXPEDITED .
97 Upon return from the system call, the calling thread has a guarantee that all
98 running threads have passed through a state where all memory accesses to
99 user-space addresses match program order between entry to and return
100 from the system call (non-running threads are de facto in such a state).
101 This guarantee is provided only for the threads of processes that
102 previously registered with
103 .BR MEMBARRIER_CMD_REGISTER_GLOBAL_EXPEDITED .
105 Given that registration is about the intent to receive the barriers, it
107 .B MEMBARRIER_CMD_GLOBAL_EXPEDITED
108 from a process that has not employed
109 .BR MEMBARRIER_CMD_REGISTER_GLOBAL_EXPEDITED .
111 The "expedited" commands complete faster than the non-expedited ones;
112 they never block, but have the downside of causing extra overhead.
114 .BR MEMBARRIER_CMD_REGISTER_GLOBAL_EXPEDITED " (since Linux 4.16)"
115 Register the process's intent to receive
116 .B MEMBARRIER_CMD_GLOBAL_EXPEDITED
119 .BR MEMBARRIER_CMD_PRIVATE_EXPEDITED " (since Linux 4.14)"
120 Execute a memory barrier on each running thread belonging to the same
121 process as the calling thread.
123 Upon return from the system call, the calling
124 thread has a guarantee that all its running thread siblings have passed
125 through a state where all memory accesses to user-space addresses match
126 program order between entry to and return from the system call
127 (non-running threads are de facto in such a state).
128 This guarantee is provided only for threads in
129 the same process as the calling thread.
131 The "expedited" commands complete faster than the non-expedited ones;
132 they never block, but have the downside of causing extra overhead.
134 A process must register its intent to use the private
135 expedited command prior to using it.
137 .BR MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED " (since Linux 4.14)"
138 Register the process's intent to use
139 .BR MEMBARRIER_CMD_PRIVATE_EXPEDITED .
141 .BR MEMBARRIER_CMD_PRIVATE_EXPEDITED_SYNC_CORE " (since Linux 4.16)"
142 In addition to providing the memory ordering guarantees described in
143 .BR MEMBARRIER_CMD_PRIVATE_EXPEDITED ,
144 upon return from system call the calling thread has a guarantee that all its
145 running thread siblings have executed a core serializing instruction.
146 This guarantee is provided only for threads in
147 the same process as the calling thread.
149 The "expedited" commands complete faster than the non-expedited ones,
150 they never block, but have the downside of causing extra overhead.
152 A process must register its intent to use the private expedited sync
153 core command prior to using it.
155 .BR MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED_SYNC_CORE " (since Linux 4.16)"
156 Register the process's intent to use
157 .BR MEMBARRIER_CMD_PRIVATE_EXPEDITED_SYNC_CORE .
159 .BR MEMBARRIER_CMD_PRIVATE_EXPEDITED_RSEQ " (since Linux 5.10)"
160 Ensure the caller thread, upon return from system call, that all its
161 running thread siblings have any currently running rseq critical sections
167 .BR MEMBARRIER_CMD_FLAG_CPU ,
168 then this operation is performed only on CPU indicated by
170 This guarantee is provided only for threads in
171 the same process as the calling thread.
173 RSEQ membarrier is only available in the "private expedited" form.
175 A process must register its intent to use the private expedited rseq
176 command prior to using it.
178 .BR MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED_RSEQ " (since Linux 5.10)"
179 Register the process's intent to use
180 .BR MEMBARRIER_CMD_PRIVATE_EXPEDITED_RSEQ .
182 .BR MEMBARRIER_CMD_SHARED " (since Linux 4.3)"
184 .B MEMBARRIER_CMD_GLOBAL
185 that exists for header backward compatibility.
189 argument must be specified as 0 unless the command is
190 .BR MEMBARRIER_CMD_PRIVATE_EXPEDITED_RSEQ ,
194 .BR MEMBARRIER_CMD_FLAG_CPU .
198 argument is ignored unless
201 .BR MEMBARRIER_CMD_FLAG_CPU ,
202 in which case it must specify the CPU targeted by this membarrier
205 All memory accesses performed in program order from each targeted thread
206 are guaranteed to be ordered with respect to
209 If we use the semantic
211 to represent a compiler barrier forcing memory
212 accesses to be performed in program order across the barrier, and
214 to represent explicit memory barriers forcing full memory
215 ordering across the barrier, we have the following ordering table for
221 The pair ordering is detailed as (O: ordered, X: not ordered):
226 \& barrier() smp_mb() membarrier()
234 .B MEMBARRIER_CMD_QUERY
235 operation returns a bit mask of supported commands, and the
236 .BR MEMBARRIER_CMD_GLOBAL ,
237 .BR MEMBARRIER_CMD_GLOBAL_EXPEDITED ,
238 .BR MEMBARRIER_CMD_REGISTER_GLOBAL_EXPEDITED ,
239 .BR MEMBARRIER_CMD_PRIVATE_EXPEDITED ,
240 .BR MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED ,
241 .BR MEMBARRIER_CMD_PRIVATE_EXPEDITED_SYNC_CORE ,
243 .B MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED_SYNC_CORE
244 operations return zero.
245 On error, \-1 is returned,
248 is set to indicate the error.
250 For a given command, with
252 set to 0, this system call is
253 guaranteed to always return the same value until reboot.
254 Further calls with the same arguments will lead to the same result.
257 set to 0, error handling is required only for the first call to
266 .B MEMBARRIER_CMD_GLOBAL
267 command is disabled because the
269 CPU parameter has been set, or the
270 .B MEMBARRIER_CMD_PRIVATE_EXPEDITED_SYNC_CORE
272 .B MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED_SYNC_CORE
273 commands are not implemented by the architecture.
278 system call is not implemented by this kernel.
281 The current process was not registered prior to using private expedited
288 Before Linux 5.10, the prototype was:
292 .BI "int membarrier(int " cmd ", int " flags );
296 A memory barrier instruction is part of the instruction set of
297 architectures with weakly ordered memory models.
299 accesses prior to the barrier and after the barrier with respect to
300 matching barriers on other cores.
301 For instance, a load fence can order
302 loads prior to and following that fence with respect to stores ordered
305 Program order is the order in which instructions are ordered in the
306 program assembly code.
310 can be useful include implementations
311 of Read-Copy-Update libraries and garbage collectors.
313 Assuming a multithreaded application where "fast_path()" is executed
314 very frequently, and where "slow_path()" is executed infrequently, the
315 following code (x86) can be transformed using
319 .\" SRC BEGIN (membarrier.c)
323 static volatile int a, b;
326 fast_path(int *read_b)
329 asm volatile ("mfence" : : : "memory");
334 slow_path(int *read_a)
337 asm volatile ("mfence" : : : "memory");
347 * Real applications would call fast_path() and slow_path()
348 * from different threads. Call those from main() to keep
349 * this example short.
356 * read_b == 0 implies read_a == 1 and
357 * read_a == 0 implies read_b == 1.
360 if (read_b == 0 && read_a == 0)
369 The code above transformed to use
379 #include <sys/syscall.h>
380 #include <linux/membarrier.h>
382 static volatile int a, b;
385 membarrier(int cmd, unsigned int flags, int cpu_id)
387 return syscall(__NR_membarrier, cmd, flags, cpu_id);
391 init_membarrier(void)
395 /* Check that membarrier() is supported. */
397 ret = membarrier(MEMBARRIER_CMD_QUERY, 0, 0);
399 perror("membarrier");
403 if (!(ret & MEMBARRIER_CMD_GLOBAL)) {
405 "membarrier does not support MEMBARRIER_CMD_GLOBAL\en");
413 fast_path(int *read_b)
416 asm volatile ("" : : : "memory");
421 slow_path(int *read_a)
424 membarrier(MEMBARRIER_CMD_GLOBAL, 0, 0);
429 main(int argc, char *argv[])
433 if (init_membarrier())
437 * Real applications would call fast_path() and slow_path()
438 * from different threads. Call those from main() to keep
439 * this example short.
446 * read_b == 0 implies read_a == 1 and
447 * read_a == 0 implies read_b == 1.
450 if (read_b == 0 && read_a == 0)
458 .\" FIXME See if the following syscalls make it into Linux 4.15 or later