1 \input texinfo @c -*-texinfo-*-
4 @setfilename libgomp.info
10 Copyright @copyright{} 2006-2021 Free Software Foundation, Inc.
12 Permission is granted to copy, distribute and/or modify this document
13 under the terms of the GNU Free Documentation License, Version 1.3 or
14 any later version published by the Free Software Foundation; with the
15 Invariant Sections being ``Funding Free Software'', the Front-Cover
16 texts being (a) (see below), and with the Back-Cover Texts being (b)
17 (see below). A copy of the license is included in the section entitled
18 ``GNU Free Documentation License''.
20 (a) The FSF's Front-Cover Text is:
24 (b) The FSF's Back-Cover Text is:
26 You have freedom to copy and modify this GNU Manual, like GNU
27 software. Copies published by the Free Software Foundation raise
28 funds for GNU development.
32 @dircategory GNU Libraries
34 * libgomp: (libgomp). GNU Offloading and Multi Processing Runtime Library.
37 This manual documents libgomp, the GNU Offloading and Multi Processing
38 Runtime library. This is the GNU implementation of the OpenMP and
39 OpenACC APIs for parallel and accelerator programming in C/C++ and
42 Published by the Free Software Foundation
43 51 Franklin Street, Fifth Floor
44 Boston, MA 02110-1301 USA
50 @setchapternewpage odd
53 @title GNU Offloading and Multi Processing Runtime Library
54 @subtitle The GNU OpenMP and OpenACC Implementation
56 @vskip 0pt plus 1filll
57 @comment For the @value{version-GCC} Version*
59 Published by the Free Software Foundation @*
60 51 Franklin Street, Fifth Floor@*
61 Boston, MA 02110-1301, USA@*
71 @node Top, Enabling OpenMP
75 This manual documents the usage of libgomp, the GNU Offloading and
76 Multi Processing Runtime Library. This includes the GNU
77 implementation of the @uref{https://www.openmp.org, OpenMP} Application
78 Programming Interface (API) for multi-platform shared-memory parallel
79 programming in C/C++ and Fortran, and the GNU implementation of the
80 @uref{https://www.openacc.org, OpenACC} Application Programming
81 Interface (API) for offloading of code to accelerator devices in C/C++
84 Originally, libgomp implemented the GNU OpenMP Runtime Library. Based
85 on this, support for OpenACC and offloading (both OpenACC and OpenMP
86 4's target construct) has been added later on, and the library's name
87 changed to GNU Offloading and Multi Processing Runtime Library.
92 @comment When you add a new menu item, please keep the right hand
93 @comment aligned to the same column. Do not use tabs. This provides
94 @comment better formatting.
97 * Enabling OpenMP:: How to enable OpenMP for your applications.
98 * OpenMP Implementation Status:: List of implemented features by OpenMP version
99 * OpenMP Runtime Library Routines: Runtime Library Routines.
100 The OpenMP runtime application programming
102 * OpenMP Environment Variables: Environment Variables.
103 Influencing OpenMP runtime behavior with
104 environment variables.
105 * Enabling OpenACC:: How to enable OpenACC for your
107 * OpenACC Runtime Library Routines:: The OpenACC runtime application
108 programming interface.
109 * OpenACC Environment Variables:: Influencing OpenACC runtime behavior with
110 environment variables.
111 * CUDA Streams Usage:: Notes on the implementation of
112 asynchronous operations.
113 * OpenACC Library Interoperability:: OpenACC library interoperability with the
114 NVIDIA CUBLAS library.
115 * OpenACC Profiling Interface::
116 * The libgomp ABI:: Notes on the external ABI presented by libgomp.
117 * Reporting Bugs:: How to report bugs in the GNU Offloading and
118 Multi Processing Runtime Library.
119 * Copying:: GNU general public license says
120 how you can copy and share libgomp.
121 * GNU Free Documentation License::
122 How you can copy and share this manual.
123 * Funding:: How to help assure continued work for free
125 * Library Index:: Index of this documentation.
129 @c ---------------------------------------------------------------------
131 @c ---------------------------------------------------------------------
133 @node Enabling OpenMP
134 @chapter Enabling OpenMP
136 To activate the OpenMP extensions for C/C++ and Fortran, the compile-time
137 flag @command{-fopenmp} must be specified. This enables the OpenMP directive
138 @code{#pragma omp} in C/C++ and @code{!$omp} directives in free form,
139 @code{c$omp}, @code{*$omp} and @code{!$omp} directives in fixed form,
140 @code{!$} conditional compilation sentinels in free form and @code{c$},
141 @code{*$} and @code{!$} sentinels in fixed form, for Fortran. The flag also
142 arranges for automatic linking of the OpenMP runtime library
143 (@ref{Runtime Library Routines}).
145 A complete description of all OpenMP directives may be found in the
146 @uref{https://www.openmp.org, OpenMP Application Program Interface} manuals.
147 See also @ref{OpenMP Implementation Status}.
150 @c ---------------------------------------------------------------------
151 @c OpenMP Implementation Status
152 @c ---------------------------------------------------------------------
154 @node OpenMP Implementation Status
155 @chapter OpenMP Implementation Status
158 * OpenMP 4.5:: Feature completion status to 4.5 specification
159 * OpenMP 5.0:: Feature completion status to 5.0 specification
160 * OpenMP 5.1:: Feature completion status to 5.1 specification
163 The @code{_OPENMP} preprocessor macro and Fortran's @code{openmp_version}
164 parameter, provided by @code{omp_lib.h} and the @code{omp_lib} module, have
165 the value @code{201511} (i.e. OpenMP 4.5).
170 The OpenMP 4.5 specification is fully supported.
175 @unnumberedsubsec New features listed in Appendix B of the OpenMP specification
176 @c This list is sorted as in OpenMP 5.1's B.3 not as in OpenMP 5.0's B.2
178 @multitable @columnfractions .60 .10 .25
179 @headitem Description @tab Status @tab Comments
180 @item Array shaping @tab N @tab
181 @item Array sections with non-unit strides in C and C++ @tab N @tab
182 @item Iterators @tab Y @tab
183 @item @code{metadirective} directive @tab N @tab
184 @item @code{declare variant} directive
185 @tab P @tab simd traits not handled correctly
186 @item @emph{target-offload-var} ICV and @code{OMP_TARGET_OFFLOAD}
187 env variable @tab Y @tab
188 @item Nested-parallel changes to @emph{max-active-levels-var} ICV @tab Y @tab
189 @item @code{requires} directive @tab P
190 @tab Only fulfillable requirement is @code{atomic_default_mem_order}
191 @item @code{teams} construct outside an enclosing target region @tab Y @tab
192 @item Non-rectangular loop nests @tab P @tab Only C/C++
193 @item @code{!=} as relational-op in canonical loop form for C/C++ @tab Y @tab
194 @item @code{nonmonotonic} as default loop schedule modifier for worksharing-loop
195 constructs @tab Y @tab
196 @item Collapse of associated loops that are imperfectly nested loops @tab N @tab
197 @item Clauses @code{if}, @code{nontemporal} and @code{order(concurrent)} in
198 @code{simd} construct @tab Y @tab
199 @item @code{atomic} constructs in @code{simd} @tab Y @tab
200 @item @code{loop} construct @tab Y @tab
201 @item @code{order(concurrent)} clause @tab Y @tab
202 @item @code{scan} directive and @code{in_scan} modifier for the
203 @code{reduction} clause @tab Y @tab
204 @item @code{in_reduction} clause on @code{task} constructs @tab Y @tab
205 @item @code{in_reduction} clause on @code{target} constructs @tab P
206 @tab Only C/C++, @code{nowait} only stub
207 @item @code{task_reduction} clause with @code{taskgroup} @tab Y @tab
208 @item @code{task} modifier to @code{reduction} clause @tab Y @tab
209 @item @code{affinity} clause to @code{task} construct @tab Y @tab Stub only
210 @item @code{detach} clause to @code{task} construct @tab Y @tab
211 @item @code{omp_fulfill_event} runtime routine @tab Y @tab
212 @item @code{reduction} and @code{in_reduction} clauses on @code{taskloop}
213 and @code{taskloop simd} constructs @tab Y @tab
214 @item @code{taskloop} construct cancelable by @code{cancel} construct
216 @item @code{mutexinouset} @emph{dependence-type} for @code{depend} clause
218 @item Predefined memory spaces, memory allocators, allocator traits
219 @tab Y @tab Some are only stubs
220 @item Memory management routines @tab Y @tab
221 @item @code{allocate} directive @tab N @tab
222 @item @code{allocate} clause @tab P @tab initial support in C/C++ only
223 @item @code{use_device_addr} clause on @code{target data} @tab Y @tab
224 @item @code{ancestor} modifier on @code{device} clause
225 @tab P @tab Reverse offload unsupported
226 @item Implicit declare target directive @tab Y @tab
227 @item Discontiguous array section with @code{target update} construct
229 @item C/C++'s lvalue expressions in @code{to}, @code{from}
230 and @code{map} clauses @tab N @tab
231 @item C/C++'s lvalue expressions in @code{depend} clauses @tab Y @tab
232 @item Nested @code{declare target} directive @tab Y @tab
233 @item Combined @code{master} constructs @tab Y @tab
234 @item @code{depend} clause on @code{taskwait} @tab Y @tab
235 @item Weak memory ordering clauses on @code{atomic} and @code{flush} construct
237 @item @code{hint} clause on the @code{atomic} construct @tab Y @tab Stub only
238 @item @code{depobj} construct and depend objects @tab Y @tab
239 @item Lock hints were renamed to synchronization hints @tab Y @tab
240 @item @code{conditional} modifier to @code{lastprivate} clause @tab Y @tab
241 @item Map-order clarifications @tab P @tab
242 @item @code{close} @emph{map-type-modifier} @tab Y @tab
243 @item Mapping C/C++ pointer variables and to assign the address of
244 device memory mapped by an array section @tab P @tab
245 @item Mapping of Fortran pointer and allocatable variables, including pointer
246 and allocatable components of variables
247 @tab P @tab Mapping of vars with allocatable components unspported
248 @item @code{defaultmap} extensions @tab Y @tab
249 @item @code{declare mapper} directive @tab N @tab
250 @item @code{omp_get_supported_active_levels} routine @tab Y @tab
251 @item Runtime routines and environment variables to display runtime thread
252 affinity information @tab Y @tab
253 @item @code{omp_pause_resource} and @code{omp_pause_resource_all} runtime
255 @item @code{omp_get_device_num} runtime routine @tab Y @tab
256 @item OMPT interface @tab N @tab
257 @item OMPD interface @tab N @tab
260 @unnumberedsubsec Other new OpenMP 5.0 features
262 @multitable @columnfractions .60 .10 .25
263 @headitem Description @tab Status @tab Comments
264 @item Supporting C++'s range-based for loop @tab Y @tab
271 @unnumberedsubsec New features listed in Appendix B of the OpenMP specification
273 @multitable @columnfractions .60 .10 .25
274 @headitem Description @tab Status @tab Comments
275 @item OpenMP directive as C++ attribute specifiers @tab Y @tab
276 @item @code{omp_all_memory} reserved locator @tab N @tab
277 @item @emph{target_device trait} in OpenMP Context @tab N @tab
278 @item @code{target_device} selector set in context selectors @tab N @tab
279 @item C/C++'s @code{declare variant} directive: elision support of
280 preprocessed code @tab N @tab
281 @item @code{declare variant}: new clauses @code{adjust_args} and
282 @code{append_args} @tab N @tab
283 @item @code{dispatch} construct @tab N @tab
284 @item device-specific ICV settings the environment variables @tab N @tab
285 @item assume directive @tab N @tab
286 @item @code{nothing} directive @tab Y @tab
287 @item @code{error} directive @tab Y @tab
288 @item @code{masked} construct @tab Y @tab
289 @item @code{scope} directive @tab Y @tab
290 @item Loop transformation constructs @tab N @tab
291 @item @code{strict} modifier in the @code{grainsize} and @code{num_tasks}
292 clauses of the taskloop construct @tab Y @tab
293 @item @code{align} clause/modifier in @code{allocate} directive/clause
294 and @code{allocator} directive @tab P @tab C/C++ on clause only
295 @item @code{thread_limit} clause to @code{target} construct @tab Y @tab
296 @item @code{has_device_addr} clause to @code{target} construct @tab N @tab
297 @item iterators in @code{target update} motion clauses and @code{map}
299 @item indirect calls to the device version of a procedure or function in
300 @code{target} regions @tab N @tab
301 @item @code{interop} directive @tab N @tab
302 @item @code{omp_interop_t} object support in runtime routines @tab N @tab
303 @item @code{nowait} clause in @code{taskwait} directive @tab N @tab
304 @item Extensions to the @code{atomic} directive @tab Y @tab
305 @item @code{seq_cst} clause on a @code{flush} construct @tab Y @tab
306 @item @code{inoutset} argument to the @code{depend} clause @tab N @tab
307 @item @code{private} and @code{firstprivate} argument to @code{default}
308 clause in C and C++ @tab Y @tab
309 @item @code{present} argument to @code{defaultmap} clause @tab N @tab
310 @item @code{omp_set_num_teams}, @code{omp_set_teams_thread_limit},
311 @code{omp_get_max_teams}, @code{omp_get_teams_thread_limit} runtime
313 @item @code{omp_target_is_accessible} runtime routine @tab N @tab
314 @item @code{omp_target_memcpy_async} and @code{omp_target_memcpy_rect_async}
315 runtime routines @tab N @tab
316 @item @code{omp_get_mapped_ptr} runtime routine @tab N @tab
317 @item @code{omp_calloc}, @code{omp_realloc}, @code{omp_aligned_alloc} and
318 @code{omp_aligned_calloc} runtime routines @tab Y @tab
319 @item @code{omp_alloctrait_key_t} enum: @code{omp_atv_serialized} added,
320 @code{omp_atv_default} changed @tab Y @tab
321 @item @code{omp_display_env} runtime routine @tab P
322 @tab Not inside @code{target} regions
323 @item @code{ompt_scope_endpoint_t} enum: @code{ompt_scope_beginend} @tab N @tab
324 @item @code{ompt_sync_region_t} enum additions @tab N @tab
325 @item @code{ompt_state_t} enum: @code{ompt_state_wait_barrier_implementation}
326 and @code{ompt_state_wait_barrier_teams} @tab N @tab
327 @item @code{ompt_callback_target_data_op_emi_t},
328 @code{ompt_callback_target_emi_t}, @code{ompt_callback_target_map_emi_t}
329 and @code{ompt_callback_target_submit_emi_t} @tab N @tab
330 @item @code{ompt_callback_error_t} type @tab N @tab
331 @item @code{OMP_PLACES} syntax extensions @tab Y @tab
332 @item @code{OMP_NUM_TEAMS} and @code{OMP_TEAMS_THREAD_LIMIT} environment
333 variables @tab Y @tab
336 @unnumberedsubsec Other new OpenMP 5.1 features
338 @multitable @columnfractions .60 .10 .25
339 @headitem Description @tab Status @tab Comments
340 @item Support of strictly structured blocks in Fortran @tab Y @tab
341 @item Support of structured block sequences in C/C++ @tab Y @tab
342 @item @code{unconstrained} and @code{reproducible} modifiers on @code{order}
347 @c ---------------------------------------------------------------------
348 @c OpenMP Runtime Library Routines
349 @c ---------------------------------------------------------------------
351 @node Runtime Library Routines
352 @chapter OpenMP Runtime Library Routines
354 The runtime routines described here are defined by Section 3 of the OpenMP
355 specification in version 4.5. The routines are structured in following
359 Control threads, processors and the parallel environment. They have C
360 linkage, and do not throw exceptions.
362 * omp_get_active_level:: Number of active parallel regions
363 * omp_get_ancestor_thread_num:: Ancestor thread ID
364 * omp_get_cancellation:: Whether cancellation support is enabled
365 * omp_get_default_device:: Get the default device for target regions
366 * omp_get_device_num:: Get device that current thread is running on
367 * omp_get_dynamic:: Dynamic teams setting
368 * omp_get_initial_device:: Device number of host device
369 * omp_get_level:: Number of parallel regions
370 * omp_get_max_active_levels:: Current maximum number of active regions
371 * omp_get_max_task_priority:: Maximum task priority value that can be set
372 * omp_get_max_teams:: Maximum number of teams for teams region
373 * omp_get_max_threads:: Maximum number of threads of parallel region
374 * omp_get_nested:: Nested parallel regions
375 * omp_get_num_devices:: Number of target devices
376 * omp_get_num_procs:: Number of processors online
377 * omp_get_num_teams:: Number of teams
378 * omp_get_num_threads:: Size of the active team
379 * omp_get_proc_bind:: Whether theads may be moved between CPUs
380 * omp_get_schedule:: Obtain the runtime scheduling method
381 * omp_get_supported_active_levels:: Maximum number of active regions supported
382 * omp_get_team_num:: Get team number
383 * omp_get_team_size:: Number of threads in a team
384 * omp_get_teams_thread_limit:: Maximum number of threads imposed by teams
385 * omp_get_thread_limit:: Maximum number of threads
386 * omp_get_thread_num:: Current thread ID
387 * omp_in_parallel:: Whether a parallel region is active
388 * omp_in_final:: Whether in final or included task region
389 * omp_is_initial_device:: Whether executing on the host device
390 * omp_set_default_device:: Set the default device for target regions
391 * omp_set_dynamic:: Enable/disable dynamic teams
392 * omp_set_max_active_levels:: Limits the number of active parallel regions
393 * omp_set_nested:: Enable/disable nested parallel regions
394 * omp_set_num_teams:: Set upper teams limit for teams region
395 * omp_set_num_threads:: Set upper team size limit
396 * omp_set_schedule:: Set the runtime scheduling method
397 * omp_set_teams_thread_limit:: Set upper thread limit for teams construct
399 Initialize, set, test, unset and destroy simple and nested locks.
401 * omp_init_lock:: Initialize simple lock
402 * omp_set_lock:: Wait for and set simple lock
403 * omp_test_lock:: Test and set simple lock if available
404 * omp_unset_lock:: Unset simple lock
405 * omp_destroy_lock:: Destroy simple lock
406 * omp_init_nest_lock:: Initialize nested lock
407 * omp_set_nest_lock:: Wait for and set simple lock
408 * omp_test_nest_lock:: Test and set nested lock if available
409 * omp_unset_nest_lock:: Unset nested lock
410 * omp_destroy_nest_lock:: Destroy nested lock
412 Portable, thread-based, wall clock timer.
414 * omp_get_wtick:: Get timer precision.
415 * omp_get_wtime:: Elapsed wall clock time.
417 Support for event objects.
419 * omp_fulfill_event:: Fulfill and destroy an OpenMP event.
424 @node omp_get_active_level
425 @section @code{omp_get_active_level} -- Number of parallel regions
427 @item @emph{Description}:
428 This function returns the nesting level for the active parallel blocks,
429 which enclose the calling call.
432 @multitable @columnfractions .20 .80
433 @item @emph{Prototype}: @tab @code{int omp_get_active_level(void);}
436 @item @emph{Fortran}:
437 @multitable @columnfractions .20 .80
438 @item @emph{Interface}: @tab @code{integer function omp_get_active_level()}
441 @item @emph{See also}:
442 @ref{omp_get_level}, @ref{omp_get_max_active_levels}, @ref{omp_set_max_active_levels}
444 @item @emph{Reference}:
445 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.20.
450 @node omp_get_ancestor_thread_num
451 @section @code{omp_get_ancestor_thread_num} -- Ancestor thread ID
453 @item @emph{Description}:
454 This function returns the thread identification number for the given
455 nesting level of the current thread. For values of @var{level} outside
456 zero to @code{omp_get_level} -1 is returned; if @var{level} is
457 @code{omp_get_level} the result is identical to @code{omp_get_thread_num}.
460 @multitable @columnfractions .20 .80
461 @item @emph{Prototype}: @tab @code{int omp_get_ancestor_thread_num(int level);}
464 @item @emph{Fortran}:
465 @multitable @columnfractions .20 .80
466 @item @emph{Interface}: @tab @code{integer function omp_get_ancestor_thread_num(level)}
467 @item @tab @code{integer level}
470 @item @emph{See also}:
471 @ref{omp_get_level}, @ref{omp_get_thread_num}, @ref{omp_get_team_size}
473 @item @emph{Reference}:
474 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.18.
479 @node omp_get_cancellation
480 @section @code{omp_get_cancellation} -- Whether cancellation support is enabled
482 @item @emph{Description}:
483 This function returns @code{true} if cancellation is activated, @code{false}
484 otherwise. Here, @code{true} and @code{false} represent their language-specific
485 counterparts. Unless @env{OMP_CANCELLATION} is set true, cancellations are
489 @multitable @columnfractions .20 .80
490 @item @emph{Prototype}: @tab @code{int omp_get_cancellation(void);}
493 @item @emph{Fortran}:
494 @multitable @columnfractions .20 .80
495 @item @emph{Interface}: @tab @code{logical function omp_get_cancellation()}
498 @item @emph{See also}:
499 @ref{OMP_CANCELLATION}
501 @item @emph{Reference}:
502 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.9.
507 @node omp_get_default_device
508 @section @code{omp_get_default_device} -- Get the default device for target regions
510 @item @emph{Description}:
511 Get the default device for target regions without device clause.
514 @multitable @columnfractions .20 .80
515 @item @emph{Prototype}: @tab @code{int omp_get_default_device(void);}
518 @item @emph{Fortran}:
519 @multitable @columnfractions .20 .80
520 @item @emph{Interface}: @tab @code{integer function omp_get_default_device()}
523 @item @emph{See also}:
524 @ref{OMP_DEFAULT_DEVICE}, @ref{omp_set_default_device}
526 @item @emph{Reference}:
527 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.30.
532 @node omp_get_device_num
533 @section @code{omp_get_device_num} -- Return device number of current device
535 @item @emph{Description}:
536 This function returns a device number that represents the device that the
537 current thread is executing on. For OpenMP 5.0, this must be equal to the
538 value returned by the @code{omp_get_initial_device} function when called
542 @multitable @columnfractions .20 .80
543 @item @emph{Prototype}: @tab @code{int omp_get_device_num(void);}
546 @item @emph{Fortran}:
547 @multitable @columnfractions .20 .80
548 @item @emph{Interface}: @tab @code{integer function omp_get_device_num()}
551 @item @emph{See also}:
552 @ref{omp_get_initial_device}
554 @item @emph{Reference}:
555 @uref{https://www.openmp.org, OpenMP specification v5.0}, Section 3.2.37.
560 @node omp_get_dynamic
561 @section @code{omp_get_dynamic} -- Dynamic teams setting
563 @item @emph{Description}:
564 This function returns @code{true} if enabled, @code{false} otherwise.
565 Here, @code{true} and @code{false} represent their language-specific
568 The dynamic team setting may be initialized at startup by the
569 @env{OMP_DYNAMIC} environment variable or at runtime using
570 @code{omp_set_dynamic}. If undefined, dynamic adjustment is
574 @multitable @columnfractions .20 .80
575 @item @emph{Prototype}: @tab @code{int omp_get_dynamic(void);}
578 @item @emph{Fortran}:
579 @multitable @columnfractions .20 .80
580 @item @emph{Interface}: @tab @code{logical function omp_get_dynamic()}
583 @item @emph{See also}:
584 @ref{omp_set_dynamic}, @ref{OMP_DYNAMIC}
586 @item @emph{Reference}:
587 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.8.
592 @node omp_get_initial_device
593 @section @code{omp_get_initial_device} -- Return device number of initial device
595 @item @emph{Description}:
596 This function returns a device number that represents the host device.
597 For OpenMP 5.1, this must be equal to the value returned by the
598 @code{omp_get_num_devices} function.
601 @multitable @columnfractions .20 .80
602 @item @emph{Prototype}: @tab @code{int omp_get_initial_device(void);}
605 @item @emph{Fortran}:
606 @multitable @columnfractions .20 .80
607 @item @emph{Interface}: @tab @code{integer function omp_get_initial_device()}
610 @item @emph{See also}:
611 @ref{omp_get_num_devices}
613 @item @emph{Reference}:
614 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.35.
620 @section @code{omp_get_level} -- Obtain the current nesting level
622 @item @emph{Description}:
623 This function returns the nesting level for the parallel blocks,
624 which enclose the calling call.
627 @multitable @columnfractions .20 .80
628 @item @emph{Prototype}: @tab @code{int omp_get_level(void);}
631 @item @emph{Fortran}:
632 @multitable @columnfractions .20 .80
633 @item @emph{Interface}: @tab @code{integer function omp_level()}
636 @item @emph{See also}:
637 @ref{omp_get_active_level}
639 @item @emph{Reference}:
640 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.17.
645 @node omp_get_max_active_levels
646 @section @code{omp_get_max_active_levels} -- Current maximum number of active regions
648 @item @emph{Description}:
649 This function obtains the maximum allowed number of nested, active parallel regions.
652 @multitable @columnfractions .20 .80
653 @item @emph{Prototype}: @tab @code{int omp_get_max_active_levels(void);}
656 @item @emph{Fortran}:
657 @multitable @columnfractions .20 .80
658 @item @emph{Interface}: @tab @code{integer function omp_get_max_active_levels()}
661 @item @emph{See also}:
662 @ref{omp_set_max_active_levels}, @ref{omp_get_active_level}
664 @item @emph{Reference}:
665 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.16.
669 @node omp_get_max_task_priority
670 @section @code{omp_get_max_task_priority} -- Maximum priority value
671 that can be set for tasks.
673 @item @emph{Description}:
674 This function obtains the maximum allowed priority number for tasks.
677 @multitable @columnfractions .20 .80
678 @item @emph{Prototype}: @tab @code{int omp_get_max_task_priority(void);}
681 @item @emph{Fortran}:
682 @multitable @columnfractions .20 .80
683 @item @emph{Interface}: @tab @code{integer function omp_get_max_task_priority()}
686 @item @emph{Reference}:
687 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.29.
691 @node omp_get_max_teams
692 @section @code{omp_get_max_teams} -- Maximum number of teams of teams region
694 @item @emph{Description}:
695 Return the maximum number of teams used for the teams region
696 that does not use the clause @code{num_teams}.
699 @multitable @columnfractions .20 .80
700 @item @emph{Prototype}: @tab @code{int omp_get_max_teams(void);}
703 @item @emph{Fortran}:
704 @multitable @columnfractions .20 .80
705 @item @emph{Interface}: @tab @code{integer function omp_get_max_teams()}
708 @item @emph{See also}:
709 @ref{omp_set_num_teams}, @ref{omp_get_num_teams}
711 @item @emph{Reference}:
712 @uref{https://www.openmp.org, OpenMP specification v5.1}, Section 3.4.4.
717 @node omp_get_max_threads
718 @section @code{omp_get_max_threads} -- Maximum number of threads of parallel region
720 @item @emph{Description}:
721 Return the maximum number of threads used for the current parallel region
722 that does not use the clause @code{num_threads}.
725 @multitable @columnfractions .20 .80
726 @item @emph{Prototype}: @tab @code{int omp_get_max_threads(void);}
729 @item @emph{Fortran}:
730 @multitable @columnfractions .20 .80
731 @item @emph{Interface}: @tab @code{integer function omp_get_max_threads()}
734 @item @emph{See also}:
735 @ref{omp_set_num_threads}, @ref{omp_set_dynamic}, @ref{omp_get_thread_limit}
737 @item @emph{Reference}:
738 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.3.
744 @section @code{omp_get_nested} -- Nested parallel regions
746 @item @emph{Description}:
747 This function returns @code{true} if nested parallel regions are
748 enabled, @code{false} otherwise. Here, @code{true} and @code{false}
749 represent their language-specific counterparts.
751 The state of nested parallel regions at startup depends on several
752 environment variables. If @env{OMP_MAX_ACTIVE_LEVELS} is defined
753 and is set to greater than one, then nested parallel regions will be
754 enabled. If not defined, then the value of the @env{OMP_NESTED}
755 environment variable will be followed if defined. If neither are
756 defined, then if either @env{OMP_NUM_THREADS} or @env{OMP_PROC_BIND}
757 are defined with a list of more than one value, then nested parallel
758 regions are enabled. If none of these are defined, then nested parallel
759 regions are disabled by default.
761 Nested parallel regions can be enabled or disabled at runtime using
762 @code{omp_set_nested}, or by setting the maximum number of nested
763 regions with @code{omp_set_max_active_levels} to one to disable, or
767 @multitable @columnfractions .20 .80
768 @item @emph{Prototype}: @tab @code{int omp_get_nested(void);}
771 @item @emph{Fortran}:
772 @multitable @columnfractions .20 .80
773 @item @emph{Interface}: @tab @code{logical function omp_get_nested()}
776 @item @emph{See also}:
777 @ref{omp_set_max_active_levels}, @ref{omp_set_nested},
778 @ref{OMP_MAX_ACTIVE_LEVELS}, @ref{OMP_NESTED}
780 @item @emph{Reference}:
781 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.11.
786 @node omp_get_num_devices
787 @section @code{omp_get_num_devices} -- Number of target devices
789 @item @emph{Description}:
790 Returns the number of target devices.
793 @multitable @columnfractions .20 .80
794 @item @emph{Prototype}: @tab @code{int omp_get_num_devices(void);}
797 @item @emph{Fortran}:
798 @multitable @columnfractions .20 .80
799 @item @emph{Interface}: @tab @code{integer function omp_get_num_devices()}
802 @item @emph{Reference}:
803 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.31.
808 @node omp_get_num_procs
809 @section @code{omp_get_num_procs} -- Number of processors online
811 @item @emph{Description}:
812 Returns the number of processors online on that device.
815 @multitable @columnfractions .20 .80
816 @item @emph{Prototype}: @tab @code{int omp_get_num_procs(void);}
819 @item @emph{Fortran}:
820 @multitable @columnfractions .20 .80
821 @item @emph{Interface}: @tab @code{integer function omp_get_num_procs()}
824 @item @emph{Reference}:
825 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.5.
830 @node omp_get_num_teams
831 @section @code{omp_get_num_teams} -- Number of teams
833 @item @emph{Description}:
834 Returns the number of teams in the current team region.
837 @multitable @columnfractions .20 .80
838 @item @emph{Prototype}: @tab @code{int omp_get_num_teams(void);}
841 @item @emph{Fortran}:
842 @multitable @columnfractions .20 .80
843 @item @emph{Interface}: @tab @code{integer function omp_get_num_teams()}
846 @item @emph{Reference}:
847 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.32.
852 @node omp_get_num_threads
853 @section @code{omp_get_num_threads} -- Size of the active team
855 @item @emph{Description}:
856 Returns the number of threads in the current team. In a sequential section of
857 the program @code{omp_get_num_threads} returns 1.
859 The default team size may be initialized at startup by the
860 @env{OMP_NUM_THREADS} environment variable. At runtime, the size
861 of the current team may be set either by the @code{NUM_THREADS}
862 clause or by @code{omp_set_num_threads}. If none of the above were
863 used to define a specific value and @env{OMP_DYNAMIC} is disabled,
864 one thread per CPU online is used.
867 @multitable @columnfractions .20 .80
868 @item @emph{Prototype}: @tab @code{int omp_get_num_threads(void);}
871 @item @emph{Fortran}:
872 @multitable @columnfractions .20 .80
873 @item @emph{Interface}: @tab @code{integer function omp_get_num_threads()}
876 @item @emph{See also}:
877 @ref{omp_get_max_threads}, @ref{omp_set_num_threads}, @ref{OMP_NUM_THREADS}
879 @item @emph{Reference}:
880 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.2.
885 @node omp_get_proc_bind
886 @section @code{omp_get_proc_bind} -- Whether theads may be moved between CPUs
888 @item @emph{Description}:
889 This functions returns the currently active thread affinity policy, which is
890 set via @env{OMP_PROC_BIND}. Possible values are @code{omp_proc_bind_false},
891 @code{omp_proc_bind_true}, @code{omp_proc_bind_primary},
892 @code{omp_proc_bind_master}, @code{omp_proc_bind_close} and @code{omp_proc_bind_spread},
893 where @code{omp_proc_bind_master} is an alias for @code{omp_proc_bind_primary}.
896 @multitable @columnfractions .20 .80
897 @item @emph{Prototype}: @tab @code{omp_proc_bind_t omp_get_proc_bind(void);}
900 @item @emph{Fortran}:
901 @multitable @columnfractions .20 .80
902 @item @emph{Interface}: @tab @code{integer(kind=omp_proc_bind_kind) function omp_get_proc_bind()}
905 @item @emph{See also}:
906 @ref{OMP_PROC_BIND}, @ref{OMP_PLACES}, @ref{GOMP_CPU_AFFINITY},
908 @item @emph{Reference}:
909 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.22.
914 @node omp_get_schedule
915 @section @code{omp_get_schedule} -- Obtain the runtime scheduling method
917 @item @emph{Description}:
918 Obtain the runtime scheduling method. The @var{kind} argument will be
919 set to the value @code{omp_sched_static}, @code{omp_sched_dynamic},
920 @code{omp_sched_guided} or @code{omp_sched_auto}. The second argument,
921 @var{chunk_size}, is set to the chunk size.
924 @multitable @columnfractions .20 .80
925 @item @emph{Prototype}: @tab @code{void omp_get_schedule(omp_sched_t *kind, int *chunk_size);}
928 @item @emph{Fortran}:
929 @multitable @columnfractions .20 .80
930 @item @emph{Interface}: @tab @code{subroutine omp_get_schedule(kind, chunk_size)}
931 @item @tab @code{integer(kind=omp_sched_kind) kind}
932 @item @tab @code{integer chunk_size}
935 @item @emph{See also}:
936 @ref{omp_set_schedule}, @ref{OMP_SCHEDULE}
938 @item @emph{Reference}:
939 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.13.
943 @node omp_get_supported_active_levels
944 @section @code{omp_get_supported_active_levels} -- Maximum number of active regions supported
946 @item @emph{Description}:
947 This function returns the maximum number of nested, active parallel regions
948 supported by this implementation.
951 @multitable @columnfractions .20 .80
952 @item @emph{Prototype}: @tab @code{int omp_get_supported_active_levels(void);}
955 @item @emph{Fortran}:
956 @multitable @columnfractions .20 .80
957 @item @emph{Interface}: @tab @code{integer function omp_get_supported_active_levels()}
960 @item @emph{See also}:
961 @ref{omp_get_max_active_levels}, @ref{omp_set_max_active_levels}
963 @item @emph{Reference}:
964 @uref{https://www.openmp.org, OpenMP specification v5.0}, Section 3.2.15.
969 @node omp_get_team_num
970 @section @code{omp_get_team_num} -- Get team number
972 @item @emph{Description}:
973 Returns the team number of the calling thread.
976 @multitable @columnfractions .20 .80
977 @item @emph{Prototype}: @tab @code{int omp_get_team_num(void);}
980 @item @emph{Fortran}:
981 @multitable @columnfractions .20 .80
982 @item @emph{Interface}: @tab @code{integer function omp_get_team_num()}
985 @item @emph{Reference}:
986 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.33.
991 @node omp_get_team_size
992 @section @code{omp_get_team_size} -- Number of threads in a team
994 @item @emph{Description}:
995 This function returns the number of threads in a thread team to which
996 either the current thread or its ancestor belongs. For values of @var{level}
997 outside zero to @code{omp_get_level}, -1 is returned; if @var{level} is zero,
998 1 is returned, and for @code{omp_get_level}, the result is identical
999 to @code{omp_get_num_threads}.
1002 @multitable @columnfractions .20 .80
1003 @item @emph{Prototype}: @tab @code{int omp_get_team_size(int level);}
1006 @item @emph{Fortran}:
1007 @multitable @columnfractions .20 .80
1008 @item @emph{Interface}: @tab @code{integer function omp_get_team_size(level)}
1009 @item @tab @code{integer level}
1012 @item @emph{See also}:
1013 @ref{omp_get_num_threads}, @ref{omp_get_level}, @ref{omp_get_ancestor_thread_num}
1015 @item @emph{Reference}:
1016 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.19.
1021 @node omp_get_teams_thread_limit
1022 @section @code{omp_get_teams_thread_limit} -- Maximum number of threads imposed by teams
1024 @item @emph{Description}:
1025 Return the maximum number of threads that will be able to participate in
1026 each team created by a teams construct.
1029 @multitable @columnfractions .20 .80
1030 @item @emph{Prototype}: @tab @code{int omp_get_teams_thread_limit(void);}
1033 @item @emph{Fortran}:
1034 @multitable @columnfractions .20 .80
1035 @item @emph{Interface}: @tab @code{integer function omp_get_teams_thread_limit()}
1038 @item @emph{See also}:
1039 @ref{omp_set_teams_thread_limit}, @ref{OMP_TEAMS_THREAD_LIMIT}
1041 @item @emph{Reference}:
1042 @uref{https://www.openmp.org, OpenMP specification v5.1}, Section 3.4.6.
1047 @node omp_get_thread_limit
1048 @section @code{omp_get_thread_limit} -- Maximum number of threads
1050 @item @emph{Description}:
1051 Return the maximum number of threads of the program.
1054 @multitable @columnfractions .20 .80
1055 @item @emph{Prototype}: @tab @code{int omp_get_thread_limit(void);}
1058 @item @emph{Fortran}:
1059 @multitable @columnfractions .20 .80
1060 @item @emph{Interface}: @tab @code{integer function omp_get_thread_limit()}
1063 @item @emph{See also}:
1064 @ref{omp_get_max_threads}, @ref{OMP_THREAD_LIMIT}
1066 @item @emph{Reference}:
1067 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.14.
1072 @node omp_get_thread_num
1073 @section @code{omp_get_thread_num} -- Current thread ID
1075 @item @emph{Description}:
1076 Returns a unique thread identification number within the current team.
1077 In a sequential parts of the program, @code{omp_get_thread_num}
1078 always returns 0. In parallel regions the return value varies
1079 from 0 to @code{omp_get_num_threads}-1 inclusive. The return
1080 value of the primary thread of a team is always 0.
1083 @multitable @columnfractions .20 .80
1084 @item @emph{Prototype}: @tab @code{int omp_get_thread_num(void);}
1087 @item @emph{Fortran}:
1088 @multitable @columnfractions .20 .80
1089 @item @emph{Interface}: @tab @code{integer function omp_get_thread_num()}
1092 @item @emph{See also}:
1093 @ref{omp_get_num_threads}, @ref{omp_get_ancestor_thread_num}
1095 @item @emph{Reference}:
1096 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.4.
1101 @node omp_in_parallel
1102 @section @code{omp_in_parallel} -- Whether a parallel region is active
1104 @item @emph{Description}:
1105 This function returns @code{true} if currently running in parallel,
1106 @code{false} otherwise. Here, @code{true} and @code{false} represent
1107 their language-specific counterparts.
1110 @multitable @columnfractions .20 .80
1111 @item @emph{Prototype}: @tab @code{int omp_in_parallel(void);}
1114 @item @emph{Fortran}:
1115 @multitable @columnfractions .20 .80
1116 @item @emph{Interface}: @tab @code{logical function omp_in_parallel()}
1119 @item @emph{Reference}:
1120 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.6.
1125 @section @code{omp_in_final} -- Whether in final or included task region
1127 @item @emph{Description}:
1128 This function returns @code{true} if currently running in a final
1129 or included task region, @code{false} otherwise. Here, @code{true}
1130 and @code{false} represent their language-specific counterparts.
1133 @multitable @columnfractions .20 .80
1134 @item @emph{Prototype}: @tab @code{int omp_in_final(void);}
1137 @item @emph{Fortran}:
1138 @multitable @columnfractions .20 .80
1139 @item @emph{Interface}: @tab @code{logical function omp_in_final()}
1142 @item @emph{Reference}:
1143 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.21.
1148 @node omp_is_initial_device
1149 @section @code{omp_is_initial_device} -- Whether executing on the host device
1151 @item @emph{Description}:
1152 This function returns @code{true} if currently running on the host device,
1153 @code{false} otherwise. Here, @code{true} and @code{false} represent
1154 their language-specific counterparts.
1157 @multitable @columnfractions .20 .80
1158 @item @emph{Prototype}: @tab @code{int omp_is_initial_device(void);}
1161 @item @emph{Fortran}:
1162 @multitable @columnfractions .20 .80
1163 @item @emph{Interface}: @tab @code{logical function omp_is_initial_device()}
1166 @item @emph{Reference}:
1167 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.34.
1172 @node omp_set_default_device
1173 @section @code{omp_set_default_device} -- Set the default device for target regions
1175 @item @emph{Description}:
1176 Set the default device for target regions without device clause. The argument
1177 shall be a nonnegative device number.
1180 @multitable @columnfractions .20 .80
1181 @item @emph{Prototype}: @tab @code{void omp_set_default_device(int device_num);}
1184 @item @emph{Fortran}:
1185 @multitable @columnfractions .20 .80
1186 @item @emph{Interface}: @tab @code{subroutine omp_set_default_device(device_num)}
1187 @item @tab @code{integer device_num}
1190 @item @emph{See also}:
1191 @ref{OMP_DEFAULT_DEVICE}, @ref{omp_get_default_device}
1193 @item @emph{Reference}:
1194 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.29.
1199 @node omp_set_dynamic
1200 @section @code{omp_set_dynamic} -- Enable/disable dynamic teams
1202 @item @emph{Description}:
1203 Enable or disable the dynamic adjustment of the number of threads
1204 within a team. The function takes the language-specific equivalent
1205 of @code{true} and @code{false}, where @code{true} enables dynamic
1206 adjustment of team sizes and @code{false} disables it.
1209 @multitable @columnfractions .20 .80
1210 @item @emph{Prototype}: @tab @code{void omp_set_dynamic(int dynamic_threads);}
1213 @item @emph{Fortran}:
1214 @multitable @columnfractions .20 .80
1215 @item @emph{Interface}: @tab @code{subroutine omp_set_dynamic(dynamic_threads)}
1216 @item @tab @code{logical, intent(in) :: dynamic_threads}
1219 @item @emph{See also}:
1220 @ref{OMP_DYNAMIC}, @ref{omp_get_dynamic}
1222 @item @emph{Reference}:
1223 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.7.
1228 @node omp_set_max_active_levels
1229 @section @code{omp_set_max_active_levels} -- Limits the number of active parallel regions
1231 @item @emph{Description}:
1232 This function limits the maximum allowed number of nested, active
1233 parallel regions. @var{max_levels} must be less or equal to
1234 the value returned by @code{omp_get_supported_active_levels}.
1237 @multitable @columnfractions .20 .80
1238 @item @emph{Prototype}: @tab @code{void omp_set_max_active_levels(int max_levels);}
1241 @item @emph{Fortran}:
1242 @multitable @columnfractions .20 .80
1243 @item @emph{Interface}: @tab @code{subroutine omp_set_max_active_levels(max_levels)}
1244 @item @tab @code{integer max_levels}
1247 @item @emph{See also}:
1248 @ref{omp_get_max_active_levels}, @ref{omp_get_active_level},
1249 @ref{omp_get_supported_active_levels}
1251 @item @emph{Reference}:
1252 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.15.
1257 @node omp_set_nested
1258 @section @code{omp_set_nested} -- Enable/disable nested parallel regions
1260 @item @emph{Description}:
1261 Enable or disable nested parallel regions, i.e., whether team members
1262 are allowed to create new teams. The function takes the language-specific
1263 equivalent of @code{true} and @code{false}, where @code{true} enables
1264 dynamic adjustment of team sizes and @code{false} disables it.
1266 Enabling nested parallel regions will also set the maximum number of
1267 active nested regions to the maximum supported. Disabling nested parallel
1268 regions will set the maximum number of active nested regions to one.
1271 @multitable @columnfractions .20 .80
1272 @item @emph{Prototype}: @tab @code{void omp_set_nested(int nested);}
1275 @item @emph{Fortran}:
1276 @multitable @columnfractions .20 .80
1277 @item @emph{Interface}: @tab @code{subroutine omp_set_nested(nested)}
1278 @item @tab @code{logical, intent(in) :: nested}
1281 @item @emph{See also}:
1282 @ref{omp_get_nested}, @ref{omp_set_max_active_levels},
1283 @ref{OMP_MAX_ACTIVE_LEVELS}, @ref{OMP_NESTED}
1285 @item @emph{Reference}:
1286 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.10.
1291 @node omp_set_num_teams
1292 @section @code{omp_set_num_teams} -- Set upper teams limit for teams construct
1294 @item @emph{Description}:
1295 Specifies the upper bound for number of teams created by the teams construct
1296 which does not specify a @code{num_teams} clause. The
1297 argument of @code{omp_set_num_teams} shall be a positive integer.
1300 @multitable @columnfractions .20 .80
1301 @item @emph{Prototype}: @tab @code{void omp_set_num_teams(int num_teams);}
1304 @item @emph{Fortran}:
1305 @multitable @columnfractions .20 .80
1306 @item @emph{Interface}: @tab @code{subroutine omp_set_num_teams(num_teams)}
1307 @item @tab @code{integer, intent(in) :: num_teams}
1310 @item @emph{See also}:
1311 @ref{OMP_NUM_TEAMS}, @ref{omp_get_num_teams}, @ref{omp_get_max_teams}
1313 @item @emph{Reference}:
1314 @uref{https://www.openmp.org, OpenMP specification v5.1}, Section 3.4.3.
1319 @node omp_set_num_threads
1320 @section @code{omp_set_num_threads} -- Set upper team size limit
1322 @item @emph{Description}:
1323 Specifies the number of threads used by default in subsequent parallel
1324 sections, if those do not specify a @code{num_threads} clause. The
1325 argument of @code{omp_set_num_threads} shall be a positive integer.
1328 @multitable @columnfractions .20 .80
1329 @item @emph{Prototype}: @tab @code{void omp_set_num_threads(int num_threads);}
1332 @item @emph{Fortran}:
1333 @multitable @columnfractions .20 .80
1334 @item @emph{Interface}: @tab @code{subroutine omp_set_num_threads(num_threads)}
1335 @item @tab @code{integer, intent(in) :: num_threads}
1338 @item @emph{See also}:
1339 @ref{OMP_NUM_THREADS}, @ref{omp_get_num_threads}, @ref{omp_get_max_threads}
1341 @item @emph{Reference}:
1342 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.1.
1347 @node omp_set_schedule
1348 @section @code{omp_set_schedule} -- Set the runtime scheduling method
1350 @item @emph{Description}:
1351 Sets the runtime scheduling method. The @var{kind} argument can have the
1352 value @code{omp_sched_static}, @code{omp_sched_dynamic},
1353 @code{omp_sched_guided} or @code{omp_sched_auto}. Except for
1354 @code{omp_sched_auto}, the chunk size is set to the value of
1355 @var{chunk_size} if positive, or to the default value if zero or negative.
1356 For @code{omp_sched_auto} the @var{chunk_size} argument is ignored.
1359 @multitable @columnfractions .20 .80
1360 @item @emph{Prototype}: @tab @code{void omp_set_schedule(omp_sched_t kind, int chunk_size);}
1363 @item @emph{Fortran}:
1364 @multitable @columnfractions .20 .80
1365 @item @emph{Interface}: @tab @code{subroutine omp_set_schedule(kind, chunk_size)}
1366 @item @tab @code{integer(kind=omp_sched_kind) kind}
1367 @item @tab @code{integer chunk_size}
1370 @item @emph{See also}:
1371 @ref{omp_get_schedule}
1374 @item @emph{Reference}:
1375 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.12.
1380 @node omp_set_teams_thread_limit
1381 @section @code{omp_set_teams_thread_limit} -- Set upper thread limit for teams construct
1383 @item @emph{Description}:
1384 Specifies the upper bound for number of threads that will be available
1385 for each team created by the teams construct which does not specify a
1386 @code{thread_limit} clause. The argument of
1387 @code{omp_set_teams_thread_limit} shall be a positive integer.
1390 @multitable @columnfractions .20 .80
1391 @item @emph{Prototype}: @tab @code{void omp_set_teams_thread_limit(int thread_limit);}
1394 @item @emph{Fortran}:
1395 @multitable @columnfractions .20 .80
1396 @item @emph{Interface}: @tab @code{subroutine omp_set_teams_thread_limit(thread_limit)}
1397 @item @tab @code{integer, intent(in) :: thread_limit}
1400 @item @emph{See also}:
1401 @ref{OMP_TEAMS_THREAD_LIMIT}, @ref{omp_get_teams_thread_limit}, @ref{omp_get_thread_limit}
1403 @item @emph{Reference}:
1404 @uref{https://www.openmp.org, OpenMP specification v5.1}, Section 3.4.5.
1410 @section @code{omp_init_lock} -- Initialize simple lock
1412 @item @emph{Description}:
1413 Initialize a simple lock. After initialization, the lock is in
1417 @multitable @columnfractions .20 .80
1418 @item @emph{Prototype}: @tab @code{void omp_init_lock(omp_lock_t *lock);}
1421 @item @emph{Fortran}:
1422 @multitable @columnfractions .20 .80
1423 @item @emph{Interface}: @tab @code{subroutine omp_init_lock(svar)}
1424 @item @tab @code{integer(omp_lock_kind), intent(out) :: svar}
1427 @item @emph{See also}:
1428 @ref{omp_destroy_lock}
1430 @item @emph{Reference}:
1431 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.3.1.
1437 @section @code{omp_set_lock} -- Wait for and set simple lock
1439 @item @emph{Description}:
1440 Before setting a simple lock, the lock variable must be initialized by
1441 @code{omp_init_lock}. The calling thread is blocked until the lock
1442 is available. If the lock is already held by the current thread,
1446 @multitable @columnfractions .20 .80
1447 @item @emph{Prototype}: @tab @code{void omp_set_lock(omp_lock_t *lock);}
1450 @item @emph{Fortran}:
1451 @multitable @columnfractions .20 .80
1452 @item @emph{Interface}: @tab @code{subroutine omp_set_lock(svar)}
1453 @item @tab @code{integer(omp_lock_kind), intent(inout) :: svar}
1456 @item @emph{See also}:
1457 @ref{omp_init_lock}, @ref{omp_test_lock}, @ref{omp_unset_lock}
1459 @item @emph{Reference}:
1460 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.3.4.
1466 @section @code{omp_test_lock} -- Test and set simple lock if available
1468 @item @emph{Description}:
1469 Before setting a simple lock, the lock variable must be initialized by
1470 @code{omp_init_lock}. Contrary to @code{omp_set_lock}, @code{omp_test_lock}
1471 does not block if the lock is not available. This function returns
1472 @code{true} upon success, @code{false} otherwise. Here, @code{true} and
1473 @code{false} represent their language-specific counterparts.
1476 @multitable @columnfractions .20 .80
1477 @item @emph{Prototype}: @tab @code{int omp_test_lock(omp_lock_t *lock);}
1480 @item @emph{Fortran}:
1481 @multitable @columnfractions .20 .80
1482 @item @emph{Interface}: @tab @code{logical function omp_test_lock(svar)}
1483 @item @tab @code{integer(omp_lock_kind), intent(inout) :: svar}
1486 @item @emph{See also}:
1487 @ref{omp_init_lock}, @ref{omp_set_lock}, @ref{omp_set_lock}
1489 @item @emph{Reference}:
1490 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.3.6.
1495 @node omp_unset_lock
1496 @section @code{omp_unset_lock} -- Unset simple lock
1498 @item @emph{Description}:
1499 A simple lock about to be unset must have been locked by @code{omp_set_lock}
1500 or @code{omp_test_lock} before. In addition, the lock must be held by the
1501 thread calling @code{omp_unset_lock}. Then, the lock becomes unlocked. If one
1502 or more threads attempted to set the lock before, one of them is chosen to,
1503 again, set the lock to itself.
1506 @multitable @columnfractions .20 .80
1507 @item @emph{Prototype}: @tab @code{void omp_unset_lock(omp_lock_t *lock);}
1510 @item @emph{Fortran}:
1511 @multitable @columnfractions .20 .80
1512 @item @emph{Interface}: @tab @code{subroutine omp_unset_lock(svar)}
1513 @item @tab @code{integer(omp_lock_kind), intent(inout) :: svar}
1516 @item @emph{See also}:
1517 @ref{omp_set_lock}, @ref{omp_test_lock}
1519 @item @emph{Reference}:
1520 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.3.5.
1525 @node omp_destroy_lock
1526 @section @code{omp_destroy_lock} -- Destroy simple lock
1528 @item @emph{Description}:
1529 Destroy a simple lock. In order to be destroyed, a simple lock must be
1530 in the unlocked state.
1533 @multitable @columnfractions .20 .80
1534 @item @emph{Prototype}: @tab @code{void omp_destroy_lock(omp_lock_t *lock);}
1537 @item @emph{Fortran}:
1538 @multitable @columnfractions .20 .80
1539 @item @emph{Interface}: @tab @code{subroutine omp_destroy_lock(svar)}
1540 @item @tab @code{integer(omp_lock_kind), intent(inout) :: svar}
1543 @item @emph{See also}:
1546 @item @emph{Reference}:
1547 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.3.3.
1552 @node omp_init_nest_lock
1553 @section @code{omp_init_nest_lock} -- Initialize nested lock
1555 @item @emph{Description}:
1556 Initialize a nested lock. After initialization, the lock is in
1557 an unlocked state and the nesting count is set to zero.
1560 @multitable @columnfractions .20 .80
1561 @item @emph{Prototype}: @tab @code{void omp_init_nest_lock(omp_nest_lock_t *lock);}
1564 @item @emph{Fortran}:
1565 @multitable @columnfractions .20 .80
1566 @item @emph{Interface}: @tab @code{subroutine omp_init_nest_lock(nvar)}
1567 @item @tab @code{integer(omp_nest_lock_kind), intent(out) :: nvar}
1570 @item @emph{See also}:
1571 @ref{omp_destroy_nest_lock}
1573 @item @emph{Reference}:
1574 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.3.1.
1578 @node omp_set_nest_lock
1579 @section @code{omp_set_nest_lock} -- Wait for and set nested lock
1581 @item @emph{Description}:
1582 Before setting a nested lock, the lock variable must be initialized by
1583 @code{omp_init_nest_lock}. The calling thread is blocked until the lock
1584 is available. If the lock is already held by the current thread, the
1585 nesting count for the lock is incremented.
1588 @multitable @columnfractions .20 .80
1589 @item @emph{Prototype}: @tab @code{void omp_set_nest_lock(omp_nest_lock_t *lock);}
1592 @item @emph{Fortran}:
1593 @multitable @columnfractions .20 .80
1594 @item @emph{Interface}: @tab @code{subroutine omp_set_nest_lock(nvar)}
1595 @item @tab @code{integer(omp_nest_lock_kind), intent(inout) :: nvar}
1598 @item @emph{See also}:
1599 @ref{omp_init_nest_lock}, @ref{omp_unset_nest_lock}
1601 @item @emph{Reference}:
1602 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.3.4.
1607 @node omp_test_nest_lock
1608 @section @code{omp_test_nest_lock} -- Test and set nested lock if available
1610 @item @emph{Description}:
1611 Before setting a nested lock, the lock variable must be initialized by
1612 @code{omp_init_nest_lock}. Contrary to @code{omp_set_nest_lock},
1613 @code{omp_test_nest_lock} does not block if the lock is not available.
1614 If the lock is already held by the current thread, the new nesting count
1615 is returned. Otherwise, the return value equals zero.
1618 @multitable @columnfractions .20 .80
1619 @item @emph{Prototype}: @tab @code{int omp_test_nest_lock(omp_nest_lock_t *lock);}
1622 @item @emph{Fortran}:
1623 @multitable @columnfractions .20 .80
1624 @item @emph{Interface}: @tab @code{logical function omp_test_nest_lock(nvar)}
1625 @item @tab @code{integer(omp_nest_lock_kind), intent(inout) :: nvar}
1629 @item @emph{See also}:
1630 @ref{omp_init_lock}, @ref{omp_set_lock}, @ref{omp_set_lock}
1632 @item @emph{Reference}:
1633 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.3.6.
1638 @node omp_unset_nest_lock
1639 @section @code{omp_unset_nest_lock} -- Unset nested lock
1641 @item @emph{Description}:
1642 A nested lock about to be unset must have been locked by @code{omp_set_nested_lock}
1643 or @code{omp_test_nested_lock} before. In addition, the lock must be held by the
1644 thread calling @code{omp_unset_nested_lock}. If the nesting count drops to zero, the
1645 lock becomes unlocked. If one ore more threads attempted to set the lock before,
1646 one of them is chosen to, again, set the lock to itself.
1649 @multitable @columnfractions .20 .80
1650 @item @emph{Prototype}: @tab @code{void omp_unset_nest_lock(omp_nest_lock_t *lock);}
1653 @item @emph{Fortran}:
1654 @multitable @columnfractions .20 .80
1655 @item @emph{Interface}: @tab @code{subroutine omp_unset_nest_lock(nvar)}
1656 @item @tab @code{integer(omp_nest_lock_kind), intent(inout) :: nvar}
1659 @item @emph{See also}:
1660 @ref{omp_set_nest_lock}
1662 @item @emph{Reference}:
1663 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.3.5.
1668 @node omp_destroy_nest_lock
1669 @section @code{omp_destroy_nest_lock} -- Destroy nested lock
1671 @item @emph{Description}:
1672 Destroy a nested lock. In order to be destroyed, a nested lock must be
1673 in the unlocked state and its nesting count must equal zero.
1676 @multitable @columnfractions .20 .80
1677 @item @emph{Prototype}: @tab @code{void omp_destroy_nest_lock(omp_nest_lock_t *);}
1680 @item @emph{Fortran}:
1681 @multitable @columnfractions .20 .80
1682 @item @emph{Interface}: @tab @code{subroutine omp_destroy_nest_lock(nvar)}
1683 @item @tab @code{integer(omp_nest_lock_kind), intent(inout) :: nvar}
1686 @item @emph{See also}:
1689 @item @emph{Reference}:
1690 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.3.3.
1696 @section @code{omp_get_wtick} -- Get timer precision
1698 @item @emph{Description}:
1699 Gets the timer precision, i.e., the number of seconds between two
1700 successive clock ticks.
1703 @multitable @columnfractions .20 .80
1704 @item @emph{Prototype}: @tab @code{double omp_get_wtick(void);}
1707 @item @emph{Fortran}:
1708 @multitable @columnfractions .20 .80
1709 @item @emph{Interface}: @tab @code{double precision function omp_get_wtick()}
1712 @item @emph{See also}:
1715 @item @emph{Reference}:
1716 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.4.2.
1722 @section @code{omp_get_wtime} -- Elapsed wall clock time
1724 @item @emph{Description}:
1725 Elapsed wall clock time in seconds. The time is measured per thread, no
1726 guarantee can be made that two distinct threads measure the same time.
1727 Time is measured from some "time in the past", which is an arbitrary time
1728 guaranteed not to change during the execution of the program.
1731 @multitable @columnfractions .20 .80
1732 @item @emph{Prototype}: @tab @code{double omp_get_wtime(void);}
1735 @item @emph{Fortran}:
1736 @multitable @columnfractions .20 .80
1737 @item @emph{Interface}: @tab @code{double precision function omp_get_wtime()}
1740 @item @emph{See also}:
1743 @item @emph{Reference}:
1744 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.4.1.
1749 @node omp_fulfill_event
1750 @section @code{omp_fulfill_event} -- Fulfill and destroy an OpenMP event
1752 @item @emph{Description}:
1753 Fulfill the event associated with the event handle argument. Currently, it
1754 is only used to fulfill events generated by detach clauses on task
1755 constructs - the effect of fulfilling the event is to allow the task to
1758 The result of calling @code{omp_fulfill_event} with an event handle other
1759 than that generated by a detach clause is undefined. Calling it with an
1760 event handle that has already been fulfilled is also undefined.
1763 @multitable @columnfractions .20 .80
1764 @item @emph{Prototype}: @tab @code{void omp_fulfill_event(omp_event_handle_t event);}
1767 @item @emph{Fortran}:
1768 @multitable @columnfractions .20 .80
1769 @item @emph{Interface}: @tab @code{subroutine omp_fulfill_event(event)}
1770 @item @tab @code{integer (kind=omp_event_handle_kind) :: event}
1773 @item @emph{Reference}:
1774 @uref{https://www.openmp.org, OpenMP specification v5.0}, Section 3.5.1.
1779 @c ---------------------------------------------------------------------
1780 @c OpenMP Environment Variables
1781 @c ---------------------------------------------------------------------
1783 @node Environment Variables
1784 @chapter OpenMP Environment Variables
1786 The environment variables which beginning with @env{OMP_} are defined by
1787 section 4 of the OpenMP specification in version 4.5, while those
1788 beginning with @env{GOMP_} are GNU extensions.
1791 * OMP_CANCELLATION:: Set whether cancellation is activated
1792 * OMP_DISPLAY_ENV:: Show OpenMP version and environment variables
1793 * OMP_DEFAULT_DEVICE:: Set the device used in target regions
1794 * OMP_DYNAMIC:: Dynamic adjustment of threads
1795 * OMP_MAX_ACTIVE_LEVELS:: Set the maximum number of nested parallel regions
1796 * OMP_MAX_TASK_PRIORITY:: Set the maximum task priority value
1797 * OMP_NESTED:: Nested parallel regions
1798 * OMP_NUM_TEAMS:: Specifies the number of teams to use by teams region
1799 * OMP_NUM_THREADS:: Specifies the number of threads to use
1800 * OMP_PROC_BIND:: Whether theads may be moved between CPUs
1801 * OMP_PLACES:: Specifies on which CPUs the theads should be placed
1802 * OMP_STACKSIZE:: Set default thread stack size
1803 * OMP_SCHEDULE:: How threads are scheduled
1804 * OMP_TARGET_OFFLOAD:: Controls offloading behaviour
1805 * OMP_TEAMS_THREAD_LIMIT:: Set the maximum number of threads imposed by teams
1806 * OMP_THREAD_LIMIT:: Set the maximum number of threads
1807 * OMP_WAIT_POLICY:: How waiting threads are handled
1808 * GOMP_CPU_AFFINITY:: Bind threads to specific CPUs
1809 * GOMP_DEBUG:: Enable debugging output
1810 * GOMP_STACKSIZE:: Set default thread stack size
1811 * GOMP_SPINCOUNT:: Set the busy-wait spin count
1812 * GOMP_RTEMS_THREAD_POOLS:: Set the RTEMS specific thread pools
1816 @node OMP_CANCELLATION
1817 @section @env{OMP_CANCELLATION} -- Set whether cancellation is activated
1818 @cindex Environment Variable
1820 @item @emph{Description}:
1821 If set to @code{TRUE}, the cancellation is activated. If set to @code{FALSE} or
1822 if unset, cancellation is disabled and the @code{cancel} construct is ignored.
1824 @item @emph{See also}:
1825 @ref{omp_get_cancellation}
1827 @item @emph{Reference}:
1828 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 4.11
1833 @node OMP_DISPLAY_ENV
1834 @section @env{OMP_DISPLAY_ENV} -- Show OpenMP version and environment variables
1835 @cindex Environment Variable
1837 @item @emph{Description}:
1838 If set to @code{TRUE}, the OpenMP version number and the values
1839 associated with the OpenMP environment variables are printed to @code{stderr}.
1840 If set to @code{VERBOSE}, it additionally shows the value of the environment
1841 variables which are GNU extensions. If undefined or set to @code{FALSE},
1842 this information will not be shown.
1845 @item @emph{Reference}:
1846 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 4.12
1851 @node OMP_DEFAULT_DEVICE
1852 @section @env{OMP_DEFAULT_DEVICE} -- Set the device used in target regions
1853 @cindex Environment Variable
1855 @item @emph{Description}:
1856 Set to choose the device which is used in a @code{target} region, unless the
1857 value is overridden by @code{omp_set_default_device} or by a @code{device}
1858 clause. The value shall be the nonnegative device number. If no device with
1859 the given device number exists, the code is executed on the host. If unset,
1860 device number 0 will be used.
1863 @item @emph{See also}:
1864 @ref{omp_get_default_device}, @ref{omp_set_default_device},
1866 @item @emph{Reference}:
1867 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 4.13
1873 @section @env{OMP_DYNAMIC} -- Dynamic adjustment of threads
1874 @cindex Environment Variable
1876 @item @emph{Description}:
1877 Enable or disable the dynamic adjustment of the number of threads
1878 within a team. The value of this environment variable shall be
1879 @code{TRUE} or @code{FALSE}. If undefined, dynamic adjustment is
1880 disabled by default.
1882 @item @emph{See also}:
1883 @ref{omp_set_dynamic}
1885 @item @emph{Reference}:
1886 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 4.3
1891 @node OMP_MAX_ACTIVE_LEVELS
1892 @section @env{OMP_MAX_ACTIVE_LEVELS} -- Set the maximum number of nested parallel regions
1893 @cindex Environment Variable
1895 @item @emph{Description}:
1896 Specifies the initial value for the maximum number of nested parallel
1897 regions. The value of this variable shall be a positive integer.
1898 If undefined, then if @env{OMP_NESTED} is defined and set to true, or
1899 if @env{OMP_NUM_THREADS} or @env{OMP_PROC_BIND} are defined and set to
1900 a list with more than one item, the maximum number of nested parallel
1901 regions will be initialized to the largest number supported, otherwise
1902 it will be set to one.
1904 @item @emph{See also}:
1905 @ref{omp_set_max_active_levels}, @ref{OMP_NESTED}
1907 @item @emph{Reference}:
1908 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 4.9
1913 @node OMP_MAX_TASK_PRIORITY
1914 @section @env{OMP_MAX_TASK_PRIORITY} -- Set the maximum priority
1915 number that can be set for a task.
1916 @cindex Environment Variable
1918 @item @emph{Description}:
1919 Specifies the initial value for the maximum priority value that can be
1920 set for a task. The value of this variable shall be a non-negative
1921 integer, and zero is allowed. If undefined, the default priority is
1924 @item @emph{See also}:
1925 @ref{omp_get_max_task_priority}
1927 @item @emph{Reference}:
1928 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 4.14
1934 @section @env{OMP_NESTED} -- Nested parallel regions
1935 @cindex Environment Variable
1936 @cindex Implementation specific setting
1938 @item @emph{Description}:
1939 Enable or disable nested parallel regions, i.e., whether team members
1940 are allowed to create new teams. The value of this environment variable
1941 shall be @code{TRUE} or @code{FALSE}. If set to @code{TRUE}, the number
1942 of maximum active nested regions supported will by default be set to the
1943 maximum supported, otherwise it will be set to one. If
1944 @env{OMP_MAX_ACTIVE_LEVELS} is defined, its setting will override this
1945 setting. If both are undefined, nested parallel regions are enabled if
1946 @env{OMP_NUM_THREADS} or @env{OMP_PROC_BINDS} are defined to a list with
1947 more than one item, otherwise they are disabled by default.
1949 @item @emph{See also}:
1950 @ref{omp_set_max_active_levels}, @ref{omp_set_nested}
1952 @item @emph{Reference}:
1953 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 4.6
1959 @section @env{OMP_NUM_TEAMS} -- Specifies the number of teams to use by teams region
1960 @cindex Environment Variable
1962 @item @emph{Description}:
1963 Specifies the upper bound for number of teams to use in teams regions
1964 without explicit @code{num_teams} clause. The value of this variable shall
1965 be a positive integer. If undefined it defaults to 0 which means
1966 implementation defined upper bound.
1968 @item @emph{See also}:
1969 @ref{omp_set_num_teams}
1971 @item @emph{Reference}:
1972 @uref{https://www.openmp.org, OpenMP specification v5.1}, Section 6.23
1977 @node OMP_NUM_THREADS
1978 @section @env{OMP_NUM_THREADS} -- Specifies the number of threads to use
1979 @cindex Environment Variable
1980 @cindex Implementation specific setting
1982 @item @emph{Description}:
1983 Specifies the default number of threads to use in parallel regions. The
1984 value of this variable shall be a comma-separated list of positive integers;
1985 the value specifies the number of threads to use for the corresponding nested
1986 level. Specifying more than one item in the list will automatically enable
1987 nesting by default. If undefined one thread per CPU is used.
1989 @item @emph{See also}:
1990 @ref{omp_set_num_threads}, @ref{OMP_NESTED}
1992 @item @emph{Reference}:
1993 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 4.2
1999 @section @env{OMP_PROC_BIND} -- Whether theads may be moved between CPUs
2000 @cindex Environment Variable
2002 @item @emph{Description}:
2003 Specifies whether threads may be moved between processors. If set to
2004 @code{TRUE}, OpenMP theads should not be moved; if set to @code{FALSE}
2005 they may be moved. Alternatively, a comma separated list with the
2006 values @code{PRIMARY}, @code{MASTER}, @code{CLOSE} and @code{SPREAD} can
2007 be used to specify the thread affinity policy for the corresponding nesting
2008 level. With @code{PRIMARY} and @code{MASTER} the worker threads are in the
2009 same place partition as the primary thread. With @code{CLOSE} those are
2010 kept close to the primary thread in contiguous place partitions. And
2011 with @code{SPREAD} a sparse distribution
2012 across the place partitions is used. Specifying more than one item in the
2013 list will automatically enable nesting by default.
2015 When undefined, @env{OMP_PROC_BIND} defaults to @code{TRUE} when
2016 @env{OMP_PLACES} or @env{GOMP_CPU_AFFINITY} is set and @code{FALSE} otherwise.
2018 @item @emph{See also}:
2019 @ref{omp_get_proc_bind}, @ref{GOMP_CPU_AFFINITY},
2020 @ref{OMP_NESTED}, @ref{OMP_PLACES}
2022 @item @emph{Reference}:
2023 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 4.4
2029 @section @env{OMP_PLACES} -- Specifies on which CPUs the theads should be placed
2030 @cindex Environment Variable
2032 @item @emph{Description}:
2033 The thread placement can be either specified using an abstract name or by an
2034 explicit list of the places. The abstract names @code{threads}, @code{cores},
2035 @code{sockets}, @code{ll_caches} and @code{numa_domains} can be optionally
2036 followed by a positive number in parentheses, which denotes the how many places
2037 shall be created. With @code{threads} each place corresponds to a single
2038 hardware thread; @code{cores} to a single core with the corresponding number of
2039 hardware threads; with @code{sockets} the place corresponds to a single
2040 socket; with @code{ll_caches} to a set of cores that shares the last level
2041 cache on the device; and @code{numa_domains} to a set of cores for which their
2042 closest memory on the device is the same memory and at a similar distance from
2043 the cores. The resulting placement can be shown by setting the
2044 @env{OMP_DISPLAY_ENV} environment variable.
2046 Alternatively, the placement can be specified explicitly as comma-separated
2047 list of places. A place is specified by set of nonnegative numbers in curly
2048 braces, denoting the hardware threads. The curly braces can be omitted
2049 when only a single number has been specified. The hardware threads
2050 belonging to a place can either be specified as comma-separated list of
2051 nonnegative thread numbers or using an interval. Multiple places can also be
2052 either specified by a comma-separated list of places or by an interval. To
2053 specify an interval, a colon followed by the count is placed after
2054 the hardware thread number or the place. Optionally, the length can be
2055 followed by a colon and the stride number -- otherwise a unit stride is
2056 assumed. Placing an exclamation mark (@code{!}) directly before a curly
2057 brace or numbers inside the curly braces (excluding intervals) will
2058 exclude those hardware threads.
2060 For instance, the following specifies the same places list:
2061 @code{"@{0,1,2@}, @{3,4,6@}, @{7,8,9@}, @{10,11,12@}"};
2062 @code{"@{0:3@}, @{3:3@}, @{7:3@}, @{10:3@}"}; and @code{"@{0:2@}:4:3"}.
2064 If @env{OMP_PLACES} and @env{GOMP_CPU_AFFINITY} are unset and
2065 @env{OMP_PROC_BIND} is either unset or @code{false}, threads may be moved
2066 between CPUs following no placement policy.
2068 @item @emph{See also}:
2069 @ref{OMP_PROC_BIND}, @ref{GOMP_CPU_AFFINITY}, @ref{omp_get_proc_bind},
2070 @ref{OMP_DISPLAY_ENV}
2072 @item @emph{Reference}:
2073 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 4.5
2079 @section @env{OMP_STACKSIZE} -- Set default thread stack size
2080 @cindex Environment Variable
2082 @item @emph{Description}:
2083 Set the default thread stack size in kilobytes, unless the number
2084 is suffixed by @code{B}, @code{K}, @code{M} or @code{G}, in which
2085 case the size is, respectively, in bytes, kilobytes, megabytes
2086 or gigabytes. This is different from @code{pthread_attr_setstacksize}
2087 which gets the number of bytes as an argument. If the stack size cannot
2088 be set due to system constraints, an error is reported and the initial
2089 stack size is left unchanged. If undefined, the stack size is system
2092 @item @emph{Reference}:
2093 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 4.7
2099 @section @env{OMP_SCHEDULE} -- How threads are scheduled
2100 @cindex Environment Variable
2101 @cindex Implementation specific setting
2103 @item @emph{Description}:
2104 Allows to specify @code{schedule type} and @code{chunk size}.
2105 The value of the variable shall have the form: @code{type[,chunk]} where
2106 @code{type} is one of @code{static}, @code{dynamic}, @code{guided} or @code{auto}
2107 The optional @code{chunk} size shall be a positive integer. If undefined,
2108 dynamic scheduling and a chunk size of 1 is used.
2110 @item @emph{See also}:
2111 @ref{omp_set_schedule}
2113 @item @emph{Reference}:
2114 @uref{https://www.openmp.org, OpenMP specification v4.5}, Sections 2.7.1.1 and 4.1
2119 @node OMP_TARGET_OFFLOAD
2120 @section @env{OMP_TARGET_OFFLOAD} -- Controls offloading behaviour
2121 @cindex Environment Variable
2122 @cindex Implementation specific setting
2124 @item @emph{Description}:
2125 Specifies the behaviour with regard to offloading code to a device. This
2126 variable can be set to one of three values - @code{MANDATORY}, @code{DISABLED}
2129 If set to @code{MANDATORY}, the program will terminate with an error if
2130 the offload device is not present or is not supported. If set to
2131 @code{DISABLED}, then offloading is disabled and all code will run on the
2132 host. If set to @code{DEFAULT}, the program will try offloading to the
2133 device first, then fall back to running code on the host if it cannot.
2135 If undefined, then the program will behave as if @code{DEFAULT} was set.
2137 @item @emph{Reference}:
2138 @uref{https://www.openmp.org, OpenMP specification v5.0}, Section 6.17
2143 @node OMP_TEAMS_THREAD_LIMIT
2144 @section @env{OMP_TEAMS_THREAD_LIMIT} -- Set the maximum number of threads imposed by teams
2145 @cindex Environment Variable
2147 @item @emph{Description}:
2148 Specifies an upper bound for the number of threads to use by each contention
2149 group created by a teams construct without explicit @code{thread_limit}
2150 clause. The value of this variable shall be a positive integer. If undefined,
2151 the value of 0 is used which stands for an implementation defined upper
2154 @item @emph{See also}:
2155 @ref{OMP_THREAD_LIMIT}, @ref{omp_set_teams_thread_limit}
2157 @item @emph{Reference}:
2158 @uref{https://www.openmp.org, OpenMP specification v5.1}, Section 6.24
2163 @node OMP_THREAD_LIMIT
2164 @section @env{OMP_THREAD_LIMIT} -- Set the maximum number of threads
2165 @cindex Environment Variable
2167 @item @emph{Description}:
2168 Specifies the number of threads to use for the whole program. The
2169 value of this variable shall be a positive integer. If undefined,
2170 the number of threads is not limited.
2172 @item @emph{See also}:
2173 @ref{OMP_NUM_THREADS}, @ref{omp_get_thread_limit}
2175 @item @emph{Reference}:
2176 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 4.10
2181 @node OMP_WAIT_POLICY
2182 @section @env{OMP_WAIT_POLICY} -- How waiting threads are handled
2183 @cindex Environment Variable
2185 @item @emph{Description}:
2186 Specifies whether waiting threads should be active or passive. If
2187 the value is @code{PASSIVE}, waiting threads should not consume CPU
2188 power while waiting; while the value is @code{ACTIVE} specifies that
2189 they should. If undefined, threads wait actively for a short time
2190 before waiting passively.
2192 @item @emph{See also}:
2193 @ref{GOMP_SPINCOUNT}
2195 @item @emph{Reference}:
2196 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 4.8
2201 @node GOMP_CPU_AFFINITY
2202 @section @env{GOMP_CPU_AFFINITY} -- Bind threads to specific CPUs
2203 @cindex Environment Variable
2205 @item @emph{Description}:
2206 Binds threads to specific CPUs. The variable should contain a space-separated
2207 or comma-separated list of CPUs. This list may contain different kinds of
2208 entries: either single CPU numbers in any order, a range of CPUs (M-N)
2209 or a range with some stride (M-N:S). CPU numbers are zero based. For example,
2210 @code{GOMP_CPU_AFFINITY="0 3 1-2 4-15:2"} will bind the initial thread
2211 to CPU 0, the second to CPU 3, the third to CPU 1, the fourth to
2212 CPU 2, the fifth to CPU 4, the sixth through tenth to CPUs 6, 8, 10, 12,
2213 and 14 respectively and then start assigning back from the beginning of
2214 the list. @code{GOMP_CPU_AFFINITY=0} binds all threads to CPU 0.
2216 There is no libgomp library routine to determine whether a CPU affinity
2217 specification is in effect. As a workaround, language-specific library
2218 functions, e.g., @code{getenv} in C or @code{GET_ENVIRONMENT_VARIABLE} in
2219 Fortran, may be used to query the setting of the @code{GOMP_CPU_AFFINITY}
2220 environment variable. A defined CPU affinity on startup cannot be changed
2221 or disabled during the runtime of the application.
2223 If both @env{GOMP_CPU_AFFINITY} and @env{OMP_PROC_BIND} are set,
2224 @env{OMP_PROC_BIND} has a higher precedence. If neither has been set and
2225 @env{OMP_PROC_BIND} is unset, or when @env{OMP_PROC_BIND} is set to
2226 @code{FALSE}, the host system will handle the assignment of threads to CPUs.
2228 @item @emph{See also}:
2229 @ref{OMP_PLACES}, @ref{OMP_PROC_BIND}
2235 @section @env{GOMP_DEBUG} -- Enable debugging output
2236 @cindex Environment Variable
2238 @item @emph{Description}:
2239 Enable debugging output. The variable should be set to @code{0}
2240 (disabled, also the default if not set), or @code{1} (enabled).
2242 If enabled, some debugging output will be printed during execution.
2243 This is currently not specified in more detail, and subject to change.
2248 @node GOMP_STACKSIZE
2249 @section @env{GOMP_STACKSIZE} -- Set default thread stack size
2250 @cindex Environment Variable
2251 @cindex Implementation specific setting
2253 @item @emph{Description}:
2254 Set the default thread stack size in kilobytes. This is different from
2255 @code{pthread_attr_setstacksize} which gets the number of bytes as an
2256 argument. If the stack size cannot be set due to system constraints, an
2257 error is reported and the initial stack size is left unchanged. If undefined,
2258 the stack size is system dependent.
2260 @item @emph{See also}:
2263 @item @emph{Reference}:
2264 @uref{https://gcc.gnu.org/ml/gcc-patches/2006-06/msg00493.html,
2265 GCC Patches Mailinglist},
2266 @uref{https://gcc.gnu.org/ml/gcc-patches/2006-06/msg00496.html,
2267 GCC Patches Mailinglist}
2272 @node GOMP_SPINCOUNT
2273 @section @env{GOMP_SPINCOUNT} -- Set the busy-wait spin count
2274 @cindex Environment Variable
2275 @cindex Implementation specific setting
2277 @item @emph{Description}:
2278 Determines how long a threads waits actively with consuming CPU power
2279 before waiting passively without consuming CPU power. The value may be
2280 either @code{INFINITE}, @code{INFINITY} to always wait actively or an
2281 integer which gives the number of spins of the busy-wait loop. The
2282 integer may optionally be followed by the following suffixes acting
2283 as multiplication factors: @code{k} (kilo, thousand), @code{M} (mega,
2284 million), @code{G} (giga, billion), or @code{T} (tera, trillion).
2285 If undefined, 0 is used when @env{OMP_WAIT_POLICY} is @code{PASSIVE},
2286 300,000 is used when @env{OMP_WAIT_POLICY} is undefined and
2287 30 billion is used when @env{OMP_WAIT_POLICY} is @code{ACTIVE}.
2288 If there are more OpenMP threads than available CPUs, 1000 and 100
2289 spins are used for @env{OMP_WAIT_POLICY} being @code{ACTIVE} or
2290 undefined, respectively; unless the @env{GOMP_SPINCOUNT} is lower
2291 or @env{OMP_WAIT_POLICY} is @code{PASSIVE}.
2293 @item @emph{See also}:
2294 @ref{OMP_WAIT_POLICY}
2299 @node GOMP_RTEMS_THREAD_POOLS
2300 @section @env{GOMP_RTEMS_THREAD_POOLS} -- Set the RTEMS specific thread pools
2301 @cindex Environment Variable
2302 @cindex Implementation specific setting
2304 @item @emph{Description}:
2305 This environment variable is only used on the RTEMS real-time operating system.
2306 It determines the scheduler instance specific thread pools. The format for
2307 @env{GOMP_RTEMS_THREAD_POOLS} is a list of optional
2308 @code{<thread-pool-count>[$<priority>]@@<scheduler-name>} configurations
2309 separated by @code{:} where:
2311 @item @code{<thread-pool-count>} is the thread pool count for this scheduler
2313 @item @code{$<priority>} is an optional priority for the worker threads of a
2314 thread pool according to @code{pthread_setschedparam}. In case a priority
2315 value is omitted, then a worker thread will inherit the priority of the OpenMP
2316 primary thread that created it. The priority of the worker thread is not
2317 changed after creation, even if a new OpenMP primary thread using the worker has
2318 a different priority.
2319 @item @code{@@<scheduler-name>} is the scheduler instance name according to the
2320 RTEMS application configuration.
2322 In case no thread pool configuration is specified for a scheduler instance,
2323 then each OpenMP primary thread of this scheduler instance will use its own
2324 dynamically allocated thread pool. To limit the worker thread count of the
2325 thread pools, each OpenMP primary thread must call @code{omp_set_num_threads}.
2326 @item @emph{Example}:
2327 Lets suppose we have three scheduler instances @code{IO}, @code{WRK0}, and
2328 @code{WRK1} with @env{GOMP_RTEMS_THREAD_POOLS} set to
2329 @code{"1@@WRK0:3$4@@WRK1"}. Then there are no thread pool restrictions for
2330 scheduler instance @code{IO}. In the scheduler instance @code{WRK0} there is
2331 one thread pool available. Since no priority is specified for this scheduler
2332 instance, the worker thread inherits the priority of the OpenMP primary thread
2333 that created it. In the scheduler instance @code{WRK1} there are three thread
2334 pools available and their worker threads run at priority four.
2339 @c ---------------------------------------------------------------------
2341 @c ---------------------------------------------------------------------
2343 @node Enabling OpenACC
2344 @chapter Enabling OpenACC
2346 To activate the OpenACC extensions for C/C++ and Fortran, the compile-time
2347 flag @option{-fopenacc} must be specified. This enables the OpenACC directive
2348 @code{#pragma acc} in C/C++ and @code{!$acc} directives in free form,
2349 @code{c$acc}, @code{*$acc} and @code{!$acc} directives in fixed form,
2350 @code{!$} conditional compilation sentinels in free form and @code{c$},
2351 @code{*$} and @code{!$} sentinels in fixed form, for Fortran. The flag also
2352 arranges for automatic linking of the OpenACC runtime library
2353 (@ref{OpenACC Runtime Library Routines}).
2355 See @uref{https://gcc.gnu.org/wiki/OpenACC} for more information.
2357 A complete description of all OpenACC directives accepted may be found in
2358 the @uref{https://www.openacc.org, OpenACC} Application Programming
2359 Interface manual, version 2.6.
2363 @c ---------------------------------------------------------------------
2364 @c OpenACC Runtime Library Routines
2365 @c ---------------------------------------------------------------------
2367 @node OpenACC Runtime Library Routines
2368 @chapter OpenACC Runtime Library Routines
2370 The runtime routines described here are defined by section 3 of the OpenACC
2371 specifications in version 2.6.
2372 They have C linkage, and do not throw exceptions.
2373 Generally, they are available only for the host, with the exception of
2374 @code{acc_on_device}, which is available for both the host and the
2375 acceleration device.
2378 * acc_get_num_devices:: Get number of devices for the given device
2380 * acc_set_device_type:: Set type of device accelerator to use.
2381 * acc_get_device_type:: Get type of device accelerator to be used.
2382 * acc_set_device_num:: Set device number to use.
2383 * acc_get_device_num:: Get device number to be used.
2384 * acc_get_property:: Get device property.
2385 * acc_async_test:: Tests for completion of a specific asynchronous
2387 * acc_async_test_all:: Tests for completion of all asynchronous
2389 * acc_wait:: Wait for completion of a specific asynchronous
2391 * acc_wait_all:: Waits for completion of all asynchronous
2393 * acc_wait_all_async:: Wait for completion of all asynchronous
2395 * acc_wait_async:: Wait for completion of asynchronous operations.
2396 * acc_init:: Initialize runtime for a specific device type.
2397 * acc_shutdown:: Shuts down the runtime for a specific device
2399 * acc_on_device:: Whether executing on a particular device
2400 * acc_malloc:: Allocate device memory.
2401 * acc_free:: Free device memory.
2402 * acc_copyin:: Allocate device memory and copy host memory to
2404 * acc_present_or_copyin:: If the data is not present on the device,
2405 allocate device memory and copy from host
2407 * acc_create:: Allocate device memory and map it to host
2409 * acc_present_or_create:: If the data is not present on the device,
2410 allocate device memory and map it to host
2412 * acc_copyout:: Copy device memory to host memory.
2413 * acc_delete:: Free device memory.
2414 * acc_update_device:: Update device memory from mapped host memory.
2415 * acc_update_self:: Update host memory from mapped device memory.
2416 * acc_map_data:: Map previously allocated device memory to host
2418 * acc_unmap_data:: Unmap device memory from host memory.
2419 * acc_deviceptr:: Get device pointer associated with specific
2421 * acc_hostptr:: Get host pointer associated with specific
2423 * acc_is_present:: Indicate whether host variable / array is
2425 * acc_memcpy_to_device:: Copy host memory to device memory.
2426 * acc_memcpy_from_device:: Copy device memory to host memory.
2427 * acc_attach:: Let device pointer point to device-pointer target.
2428 * acc_detach:: Let device pointer point to host-pointer target.
2430 API routines for target platforms.
2432 * acc_get_current_cuda_device:: Get CUDA device handle.
2433 * acc_get_current_cuda_context::Get CUDA context handle.
2434 * acc_get_cuda_stream:: Get CUDA stream handle.
2435 * acc_set_cuda_stream:: Set CUDA stream handle.
2437 API routines for the OpenACC Profiling Interface.
2439 * acc_prof_register:: Register callbacks.
2440 * acc_prof_unregister:: Unregister callbacks.
2441 * acc_prof_lookup:: Obtain inquiry functions.
2442 * acc_register_library:: Library registration.
2447 @node acc_get_num_devices
2448 @section @code{acc_get_num_devices} -- Get number of devices for given device type
2450 @item @emph{Description}
2451 This function returns a value indicating the number of devices available
2452 for the device type specified in @var{devicetype}.
2455 @multitable @columnfractions .20 .80
2456 @item @emph{Prototype}: @tab @code{int acc_get_num_devices(acc_device_t devicetype);}
2459 @item @emph{Fortran}:
2460 @multitable @columnfractions .20 .80
2461 @item @emph{Interface}: @tab @code{integer function acc_get_num_devices(devicetype)}
2462 @item @tab @code{integer(kind=acc_device_kind) devicetype}
2465 @item @emph{Reference}:
2466 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
2472 @node acc_set_device_type
2473 @section @code{acc_set_device_type} -- Set type of device accelerator to use.
2475 @item @emph{Description}
2476 This function indicates to the runtime library which device type, specified
2477 in @var{devicetype}, to use when executing a parallel or kernels region.
2480 @multitable @columnfractions .20 .80
2481 @item @emph{Prototype}: @tab @code{acc_set_device_type(acc_device_t devicetype);}
2484 @item @emph{Fortran}:
2485 @multitable @columnfractions .20 .80
2486 @item @emph{Interface}: @tab @code{subroutine acc_set_device_type(devicetype)}
2487 @item @tab @code{integer(kind=acc_device_kind) devicetype}
2490 @item @emph{Reference}:
2491 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
2497 @node acc_get_device_type
2498 @section @code{acc_get_device_type} -- Get type of device accelerator to be used.
2500 @item @emph{Description}
2501 This function returns what device type will be used when executing a
2502 parallel or kernels region.
2504 This function returns @code{acc_device_none} if
2505 @code{acc_get_device_type} is called from
2506 @code{acc_ev_device_init_start}, @code{acc_ev_device_init_end}
2507 callbacks of the OpenACC Profiling Interface (@ref{OpenACC Profiling
2508 Interface}), that is, if the device is currently being initialized.
2511 @multitable @columnfractions .20 .80
2512 @item @emph{Prototype}: @tab @code{acc_device_t acc_get_device_type(void);}
2515 @item @emph{Fortran}:
2516 @multitable @columnfractions .20 .80
2517 @item @emph{Interface}: @tab @code{function acc_get_device_type(void)}
2518 @item @tab @code{integer(kind=acc_device_kind) acc_get_device_type}
2521 @item @emph{Reference}:
2522 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
2528 @node acc_set_device_num
2529 @section @code{acc_set_device_num} -- Set device number to use.
2531 @item @emph{Description}
2532 This function will indicate to the runtime which device number,
2533 specified by @var{devicenum}, associated with the specified device
2534 type @var{devicetype}.
2537 @multitable @columnfractions .20 .80
2538 @item @emph{Prototype}: @tab @code{acc_set_device_num(int devicenum, acc_device_t devicetype);}
2541 @item @emph{Fortran}:
2542 @multitable @columnfractions .20 .80
2543 @item @emph{Interface}: @tab @code{subroutine acc_set_device_num(devicenum, devicetype)}
2544 @item @tab @code{integer devicenum}
2545 @item @tab @code{integer(kind=acc_device_kind) devicetype}
2548 @item @emph{Reference}:
2549 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
2555 @node acc_get_device_num
2556 @section @code{acc_get_device_num} -- Get device number to be used.
2558 @item @emph{Description}
2559 This function returns which device number associated with the specified device
2560 type @var{devicetype}, will be used when executing a parallel or kernels
2564 @multitable @columnfractions .20 .80
2565 @item @emph{Prototype}: @tab @code{int acc_get_device_num(acc_device_t devicetype);}
2568 @item @emph{Fortran}:
2569 @multitable @columnfractions .20 .80
2570 @item @emph{Interface}: @tab @code{function acc_get_device_num(devicetype)}
2571 @item @tab @code{integer(kind=acc_device_kind) devicetype}
2572 @item @tab @code{integer acc_get_device_num}
2575 @item @emph{Reference}:
2576 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
2582 @node acc_get_property
2583 @section @code{acc_get_property} -- Get device property.
2584 @cindex acc_get_property
2585 @cindex acc_get_property_string
2587 @item @emph{Description}
2588 These routines return the value of the specified @var{property} for the
2589 device being queried according to @var{devicenum} and @var{devicetype}.
2590 Integer-valued and string-valued properties are returned by
2591 @code{acc_get_property} and @code{acc_get_property_string} respectively.
2592 The Fortran @code{acc_get_property_string} subroutine returns the string
2593 retrieved in its fourth argument while the remaining entry points are
2594 functions, which pass the return value as their result.
2596 Note for Fortran, only: the OpenACC technical committee corrected and, hence,
2597 modified the interface introduced in OpenACC 2.6. The kind-value parameter
2598 @code{acc_device_property} has been renamed to @code{acc_device_property_kind}
2599 for consistency and the return type of the @code{acc_get_property} function is
2600 now a @code{c_size_t} integer instead of a @code{acc_device_property} integer.
2601 The parameter @code{acc_device_property} will continue to be provided,
2602 but might be removed in a future version of GCC.
2605 @multitable @columnfractions .20 .80
2606 @item @emph{Prototype}: @tab @code{size_t acc_get_property(int devicenum, acc_device_t devicetype, acc_device_property_t property);}
2607 @item @emph{Prototype}: @tab @code{const char *acc_get_property_string(int devicenum, acc_device_t devicetype, acc_device_property_t property);}
2610 @item @emph{Fortran}:
2611 @multitable @columnfractions .20 .80
2612 @item @emph{Interface}: @tab @code{function acc_get_property(devicenum, devicetype, property)}
2613 @item @emph{Interface}: @tab @code{subroutine acc_get_property_string(devicenum, devicetype, property, string)}
2614 @item @tab @code{use ISO_C_Binding, only: c_size_t}
2615 @item @tab @code{integer devicenum}
2616 @item @tab @code{integer(kind=acc_device_kind) devicetype}
2617 @item @tab @code{integer(kind=acc_device_property_kind) property}
2618 @item @tab @code{integer(kind=c_size_t) acc_get_property}
2619 @item @tab @code{character(*) string}
2622 @item @emph{Reference}:
2623 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
2629 @node acc_async_test
2630 @section @code{acc_async_test} -- Test for completion of a specific asynchronous operation.
2632 @item @emph{Description}
2633 This function tests for completion of the asynchronous operation specified
2634 in @var{arg}. In C/C++, a non-zero value will be returned to indicate
2635 the specified asynchronous operation has completed. While Fortran will return
2636 a @code{true}. If the asynchronous operation has not completed, C/C++ returns
2637 a zero and Fortran returns a @code{false}.
2640 @multitable @columnfractions .20 .80
2641 @item @emph{Prototype}: @tab @code{int acc_async_test(int arg);}
2644 @item @emph{Fortran}:
2645 @multitable @columnfractions .20 .80
2646 @item @emph{Interface}: @tab @code{function acc_async_test(arg)}
2647 @item @tab @code{integer(kind=acc_handle_kind) arg}
2648 @item @tab @code{logical acc_async_test}
2651 @item @emph{Reference}:
2652 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
2658 @node acc_async_test_all
2659 @section @code{acc_async_test_all} -- Tests for completion of all asynchronous operations.
2661 @item @emph{Description}
2662 This function tests for completion of all asynchronous operations.
2663 In C/C++, a non-zero value will be returned to indicate all asynchronous
2664 operations have completed. While Fortran will return a @code{true}. If
2665 any asynchronous operation has not completed, C/C++ returns a zero and
2666 Fortran returns a @code{false}.
2669 @multitable @columnfractions .20 .80
2670 @item @emph{Prototype}: @tab @code{int acc_async_test_all(void);}
2673 @item @emph{Fortran}:
2674 @multitable @columnfractions .20 .80
2675 @item @emph{Interface}: @tab @code{function acc_async_test()}
2676 @item @tab @code{logical acc_get_device_num}
2679 @item @emph{Reference}:
2680 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
2687 @section @code{acc_wait} -- Wait for completion of a specific asynchronous operation.
2689 @item @emph{Description}
2690 This function waits for completion of the asynchronous operation
2691 specified in @var{arg}.
2694 @multitable @columnfractions .20 .80
2695 @item @emph{Prototype}: @tab @code{acc_wait(arg);}
2696 @item @emph{Prototype (OpenACC 1.0 compatibility)}: @tab @code{acc_async_wait(arg);}
2699 @item @emph{Fortran}:
2700 @multitable @columnfractions .20 .80
2701 @item @emph{Interface}: @tab @code{subroutine acc_wait(arg)}
2702 @item @tab @code{integer(acc_handle_kind) arg}
2703 @item @emph{Interface (OpenACC 1.0 compatibility)}: @tab @code{subroutine acc_async_wait(arg)}
2704 @item @tab @code{integer(acc_handle_kind) arg}
2707 @item @emph{Reference}:
2708 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
2715 @section @code{acc_wait_all} -- Waits for completion of all asynchronous operations.
2717 @item @emph{Description}
2718 This function waits for the completion of all asynchronous operations.
2721 @multitable @columnfractions .20 .80
2722 @item @emph{Prototype}: @tab @code{acc_wait_all(void);}
2723 @item @emph{Prototype (OpenACC 1.0 compatibility)}: @tab @code{acc_async_wait_all(void);}
2726 @item @emph{Fortran}:
2727 @multitable @columnfractions .20 .80
2728 @item @emph{Interface}: @tab @code{subroutine acc_wait_all()}
2729 @item @emph{Interface (OpenACC 1.0 compatibility)}: @tab @code{subroutine acc_async_wait_all()}
2732 @item @emph{Reference}:
2733 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
2739 @node acc_wait_all_async
2740 @section @code{acc_wait_all_async} -- Wait for completion of all asynchronous operations.
2742 @item @emph{Description}
2743 This function enqueues a wait operation on the queue @var{async} for any
2744 and all asynchronous operations that have been previously enqueued on
2748 @multitable @columnfractions .20 .80
2749 @item @emph{Prototype}: @tab @code{acc_wait_all_async(int async);}
2752 @item @emph{Fortran}:
2753 @multitable @columnfractions .20 .80
2754 @item @emph{Interface}: @tab @code{subroutine acc_wait_all_async(async)}
2755 @item @tab @code{integer(acc_handle_kind) async}
2758 @item @emph{Reference}:
2759 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
2765 @node acc_wait_async
2766 @section @code{acc_wait_async} -- Wait for completion of asynchronous operations.
2768 @item @emph{Description}
2769 This function enqueues a wait operation on queue @var{async} for any and all
2770 asynchronous operations enqueued on queue @var{arg}.
2773 @multitable @columnfractions .20 .80
2774 @item @emph{Prototype}: @tab @code{acc_wait_async(int arg, int async);}
2777 @item @emph{Fortran}:
2778 @multitable @columnfractions .20 .80
2779 @item @emph{Interface}: @tab @code{subroutine acc_wait_async(arg, async)}
2780 @item @tab @code{integer(acc_handle_kind) arg, async}
2783 @item @emph{Reference}:
2784 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
2791 @section @code{acc_init} -- Initialize runtime for a specific device type.
2793 @item @emph{Description}
2794 This function initializes the runtime for the device type specified in
2798 @multitable @columnfractions .20 .80
2799 @item @emph{Prototype}: @tab @code{acc_init(acc_device_t devicetype);}
2802 @item @emph{Fortran}:
2803 @multitable @columnfractions .20 .80
2804 @item @emph{Interface}: @tab @code{subroutine acc_init(devicetype)}
2805 @item @tab @code{integer(acc_device_kind) devicetype}
2808 @item @emph{Reference}:
2809 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
2816 @section @code{acc_shutdown} -- Shuts down the runtime for a specific device type.
2818 @item @emph{Description}
2819 This function shuts down the runtime for the device type specified in
2823 @multitable @columnfractions .20 .80
2824 @item @emph{Prototype}: @tab @code{acc_shutdown(acc_device_t devicetype);}
2827 @item @emph{Fortran}:
2828 @multitable @columnfractions .20 .80
2829 @item @emph{Interface}: @tab @code{subroutine acc_shutdown(devicetype)}
2830 @item @tab @code{integer(acc_device_kind) devicetype}
2833 @item @emph{Reference}:
2834 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
2841 @section @code{acc_on_device} -- Whether executing on a particular device
2843 @item @emph{Description}:
2844 This function returns whether the program is executing on a particular
2845 device specified in @var{devicetype}. In C/C++ a non-zero value is
2846 returned to indicate the device is executing on the specified device type.
2847 In Fortran, @code{true} will be returned. If the program is not executing
2848 on the specified device type C/C++ will return a zero, while Fortran will
2849 return @code{false}.
2852 @multitable @columnfractions .20 .80
2853 @item @emph{Prototype}: @tab @code{acc_on_device(acc_device_t devicetype);}
2856 @item @emph{Fortran}:
2857 @multitable @columnfractions .20 .80
2858 @item @emph{Interface}: @tab @code{function acc_on_device(devicetype)}
2859 @item @tab @code{integer(acc_device_kind) devicetype}
2860 @item @tab @code{logical acc_on_device}
2864 @item @emph{Reference}:
2865 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
2872 @section @code{acc_malloc} -- Allocate device memory.
2874 @item @emph{Description}
2875 This function allocates @var{len} bytes of device memory. It returns
2876 the device address of the allocated memory.
2879 @multitable @columnfractions .20 .80
2880 @item @emph{Prototype}: @tab @code{d_void* acc_malloc(size_t len);}
2883 @item @emph{Reference}:
2884 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
2891 @section @code{acc_free} -- Free device memory.
2893 @item @emph{Description}
2894 Free previously allocated device memory at the device address @code{a}.
2897 @multitable @columnfractions .20 .80
2898 @item @emph{Prototype}: @tab @code{acc_free(d_void *a);}
2901 @item @emph{Reference}:
2902 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
2909 @section @code{acc_copyin} -- Allocate device memory and copy host memory to it.
2911 @item @emph{Description}
2912 In C/C++, this function allocates @var{len} bytes of device memory
2913 and maps it to the specified host address in @var{a}. The device
2914 address of the newly allocated device memory is returned.
2916 In Fortran, two (2) forms are supported. In the first form, @var{a} specifies
2917 a contiguous array section. The second form @var{a} specifies a
2918 variable or array element and @var{len} specifies the length in bytes.
2921 @multitable @columnfractions .20 .80
2922 @item @emph{Prototype}: @tab @code{void *acc_copyin(h_void *a, size_t len);}
2923 @item @emph{Prototype}: @tab @code{void *acc_copyin_async(h_void *a, size_t len, int async);}
2926 @item @emph{Fortran}:
2927 @multitable @columnfractions .20 .80
2928 @item @emph{Interface}: @tab @code{subroutine acc_copyin(a)}
2929 @item @tab @code{type, dimension(:[,:]...) :: a}
2930 @item @emph{Interface}: @tab @code{subroutine acc_copyin(a, len)}
2931 @item @tab @code{type, dimension(:[,:]...) :: a}
2932 @item @tab @code{integer len}
2933 @item @emph{Interface}: @tab @code{subroutine acc_copyin_async(a, async)}
2934 @item @tab @code{type, dimension(:[,:]...) :: a}
2935 @item @tab @code{integer(acc_handle_kind) :: async}
2936 @item @emph{Interface}: @tab @code{subroutine acc_copyin_async(a, len, async)}
2937 @item @tab @code{type, dimension(:[,:]...) :: a}
2938 @item @tab @code{integer len}
2939 @item @tab @code{integer(acc_handle_kind) :: async}
2942 @item @emph{Reference}:
2943 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
2949 @node acc_present_or_copyin
2950 @section @code{acc_present_or_copyin} -- If the data is not present on the device, allocate device memory and copy from host memory.
2952 @item @emph{Description}
2953 This function tests if the host data specified by @var{a} and of length
2954 @var{len} is present or not. If it is not present, then device memory
2955 will be allocated and the host memory copied. The device address of
2956 the newly allocated device memory is returned.
2958 In Fortran, two (2) forms are supported. In the first form, @var{a} specifies
2959 a contiguous array section. The second form @var{a} specifies a variable or
2960 array element and @var{len} specifies the length in bytes.
2962 Note that @code{acc_present_or_copyin} and @code{acc_pcopyin} exist for
2963 backward compatibility with OpenACC 2.0; use @ref{acc_copyin} instead.
2966 @multitable @columnfractions .20 .80
2967 @item @emph{Prototype}: @tab @code{void *acc_present_or_copyin(h_void *a, size_t len);}
2968 @item @emph{Prototype}: @tab @code{void *acc_pcopyin(h_void *a, size_t len);}
2971 @item @emph{Fortran}:
2972 @multitable @columnfractions .20 .80
2973 @item @emph{Interface}: @tab @code{subroutine acc_present_or_copyin(a)}
2974 @item @tab @code{type, dimension(:[,:]...) :: a}
2975 @item @emph{Interface}: @tab @code{subroutine acc_present_or_copyin(a, len)}
2976 @item @tab @code{type, dimension(:[,:]...) :: a}
2977 @item @tab @code{integer len}
2978 @item @emph{Interface}: @tab @code{subroutine acc_pcopyin(a)}
2979 @item @tab @code{type, dimension(:[,:]...) :: a}
2980 @item @emph{Interface}: @tab @code{subroutine acc_pcopyin(a, len)}
2981 @item @tab @code{type, dimension(:[,:]...) :: a}
2982 @item @tab @code{integer len}
2985 @item @emph{Reference}:
2986 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
2993 @section @code{acc_create} -- Allocate device memory and map it to host memory.
2995 @item @emph{Description}
2996 This function allocates device memory and maps it to host memory specified
2997 by the host address @var{a} with a length of @var{len} bytes. In C/C++,
2998 the function returns the device address of the allocated device memory.
3000 In Fortran, two (2) forms are supported. In the first form, @var{a} specifies
3001 a contiguous array section. The second form @var{a} specifies a variable or
3002 array element and @var{len} specifies the length in bytes.
3005 @multitable @columnfractions .20 .80
3006 @item @emph{Prototype}: @tab @code{void *acc_create(h_void *a, size_t len);}
3007 @item @emph{Prototype}: @tab @code{void *acc_create_async(h_void *a, size_t len, int async);}
3010 @item @emph{Fortran}:
3011 @multitable @columnfractions .20 .80
3012 @item @emph{Interface}: @tab @code{subroutine acc_create(a)}
3013 @item @tab @code{type, dimension(:[,:]...) :: a}
3014 @item @emph{Interface}: @tab @code{subroutine acc_create(a, len)}
3015 @item @tab @code{type, dimension(:[,:]...) :: a}
3016 @item @tab @code{integer len}
3017 @item @emph{Interface}: @tab @code{subroutine acc_create_async(a, async)}
3018 @item @tab @code{type, dimension(:[,:]...) :: a}
3019 @item @tab @code{integer(acc_handle_kind) :: async}
3020 @item @emph{Interface}: @tab @code{subroutine acc_create_async(a, len, async)}
3021 @item @tab @code{type, dimension(:[,:]...) :: a}
3022 @item @tab @code{integer len}
3023 @item @tab @code{integer(acc_handle_kind) :: async}
3026 @item @emph{Reference}:
3027 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3033 @node acc_present_or_create
3034 @section @code{acc_present_or_create} -- If the data is not present on the device, allocate device memory and map it to host memory.
3036 @item @emph{Description}
3037 This function tests if the host data specified by @var{a} and of length
3038 @var{len} is present or not. If it is not present, then device memory
3039 will be allocated and mapped to host memory. In C/C++, the device address
3040 of the newly allocated device memory is returned.
3042 In Fortran, two (2) forms are supported. In the first form, @var{a} specifies
3043 a contiguous array section. The second form @var{a} specifies a variable or
3044 array element and @var{len} specifies the length in bytes.
3046 Note that @code{acc_present_or_create} and @code{acc_pcreate} exist for
3047 backward compatibility with OpenACC 2.0; use @ref{acc_create} instead.
3050 @multitable @columnfractions .20 .80
3051 @item @emph{Prototype}: @tab @code{void *acc_present_or_create(h_void *a, size_t len)}
3052 @item @emph{Prototype}: @tab @code{void *acc_pcreate(h_void *a, size_t len)}
3055 @item @emph{Fortran}:
3056 @multitable @columnfractions .20 .80
3057 @item @emph{Interface}: @tab @code{subroutine acc_present_or_create(a)}
3058 @item @tab @code{type, dimension(:[,:]...) :: a}
3059 @item @emph{Interface}: @tab @code{subroutine acc_present_or_create(a, len)}
3060 @item @tab @code{type, dimension(:[,:]...) :: a}
3061 @item @tab @code{integer len}
3062 @item @emph{Interface}: @tab @code{subroutine acc_pcreate(a)}
3063 @item @tab @code{type, dimension(:[,:]...) :: a}
3064 @item @emph{Interface}: @tab @code{subroutine acc_pcreate(a, len)}
3065 @item @tab @code{type, dimension(:[,:]...) :: a}
3066 @item @tab @code{integer len}
3069 @item @emph{Reference}:
3070 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3077 @section @code{acc_copyout} -- Copy device memory to host memory.
3079 @item @emph{Description}
3080 This function copies mapped device memory to host memory which is specified
3081 by host address @var{a} for a length @var{len} bytes in C/C++.
3083 In Fortran, two (2) forms are supported. In the first form, @var{a} specifies
3084 a contiguous array section. The second form @var{a} specifies a variable or
3085 array element and @var{len} specifies the length in bytes.
3088 @multitable @columnfractions .20 .80
3089 @item @emph{Prototype}: @tab @code{acc_copyout(h_void *a, size_t len);}
3090 @item @emph{Prototype}: @tab @code{acc_copyout_async(h_void *a, size_t len, int async);}
3091 @item @emph{Prototype}: @tab @code{acc_copyout_finalize(h_void *a, size_t len);}
3092 @item @emph{Prototype}: @tab @code{acc_copyout_finalize_async(h_void *a, size_t len, int async);}
3095 @item @emph{Fortran}:
3096 @multitable @columnfractions .20 .80
3097 @item @emph{Interface}: @tab @code{subroutine acc_copyout(a)}
3098 @item @tab @code{type, dimension(:[,:]...) :: a}
3099 @item @emph{Interface}: @tab @code{subroutine acc_copyout(a, len)}
3100 @item @tab @code{type, dimension(:[,:]...) :: a}
3101 @item @tab @code{integer len}
3102 @item @emph{Interface}: @tab @code{subroutine acc_copyout_async(a, async)}
3103 @item @tab @code{type, dimension(:[,:]...) :: a}
3104 @item @tab @code{integer(acc_handle_kind) :: async}
3105 @item @emph{Interface}: @tab @code{subroutine acc_copyout_async(a, len, async)}
3106 @item @tab @code{type, dimension(:[,:]...) :: a}
3107 @item @tab @code{integer len}
3108 @item @tab @code{integer(acc_handle_kind) :: async}
3109 @item @emph{Interface}: @tab @code{subroutine acc_copyout_finalize(a)}
3110 @item @tab @code{type, dimension(:[,:]...) :: a}
3111 @item @emph{Interface}: @tab @code{subroutine acc_copyout_finalize(a, len)}
3112 @item @tab @code{type, dimension(:[,:]...) :: a}
3113 @item @tab @code{integer len}
3114 @item @emph{Interface}: @tab @code{subroutine acc_copyout_finalize_async(a, async)}
3115 @item @tab @code{type, dimension(:[,:]...) :: a}
3116 @item @tab @code{integer(acc_handle_kind) :: async}
3117 @item @emph{Interface}: @tab @code{subroutine acc_copyout_finalize_async(a, len, async)}
3118 @item @tab @code{type, dimension(:[,:]...) :: a}
3119 @item @tab @code{integer len}
3120 @item @tab @code{integer(acc_handle_kind) :: async}
3123 @item @emph{Reference}:
3124 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3131 @section @code{acc_delete} -- Free device memory.
3133 @item @emph{Description}
3134 This function frees previously allocated device memory specified by
3135 the device address @var{a} and the length of @var{len} bytes.
3137 In Fortran, two (2) forms are supported. In the first form, @var{a} specifies
3138 a contiguous array section. The second form @var{a} specifies a variable or
3139 array element and @var{len} specifies the length in bytes.
3142 @multitable @columnfractions .20 .80
3143 @item @emph{Prototype}: @tab @code{acc_delete(h_void *a, size_t len);}
3144 @item @emph{Prototype}: @tab @code{acc_delete_async(h_void *a, size_t len, int async);}
3145 @item @emph{Prototype}: @tab @code{acc_delete_finalize(h_void *a, size_t len);}
3146 @item @emph{Prototype}: @tab @code{acc_delete_finalize_async(h_void *a, size_t len, int async);}
3149 @item @emph{Fortran}:
3150 @multitable @columnfractions .20 .80
3151 @item @emph{Interface}: @tab @code{subroutine acc_delete(a)}
3152 @item @tab @code{type, dimension(:[,:]...) :: a}
3153 @item @emph{Interface}: @tab @code{subroutine acc_delete(a, len)}
3154 @item @tab @code{type, dimension(:[,:]...) :: a}
3155 @item @tab @code{integer len}
3156 @item @emph{Interface}: @tab @code{subroutine acc_delete_async(a, async)}
3157 @item @tab @code{type, dimension(:[,:]...) :: a}
3158 @item @tab @code{integer(acc_handle_kind) :: async}
3159 @item @emph{Interface}: @tab @code{subroutine acc_delete_async(a, len, async)}
3160 @item @tab @code{type, dimension(:[,:]...) :: a}
3161 @item @tab @code{integer len}
3162 @item @tab @code{integer(acc_handle_kind) :: async}
3163 @item @emph{Interface}: @tab @code{subroutine acc_delete_finalize(a)}
3164 @item @tab @code{type, dimension(:[,:]...) :: a}
3165 @item @emph{Interface}: @tab @code{subroutine acc_delete_finalize(a, len)}
3166 @item @tab @code{type, dimension(:[,:]...) :: a}
3167 @item @tab @code{integer len}
3168 @item @emph{Interface}: @tab @code{subroutine acc_delete_async_finalize(a, async)}
3169 @item @tab @code{type, dimension(:[,:]...) :: a}
3170 @item @tab @code{integer(acc_handle_kind) :: async}
3171 @item @emph{Interface}: @tab @code{subroutine acc_delete_async_finalize(a, len, async)}
3172 @item @tab @code{type, dimension(:[,:]...) :: a}
3173 @item @tab @code{integer len}
3174 @item @tab @code{integer(acc_handle_kind) :: async}
3177 @item @emph{Reference}:
3178 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3184 @node acc_update_device
3185 @section @code{acc_update_device} -- Update device memory from mapped host memory.
3187 @item @emph{Description}
3188 This function updates the device copy from the previously mapped host memory.
3189 The host memory is specified with the host address @var{a} and a length of
3192 In Fortran, two (2) forms are supported. In the first form, @var{a} specifies
3193 a contiguous array section. The second form @var{a} specifies a variable or
3194 array element and @var{len} specifies the length in bytes.
3197 @multitable @columnfractions .20 .80
3198 @item @emph{Prototype}: @tab @code{acc_update_device(h_void *a, size_t len);}
3199 @item @emph{Prototype}: @tab @code{acc_update_device(h_void *a, size_t len, async);}
3202 @item @emph{Fortran}:
3203 @multitable @columnfractions .20 .80
3204 @item @emph{Interface}: @tab @code{subroutine acc_update_device(a)}
3205 @item @tab @code{type, dimension(:[,:]...) :: a}
3206 @item @emph{Interface}: @tab @code{subroutine acc_update_device(a, len)}
3207 @item @tab @code{type, dimension(:[,:]...) :: a}
3208 @item @tab @code{integer len}
3209 @item @emph{Interface}: @tab @code{subroutine acc_update_device_async(a, async)}
3210 @item @tab @code{type, dimension(:[,:]...) :: a}
3211 @item @tab @code{integer(acc_handle_kind) :: async}
3212 @item @emph{Interface}: @tab @code{subroutine acc_update_device_async(a, len, async)}
3213 @item @tab @code{type, dimension(:[,:]...) :: a}
3214 @item @tab @code{integer len}
3215 @item @tab @code{integer(acc_handle_kind) :: async}
3218 @item @emph{Reference}:
3219 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3225 @node acc_update_self
3226 @section @code{acc_update_self} -- Update host memory from mapped device memory.
3228 @item @emph{Description}
3229 This function updates the host copy from the previously mapped device memory.
3230 The host memory is specified with the host address @var{a} and a length of
3233 In Fortran, two (2) forms are supported. In the first form, @var{a} specifies
3234 a contiguous array section. The second form @var{a} specifies a variable or
3235 array element and @var{len} specifies the length in bytes.
3238 @multitable @columnfractions .20 .80
3239 @item @emph{Prototype}: @tab @code{acc_update_self(h_void *a, size_t len);}
3240 @item @emph{Prototype}: @tab @code{acc_update_self_async(h_void *a, size_t len, int async);}
3243 @item @emph{Fortran}:
3244 @multitable @columnfractions .20 .80
3245 @item @emph{Interface}: @tab @code{subroutine acc_update_self(a)}
3246 @item @tab @code{type, dimension(:[,:]...) :: a}
3247 @item @emph{Interface}: @tab @code{subroutine acc_update_self(a, len)}
3248 @item @tab @code{type, dimension(:[,:]...) :: a}
3249 @item @tab @code{integer len}
3250 @item @emph{Interface}: @tab @code{subroutine acc_update_self_async(a, async)}
3251 @item @tab @code{type, dimension(:[,:]...) :: a}
3252 @item @tab @code{integer(acc_handle_kind) :: async}
3253 @item @emph{Interface}: @tab @code{subroutine acc_update_self_async(a, len, async)}
3254 @item @tab @code{type, dimension(:[,:]...) :: a}
3255 @item @tab @code{integer len}
3256 @item @tab @code{integer(acc_handle_kind) :: async}
3259 @item @emph{Reference}:
3260 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3267 @section @code{acc_map_data} -- Map previously allocated device memory to host memory.
3269 @item @emph{Description}
3270 This function maps previously allocated device and host memory. The device
3271 memory is specified with the device address @var{d}. The host memory is
3272 specified with the host address @var{h} and a length of @var{len}.
3275 @multitable @columnfractions .20 .80
3276 @item @emph{Prototype}: @tab @code{acc_map_data(h_void *h, d_void *d, size_t len);}
3279 @item @emph{Reference}:
3280 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3286 @node acc_unmap_data
3287 @section @code{acc_unmap_data} -- Unmap device memory from host memory.
3289 @item @emph{Description}
3290 This function unmaps previously mapped device and host memory. The latter
3291 specified by @var{h}.
3294 @multitable @columnfractions .20 .80
3295 @item @emph{Prototype}: @tab @code{acc_unmap_data(h_void *h);}
3298 @item @emph{Reference}:
3299 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3306 @section @code{acc_deviceptr} -- Get device pointer associated with specific host address.
3308 @item @emph{Description}
3309 This function returns the device address that has been mapped to the
3310 host address specified by @var{h}.
3313 @multitable @columnfractions .20 .80
3314 @item @emph{Prototype}: @tab @code{void *acc_deviceptr(h_void *h);}
3317 @item @emph{Reference}:
3318 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3325 @section @code{acc_hostptr} -- Get host pointer associated with specific device address.
3327 @item @emph{Description}
3328 This function returns the host address that has been mapped to the
3329 device address specified by @var{d}.
3332 @multitable @columnfractions .20 .80
3333 @item @emph{Prototype}: @tab @code{void *acc_hostptr(d_void *d);}
3336 @item @emph{Reference}:
3337 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3343 @node acc_is_present
3344 @section @code{acc_is_present} -- Indicate whether host variable / array is present on device.
3346 @item @emph{Description}
3347 This function indicates whether the specified host address in @var{a} and a
3348 length of @var{len} bytes is present on the device. In C/C++, a non-zero
3349 value is returned to indicate the presence of the mapped memory on the
3350 device. A zero is returned to indicate the memory is not mapped on the
3353 In Fortran, two (2) forms are supported. In the first form, @var{a} specifies
3354 a contiguous array section. The second form @var{a} specifies a variable or
3355 array element and @var{len} specifies the length in bytes. If the host
3356 memory is mapped to device memory, then a @code{true} is returned. Otherwise,
3357 a @code{false} is return to indicate the mapped memory is not present.
3360 @multitable @columnfractions .20 .80
3361 @item @emph{Prototype}: @tab @code{int acc_is_present(h_void *a, size_t len);}
3364 @item @emph{Fortran}:
3365 @multitable @columnfractions .20 .80
3366 @item @emph{Interface}: @tab @code{function acc_is_present(a)}
3367 @item @tab @code{type, dimension(:[,:]...) :: a}
3368 @item @tab @code{logical acc_is_present}
3369 @item @emph{Interface}: @tab @code{function acc_is_present(a, len)}
3370 @item @tab @code{type, dimension(:[,:]...) :: a}
3371 @item @tab @code{integer len}
3372 @item @tab @code{logical acc_is_present}
3375 @item @emph{Reference}:
3376 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3382 @node acc_memcpy_to_device
3383 @section @code{acc_memcpy_to_device} -- Copy host memory to device memory.
3385 @item @emph{Description}
3386 This function copies host memory specified by host address of @var{src} to
3387 device memory specified by the device address @var{dest} for a length of
3391 @multitable @columnfractions .20 .80
3392 @item @emph{Prototype}: @tab @code{acc_memcpy_to_device(d_void *dest, h_void *src, size_t bytes);}
3395 @item @emph{Reference}:
3396 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3402 @node acc_memcpy_from_device
3403 @section @code{acc_memcpy_from_device} -- Copy device memory to host memory.
3405 @item @emph{Description}
3406 This function copies host memory specified by host address of @var{src} from
3407 device memory specified by the device address @var{dest} for a length of
3411 @multitable @columnfractions .20 .80
3412 @item @emph{Prototype}: @tab @code{acc_memcpy_from_device(d_void *dest, h_void *src, size_t bytes);}
3415 @item @emph{Reference}:
3416 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3423 @section @code{acc_attach} -- Let device pointer point to device-pointer target.
3425 @item @emph{Description}
3426 This function updates a pointer on the device from pointing to a host-pointer
3427 address to pointing to the corresponding device data.
3430 @multitable @columnfractions .20 .80
3431 @item @emph{Prototype}: @tab @code{acc_attach(h_void **ptr);}
3432 @item @emph{Prototype}: @tab @code{acc_attach_async(h_void **ptr, int async);}
3435 @item @emph{Reference}:
3436 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3443 @section @code{acc_detach} -- Let device pointer point to host-pointer target.
3445 @item @emph{Description}
3446 This function updates a pointer on the device from pointing to a device-pointer
3447 address to pointing to the corresponding host data.
3450 @multitable @columnfractions .20 .80
3451 @item @emph{Prototype}: @tab @code{acc_detach(h_void **ptr);}
3452 @item @emph{Prototype}: @tab @code{acc_detach_async(h_void **ptr, int async);}
3453 @item @emph{Prototype}: @tab @code{acc_detach_finalize(h_void **ptr);}
3454 @item @emph{Prototype}: @tab @code{acc_detach_finalize_async(h_void **ptr, int async);}
3457 @item @emph{Reference}:
3458 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3464 @node acc_get_current_cuda_device
3465 @section @code{acc_get_current_cuda_device} -- Get CUDA device handle.
3467 @item @emph{Description}
3468 This function returns the CUDA device handle. This handle is the same
3469 as used by the CUDA Runtime or Driver API's.
3472 @multitable @columnfractions .20 .80
3473 @item @emph{Prototype}: @tab @code{void *acc_get_current_cuda_device(void);}
3476 @item @emph{Reference}:
3477 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3483 @node acc_get_current_cuda_context
3484 @section @code{acc_get_current_cuda_context} -- Get CUDA context handle.
3486 @item @emph{Description}
3487 This function returns the CUDA context handle. This handle is the same
3488 as used by the CUDA Runtime or Driver API's.
3491 @multitable @columnfractions .20 .80
3492 @item @emph{Prototype}: @tab @code{void *acc_get_current_cuda_context(void);}
3495 @item @emph{Reference}:
3496 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3502 @node acc_get_cuda_stream
3503 @section @code{acc_get_cuda_stream} -- Get CUDA stream handle.
3505 @item @emph{Description}
3506 This function returns the CUDA stream handle for the queue @var{async}.
3507 This handle is the same as used by the CUDA Runtime or Driver API's.
3510 @multitable @columnfractions .20 .80
3511 @item @emph{Prototype}: @tab @code{void *acc_get_cuda_stream(int async);}
3514 @item @emph{Reference}:
3515 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3521 @node acc_set_cuda_stream
3522 @section @code{acc_set_cuda_stream} -- Set CUDA stream handle.
3524 @item @emph{Description}
3525 This function associates the stream handle specified by @var{stream} with
3526 the queue @var{async}.
3528 This cannot be used to change the stream handle associated with
3529 @code{acc_async_sync}.
3531 The return value is not specified.
3534 @multitable @columnfractions .20 .80
3535 @item @emph{Prototype}: @tab @code{int acc_set_cuda_stream(int async, void *stream);}
3538 @item @emph{Reference}:
3539 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3545 @node acc_prof_register
3546 @section @code{acc_prof_register} -- Register callbacks.
3548 @item @emph{Description}:
3549 This function registers callbacks.
3552 @multitable @columnfractions .20 .80
3553 @item @emph{Prototype}: @tab @code{void acc_prof_register (acc_event_t, acc_prof_callback, acc_register_t);}
3556 @item @emph{See also}:
3557 @ref{OpenACC Profiling Interface}
3559 @item @emph{Reference}:
3560 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3566 @node acc_prof_unregister
3567 @section @code{acc_prof_unregister} -- Unregister callbacks.
3569 @item @emph{Description}:
3570 This function unregisters callbacks.
3573 @multitable @columnfractions .20 .80
3574 @item @emph{Prototype}: @tab @code{void acc_prof_unregister (acc_event_t, acc_prof_callback, acc_register_t);}
3577 @item @emph{See also}:
3578 @ref{OpenACC Profiling Interface}
3580 @item @emph{Reference}:
3581 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3587 @node acc_prof_lookup
3588 @section @code{acc_prof_lookup} -- Obtain inquiry functions.
3590 @item @emph{Description}:
3591 Function to obtain inquiry functions.
3594 @multitable @columnfractions .20 .80
3595 @item @emph{Prototype}: @tab @code{acc_query_fn acc_prof_lookup (const char *);}
3598 @item @emph{See also}:
3599 @ref{OpenACC Profiling Interface}
3601 @item @emph{Reference}:
3602 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3608 @node acc_register_library
3609 @section @code{acc_register_library} -- Library registration.
3611 @item @emph{Description}:
3612 Function for library registration.
3615 @multitable @columnfractions .20 .80
3616 @item @emph{Prototype}: @tab @code{void acc_register_library (acc_prof_reg, acc_prof_reg, acc_prof_lookup_func);}
3619 @item @emph{See also}:
3620 @ref{OpenACC Profiling Interface}, @ref{ACC_PROFLIB}
3622 @item @emph{Reference}:
3623 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3629 @c ---------------------------------------------------------------------
3630 @c OpenACC Environment Variables
3631 @c ---------------------------------------------------------------------
3633 @node OpenACC Environment Variables
3634 @chapter OpenACC Environment Variables
3636 The variables @env{ACC_DEVICE_TYPE} and @env{ACC_DEVICE_NUM}
3637 are defined by section 4 of the OpenACC specification in version 2.0.
3638 The variable @env{ACC_PROFLIB}
3639 is defined by section 4 of the OpenACC specification in version 2.6.
3640 The variable @env{GCC_ACC_NOTIFY} is used for diagnostic purposes.
3651 @node ACC_DEVICE_TYPE
3652 @section @code{ACC_DEVICE_TYPE}
3654 @item @emph{Reference}:
3655 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3661 @node ACC_DEVICE_NUM
3662 @section @code{ACC_DEVICE_NUM}
3664 @item @emph{Reference}:
3665 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3672 @section @code{ACC_PROFLIB}
3674 @item @emph{See also}:
3675 @ref{acc_register_library}, @ref{OpenACC Profiling Interface}
3677 @item @emph{Reference}:
3678 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3684 @node GCC_ACC_NOTIFY
3685 @section @code{GCC_ACC_NOTIFY}
3687 @item @emph{Description}:
3688 Print debug information pertaining to the accelerator.
3693 @c ---------------------------------------------------------------------
3694 @c CUDA Streams Usage
3695 @c ---------------------------------------------------------------------
3697 @node CUDA Streams Usage
3698 @chapter CUDA Streams Usage
3700 This applies to the @code{nvptx} plugin only.
3702 The library provides elements that perform asynchronous movement of
3703 data and asynchronous operation of computing constructs. This
3704 asynchronous functionality is implemented by making use of CUDA
3705 streams@footnote{See "Stream Management" in "CUDA Driver API",
3706 TRM-06703-001, Version 5.5, for additional information}.
3708 The primary means by that the asynchronous functionality is accessed
3709 is through the use of those OpenACC directives which make use of the
3710 @code{async} and @code{wait} clauses. When the @code{async} clause is
3711 first used with a directive, it creates a CUDA stream. If an
3712 @code{async-argument} is used with the @code{async} clause, then the
3713 stream is associated with the specified @code{async-argument}.
3715 Following the creation of an association between a CUDA stream and the
3716 @code{async-argument} of an @code{async} clause, both the @code{wait}
3717 clause and the @code{wait} directive can be used. When either the
3718 clause or directive is used after stream creation, it creates a
3719 rendezvous point whereby execution waits until all operations
3720 associated with the @code{async-argument}, that is, stream, have
3723 Normally, the management of the streams that are created as a result of
3724 using the @code{async} clause, is done without any intervention by the
3725 caller. This implies the association between the @code{async-argument}
3726 and the CUDA stream will be maintained for the lifetime of the program.
3727 However, this association can be changed through the use of the library
3728 function @code{acc_set_cuda_stream}. When the function
3729 @code{acc_set_cuda_stream} is called, the CUDA stream that was
3730 originally associated with the @code{async} clause will be destroyed.
3731 Caution should be taken when changing the association as subsequent
3732 references to the @code{async-argument} refer to a different
3737 @c ---------------------------------------------------------------------
3738 @c OpenACC Library Interoperability
3739 @c ---------------------------------------------------------------------
3741 @node OpenACC Library Interoperability
3742 @chapter OpenACC Library Interoperability
3744 @section Introduction
3746 The OpenACC library uses the CUDA Driver API, and may interact with
3747 programs that use the Runtime library directly, or another library
3748 based on the Runtime library, e.g., CUBLAS@footnote{See section 2.26,
3749 "Interactions with the CUDA Driver API" in
3750 "CUDA Runtime API", Version 5.5, and section 2.27, "VDPAU
3751 Interoperability", in "CUDA Driver API", TRM-06703-001, Version 5.5,
3752 for additional information on library interoperability.}.
3753 This chapter describes the use cases and what changes are
3754 required in order to use both the OpenACC library and the CUBLAS and Runtime
3755 libraries within a program.
3757 @section First invocation: NVIDIA CUBLAS library API
3759 In this first use case (see below), a function in the CUBLAS library is called
3760 prior to any of the functions in the OpenACC library. More specifically, the
3761 function @code{cublasCreate()}.
3763 When invoked, the function initializes the library and allocates the
3764 hardware resources on the host and the device on behalf of the caller. Once
3765 the initialization and allocation has completed, a handle is returned to the
3766 caller. The OpenACC library also requires initialization and allocation of
3767 hardware resources. Since the CUBLAS library has already allocated the
3768 hardware resources for the device, all that is left to do is to initialize
3769 the OpenACC library and acquire the hardware resources on the host.
3771 Prior to calling the OpenACC function that initializes the library and
3772 allocate the host hardware resources, you need to acquire the device number
3773 that was allocated during the call to @code{cublasCreate()}. The invoking of the
3774 runtime library function @code{cudaGetDevice()} accomplishes this. Once
3775 acquired, the device number is passed along with the device type as
3776 parameters to the OpenACC library function @code{acc_set_device_num()}.
3778 Once the call to @code{acc_set_device_num()} has completed, the OpenACC
3779 library uses the context that was created during the call to
3780 @code{cublasCreate()}. In other words, both libraries will be sharing the
3784 /* Create the handle */
3785 s = cublasCreate(&h);
3786 if (s != CUBLAS_STATUS_SUCCESS)
3788 fprintf(stderr, "cublasCreate failed %d\n", s);
3792 /* Get the device number */
3793 e = cudaGetDevice(&dev);
3794 if (e != cudaSuccess)
3796 fprintf(stderr, "cudaGetDevice failed %d\n", e);
3800 /* Initialize OpenACC library and use device 'dev' */
3801 acc_set_device_num(dev, acc_device_nvidia);
3806 @section First invocation: OpenACC library API
3808 In this second use case (see below), a function in the OpenACC library is
3809 called prior to any of the functions in the CUBLAS library. More specificially,
3810 the function @code{acc_set_device_num()}.
3812 In the use case presented here, the function @code{acc_set_device_num()}
3813 is used to both initialize the OpenACC library and allocate the hardware
3814 resources on the host and the device. In the call to the function, the
3815 call parameters specify which device to use and what device
3816 type to use, i.e., @code{acc_device_nvidia}. It should be noted that this
3817 is but one method to initialize the OpenACC library and allocate the
3818 appropriate hardware resources. Other methods are available through the
3819 use of environment variables and these will be discussed in the next section.
3821 Once the call to @code{acc_set_device_num()} has completed, other OpenACC
3822 functions can be called as seen with multiple calls being made to
3823 @code{acc_copyin()}. In addition, calls can be made to functions in the
3824 CUBLAS library. In the use case a call to @code{cublasCreate()} is made
3825 subsequent to the calls to @code{acc_copyin()}.
3826 As seen in the previous use case, a call to @code{cublasCreate()}
3827 initializes the CUBLAS library and allocates the hardware resources on the
3828 host and the device. However, since the device has already been allocated,
3829 @code{cublasCreate()} will only initialize the CUBLAS library and allocate
3830 the appropriate hardware resources on the host. The context that was created
3831 as part of the OpenACC initialization is shared with the CUBLAS library,
3832 similarly to the first use case.
3837 acc_set_device_num(dev, acc_device_nvidia);
3839 /* Copy the first set to the device */
3840 d_X = acc_copyin(&h_X[0], N * sizeof (float));
3843 fprintf(stderr, "copyin error h_X\n");
3847 /* Copy the second set to the device */
3848 d_Y = acc_copyin(&h_Y1[0], N * sizeof (float));
3851 fprintf(stderr, "copyin error h_Y1\n");
3855 /* Create the handle */
3856 s = cublasCreate(&h);
3857 if (s != CUBLAS_STATUS_SUCCESS)
3859 fprintf(stderr, "cublasCreate failed %d\n", s);
3863 /* Perform saxpy using CUBLAS library function */
3864 s = cublasSaxpy(h, N, &alpha, d_X, 1, d_Y, 1);
3865 if (s != CUBLAS_STATUS_SUCCESS)
3867 fprintf(stderr, "cublasSaxpy failed %d\n", s);
3871 /* Copy the results from the device */
3872 acc_memcpy_from_device(&h_Y1[0], d_Y, N * sizeof (float));
3877 @section OpenACC library and environment variables
3879 There are two environment variables associated with the OpenACC library
3880 that may be used to control the device type and device number:
3881 @env{ACC_DEVICE_TYPE} and @env{ACC_DEVICE_NUM}, respectively. These two
3882 environment variables can be used as an alternative to calling
3883 @code{acc_set_device_num()}. As seen in the second use case, the device
3884 type and device number were specified using @code{acc_set_device_num()}.
3885 If however, the aforementioned environment variables were set, then the
3886 call to @code{acc_set_device_num()} would not be required.
3889 The use of the environment variables is only relevant when an OpenACC function
3890 is called prior to a call to @code{cudaCreate()}. If @code{cudaCreate()}
3891 is called prior to a call to an OpenACC function, then you must call
3892 @code{acc_set_device_num()}@footnote{More complete information
3893 about @env{ACC_DEVICE_TYPE} and @env{ACC_DEVICE_NUM} can be found in
3894 sections 4.1 and 4.2 of the @uref{https://www.openacc.org, OpenACC}
3895 Application Programming Interface”, Version 2.6.}
3899 @c ---------------------------------------------------------------------
3900 @c OpenACC Profiling Interface
3901 @c ---------------------------------------------------------------------
3903 @node OpenACC Profiling Interface
3904 @chapter OpenACC Profiling Interface
3906 @section Implementation Status and Implementation-Defined Behavior
3908 We're implementing the OpenACC Profiling Interface as defined by the
3909 OpenACC 2.6 specification. We're clarifying some aspects here as
3910 @emph{implementation-defined behavior}, while they're still under
3911 discussion within the OpenACC Technical Committee.
3913 This implementation is tuned to keep the performance impact as low as
3914 possible for the (very common) case that the Profiling Interface is
3915 not enabled. This is relevant, as the Profiling Interface affects all
3916 the @emph{hot} code paths (in the target code, not in the offloaded
3917 code). Users of the OpenACC Profiling Interface can be expected to
3918 understand that performance will be impacted to some degree once the
3919 Profiling Interface has gotten enabled: for example, because of the
3920 @emph{runtime} (libgomp) calling into a third-party @emph{library} for
3921 every event that has been registered.
3923 We're not yet accounting for the fact that @cite{OpenACC events may
3924 occur during event processing}.
3925 We just handle one case specially, as required by CUDA 9.0
3926 @command{nvprof}, that @code{acc_get_device_type}
3927 (@ref{acc_get_device_type})) may be called from
3928 @code{acc_ev_device_init_start}, @code{acc_ev_device_init_end}
3931 We're not yet implementing initialization via a
3932 @code{acc_register_library} function that is either statically linked
3933 in, or dynamically via @env{LD_PRELOAD}.
3934 Initialization via @code{acc_register_library} functions dynamically
3935 loaded via the @env{ACC_PROFLIB} environment variable does work, as
3936 does directly calling @code{acc_prof_register},
3937 @code{acc_prof_unregister}, @code{acc_prof_lookup}.
3939 As currently there are no inquiry functions defined, calls to
3940 @code{acc_prof_lookup} will always return @code{NULL}.
3942 There aren't separate @emph{start}, @emph{stop} events defined for the
3943 event types @code{acc_ev_create}, @code{acc_ev_delete},
3944 @code{acc_ev_alloc}, @code{acc_ev_free}. It's not clear if these
3945 should be triggered before or after the actual device-specific call is
3946 made. We trigger them after.
3948 Remarks about data provided to callbacks:
3952 @item @code{acc_prof_info.event_type}
3953 It's not clear if for @emph{nested} event callbacks (for example,
3954 @code{acc_ev_enqueue_launch_start} as part of a parent compute
3955 construct), this should be set for the nested event
3956 (@code{acc_ev_enqueue_launch_start}), or if the value of the parent
3957 construct should remain (@code{acc_ev_compute_construct_start}). In
3958 this implementation, the value will generally correspond to the
3959 innermost nested event type.
3961 @item @code{acc_prof_info.device_type}
3965 For @code{acc_ev_compute_construct_start}, and in presence of an
3966 @code{if} clause with @emph{false} argument, this will still refer to
3967 the offloading device type.
3968 It's not clear if that's the expected behavior.
3971 Complementary to the item before, for
3972 @code{acc_ev_compute_construct_end}, this is set to
3973 @code{acc_device_host} in presence of an @code{if} clause with
3974 @emph{false} argument.
3975 It's not clear if that's the expected behavior.
3979 @item @code{acc_prof_info.thread_id}
3980 Always @code{-1}; not yet implemented.
3982 @item @code{acc_prof_info.async}
3986 Not yet implemented correctly for
3987 @code{acc_ev_compute_construct_start}.
3990 In a compute construct, for host-fallback
3991 execution/@code{acc_device_host} it will always be
3992 @code{acc_async_sync}.
3993 It's not clear if that's the expected behavior.
3996 For @code{acc_ev_device_init_start} and @code{acc_ev_device_init_end},
3997 it will always be @code{acc_async_sync}.
3998 It's not clear if that's the expected behavior.
4002 @item @code{acc_prof_info.async_queue}
4003 There is no @cite{limited number of asynchronous queues} in libgomp.
4004 This will always have the same value as @code{acc_prof_info.async}.
4006 @item @code{acc_prof_info.src_file}
4007 Always @code{NULL}; not yet implemented.
4009 @item @code{acc_prof_info.func_name}
4010 Always @code{NULL}; not yet implemented.
4012 @item @code{acc_prof_info.line_no}
4013 Always @code{-1}; not yet implemented.
4015 @item @code{acc_prof_info.end_line_no}
4016 Always @code{-1}; not yet implemented.
4018 @item @code{acc_prof_info.func_line_no}
4019 Always @code{-1}; not yet implemented.
4021 @item @code{acc_prof_info.func_end_line_no}
4022 Always @code{-1}; not yet implemented.
4024 @item @code{acc_event_info.event_type}, @code{acc_event_info.*.event_type}
4025 Relating to @code{acc_prof_info.event_type} discussed above, in this
4026 implementation, this will always be the same value as
4027 @code{acc_prof_info.event_type}.
4029 @item @code{acc_event_info.*.parent_construct}
4033 Will be @code{acc_construct_parallel} for all OpenACC compute
4034 constructs as well as many OpenACC Runtime API calls; should be the
4035 one matching the actual construct, or
4036 @code{acc_construct_runtime_api}, respectively.
4039 Will be @code{acc_construct_enter_data} or
4040 @code{acc_construct_exit_data} when processing variable mappings
4041 specified in OpenACC @emph{declare} directives; should be
4042 @code{acc_construct_declare}.
4045 For implicit @code{acc_ev_device_init_start},
4046 @code{acc_ev_device_init_end}, and explicit as well as implicit
4047 @code{acc_ev_alloc}, @code{acc_ev_free},
4048 @code{acc_ev_enqueue_upload_start}, @code{acc_ev_enqueue_upload_end},
4049 @code{acc_ev_enqueue_download_start}, and
4050 @code{acc_ev_enqueue_download_end}, will be
4051 @code{acc_construct_parallel}; should reflect the real parent
4056 @item @code{acc_event_info.*.implicit}
4057 For @code{acc_ev_alloc}, @code{acc_ev_free},
4058 @code{acc_ev_enqueue_upload_start}, @code{acc_ev_enqueue_upload_end},
4059 @code{acc_ev_enqueue_download_start}, and
4060 @code{acc_ev_enqueue_download_end}, this currently will be @code{1}
4061 also for explicit usage.
4063 @item @code{acc_event_info.data_event.var_name}
4064 Always @code{NULL}; not yet implemented.
4066 @item @code{acc_event_info.data_event.host_ptr}
4067 For @code{acc_ev_alloc}, and @code{acc_ev_free}, this is always
4070 @item @code{typedef union acc_api_info}
4071 @dots{} as printed in @cite{5.2.3. Third Argument: API-Specific
4072 Information}. This should obviously be @code{typedef @emph{struct}
4075 @item @code{acc_api_info.device_api}
4076 Possibly not yet implemented correctly for
4077 @code{acc_ev_compute_construct_start},
4078 @code{acc_ev_device_init_start}, @code{acc_ev_device_init_end}:
4079 will always be @code{acc_device_api_none} for these event types.
4080 For @code{acc_ev_enter_data_start}, it will be
4081 @code{acc_device_api_none} in some cases.
4083 @item @code{acc_api_info.device_type}
4084 Always the same as @code{acc_prof_info.device_type}.
4086 @item @code{acc_api_info.vendor}
4087 Always @code{-1}; not yet implemented.
4089 @item @code{acc_api_info.device_handle}
4090 Always @code{NULL}; not yet implemented.
4092 @item @code{acc_api_info.context_handle}
4093 Always @code{NULL}; not yet implemented.
4095 @item @code{acc_api_info.async_handle}
4096 Always @code{NULL}; not yet implemented.
4100 Remarks about certain event types:
4104 @item @code{acc_ev_device_init_start}, @code{acc_ev_device_init_end}
4108 @c See 'DEVICE_INIT_INSIDE_COMPUTE_CONSTRUCT' in
4109 @c 'libgomp.oacc-c-c++-common/acc_prof-kernels-1.c',
4110 @c 'libgomp.oacc-c-c++-common/acc_prof-parallel-1.c'.
4111 When a compute construct triggers implicit
4112 @code{acc_ev_device_init_start} and @code{acc_ev_device_init_end}
4113 events, they currently aren't @emph{nested within} the corresponding
4114 @code{acc_ev_compute_construct_start} and
4115 @code{acc_ev_compute_construct_end}, but they're currently observed
4116 @emph{before} @code{acc_ev_compute_construct_start}.
4117 It's not clear what to do: the standard asks us provide a lot of
4118 details to the @code{acc_ev_compute_construct_start} callback, without
4119 (implicitly) initializing a device before?
4122 Callbacks for these event types will not be invoked for calls to the
4123 @code{acc_set_device_type} and @code{acc_set_device_num} functions.
4124 It's not clear if they should be.
4128 @item @code{acc_ev_enter_data_start}, @code{acc_ev_enter_data_end}, @code{acc_ev_exit_data_start}, @code{acc_ev_exit_data_end}
4132 Callbacks for these event types will also be invoked for OpenACC
4133 @emph{host_data} constructs.
4134 It's not clear if they should be.
4137 Callbacks for these event types will also be invoked when processing
4138 variable mappings specified in OpenACC @emph{declare} directives.
4139 It's not clear if they should be.
4145 Callbacks for the following event types will be invoked, but dispatch
4146 and information provided therein has not yet been thoroughly reviewed:
4149 @item @code{acc_ev_alloc}
4150 @item @code{acc_ev_free}
4151 @item @code{acc_ev_update_start}, @code{acc_ev_update_end}
4152 @item @code{acc_ev_enqueue_upload_start}, @code{acc_ev_enqueue_upload_end}
4153 @item @code{acc_ev_enqueue_download_start}, @code{acc_ev_enqueue_download_end}
4156 During device initialization, and finalization, respectively,
4157 callbacks for the following event types will not yet be invoked:
4160 @item @code{acc_ev_alloc}
4161 @item @code{acc_ev_free}
4164 Callbacks for the following event types have not yet been implemented,
4165 so currently won't be invoked:
4168 @item @code{acc_ev_device_shutdown_start}, @code{acc_ev_device_shutdown_end}
4169 @item @code{acc_ev_runtime_shutdown}
4170 @item @code{acc_ev_create}, @code{acc_ev_delete}
4171 @item @code{acc_ev_wait_start}, @code{acc_ev_wait_end}
4174 For the following runtime library functions, not all expected
4175 callbacks will be invoked (mostly concerning implicit device
4179 @item @code{acc_get_num_devices}
4180 @item @code{acc_set_device_type}
4181 @item @code{acc_get_device_type}
4182 @item @code{acc_set_device_num}
4183 @item @code{acc_get_device_num}
4184 @item @code{acc_init}
4185 @item @code{acc_shutdown}
4188 Aside from implicit device initialization, for the following runtime
4189 library functions, no callbacks will be invoked for shared-memory
4190 offloading devices (it's not clear if they should be):
4193 @item @code{acc_malloc}
4194 @item @code{acc_free}
4195 @item @code{acc_copyin}, @code{acc_present_or_copyin}, @code{acc_copyin_async}
4196 @item @code{acc_create}, @code{acc_present_or_create}, @code{acc_create_async}
4197 @item @code{acc_copyout}, @code{acc_copyout_async}, @code{acc_copyout_finalize}, @code{acc_copyout_finalize_async}
4198 @item @code{acc_delete}, @code{acc_delete_async}, @code{acc_delete_finalize}, @code{acc_delete_finalize_async}
4199 @item @code{acc_update_device}, @code{acc_update_device_async}
4200 @item @code{acc_update_self}, @code{acc_update_self_async}
4201 @item @code{acc_map_data}, @code{acc_unmap_data}
4202 @item @code{acc_memcpy_to_device}, @code{acc_memcpy_to_device_async}
4203 @item @code{acc_memcpy_from_device}, @code{acc_memcpy_from_device_async}
4208 @c ---------------------------------------------------------------------
4210 @c ---------------------------------------------------------------------
4212 @node The libgomp ABI
4213 @chapter The libgomp ABI
4215 The following sections present notes on the external ABI as
4216 presented by libgomp. Only maintainers should need them.
4219 * Implementing MASTER construct::
4220 * Implementing CRITICAL construct::
4221 * Implementing ATOMIC construct::
4222 * Implementing FLUSH construct::
4223 * Implementing BARRIER construct::
4224 * Implementing THREADPRIVATE construct::
4225 * Implementing PRIVATE clause::
4226 * Implementing FIRSTPRIVATE LASTPRIVATE COPYIN and COPYPRIVATE clauses::
4227 * Implementing REDUCTION clause::
4228 * Implementing PARALLEL construct::
4229 * Implementing FOR construct::
4230 * Implementing ORDERED construct::
4231 * Implementing SECTIONS construct::
4232 * Implementing SINGLE construct::
4233 * Implementing OpenACC's PARALLEL construct::
4237 @node Implementing MASTER construct
4238 @section Implementing MASTER construct
4241 if (omp_get_thread_num () == 0)
4245 Alternately, we generate two copies of the parallel subfunction
4246 and only include this in the version run by the primary thread.
4247 Surely this is not worthwhile though...
4251 @node Implementing CRITICAL construct
4252 @section Implementing CRITICAL construct
4254 Without a specified name,
4257 void GOMP_critical_start (void);
4258 void GOMP_critical_end (void);
4261 so that we don't get COPY relocations from libgomp to the main
4264 With a specified name, use omp_set_lock and omp_unset_lock with
4265 name being transformed into a variable declared like
4268 omp_lock_t gomp_critical_user_<name> __attribute__((common))
4271 Ideally the ABI would specify that all zero is a valid unlocked
4272 state, and so we wouldn't need to initialize this at
4277 @node Implementing ATOMIC construct
4278 @section Implementing ATOMIC construct
4280 The target should implement the @code{__sync} builtins.
4282 Failing that we could add
4285 void GOMP_atomic_enter (void)
4286 void GOMP_atomic_exit (void)
4289 which reuses the regular lock code, but with yet another lock
4290 object private to the library.
4294 @node Implementing FLUSH construct
4295 @section Implementing FLUSH construct
4297 Expands to the @code{__sync_synchronize} builtin.
4301 @node Implementing BARRIER construct
4302 @section Implementing BARRIER construct
4305 void GOMP_barrier (void)
4309 @node Implementing THREADPRIVATE construct
4310 @section Implementing THREADPRIVATE construct
4312 In _most_ cases we can map this directly to @code{__thread}. Except
4313 that OMP allows constructors for C++ objects. We can either
4314 refuse to support this (how often is it used?) or we can
4315 implement something akin to .ctors.
4317 Even more ideally, this ctor feature is handled by extensions
4318 to the main pthreads library. Failing that, we can have a set
4319 of entry points to register ctor functions to be called.
4323 @node Implementing PRIVATE clause
4324 @section Implementing PRIVATE clause
4326 In association with a PARALLEL, or within the lexical extent
4327 of a PARALLEL block, the variable becomes a local variable in
4328 the parallel subfunction.
4330 In association with FOR or SECTIONS blocks, create a new
4331 automatic variable within the current function. This preserves
4332 the semantic of new variable creation.
4336 @node Implementing FIRSTPRIVATE LASTPRIVATE COPYIN and COPYPRIVATE clauses
4337 @section Implementing FIRSTPRIVATE LASTPRIVATE COPYIN and COPYPRIVATE clauses
4339 This seems simple enough for PARALLEL blocks. Create a private
4340 struct for communicating between the parent and subfunction.
4341 In the parent, copy in values for scalar and "small" structs;
4342 copy in addresses for others TREE_ADDRESSABLE types. In the
4343 subfunction, copy the value into the local variable.
4345 It is not clear what to do with bare FOR or SECTION blocks.
4346 The only thing I can figure is that we do something like:
4349 #pragma omp for firstprivate(x) lastprivate(y)
4350 for (int i = 0; i < n; ++i)
4367 where the "x=x" and "y=y" assignments actually have different
4368 uids for the two variables, i.e. not something you could write
4369 directly in C. Presumably this only makes sense if the "outer"
4370 x and y are global variables.
4372 COPYPRIVATE would work the same way, except the structure
4373 broadcast would have to happen via SINGLE machinery instead.
4377 @node Implementing REDUCTION clause
4378 @section Implementing REDUCTION clause
4380 The private struct mentioned in the previous section should have
4381 a pointer to an array of the type of the variable, indexed by the
4382 thread's @var{team_id}. The thread stores its final value into the
4383 array, and after the barrier, the primary thread iterates over the
4384 array to collect the values.
4387 @node Implementing PARALLEL construct
4388 @section Implementing PARALLEL construct
4391 #pragma omp parallel
4400 void subfunction (void *data)
4407 GOMP_parallel_start (subfunction, &data, num_threads);
4408 subfunction (&data);
4409 GOMP_parallel_end ();
4413 void GOMP_parallel_start (void (*fn)(void *), void *data, unsigned num_threads)
4416 The @var{FN} argument is the subfunction to be run in parallel.
4418 The @var{DATA} argument is a pointer to a structure used to
4419 communicate data in and out of the subfunction, as discussed
4420 above with respect to FIRSTPRIVATE et al.
4422 The @var{NUM_THREADS} argument is 1 if an IF clause is present
4423 and false, or the value of the NUM_THREADS clause, if
4426 The function needs to create the appropriate number of
4427 threads and/or launch them from the dock. It needs to
4428 create the team structure and assign team ids.
4431 void GOMP_parallel_end (void)
4434 Tears down the team and returns us to the previous @code{omp_in_parallel()} state.
4438 @node Implementing FOR construct
4439 @section Implementing FOR construct
4442 #pragma omp parallel for
4443 for (i = lb; i <= ub; i++)
4450 void subfunction (void *data)
4453 while (GOMP_loop_static_next (&_s0, &_e0))
4456 for (i = _s0; i < _e1; i++)
4459 GOMP_loop_end_nowait ();
4462 GOMP_parallel_loop_static (subfunction, NULL, 0, lb, ub+1, 1, 0);
4464 GOMP_parallel_end ();
4468 #pragma omp for schedule(runtime)
4469 for (i = 0; i < n; i++)
4478 if (GOMP_loop_runtime_start (0, n, 1, &_s0, &_e0))
4481 for (i = _s0, i < _e0; i++)
4483 @} while (GOMP_loop_runtime_next (&_s0, _&e0));
4488 Note that while it looks like there is trickiness to propagating
4489 a non-constant STEP, there isn't really. We're explicitly allowed
4490 to evaluate it as many times as we want, and any variables involved
4491 should automatically be handled as PRIVATE or SHARED like any other
4492 variables. So the expression should remain evaluable in the
4493 subfunction. We can also pull it into a local variable if we like,
4494 but since its supposed to remain unchanged, we can also not if we like.
4496 If we have SCHEDULE(STATIC), and no ORDERED, then we ought to be
4497 able to get away with no work-sharing context at all, since we can
4498 simply perform the arithmetic directly in each thread to divide up
4499 the iterations. Which would mean that we wouldn't need to call any
4502 There are separate routines for handling loops with an ORDERED
4503 clause. Bookkeeping for that is non-trivial...
4507 @node Implementing ORDERED construct
4508 @section Implementing ORDERED construct
4511 void GOMP_ordered_start (void)
4512 void GOMP_ordered_end (void)
4517 @node Implementing SECTIONS construct
4518 @section Implementing SECTIONS construct
4523 #pragma omp sections
4537 for (i = GOMP_sections_start (3); i != 0; i = GOMP_sections_next ())
4554 @node Implementing SINGLE construct
4555 @section Implementing SINGLE construct
4569 if (GOMP_single_start ())
4577 #pragma omp single copyprivate(x)
4584 datap = GOMP_single_copy_start ();
4589 GOMP_single_copy_end (&data);
4598 @node Implementing OpenACC's PARALLEL construct
4599 @section Implementing OpenACC's PARALLEL construct
4602 void GOACC_parallel ()
4607 @c ---------------------------------------------------------------------
4609 @c ---------------------------------------------------------------------
4611 @node Reporting Bugs
4612 @chapter Reporting Bugs
4614 Bugs in the GNU Offloading and Multi Processing Runtime Library should
4615 be reported via @uref{https://gcc.gnu.org/bugzilla/, Bugzilla}. Please add
4616 "openacc", or "openmp", or both to the keywords field in the bug
4617 report, as appropriate.
4621 @c ---------------------------------------------------------------------
4622 @c GNU General Public License
4623 @c ---------------------------------------------------------------------
4625 @include gpl_v3.texi
4629 @c ---------------------------------------------------------------------
4630 @c GNU Free Documentation License
4631 @c ---------------------------------------------------------------------
4637 @c ---------------------------------------------------------------------
4638 @c Funding Free Software
4639 @c ---------------------------------------------------------------------
4641 @include funding.texi
4643 @c ---------------------------------------------------------------------
4645 @c ---------------------------------------------------------------------
4648 @unnumbered Library Index