1 \input texinfo @c -*-texinfo-*-
4 @setfilename libgomp.info
10 Copyright @copyright{} 2006-2022 Free Software Foundation, Inc.
12 Permission is granted to copy, distribute and/or modify this document
13 under the terms of the GNU Free Documentation License, Version 1.3 or
14 any later version published by the Free Software Foundation; with the
15 Invariant Sections being ``Funding Free Software'', the Front-Cover
16 texts being (a) (see below), and with the Back-Cover Texts being (b)
17 (see below). A copy of the license is included in the section entitled
18 ``GNU Free Documentation License''.
20 (a) The FSF's Front-Cover Text is:
24 (b) The FSF's Back-Cover Text is:
26 You have freedom to copy and modify this GNU Manual, like GNU
27 software. Copies published by the Free Software Foundation raise
28 funds for GNU development.
32 @dircategory GNU Libraries
34 * libgomp: (libgomp). GNU Offloading and Multi Processing Runtime Library.
37 This manual documents libgomp, the GNU Offloading and Multi Processing
38 Runtime library. This is the GNU implementation of the OpenMP and
39 OpenACC APIs for parallel and accelerator programming in C/C++ and
42 Published by the Free Software Foundation
43 51 Franklin Street, Fifth Floor
44 Boston, MA 02110-1301 USA
50 @setchapternewpage odd
53 @title GNU Offloading and Multi Processing Runtime Library
54 @subtitle The GNU OpenMP and OpenACC Implementation
56 @vskip 0pt plus 1filll
57 @comment For the @value{version-GCC} Version*
59 Published by the Free Software Foundation @*
60 51 Franklin Street, Fifth Floor@*
61 Boston, MA 02110-1301, USA@*
71 @node Top, Enabling OpenMP
75 This manual documents the usage of libgomp, the GNU Offloading and
76 Multi Processing Runtime Library. This includes the GNU
77 implementation of the @uref{https://www.openmp.org, OpenMP} Application
78 Programming Interface (API) for multi-platform shared-memory parallel
79 programming in C/C++ and Fortran, and the GNU implementation of the
80 @uref{https://www.openacc.org, OpenACC} Application Programming
81 Interface (API) for offloading of code to accelerator devices in C/C++
84 Originally, libgomp implemented the GNU OpenMP Runtime Library. Based
85 on this, support for OpenACC and offloading (both OpenACC and OpenMP
86 4's target construct) has been added later on, and the library's name
87 changed to GNU Offloading and Multi Processing Runtime Library.
92 @comment When you add a new menu item, please keep the right hand
93 @comment aligned to the same column. Do not use tabs. This provides
94 @comment better formatting.
97 * Enabling OpenMP:: How to enable OpenMP for your applications.
98 * OpenMP Implementation Status:: List of implemented features by OpenMP version
99 * OpenMP Runtime Library Routines: Runtime Library Routines.
100 The OpenMP runtime application programming
102 * OpenMP Environment Variables: Environment Variables.
103 Influencing OpenMP runtime behavior with
104 environment variables.
105 * Enabling OpenACC:: How to enable OpenACC for your
107 * OpenACC Runtime Library Routines:: The OpenACC runtime application
108 programming interface.
109 * OpenACC Environment Variables:: Influencing OpenACC runtime behavior with
110 environment variables.
111 * CUDA Streams Usage:: Notes on the implementation of
112 asynchronous operations.
113 * OpenACC Library Interoperability:: OpenACC library interoperability with the
114 NVIDIA CUBLAS library.
115 * OpenACC Profiling Interface::
116 * OpenMP-Implementation Specifics:: Notes specifics of this OpenMP
118 * Offload-Target Specifics:: Notes on offload-target specific internals
119 * The libgomp ABI:: Notes on the external ABI presented by libgomp.
120 * Reporting Bugs:: How to report bugs in the GNU Offloading and
121 Multi Processing Runtime Library.
122 * Copying:: GNU general public license says
123 how you can copy and share libgomp.
124 * GNU Free Documentation License::
125 How you can copy and share this manual.
126 * Funding:: How to help assure continued work for free
128 * Library Index:: Index of this documentation.
132 @c ---------------------------------------------------------------------
134 @c ---------------------------------------------------------------------
136 @node Enabling OpenMP
137 @chapter Enabling OpenMP
139 To activate the OpenMP extensions for C/C++ and Fortran, the compile-time
140 flag @command{-fopenmp} must be specified. This enables the OpenMP directive
141 @code{#pragma omp} in C/C++ and @code{!$omp} directives in free form,
142 @code{c$omp}, @code{*$omp} and @code{!$omp} directives in fixed form,
143 @code{!$} conditional compilation sentinels in free form and @code{c$},
144 @code{*$} and @code{!$} sentinels in fixed form, for Fortran. The flag also
145 arranges for automatic linking of the OpenMP runtime library
146 (@ref{Runtime Library Routines}).
148 A complete description of all OpenMP directives may be found in the
149 @uref{https://www.openmp.org, OpenMP Application Program Interface} manuals.
150 See also @ref{OpenMP Implementation Status}.
153 @c ---------------------------------------------------------------------
154 @c OpenMP Implementation Status
155 @c ---------------------------------------------------------------------
157 @node OpenMP Implementation Status
158 @chapter OpenMP Implementation Status
161 * OpenMP 4.5:: Feature completion status to 4.5 specification
162 * OpenMP 5.0:: Feature completion status to 5.0 specification
163 * OpenMP 5.1:: Feature completion status to 5.1 specification
164 * OpenMP 5.2:: Feature completion status to 5.2 specification
167 The @code{_OPENMP} preprocessor macro and Fortran's @code{openmp_version}
168 parameter, provided by @code{omp_lib.h} and the @code{omp_lib} module, have
169 the value @code{201511} (i.e. OpenMP 4.5).
174 The OpenMP 4.5 specification is fully supported.
179 @unnumberedsubsec New features listed in Appendix B of the OpenMP specification
180 @c This list is sorted as in OpenMP 5.1's B.3 not as in OpenMP 5.0's B.2
182 @multitable @columnfractions .60 .10 .25
183 @headitem Description @tab Status @tab Comments
184 @item Array shaping @tab N @tab
185 @item Array sections with non-unit strides in C and C++ @tab N @tab
186 @item Iterators @tab Y @tab
187 @item @code{metadirective} directive @tab N @tab
188 @item @code{declare variant} directive
189 @tab P @tab @emph{simd} traits not handled correctly
190 @item @emph{target-offload-var} ICV and @code{OMP_TARGET_OFFLOAD}
191 env variable @tab Y @tab
192 @item Nested-parallel changes to @emph{max-active-levels-var} ICV @tab Y @tab
193 @item @code{requires} directive @tab P
194 @tab complete but no non-host devices provides @code{unified_address},
195 @code{unified_shared_memory} or @code{reverse_offload}
196 @item @code{teams} construct outside an enclosing target region @tab Y @tab
197 @item Non-rectangular loop nests @tab Y @tab
198 @item @code{!=} as relational-op in canonical loop form for C/C++ @tab Y @tab
199 @item @code{nonmonotonic} as default loop schedule modifier for worksharing-loop
200 constructs @tab Y @tab
201 @item Collapse of associated loops that are imperfectly nested loops @tab N @tab
202 @item Clauses @code{if}, @code{nontemporal} and @code{order(concurrent)} in
203 @code{simd} construct @tab Y @tab
204 @item @code{atomic} constructs in @code{simd} @tab Y @tab
205 @item @code{loop} construct @tab Y @tab
206 @item @code{order(concurrent)} clause @tab Y @tab
207 @item @code{scan} directive and @code{in_scan} modifier for the
208 @code{reduction} clause @tab Y @tab
209 @item @code{in_reduction} clause on @code{task} constructs @tab Y @tab
210 @item @code{in_reduction} clause on @code{target} constructs @tab P
211 @tab @code{nowait} only stub
212 @item @code{task_reduction} clause with @code{taskgroup} @tab Y @tab
213 @item @code{task} modifier to @code{reduction} clause @tab Y @tab
214 @item @code{affinity} clause to @code{task} construct @tab Y @tab Stub only
215 @item @code{detach} clause to @code{task} construct @tab Y @tab
216 @item @code{omp_fulfill_event} runtime routine @tab Y @tab
217 @item @code{reduction} and @code{in_reduction} clauses on @code{taskloop}
218 and @code{taskloop simd} constructs @tab Y @tab
219 @item @code{taskloop} construct cancelable by @code{cancel} construct
221 @item @code{mutexinoutset} @emph{dependence-type} for @code{depend} clause
223 @item Predefined memory spaces, memory allocators, allocator traits
224 @tab Y @tab Some are only stubs
225 @item Memory management routines @tab Y @tab
226 @item @code{allocate} directive @tab N @tab
227 @item @code{allocate} clause @tab P @tab Initial support
228 @item @code{use_device_addr} clause on @code{target data} @tab Y @tab
229 @item @code{ancestor} modifier on @code{device} clause
230 @tab Y @tab See comment for @code{requires}
231 @item Implicit declare target directive @tab Y @tab
232 @item Discontiguous array section with @code{target update} construct
234 @item C/C++'s lvalue expressions in @code{to}, @code{from}
235 and @code{map} clauses @tab N @tab
236 @item C/C++'s lvalue expressions in @code{depend} clauses @tab Y @tab
237 @item Nested @code{declare target} directive @tab Y @tab
238 @item Combined @code{master} constructs @tab Y @tab
239 @item @code{depend} clause on @code{taskwait} @tab Y @tab
240 @item Weak memory ordering clauses on @code{atomic} and @code{flush} construct
242 @item @code{hint} clause on the @code{atomic} construct @tab Y @tab Stub only
243 @item @code{depobj} construct and depend objects @tab Y @tab
244 @item Lock hints were renamed to synchronization hints @tab Y @tab
245 @item @code{conditional} modifier to @code{lastprivate} clause @tab Y @tab
246 @item Map-order clarifications @tab P @tab
247 @item @code{close} @emph{map-type-modifier} @tab Y @tab
248 @item Mapping C/C++ pointer variables and to assign the address of
249 device memory mapped by an array section @tab P @tab
250 @item Mapping of Fortran pointer and allocatable variables, including pointer
251 and allocatable components of variables
252 @tab P @tab Mapping of vars with allocatable components unsupported
253 @item @code{defaultmap} extensions @tab Y @tab
254 @item @code{declare mapper} directive @tab N @tab
255 @item @code{omp_get_supported_active_levels} routine @tab Y @tab
256 @item Runtime routines and environment variables to display runtime thread
257 affinity information @tab Y @tab
258 @item @code{omp_pause_resource} and @code{omp_pause_resource_all} runtime
260 @item @code{omp_get_device_num} runtime routine @tab Y @tab
261 @item OMPT interface @tab N @tab
262 @item OMPD interface @tab N @tab
265 @unnumberedsubsec Other new OpenMP 5.0 features
267 @multitable @columnfractions .60 .10 .25
268 @headitem Description @tab Status @tab Comments
269 @item Supporting C++'s range-based for loop @tab Y @tab
276 @unnumberedsubsec New features listed in Appendix B of the OpenMP specification
278 @multitable @columnfractions .60 .10 .25
279 @headitem Description @tab Status @tab Comments
280 @item OpenMP directive as C++ attribute specifiers @tab Y @tab
281 @item @code{omp_all_memory} reserved locator @tab Y @tab
282 @item @emph{target_device trait} in OpenMP Context @tab N @tab
283 @item @code{target_device} selector set in context selectors @tab N @tab
284 @item C/C++'s @code{declare variant} directive: elision support of
285 preprocessed code @tab N @tab
286 @item @code{declare variant}: new clauses @code{adjust_args} and
287 @code{append_args} @tab N @tab
288 @item @code{dispatch} construct @tab N @tab
289 @item device-specific ICV settings with environment variables @tab Y @tab
290 @item @code{assume} directive @tab Y @tab
291 @item @code{nothing} directive @tab Y @tab
292 @item @code{error} directive @tab Y @tab
293 @item @code{masked} construct @tab Y @tab
294 @item @code{scope} directive @tab Y @tab
295 @item Loop transformation constructs @tab N @tab
296 @item @code{strict} modifier in the @code{grainsize} and @code{num_tasks}
297 clauses of the @code{taskloop} construct @tab Y @tab
298 @item @code{align} clause/modifier in @code{allocate} directive/clause
299 and @code{allocator} directive @tab P @tab C/C++ on clause only
300 @item @code{thread_limit} clause to @code{target} construct @tab Y @tab
301 @item @code{has_device_addr} clause to @code{target} construct @tab Y @tab
302 @item Iterators in @code{target update} motion clauses and @code{map}
304 @item Indirect calls to the device version of a procedure or function in
305 @code{target} regions @tab N @tab
306 @item @code{interop} directive @tab N @tab
307 @item @code{omp_interop_t} object support in runtime routines @tab N @tab
308 @item @code{nowait} clause in @code{taskwait} directive @tab Y @tab
309 @item Extensions to the @code{atomic} directive @tab Y @tab
310 @item @code{seq_cst} clause on a @code{flush} construct @tab Y @tab
311 @item @code{inoutset} argument to the @code{depend} clause @tab Y @tab
312 @item @code{private} and @code{firstprivate} argument to @code{default}
313 clause in C and C++ @tab Y @tab
314 @item @code{present} argument to @code{defaultmap} clause @tab N @tab
315 @item @code{omp_set_num_teams}, @code{omp_set_teams_thread_limit},
316 @code{omp_get_max_teams}, @code{omp_get_teams_thread_limit} runtime
318 @item @code{omp_target_is_accessible} runtime routine @tab Y @tab
319 @item @code{omp_target_memcpy_async} and @code{omp_target_memcpy_rect_async}
320 runtime routines @tab Y @tab
321 @item @code{omp_get_mapped_ptr} runtime routine @tab Y @tab
322 @item @code{omp_calloc}, @code{omp_realloc}, @code{omp_aligned_alloc} and
323 @code{omp_aligned_calloc} runtime routines @tab Y @tab
324 @item @code{omp_alloctrait_key_t} enum: @code{omp_atv_serialized} added,
325 @code{omp_atv_default} changed @tab Y @tab
326 @item @code{omp_display_env} runtime routine @tab Y @tab
327 @item @code{ompt_scope_endpoint_t} enum: @code{ompt_scope_beginend} @tab N @tab
328 @item @code{ompt_sync_region_t} enum additions @tab N @tab
329 @item @code{ompt_state_t} enum: @code{ompt_state_wait_barrier_implementation}
330 and @code{ompt_state_wait_barrier_teams} @tab N @tab
331 @item @code{ompt_callback_target_data_op_emi_t},
332 @code{ompt_callback_target_emi_t}, @code{ompt_callback_target_map_emi_t}
333 and @code{ompt_callback_target_submit_emi_t} @tab N @tab
334 @item @code{ompt_callback_error_t} type @tab N @tab
335 @item @code{OMP_PLACES} syntax extensions @tab Y @tab
336 @item @code{OMP_NUM_TEAMS} and @code{OMP_TEAMS_THREAD_LIMIT} environment
337 variables @tab Y @tab
340 @unnumberedsubsec Other new OpenMP 5.1 features
342 @multitable @columnfractions .60 .10 .25
343 @headitem Description @tab Status @tab Comments
344 @item Support of strictly structured blocks in Fortran @tab Y @tab
345 @item Support of structured block sequences in C/C++ @tab Y @tab
346 @item @code{unconstrained} and @code{reproducible} modifiers on @code{order}
348 @item Support @code{begin/end declare target} syntax in C/C++ @tab Y @tab
349 @item Pointer predetermined firstprivate getting initialized
350 to address of matching mapped list item per 5.1, Sect. 2.21.7.2 @tab N @tab
351 @item For Fortran, diagnose placing declarative before/between @code{USE},
352 @code{IMPORT}, and @code{IMPLICIT} as invalid @tab N @tab
359 @unnumberedsubsec New features listed in Appendix B of the OpenMP specification
361 @multitable @columnfractions .60 .10 .25
362 @headitem Description @tab Status @tab Comments
363 @item @code{omp_in_explicit_task} routine and @emph{explicit-task-var} ICV
365 @item @code{omp}/@code{ompx}/@code{omx} sentinels and @code{omp_}/@code{ompx_}
367 @tab warning for @code{ompx/omx} sentinels@footnote{The @code{ompx}
368 sentinel as C/C++ pragma and C++ attributes are warned for with
369 @code{-Wunknown-pragmas} (implied by @code{-Wall}) and @code{-Wattributes}
370 (enabled by default), respectively; for Fortran free-source code, there is
371 a warning enabled by default and, for fixed-source code, the @code{omx}
372 sentinel is warned for with with @code{-Wsurprising} (enabled by
373 @code{-Wall}). Unknown clauses are always rejected with an error.}
374 @item Clauses on @code{end} directive can be on directive @tab N @tab
375 @item Deprecation of no-argument @code{destroy} clause on @code{depobj}
377 @item @code{linear} clause syntax changes and @code{step} modifier @tab Y @tab
378 @item Deprecation of minus operator for reductions @tab N @tab
379 @item Deprecation of separating @code{map} modifiers without comma @tab N @tab
380 @item @code{declare mapper} with iterator and @code{present} modifiers
382 @item If a matching mapped list item is not found in the data environment, the
383 pointer retains its original value @tab N @tab
384 @item New @code{enter} clause as alias for @code{to} on declare target directive
386 @item Deprecation of @code{to} clause on declare target directive @tab N @tab
387 @item Extended list of directives permitted in Fortran pure procedures
389 @item New @code{allocators} directive for Fortran @tab N @tab
390 @item Deprecation of @code{allocate} directive for Fortran
391 allocatables/pointers @tab N @tab
392 @item Optional paired @code{end} directive with @code{dispatch} @tab N @tab
393 @item New @code{memspace} and @code{traits} modifiers for @code{uses_allocators}
395 @item Deprecation of traits array following the allocator_handle expression in
396 @code{uses_allocators} @tab N @tab
397 @item New @code{otherwise} clause as alias for @code{default} on metadirectives
399 @item Deprecation of @code{default} clause on metadirectives @tab N @tab
400 @item Deprecation of delimited form of @code{declare target} @tab N @tab
401 @item Reproducible semantics changed for @code{order(concurrent)} @tab N @tab
402 @item @code{allocate} and @code{firstprivate} clauses on @code{scope}
404 @item @code{ompt_callback_work} @tab N @tab
405 @item Default map-type for @code{map} clause in @code{target enter/exit data}
407 @item New @code{doacross} clause as alias for @code{depend} with
408 @code{source}/@code{sink} modifier @tab Y @tab
409 @item Deprecation of @code{depend} with @code{source}/@code{sink} modifier
411 @item @code{omp_cur_iteration} keyword @tab Y @tab
414 @unnumberedsubsec Other new OpenMP 5.2 features
416 @multitable @columnfractions .60 .10 .25
417 @headitem Description @tab Status @tab Comments
418 @item For Fortran, optional comma between directive and clause @tab N @tab
419 @item Conforming device numbers and @code{omp_initial_device} and
420 @code{omp_invalid_device} enum/PARAMETER @tab Y @tab
421 @item Initial value of @emph{default-device-var} ICV with
422 @code{OMP_TARGET_OFFLOAD=mandatory} @tab N @tab
423 @item @emph{interop_types} in any position of the modifier list for the @code{init} clause
424 of the @code{interop} construct @tab N @tab
428 @c ---------------------------------------------------------------------
429 @c OpenMP Runtime Library Routines
430 @c ---------------------------------------------------------------------
432 @node Runtime Library Routines
433 @chapter OpenMP Runtime Library Routines
435 The runtime routines described here are defined by Section 3 of the OpenMP
436 specification in version 4.5. The routines are structured in following
440 Control threads, processors and the parallel environment. They have C
441 linkage, and do not throw exceptions.
443 * omp_get_active_level:: Number of active parallel regions
444 * omp_get_ancestor_thread_num:: Ancestor thread ID
445 * omp_get_cancellation:: Whether cancellation support is enabled
446 * omp_get_default_device:: Get the default device for target regions
447 * omp_get_device_num:: Get device that current thread is running on
448 * omp_get_dynamic:: Dynamic teams setting
449 * omp_get_initial_device:: Device number of host device
450 * omp_get_level:: Number of parallel regions
451 * omp_get_max_active_levels:: Current maximum number of active regions
452 * omp_get_max_task_priority:: Maximum task priority value that can be set
453 * omp_get_max_teams:: Maximum number of teams for teams region
454 * omp_get_max_threads:: Maximum number of threads of parallel region
455 * omp_get_nested:: Nested parallel regions
456 * omp_get_num_devices:: Number of target devices
457 * omp_get_num_procs:: Number of processors online
458 * omp_get_num_teams:: Number of teams
459 * omp_get_num_threads:: Size of the active team
460 * omp_get_proc_bind:: Whether theads may be moved between CPUs
461 * omp_get_schedule:: Obtain the runtime scheduling method
462 * omp_get_supported_active_levels:: Maximum number of active regions supported
463 * omp_get_team_num:: Get team number
464 * omp_get_team_size:: Number of threads in a team
465 * omp_get_teams_thread_limit:: Maximum number of threads imposed by teams
466 * omp_get_thread_limit:: Maximum number of threads
467 * omp_get_thread_num:: Current thread ID
468 * omp_in_parallel:: Whether a parallel region is active
469 * omp_in_final:: Whether in final or included task region
470 * omp_is_initial_device:: Whether executing on the host device
471 * omp_set_default_device:: Set the default device for target regions
472 * omp_set_dynamic:: Enable/disable dynamic teams
473 * omp_set_max_active_levels:: Limits the number of active parallel regions
474 * omp_set_nested:: Enable/disable nested parallel regions
475 * omp_set_num_teams:: Set upper teams limit for teams region
476 * omp_set_num_threads:: Set upper team size limit
477 * omp_set_schedule:: Set the runtime scheduling method
478 * omp_set_teams_thread_limit:: Set upper thread limit for teams construct
480 Initialize, set, test, unset and destroy simple and nested locks.
482 * omp_init_lock:: Initialize simple lock
483 * omp_set_lock:: Wait for and set simple lock
484 * omp_test_lock:: Test and set simple lock if available
485 * omp_unset_lock:: Unset simple lock
486 * omp_destroy_lock:: Destroy simple lock
487 * omp_init_nest_lock:: Initialize nested lock
488 * omp_set_nest_lock:: Wait for and set simple lock
489 * omp_test_nest_lock:: Test and set nested lock if available
490 * omp_unset_nest_lock:: Unset nested lock
491 * omp_destroy_nest_lock:: Destroy nested lock
493 Portable, thread-based, wall clock timer.
495 * omp_get_wtick:: Get timer precision.
496 * omp_get_wtime:: Elapsed wall clock time.
498 Support for event objects.
500 * omp_fulfill_event:: Fulfill and destroy an OpenMP event.
505 @node omp_get_active_level
506 @section @code{omp_get_active_level} -- Number of parallel regions
508 @item @emph{Description}:
509 This function returns the nesting level for the active parallel blocks,
510 which enclose the calling call.
513 @multitable @columnfractions .20 .80
514 @item @emph{Prototype}: @tab @code{int omp_get_active_level(void);}
517 @item @emph{Fortran}:
518 @multitable @columnfractions .20 .80
519 @item @emph{Interface}: @tab @code{integer function omp_get_active_level()}
522 @item @emph{See also}:
523 @ref{omp_get_level}, @ref{omp_get_max_active_levels}, @ref{omp_set_max_active_levels}
525 @item @emph{Reference}:
526 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.20.
531 @node omp_get_ancestor_thread_num
532 @section @code{omp_get_ancestor_thread_num} -- Ancestor thread ID
534 @item @emph{Description}:
535 This function returns the thread identification number for the given
536 nesting level of the current thread. For values of @var{level} outside
537 zero to @code{omp_get_level} -1 is returned; if @var{level} is
538 @code{omp_get_level} the result is identical to @code{omp_get_thread_num}.
541 @multitable @columnfractions .20 .80
542 @item @emph{Prototype}: @tab @code{int omp_get_ancestor_thread_num(int level);}
545 @item @emph{Fortran}:
546 @multitable @columnfractions .20 .80
547 @item @emph{Interface}: @tab @code{integer function omp_get_ancestor_thread_num(level)}
548 @item @tab @code{integer level}
551 @item @emph{See also}:
552 @ref{omp_get_level}, @ref{omp_get_thread_num}, @ref{omp_get_team_size}
554 @item @emph{Reference}:
555 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.18.
560 @node omp_get_cancellation
561 @section @code{omp_get_cancellation} -- Whether cancellation support is enabled
563 @item @emph{Description}:
564 This function returns @code{true} if cancellation is activated, @code{false}
565 otherwise. Here, @code{true} and @code{false} represent their language-specific
566 counterparts. Unless @env{OMP_CANCELLATION} is set true, cancellations are
570 @multitable @columnfractions .20 .80
571 @item @emph{Prototype}: @tab @code{int omp_get_cancellation(void);}
574 @item @emph{Fortran}:
575 @multitable @columnfractions .20 .80
576 @item @emph{Interface}: @tab @code{logical function omp_get_cancellation()}
579 @item @emph{See also}:
580 @ref{OMP_CANCELLATION}
582 @item @emph{Reference}:
583 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.9.
588 @node omp_get_default_device
589 @section @code{omp_get_default_device} -- Get the default device for target regions
591 @item @emph{Description}:
592 Get the default device for target regions without device clause.
595 @multitable @columnfractions .20 .80
596 @item @emph{Prototype}: @tab @code{int omp_get_default_device(void);}
599 @item @emph{Fortran}:
600 @multitable @columnfractions .20 .80
601 @item @emph{Interface}: @tab @code{integer function omp_get_default_device()}
604 @item @emph{See also}:
605 @ref{OMP_DEFAULT_DEVICE}, @ref{omp_set_default_device}
607 @item @emph{Reference}:
608 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.30.
613 @node omp_get_device_num
614 @section @code{omp_get_device_num} -- Return device number of current device
616 @item @emph{Description}:
617 This function returns a device number that represents the device that the
618 current thread is executing on. For OpenMP 5.0, this must be equal to the
619 value returned by the @code{omp_get_initial_device} function when called
623 @multitable @columnfractions .20 .80
624 @item @emph{Prototype}: @tab @code{int omp_get_device_num(void);}
627 @item @emph{Fortran}:
628 @multitable @columnfractions .20 .80
629 @item @emph{Interface}: @tab @code{integer function omp_get_device_num()}
632 @item @emph{See also}:
633 @ref{omp_get_initial_device}
635 @item @emph{Reference}:
636 @uref{https://www.openmp.org, OpenMP specification v5.0}, Section 3.2.37.
641 @node omp_get_dynamic
642 @section @code{omp_get_dynamic} -- Dynamic teams setting
644 @item @emph{Description}:
645 This function returns @code{true} if enabled, @code{false} otherwise.
646 Here, @code{true} and @code{false} represent their language-specific
649 The dynamic team setting may be initialized at startup by the
650 @env{OMP_DYNAMIC} environment variable or at runtime using
651 @code{omp_set_dynamic}. If undefined, dynamic adjustment is
655 @multitable @columnfractions .20 .80
656 @item @emph{Prototype}: @tab @code{int omp_get_dynamic(void);}
659 @item @emph{Fortran}:
660 @multitable @columnfractions .20 .80
661 @item @emph{Interface}: @tab @code{logical function omp_get_dynamic()}
664 @item @emph{See also}:
665 @ref{omp_set_dynamic}, @ref{OMP_DYNAMIC}
667 @item @emph{Reference}:
668 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.8.
673 @node omp_get_initial_device
674 @section @code{omp_get_initial_device} -- Return device number of initial device
676 @item @emph{Description}:
677 This function returns a device number that represents the host device.
678 For OpenMP 5.1, this must be equal to the value returned by the
679 @code{omp_get_num_devices} function.
682 @multitable @columnfractions .20 .80
683 @item @emph{Prototype}: @tab @code{int omp_get_initial_device(void);}
686 @item @emph{Fortran}:
687 @multitable @columnfractions .20 .80
688 @item @emph{Interface}: @tab @code{integer function omp_get_initial_device()}
691 @item @emph{See also}:
692 @ref{omp_get_num_devices}
694 @item @emph{Reference}:
695 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.35.
701 @section @code{omp_get_level} -- Obtain the current nesting level
703 @item @emph{Description}:
704 This function returns the nesting level for the parallel blocks,
705 which enclose the calling call.
708 @multitable @columnfractions .20 .80
709 @item @emph{Prototype}: @tab @code{int omp_get_level(void);}
712 @item @emph{Fortran}:
713 @multitable @columnfractions .20 .80
714 @item @emph{Interface}: @tab @code{integer function omp_level()}
717 @item @emph{See also}:
718 @ref{omp_get_active_level}
720 @item @emph{Reference}:
721 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.17.
726 @node omp_get_max_active_levels
727 @section @code{omp_get_max_active_levels} -- Current maximum number of active regions
729 @item @emph{Description}:
730 This function obtains the maximum allowed number of nested, active parallel regions.
733 @multitable @columnfractions .20 .80
734 @item @emph{Prototype}: @tab @code{int omp_get_max_active_levels(void);}
737 @item @emph{Fortran}:
738 @multitable @columnfractions .20 .80
739 @item @emph{Interface}: @tab @code{integer function omp_get_max_active_levels()}
742 @item @emph{See also}:
743 @ref{omp_set_max_active_levels}, @ref{omp_get_active_level}
745 @item @emph{Reference}:
746 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.16.
750 @node omp_get_max_task_priority
751 @section @code{omp_get_max_task_priority} -- Maximum priority value
752 that can be set for tasks.
754 @item @emph{Description}:
755 This function obtains the maximum allowed priority number for tasks.
758 @multitable @columnfractions .20 .80
759 @item @emph{Prototype}: @tab @code{int omp_get_max_task_priority(void);}
762 @item @emph{Fortran}:
763 @multitable @columnfractions .20 .80
764 @item @emph{Interface}: @tab @code{integer function omp_get_max_task_priority()}
767 @item @emph{Reference}:
768 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.29.
772 @node omp_get_max_teams
773 @section @code{omp_get_max_teams} -- Maximum number of teams of teams region
775 @item @emph{Description}:
776 Return the maximum number of teams used for the teams region
777 that does not use the clause @code{num_teams}.
780 @multitable @columnfractions .20 .80
781 @item @emph{Prototype}: @tab @code{int omp_get_max_teams(void);}
784 @item @emph{Fortran}:
785 @multitable @columnfractions .20 .80
786 @item @emph{Interface}: @tab @code{integer function omp_get_max_teams()}
789 @item @emph{See also}:
790 @ref{omp_set_num_teams}, @ref{omp_get_num_teams}
792 @item @emph{Reference}:
793 @uref{https://www.openmp.org, OpenMP specification v5.1}, Section 3.4.4.
798 @node omp_get_max_threads
799 @section @code{omp_get_max_threads} -- Maximum number of threads of parallel region
801 @item @emph{Description}:
802 Return the maximum number of threads used for the current parallel region
803 that does not use the clause @code{num_threads}.
806 @multitable @columnfractions .20 .80
807 @item @emph{Prototype}: @tab @code{int omp_get_max_threads(void);}
810 @item @emph{Fortran}:
811 @multitable @columnfractions .20 .80
812 @item @emph{Interface}: @tab @code{integer function omp_get_max_threads()}
815 @item @emph{See also}:
816 @ref{omp_set_num_threads}, @ref{omp_set_dynamic}, @ref{omp_get_thread_limit}
818 @item @emph{Reference}:
819 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.3.
825 @section @code{omp_get_nested} -- Nested parallel regions
827 @item @emph{Description}:
828 This function returns @code{true} if nested parallel regions are
829 enabled, @code{false} otherwise. Here, @code{true} and @code{false}
830 represent their language-specific counterparts.
832 The state of nested parallel regions at startup depends on several
833 environment variables. If @env{OMP_MAX_ACTIVE_LEVELS} is defined
834 and is set to greater than one, then nested parallel regions will be
835 enabled. If not defined, then the value of the @env{OMP_NESTED}
836 environment variable will be followed if defined. If neither are
837 defined, then if either @env{OMP_NUM_THREADS} or @env{OMP_PROC_BIND}
838 are defined with a list of more than one value, then nested parallel
839 regions are enabled. If none of these are defined, then nested parallel
840 regions are disabled by default.
842 Nested parallel regions can be enabled or disabled at runtime using
843 @code{omp_set_nested}, or by setting the maximum number of nested
844 regions with @code{omp_set_max_active_levels} to one to disable, or
848 @multitable @columnfractions .20 .80
849 @item @emph{Prototype}: @tab @code{int omp_get_nested(void);}
852 @item @emph{Fortran}:
853 @multitable @columnfractions .20 .80
854 @item @emph{Interface}: @tab @code{logical function omp_get_nested()}
857 @item @emph{See also}:
858 @ref{omp_set_max_active_levels}, @ref{omp_set_nested},
859 @ref{OMP_MAX_ACTIVE_LEVELS}, @ref{OMP_NESTED}
861 @item @emph{Reference}:
862 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.11.
867 @node omp_get_num_devices
868 @section @code{omp_get_num_devices} -- Number of target devices
870 @item @emph{Description}:
871 Returns the number of target devices.
874 @multitable @columnfractions .20 .80
875 @item @emph{Prototype}: @tab @code{int omp_get_num_devices(void);}
878 @item @emph{Fortran}:
879 @multitable @columnfractions .20 .80
880 @item @emph{Interface}: @tab @code{integer function omp_get_num_devices()}
883 @item @emph{Reference}:
884 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.31.
889 @node omp_get_num_procs
890 @section @code{omp_get_num_procs} -- Number of processors online
892 @item @emph{Description}:
893 Returns the number of processors online on that device.
896 @multitable @columnfractions .20 .80
897 @item @emph{Prototype}: @tab @code{int omp_get_num_procs(void);}
900 @item @emph{Fortran}:
901 @multitable @columnfractions .20 .80
902 @item @emph{Interface}: @tab @code{integer function omp_get_num_procs()}
905 @item @emph{Reference}:
906 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.5.
911 @node omp_get_num_teams
912 @section @code{omp_get_num_teams} -- Number of teams
914 @item @emph{Description}:
915 Returns the number of teams in the current team region.
918 @multitable @columnfractions .20 .80
919 @item @emph{Prototype}: @tab @code{int omp_get_num_teams(void);}
922 @item @emph{Fortran}:
923 @multitable @columnfractions .20 .80
924 @item @emph{Interface}: @tab @code{integer function omp_get_num_teams()}
927 @item @emph{Reference}:
928 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.32.
933 @node omp_get_num_threads
934 @section @code{omp_get_num_threads} -- Size of the active team
936 @item @emph{Description}:
937 Returns the number of threads in the current team. In a sequential section of
938 the program @code{omp_get_num_threads} returns 1.
940 The default team size may be initialized at startup by the
941 @env{OMP_NUM_THREADS} environment variable. At runtime, the size
942 of the current team may be set either by the @code{NUM_THREADS}
943 clause or by @code{omp_set_num_threads}. If none of the above were
944 used to define a specific value and @env{OMP_DYNAMIC} is disabled,
945 one thread per CPU online is used.
948 @multitable @columnfractions .20 .80
949 @item @emph{Prototype}: @tab @code{int omp_get_num_threads(void);}
952 @item @emph{Fortran}:
953 @multitable @columnfractions .20 .80
954 @item @emph{Interface}: @tab @code{integer function omp_get_num_threads()}
957 @item @emph{See also}:
958 @ref{omp_get_max_threads}, @ref{omp_set_num_threads}, @ref{OMP_NUM_THREADS}
960 @item @emph{Reference}:
961 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.2.
966 @node omp_get_proc_bind
967 @section @code{omp_get_proc_bind} -- Whether theads may be moved between CPUs
969 @item @emph{Description}:
970 This functions returns the currently active thread affinity policy, which is
971 set via @env{OMP_PROC_BIND}. Possible values are @code{omp_proc_bind_false},
972 @code{omp_proc_bind_true}, @code{omp_proc_bind_primary},
973 @code{omp_proc_bind_master}, @code{omp_proc_bind_close} and @code{omp_proc_bind_spread},
974 where @code{omp_proc_bind_master} is an alias for @code{omp_proc_bind_primary}.
977 @multitable @columnfractions .20 .80
978 @item @emph{Prototype}: @tab @code{omp_proc_bind_t omp_get_proc_bind(void);}
981 @item @emph{Fortran}:
982 @multitable @columnfractions .20 .80
983 @item @emph{Interface}: @tab @code{integer(kind=omp_proc_bind_kind) function omp_get_proc_bind()}
986 @item @emph{See also}:
987 @ref{OMP_PROC_BIND}, @ref{OMP_PLACES}, @ref{GOMP_CPU_AFFINITY},
989 @item @emph{Reference}:
990 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.22.
995 @node omp_get_schedule
996 @section @code{omp_get_schedule} -- Obtain the runtime scheduling method
998 @item @emph{Description}:
999 Obtain the runtime scheduling method. The @var{kind} argument will be
1000 set to the value @code{omp_sched_static}, @code{omp_sched_dynamic},
1001 @code{omp_sched_guided} or @code{omp_sched_auto}. The second argument,
1002 @var{chunk_size}, is set to the chunk size.
1005 @multitable @columnfractions .20 .80
1006 @item @emph{Prototype}: @tab @code{void omp_get_schedule(omp_sched_t *kind, int *chunk_size);}
1009 @item @emph{Fortran}:
1010 @multitable @columnfractions .20 .80
1011 @item @emph{Interface}: @tab @code{subroutine omp_get_schedule(kind, chunk_size)}
1012 @item @tab @code{integer(kind=omp_sched_kind) kind}
1013 @item @tab @code{integer chunk_size}
1016 @item @emph{See also}:
1017 @ref{omp_set_schedule}, @ref{OMP_SCHEDULE}
1019 @item @emph{Reference}:
1020 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.13.
1024 @node omp_get_supported_active_levels
1025 @section @code{omp_get_supported_active_levels} -- Maximum number of active regions supported
1027 @item @emph{Description}:
1028 This function returns the maximum number of nested, active parallel regions
1029 supported by this implementation.
1032 @multitable @columnfractions .20 .80
1033 @item @emph{Prototype}: @tab @code{int omp_get_supported_active_levels(void);}
1036 @item @emph{Fortran}:
1037 @multitable @columnfractions .20 .80
1038 @item @emph{Interface}: @tab @code{integer function omp_get_supported_active_levels()}
1041 @item @emph{See also}:
1042 @ref{omp_get_max_active_levels}, @ref{omp_set_max_active_levels}
1044 @item @emph{Reference}:
1045 @uref{https://www.openmp.org, OpenMP specification v5.0}, Section 3.2.15.
1050 @node omp_get_team_num
1051 @section @code{omp_get_team_num} -- Get team number
1053 @item @emph{Description}:
1054 Returns the team number of the calling thread.
1057 @multitable @columnfractions .20 .80
1058 @item @emph{Prototype}: @tab @code{int omp_get_team_num(void);}
1061 @item @emph{Fortran}:
1062 @multitable @columnfractions .20 .80
1063 @item @emph{Interface}: @tab @code{integer function omp_get_team_num()}
1066 @item @emph{Reference}:
1067 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.33.
1072 @node omp_get_team_size
1073 @section @code{omp_get_team_size} -- Number of threads in a team
1075 @item @emph{Description}:
1076 This function returns the number of threads in a thread team to which
1077 either the current thread or its ancestor belongs. For values of @var{level}
1078 outside zero to @code{omp_get_level}, -1 is returned; if @var{level} is zero,
1079 1 is returned, and for @code{omp_get_level}, the result is identical
1080 to @code{omp_get_num_threads}.
1083 @multitable @columnfractions .20 .80
1084 @item @emph{Prototype}: @tab @code{int omp_get_team_size(int level);}
1087 @item @emph{Fortran}:
1088 @multitable @columnfractions .20 .80
1089 @item @emph{Interface}: @tab @code{integer function omp_get_team_size(level)}
1090 @item @tab @code{integer level}
1093 @item @emph{See also}:
1094 @ref{omp_get_num_threads}, @ref{omp_get_level}, @ref{omp_get_ancestor_thread_num}
1096 @item @emph{Reference}:
1097 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.19.
1102 @node omp_get_teams_thread_limit
1103 @section @code{omp_get_teams_thread_limit} -- Maximum number of threads imposed by teams
1105 @item @emph{Description}:
1106 Return the maximum number of threads that will be able to participate in
1107 each team created by a teams construct.
1110 @multitable @columnfractions .20 .80
1111 @item @emph{Prototype}: @tab @code{int omp_get_teams_thread_limit(void);}
1114 @item @emph{Fortran}:
1115 @multitable @columnfractions .20 .80
1116 @item @emph{Interface}: @tab @code{integer function omp_get_teams_thread_limit()}
1119 @item @emph{See also}:
1120 @ref{omp_set_teams_thread_limit}, @ref{OMP_TEAMS_THREAD_LIMIT}
1122 @item @emph{Reference}:
1123 @uref{https://www.openmp.org, OpenMP specification v5.1}, Section 3.4.6.
1128 @node omp_get_thread_limit
1129 @section @code{omp_get_thread_limit} -- Maximum number of threads
1131 @item @emph{Description}:
1132 Return the maximum number of threads of the program.
1135 @multitable @columnfractions .20 .80
1136 @item @emph{Prototype}: @tab @code{int omp_get_thread_limit(void);}
1139 @item @emph{Fortran}:
1140 @multitable @columnfractions .20 .80
1141 @item @emph{Interface}: @tab @code{integer function omp_get_thread_limit()}
1144 @item @emph{See also}:
1145 @ref{omp_get_max_threads}, @ref{OMP_THREAD_LIMIT}
1147 @item @emph{Reference}:
1148 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.14.
1153 @node omp_get_thread_num
1154 @section @code{omp_get_thread_num} -- Current thread ID
1156 @item @emph{Description}:
1157 Returns a unique thread identification number within the current team.
1158 In a sequential parts of the program, @code{omp_get_thread_num}
1159 always returns 0. In parallel regions the return value varies
1160 from 0 to @code{omp_get_num_threads}-1 inclusive. The return
1161 value of the primary thread of a team is always 0.
1164 @multitable @columnfractions .20 .80
1165 @item @emph{Prototype}: @tab @code{int omp_get_thread_num(void);}
1168 @item @emph{Fortran}:
1169 @multitable @columnfractions .20 .80
1170 @item @emph{Interface}: @tab @code{integer function omp_get_thread_num()}
1173 @item @emph{See also}:
1174 @ref{omp_get_num_threads}, @ref{omp_get_ancestor_thread_num}
1176 @item @emph{Reference}:
1177 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.4.
1182 @node omp_in_parallel
1183 @section @code{omp_in_parallel} -- Whether a parallel region is active
1185 @item @emph{Description}:
1186 This function returns @code{true} if currently running in parallel,
1187 @code{false} otherwise. Here, @code{true} and @code{false} represent
1188 their language-specific counterparts.
1191 @multitable @columnfractions .20 .80
1192 @item @emph{Prototype}: @tab @code{int omp_in_parallel(void);}
1195 @item @emph{Fortran}:
1196 @multitable @columnfractions .20 .80
1197 @item @emph{Interface}: @tab @code{logical function omp_in_parallel()}
1200 @item @emph{Reference}:
1201 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.6.
1206 @section @code{omp_in_final} -- Whether in final or included task region
1208 @item @emph{Description}:
1209 This function returns @code{true} if currently running in a final
1210 or included task region, @code{false} otherwise. Here, @code{true}
1211 and @code{false} represent their language-specific counterparts.
1214 @multitable @columnfractions .20 .80
1215 @item @emph{Prototype}: @tab @code{int omp_in_final(void);}
1218 @item @emph{Fortran}:
1219 @multitable @columnfractions .20 .80
1220 @item @emph{Interface}: @tab @code{logical function omp_in_final()}
1223 @item @emph{Reference}:
1224 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.21.
1229 @node omp_is_initial_device
1230 @section @code{omp_is_initial_device} -- Whether executing on the host device
1232 @item @emph{Description}:
1233 This function returns @code{true} if currently running on the host device,
1234 @code{false} otherwise. Here, @code{true} and @code{false} represent
1235 their language-specific counterparts.
1238 @multitable @columnfractions .20 .80
1239 @item @emph{Prototype}: @tab @code{int omp_is_initial_device(void);}
1242 @item @emph{Fortran}:
1243 @multitable @columnfractions .20 .80
1244 @item @emph{Interface}: @tab @code{logical function omp_is_initial_device()}
1247 @item @emph{Reference}:
1248 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.34.
1253 @node omp_set_default_device
1254 @section @code{omp_set_default_device} -- Set the default device for target regions
1256 @item @emph{Description}:
1257 Set the default device for target regions without device clause. The argument
1258 shall be a nonnegative device number.
1261 @multitable @columnfractions .20 .80
1262 @item @emph{Prototype}: @tab @code{void omp_set_default_device(int device_num);}
1265 @item @emph{Fortran}:
1266 @multitable @columnfractions .20 .80
1267 @item @emph{Interface}: @tab @code{subroutine omp_set_default_device(device_num)}
1268 @item @tab @code{integer device_num}
1271 @item @emph{See also}:
1272 @ref{OMP_DEFAULT_DEVICE}, @ref{omp_get_default_device}
1274 @item @emph{Reference}:
1275 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.29.
1280 @node omp_set_dynamic
1281 @section @code{omp_set_dynamic} -- Enable/disable dynamic teams
1283 @item @emph{Description}:
1284 Enable or disable the dynamic adjustment of the number of threads
1285 within a team. The function takes the language-specific equivalent
1286 of @code{true} and @code{false}, where @code{true} enables dynamic
1287 adjustment of team sizes and @code{false} disables it.
1290 @multitable @columnfractions .20 .80
1291 @item @emph{Prototype}: @tab @code{void omp_set_dynamic(int dynamic_threads);}
1294 @item @emph{Fortran}:
1295 @multitable @columnfractions .20 .80
1296 @item @emph{Interface}: @tab @code{subroutine omp_set_dynamic(dynamic_threads)}
1297 @item @tab @code{logical, intent(in) :: dynamic_threads}
1300 @item @emph{See also}:
1301 @ref{OMP_DYNAMIC}, @ref{omp_get_dynamic}
1303 @item @emph{Reference}:
1304 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.7.
1309 @node omp_set_max_active_levels
1310 @section @code{omp_set_max_active_levels} -- Limits the number of active parallel regions
1312 @item @emph{Description}:
1313 This function limits the maximum allowed number of nested, active
1314 parallel regions. @var{max_levels} must be less or equal to
1315 the value returned by @code{omp_get_supported_active_levels}.
1318 @multitable @columnfractions .20 .80
1319 @item @emph{Prototype}: @tab @code{void omp_set_max_active_levels(int max_levels);}
1322 @item @emph{Fortran}:
1323 @multitable @columnfractions .20 .80
1324 @item @emph{Interface}: @tab @code{subroutine omp_set_max_active_levels(max_levels)}
1325 @item @tab @code{integer max_levels}
1328 @item @emph{See also}:
1329 @ref{omp_get_max_active_levels}, @ref{omp_get_active_level},
1330 @ref{omp_get_supported_active_levels}
1332 @item @emph{Reference}:
1333 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.15.
1338 @node omp_set_nested
1339 @section @code{omp_set_nested} -- Enable/disable nested parallel regions
1341 @item @emph{Description}:
1342 Enable or disable nested parallel regions, i.e., whether team members
1343 are allowed to create new teams. The function takes the language-specific
1344 equivalent of @code{true} and @code{false}, where @code{true} enables
1345 dynamic adjustment of team sizes and @code{false} disables it.
1347 Enabling nested parallel regions will also set the maximum number of
1348 active nested regions to the maximum supported. Disabling nested parallel
1349 regions will set the maximum number of active nested regions to one.
1352 @multitable @columnfractions .20 .80
1353 @item @emph{Prototype}: @tab @code{void omp_set_nested(int nested);}
1356 @item @emph{Fortran}:
1357 @multitable @columnfractions .20 .80
1358 @item @emph{Interface}: @tab @code{subroutine omp_set_nested(nested)}
1359 @item @tab @code{logical, intent(in) :: nested}
1362 @item @emph{See also}:
1363 @ref{omp_get_nested}, @ref{omp_set_max_active_levels},
1364 @ref{OMP_MAX_ACTIVE_LEVELS}, @ref{OMP_NESTED}
1366 @item @emph{Reference}:
1367 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.10.
1372 @node omp_set_num_teams
1373 @section @code{omp_set_num_teams} -- Set upper teams limit for teams construct
1375 @item @emph{Description}:
1376 Specifies the upper bound for number of teams created by the teams construct
1377 which does not specify a @code{num_teams} clause. The
1378 argument of @code{omp_set_num_teams} shall be a positive integer.
1381 @multitable @columnfractions .20 .80
1382 @item @emph{Prototype}: @tab @code{void omp_set_num_teams(int num_teams);}
1385 @item @emph{Fortran}:
1386 @multitable @columnfractions .20 .80
1387 @item @emph{Interface}: @tab @code{subroutine omp_set_num_teams(num_teams)}
1388 @item @tab @code{integer, intent(in) :: num_teams}
1391 @item @emph{See also}:
1392 @ref{OMP_NUM_TEAMS}, @ref{omp_get_num_teams}, @ref{omp_get_max_teams}
1394 @item @emph{Reference}:
1395 @uref{https://www.openmp.org, OpenMP specification v5.1}, Section 3.4.3.
1400 @node omp_set_num_threads
1401 @section @code{omp_set_num_threads} -- Set upper team size limit
1403 @item @emph{Description}:
1404 Specifies the number of threads used by default in subsequent parallel
1405 sections, if those do not specify a @code{num_threads} clause. The
1406 argument of @code{omp_set_num_threads} shall be a positive integer.
1409 @multitable @columnfractions .20 .80
1410 @item @emph{Prototype}: @tab @code{void omp_set_num_threads(int num_threads);}
1413 @item @emph{Fortran}:
1414 @multitable @columnfractions .20 .80
1415 @item @emph{Interface}: @tab @code{subroutine omp_set_num_threads(num_threads)}
1416 @item @tab @code{integer, intent(in) :: num_threads}
1419 @item @emph{See also}:
1420 @ref{OMP_NUM_THREADS}, @ref{omp_get_num_threads}, @ref{omp_get_max_threads}
1422 @item @emph{Reference}:
1423 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.1.
1428 @node omp_set_schedule
1429 @section @code{omp_set_schedule} -- Set the runtime scheduling method
1431 @item @emph{Description}:
1432 Sets the runtime scheduling method. The @var{kind} argument can have the
1433 value @code{omp_sched_static}, @code{omp_sched_dynamic},
1434 @code{omp_sched_guided} or @code{omp_sched_auto}. Except for
1435 @code{omp_sched_auto}, the chunk size is set to the value of
1436 @var{chunk_size} if positive, or to the default value if zero or negative.
1437 For @code{omp_sched_auto} the @var{chunk_size} argument is ignored.
1440 @multitable @columnfractions .20 .80
1441 @item @emph{Prototype}: @tab @code{void omp_set_schedule(omp_sched_t kind, int chunk_size);}
1444 @item @emph{Fortran}:
1445 @multitable @columnfractions .20 .80
1446 @item @emph{Interface}: @tab @code{subroutine omp_set_schedule(kind, chunk_size)}
1447 @item @tab @code{integer(kind=omp_sched_kind) kind}
1448 @item @tab @code{integer chunk_size}
1451 @item @emph{See also}:
1452 @ref{omp_get_schedule}
1455 @item @emph{Reference}:
1456 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.12.
1461 @node omp_set_teams_thread_limit
1462 @section @code{omp_set_teams_thread_limit} -- Set upper thread limit for teams construct
1464 @item @emph{Description}:
1465 Specifies the upper bound for number of threads that will be available
1466 for each team created by the teams construct which does not specify a
1467 @code{thread_limit} clause. The argument of
1468 @code{omp_set_teams_thread_limit} shall be a positive integer.
1471 @multitable @columnfractions .20 .80
1472 @item @emph{Prototype}: @tab @code{void omp_set_teams_thread_limit(int thread_limit);}
1475 @item @emph{Fortran}:
1476 @multitable @columnfractions .20 .80
1477 @item @emph{Interface}: @tab @code{subroutine omp_set_teams_thread_limit(thread_limit)}
1478 @item @tab @code{integer, intent(in) :: thread_limit}
1481 @item @emph{See also}:
1482 @ref{OMP_TEAMS_THREAD_LIMIT}, @ref{omp_get_teams_thread_limit}, @ref{omp_get_thread_limit}
1484 @item @emph{Reference}:
1485 @uref{https://www.openmp.org, OpenMP specification v5.1}, Section 3.4.5.
1491 @section @code{omp_init_lock} -- Initialize simple lock
1493 @item @emph{Description}:
1494 Initialize a simple lock. After initialization, the lock is in
1498 @multitable @columnfractions .20 .80
1499 @item @emph{Prototype}: @tab @code{void omp_init_lock(omp_lock_t *lock);}
1502 @item @emph{Fortran}:
1503 @multitable @columnfractions .20 .80
1504 @item @emph{Interface}: @tab @code{subroutine omp_init_lock(svar)}
1505 @item @tab @code{integer(omp_lock_kind), intent(out) :: svar}
1508 @item @emph{See also}:
1509 @ref{omp_destroy_lock}
1511 @item @emph{Reference}:
1512 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.3.1.
1518 @section @code{omp_set_lock} -- Wait for and set simple lock
1520 @item @emph{Description}:
1521 Before setting a simple lock, the lock variable must be initialized by
1522 @code{omp_init_lock}. The calling thread is blocked until the lock
1523 is available. If the lock is already held by the current thread,
1527 @multitable @columnfractions .20 .80
1528 @item @emph{Prototype}: @tab @code{void omp_set_lock(omp_lock_t *lock);}
1531 @item @emph{Fortran}:
1532 @multitable @columnfractions .20 .80
1533 @item @emph{Interface}: @tab @code{subroutine omp_set_lock(svar)}
1534 @item @tab @code{integer(omp_lock_kind), intent(inout) :: svar}
1537 @item @emph{See also}:
1538 @ref{omp_init_lock}, @ref{omp_test_lock}, @ref{omp_unset_lock}
1540 @item @emph{Reference}:
1541 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.3.4.
1547 @section @code{omp_test_lock} -- Test and set simple lock if available
1549 @item @emph{Description}:
1550 Before setting a simple lock, the lock variable must be initialized by
1551 @code{omp_init_lock}. Contrary to @code{omp_set_lock}, @code{omp_test_lock}
1552 does not block if the lock is not available. This function returns
1553 @code{true} upon success, @code{false} otherwise. Here, @code{true} and
1554 @code{false} represent their language-specific counterparts.
1557 @multitable @columnfractions .20 .80
1558 @item @emph{Prototype}: @tab @code{int omp_test_lock(omp_lock_t *lock);}
1561 @item @emph{Fortran}:
1562 @multitable @columnfractions .20 .80
1563 @item @emph{Interface}: @tab @code{logical function omp_test_lock(svar)}
1564 @item @tab @code{integer(omp_lock_kind), intent(inout) :: svar}
1567 @item @emph{See also}:
1568 @ref{omp_init_lock}, @ref{omp_set_lock}, @ref{omp_set_lock}
1570 @item @emph{Reference}:
1571 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.3.6.
1576 @node omp_unset_lock
1577 @section @code{omp_unset_lock} -- Unset simple lock
1579 @item @emph{Description}:
1580 A simple lock about to be unset must have been locked by @code{omp_set_lock}
1581 or @code{omp_test_lock} before. In addition, the lock must be held by the
1582 thread calling @code{omp_unset_lock}. Then, the lock becomes unlocked. If one
1583 or more threads attempted to set the lock before, one of them is chosen to,
1584 again, set the lock to itself.
1587 @multitable @columnfractions .20 .80
1588 @item @emph{Prototype}: @tab @code{void omp_unset_lock(omp_lock_t *lock);}
1591 @item @emph{Fortran}:
1592 @multitable @columnfractions .20 .80
1593 @item @emph{Interface}: @tab @code{subroutine omp_unset_lock(svar)}
1594 @item @tab @code{integer(omp_lock_kind), intent(inout) :: svar}
1597 @item @emph{See also}:
1598 @ref{omp_set_lock}, @ref{omp_test_lock}
1600 @item @emph{Reference}:
1601 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.3.5.
1606 @node omp_destroy_lock
1607 @section @code{omp_destroy_lock} -- Destroy simple lock
1609 @item @emph{Description}:
1610 Destroy a simple lock. In order to be destroyed, a simple lock must be
1611 in the unlocked state.
1614 @multitable @columnfractions .20 .80
1615 @item @emph{Prototype}: @tab @code{void omp_destroy_lock(omp_lock_t *lock);}
1618 @item @emph{Fortran}:
1619 @multitable @columnfractions .20 .80
1620 @item @emph{Interface}: @tab @code{subroutine omp_destroy_lock(svar)}
1621 @item @tab @code{integer(omp_lock_kind), intent(inout) :: svar}
1624 @item @emph{See also}:
1627 @item @emph{Reference}:
1628 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.3.3.
1633 @node omp_init_nest_lock
1634 @section @code{omp_init_nest_lock} -- Initialize nested lock
1636 @item @emph{Description}:
1637 Initialize a nested lock. After initialization, the lock is in
1638 an unlocked state and the nesting count is set to zero.
1641 @multitable @columnfractions .20 .80
1642 @item @emph{Prototype}: @tab @code{void omp_init_nest_lock(omp_nest_lock_t *lock);}
1645 @item @emph{Fortran}:
1646 @multitable @columnfractions .20 .80
1647 @item @emph{Interface}: @tab @code{subroutine omp_init_nest_lock(nvar)}
1648 @item @tab @code{integer(omp_nest_lock_kind), intent(out) :: nvar}
1651 @item @emph{See also}:
1652 @ref{omp_destroy_nest_lock}
1654 @item @emph{Reference}:
1655 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.3.1.
1659 @node omp_set_nest_lock
1660 @section @code{omp_set_nest_lock} -- Wait for and set nested lock
1662 @item @emph{Description}:
1663 Before setting a nested lock, the lock variable must be initialized by
1664 @code{omp_init_nest_lock}. The calling thread is blocked until the lock
1665 is available. If the lock is already held by the current thread, the
1666 nesting count for the lock is incremented.
1669 @multitable @columnfractions .20 .80
1670 @item @emph{Prototype}: @tab @code{void omp_set_nest_lock(omp_nest_lock_t *lock);}
1673 @item @emph{Fortran}:
1674 @multitable @columnfractions .20 .80
1675 @item @emph{Interface}: @tab @code{subroutine omp_set_nest_lock(nvar)}
1676 @item @tab @code{integer(omp_nest_lock_kind), intent(inout) :: nvar}
1679 @item @emph{See also}:
1680 @ref{omp_init_nest_lock}, @ref{omp_unset_nest_lock}
1682 @item @emph{Reference}:
1683 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.3.4.
1688 @node omp_test_nest_lock
1689 @section @code{omp_test_nest_lock} -- Test and set nested lock if available
1691 @item @emph{Description}:
1692 Before setting a nested lock, the lock variable must be initialized by
1693 @code{omp_init_nest_lock}. Contrary to @code{omp_set_nest_lock},
1694 @code{omp_test_nest_lock} does not block if the lock is not available.
1695 If the lock is already held by the current thread, the new nesting count
1696 is returned. Otherwise, the return value equals zero.
1699 @multitable @columnfractions .20 .80
1700 @item @emph{Prototype}: @tab @code{int omp_test_nest_lock(omp_nest_lock_t *lock);}
1703 @item @emph{Fortran}:
1704 @multitable @columnfractions .20 .80
1705 @item @emph{Interface}: @tab @code{logical function omp_test_nest_lock(nvar)}
1706 @item @tab @code{integer(omp_nest_lock_kind), intent(inout) :: nvar}
1710 @item @emph{See also}:
1711 @ref{omp_init_lock}, @ref{omp_set_lock}, @ref{omp_set_lock}
1713 @item @emph{Reference}:
1714 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.3.6.
1719 @node omp_unset_nest_lock
1720 @section @code{omp_unset_nest_lock} -- Unset nested lock
1722 @item @emph{Description}:
1723 A nested lock about to be unset must have been locked by @code{omp_set_nested_lock}
1724 or @code{omp_test_nested_lock} before. In addition, the lock must be held by the
1725 thread calling @code{omp_unset_nested_lock}. If the nesting count drops to zero, the
1726 lock becomes unlocked. If one ore more threads attempted to set the lock before,
1727 one of them is chosen to, again, set the lock to itself.
1730 @multitable @columnfractions .20 .80
1731 @item @emph{Prototype}: @tab @code{void omp_unset_nest_lock(omp_nest_lock_t *lock);}
1734 @item @emph{Fortran}:
1735 @multitable @columnfractions .20 .80
1736 @item @emph{Interface}: @tab @code{subroutine omp_unset_nest_lock(nvar)}
1737 @item @tab @code{integer(omp_nest_lock_kind), intent(inout) :: nvar}
1740 @item @emph{See also}:
1741 @ref{omp_set_nest_lock}
1743 @item @emph{Reference}:
1744 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.3.5.
1749 @node omp_destroy_nest_lock
1750 @section @code{omp_destroy_nest_lock} -- Destroy nested lock
1752 @item @emph{Description}:
1753 Destroy a nested lock. In order to be destroyed, a nested lock must be
1754 in the unlocked state and its nesting count must equal zero.
1757 @multitable @columnfractions .20 .80
1758 @item @emph{Prototype}: @tab @code{void omp_destroy_nest_lock(omp_nest_lock_t *);}
1761 @item @emph{Fortran}:
1762 @multitable @columnfractions .20 .80
1763 @item @emph{Interface}: @tab @code{subroutine omp_destroy_nest_lock(nvar)}
1764 @item @tab @code{integer(omp_nest_lock_kind), intent(inout) :: nvar}
1767 @item @emph{See also}:
1770 @item @emph{Reference}:
1771 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.3.3.
1777 @section @code{omp_get_wtick} -- Get timer precision
1779 @item @emph{Description}:
1780 Gets the timer precision, i.e., the number of seconds between two
1781 successive clock ticks.
1784 @multitable @columnfractions .20 .80
1785 @item @emph{Prototype}: @tab @code{double omp_get_wtick(void);}
1788 @item @emph{Fortran}:
1789 @multitable @columnfractions .20 .80
1790 @item @emph{Interface}: @tab @code{double precision function omp_get_wtick()}
1793 @item @emph{See also}:
1796 @item @emph{Reference}:
1797 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.4.2.
1803 @section @code{omp_get_wtime} -- Elapsed wall clock time
1805 @item @emph{Description}:
1806 Elapsed wall clock time in seconds. The time is measured per thread, no
1807 guarantee can be made that two distinct threads measure the same time.
1808 Time is measured from some "time in the past", which is an arbitrary time
1809 guaranteed not to change during the execution of the program.
1812 @multitable @columnfractions .20 .80
1813 @item @emph{Prototype}: @tab @code{double omp_get_wtime(void);}
1816 @item @emph{Fortran}:
1817 @multitable @columnfractions .20 .80
1818 @item @emph{Interface}: @tab @code{double precision function omp_get_wtime()}
1821 @item @emph{See also}:
1824 @item @emph{Reference}:
1825 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.4.1.
1830 @node omp_fulfill_event
1831 @section @code{omp_fulfill_event} -- Fulfill and destroy an OpenMP event
1833 @item @emph{Description}:
1834 Fulfill the event associated with the event handle argument. Currently, it
1835 is only used to fulfill events generated by detach clauses on task
1836 constructs - the effect of fulfilling the event is to allow the task to
1839 The result of calling @code{omp_fulfill_event} with an event handle other
1840 than that generated by a detach clause is undefined. Calling it with an
1841 event handle that has already been fulfilled is also undefined.
1844 @multitable @columnfractions .20 .80
1845 @item @emph{Prototype}: @tab @code{void omp_fulfill_event(omp_event_handle_t event);}
1848 @item @emph{Fortran}:
1849 @multitable @columnfractions .20 .80
1850 @item @emph{Interface}: @tab @code{subroutine omp_fulfill_event(event)}
1851 @item @tab @code{integer (kind=omp_event_handle_kind) :: event}
1854 @item @emph{Reference}:
1855 @uref{https://www.openmp.org, OpenMP specification v5.0}, Section 3.5.1.
1860 @c ---------------------------------------------------------------------
1861 @c OpenMP Environment Variables
1862 @c ---------------------------------------------------------------------
1864 @node Environment Variables
1865 @chapter OpenMP Environment Variables
1867 The environment variables which beginning with @env{OMP_} are defined by
1868 section 4 of the OpenMP specification in version 4.5, while those
1869 beginning with @env{GOMP_} are GNU extensions.
1872 * OMP_CANCELLATION:: Set whether cancellation is activated
1873 * OMP_DISPLAY_ENV:: Show OpenMP version and environment variables
1874 * OMP_DEFAULT_DEVICE:: Set the device used in target regions
1875 * OMP_DYNAMIC:: Dynamic adjustment of threads
1876 * OMP_MAX_ACTIVE_LEVELS:: Set the maximum number of nested parallel regions
1877 * OMP_MAX_TASK_PRIORITY:: Set the maximum task priority value
1878 * OMP_NESTED:: Nested parallel regions
1879 * OMP_NUM_TEAMS:: Specifies the number of teams to use by teams region
1880 * OMP_NUM_THREADS:: Specifies the number of threads to use
1881 * OMP_PROC_BIND:: Whether theads may be moved between CPUs
1882 * OMP_PLACES:: Specifies on which CPUs the theads should be placed
1883 * OMP_STACKSIZE:: Set default thread stack size
1884 * OMP_SCHEDULE:: How threads are scheduled
1885 * OMP_TARGET_OFFLOAD:: Controls offloading behaviour
1886 * OMP_TEAMS_THREAD_LIMIT:: Set the maximum number of threads imposed by teams
1887 * OMP_THREAD_LIMIT:: Set the maximum number of threads
1888 * OMP_WAIT_POLICY:: How waiting threads are handled
1889 * GOMP_CPU_AFFINITY:: Bind threads to specific CPUs
1890 * GOMP_DEBUG:: Enable debugging output
1891 * GOMP_STACKSIZE:: Set default thread stack size
1892 * GOMP_SPINCOUNT:: Set the busy-wait spin count
1893 * GOMP_RTEMS_THREAD_POOLS:: Set the RTEMS specific thread pools
1897 @node OMP_CANCELLATION
1898 @section @env{OMP_CANCELLATION} -- Set whether cancellation is activated
1899 @cindex Environment Variable
1901 @item @emph{Description}:
1902 If set to @code{TRUE}, the cancellation is activated. If set to @code{FALSE} or
1903 if unset, cancellation is disabled and the @code{cancel} construct is ignored.
1905 @item @emph{See also}:
1906 @ref{omp_get_cancellation}
1908 @item @emph{Reference}:
1909 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 4.11
1914 @node OMP_DISPLAY_ENV
1915 @section @env{OMP_DISPLAY_ENV} -- Show OpenMP version and environment variables
1916 @cindex Environment Variable
1918 @item @emph{Description}:
1919 If set to @code{TRUE}, the OpenMP version number and the values
1920 associated with the OpenMP environment variables are printed to @code{stderr}.
1921 If set to @code{VERBOSE}, it additionally shows the value of the environment
1922 variables which are GNU extensions. If undefined or set to @code{FALSE},
1923 this information will not be shown.
1926 @item @emph{Reference}:
1927 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 4.12
1932 @node OMP_DEFAULT_DEVICE
1933 @section @env{OMP_DEFAULT_DEVICE} -- Set the device used in target regions
1934 @cindex Environment Variable
1936 @item @emph{Description}:
1937 Set to choose the device which is used in a @code{target} region, unless the
1938 value is overridden by @code{omp_set_default_device} or by a @code{device}
1939 clause. The value shall be the nonnegative device number. If no device with
1940 the given device number exists, the code is executed on the host. If unset,
1941 device number 0 will be used.
1944 @item @emph{See also}:
1945 @ref{omp_get_default_device}, @ref{omp_set_default_device},
1947 @item @emph{Reference}:
1948 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 4.13
1954 @section @env{OMP_DYNAMIC} -- Dynamic adjustment of threads
1955 @cindex Environment Variable
1957 @item @emph{Description}:
1958 Enable or disable the dynamic adjustment of the number of threads
1959 within a team. The value of this environment variable shall be
1960 @code{TRUE} or @code{FALSE}. If undefined, dynamic adjustment is
1961 disabled by default.
1963 @item @emph{See also}:
1964 @ref{omp_set_dynamic}
1966 @item @emph{Reference}:
1967 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 4.3
1972 @node OMP_MAX_ACTIVE_LEVELS
1973 @section @env{OMP_MAX_ACTIVE_LEVELS} -- Set the maximum number of nested parallel regions
1974 @cindex Environment Variable
1976 @item @emph{Description}:
1977 Specifies the initial value for the maximum number of nested parallel
1978 regions. The value of this variable shall be a positive integer.
1979 If undefined, then if @env{OMP_NESTED} is defined and set to true, or
1980 if @env{OMP_NUM_THREADS} or @env{OMP_PROC_BIND} are defined and set to
1981 a list with more than one item, the maximum number of nested parallel
1982 regions will be initialized to the largest number supported, otherwise
1983 it will be set to one.
1985 @item @emph{See also}:
1986 @ref{omp_set_max_active_levels}, @ref{OMP_NESTED}
1988 @item @emph{Reference}:
1989 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 4.9
1994 @node OMP_MAX_TASK_PRIORITY
1995 @section @env{OMP_MAX_TASK_PRIORITY} -- Set the maximum priority
1996 number that can be set for a task.
1997 @cindex Environment Variable
1999 @item @emph{Description}:
2000 Specifies the initial value for the maximum priority value that can be
2001 set for a task. The value of this variable shall be a non-negative
2002 integer, and zero is allowed. If undefined, the default priority is
2005 @item @emph{See also}:
2006 @ref{omp_get_max_task_priority}
2008 @item @emph{Reference}:
2009 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 4.14
2015 @section @env{OMP_NESTED} -- Nested parallel regions
2016 @cindex Environment Variable
2017 @cindex Implementation specific setting
2019 @item @emph{Description}:
2020 Enable or disable nested parallel regions, i.e., whether team members
2021 are allowed to create new teams. The value of this environment variable
2022 shall be @code{TRUE} or @code{FALSE}. If set to @code{TRUE}, the number
2023 of maximum active nested regions supported will by default be set to the
2024 maximum supported, otherwise it will be set to one. If
2025 @env{OMP_MAX_ACTIVE_LEVELS} is defined, its setting will override this
2026 setting. If both are undefined, nested parallel regions are enabled if
2027 @env{OMP_NUM_THREADS} or @env{OMP_PROC_BINDS} are defined to a list with
2028 more than one item, otherwise they are disabled by default.
2030 @item @emph{See also}:
2031 @ref{omp_set_max_active_levels}, @ref{omp_set_nested}
2033 @item @emph{Reference}:
2034 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 4.6
2040 @section @env{OMP_NUM_TEAMS} -- Specifies the number of teams to use by teams region
2041 @cindex Environment Variable
2043 @item @emph{Description}:
2044 Specifies the upper bound for number of teams to use in teams regions
2045 without explicit @code{num_teams} clause. The value of this variable shall
2046 be a positive integer. If undefined it defaults to 0 which means
2047 implementation defined upper bound.
2049 @item @emph{See also}:
2050 @ref{omp_set_num_teams}
2052 @item @emph{Reference}:
2053 @uref{https://www.openmp.org, OpenMP specification v5.1}, Section 6.23
2058 @node OMP_NUM_THREADS
2059 @section @env{OMP_NUM_THREADS} -- Specifies the number of threads to use
2060 @cindex Environment Variable
2061 @cindex Implementation specific setting
2063 @item @emph{Description}:
2064 Specifies the default number of threads to use in parallel regions. The
2065 value of this variable shall be a comma-separated list of positive integers;
2066 the value specifies the number of threads to use for the corresponding nested
2067 level. Specifying more than one item in the list will automatically enable
2068 nesting by default. If undefined one thread per CPU is used.
2070 @item @emph{See also}:
2071 @ref{omp_set_num_threads}, @ref{OMP_NESTED}
2073 @item @emph{Reference}:
2074 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 4.2
2080 @section @env{OMP_PROC_BIND} -- Whether theads may be moved between CPUs
2081 @cindex Environment Variable
2083 @item @emph{Description}:
2084 Specifies whether threads may be moved between processors. If set to
2085 @code{TRUE}, OpenMP theads should not be moved; if set to @code{FALSE}
2086 they may be moved. Alternatively, a comma separated list with the
2087 values @code{PRIMARY}, @code{MASTER}, @code{CLOSE} and @code{SPREAD} can
2088 be used to specify the thread affinity policy for the corresponding nesting
2089 level. With @code{PRIMARY} and @code{MASTER} the worker threads are in the
2090 same place partition as the primary thread. With @code{CLOSE} those are
2091 kept close to the primary thread in contiguous place partitions. And
2092 with @code{SPREAD} a sparse distribution
2093 across the place partitions is used. Specifying more than one item in the
2094 list will automatically enable nesting by default.
2096 When undefined, @env{OMP_PROC_BIND} defaults to @code{TRUE} when
2097 @env{OMP_PLACES} or @env{GOMP_CPU_AFFINITY} is set and @code{FALSE} otherwise.
2099 @item @emph{See also}:
2100 @ref{omp_get_proc_bind}, @ref{GOMP_CPU_AFFINITY},
2101 @ref{OMP_NESTED}, @ref{OMP_PLACES}
2103 @item @emph{Reference}:
2104 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 4.4
2110 @section @env{OMP_PLACES} -- Specifies on which CPUs the theads should be placed
2111 @cindex Environment Variable
2113 @item @emph{Description}:
2114 The thread placement can be either specified using an abstract name or by an
2115 explicit list of the places. The abstract names @code{threads}, @code{cores},
2116 @code{sockets}, @code{ll_caches} and @code{numa_domains} can be optionally
2117 followed by a positive number in parentheses, which denotes the how many places
2118 shall be created. With @code{threads} each place corresponds to a single
2119 hardware thread; @code{cores} to a single core with the corresponding number of
2120 hardware threads; with @code{sockets} the place corresponds to a single
2121 socket; with @code{ll_caches} to a set of cores that shares the last level
2122 cache on the device; and @code{numa_domains} to a set of cores for which their
2123 closest memory on the device is the same memory and at a similar distance from
2124 the cores. The resulting placement can be shown by setting the
2125 @env{OMP_DISPLAY_ENV} environment variable.
2127 Alternatively, the placement can be specified explicitly as comma-separated
2128 list of places. A place is specified by set of nonnegative numbers in curly
2129 braces, denoting the hardware threads. The curly braces can be omitted
2130 when only a single number has been specified. The hardware threads
2131 belonging to a place can either be specified as comma-separated list of
2132 nonnegative thread numbers or using an interval. Multiple places can also be
2133 either specified by a comma-separated list of places or by an interval. To
2134 specify an interval, a colon followed by the count is placed after
2135 the hardware thread number or the place. Optionally, the length can be
2136 followed by a colon and the stride number -- otherwise a unit stride is
2137 assumed. Placing an exclamation mark (@code{!}) directly before a curly
2138 brace or numbers inside the curly braces (excluding intervals) will
2139 exclude those hardware threads.
2141 For instance, the following specifies the same places list:
2142 @code{"@{0,1,2@}, @{3,4,6@}, @{7,8,9@}, @{10,11,12@}"};
2143 @code{"@{0:3@}, @{3:3@}, @{7:3@}, @{10:3@}"}; and @code{"@{0:2@}:4:3"}.
2145 If @env{OMP_PLACES} and @env{GOMP_CPU_AFFINITY} are unset and
2146 @env{OMP_PROC_BIND} is either unset or @code{false}, threads may be moved
2147 between CPUs following no placement policy.
2149 @item @emph{See also}:
2150 @ref{OMP_PROC_BIND}, @ref{GOMP_CPU_AFFINITY}, @ref{omp_get_proc_bind},
2151 @ref{OMP_DISPLAY_ENV}
2153 @item @emph{Reference}:
2154 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 4.5
2160 @section @env{OMP_STACKSIZE} -- Set default thread stack size
2161 @cindex Environment Variable
2163 @item @emph{Description}:
2164 Set the default thread stack size in kilobytes, unless the number
2165 is suffixed by @code{B}, @code{K}, @code{M} or @code{G}, in which
2166 case the size is, respectively, in bytes, kilobytes, megabytes
2167 or gigabytes. This is different from @code{pthread_attr_setstacksize}
2168 which gets the number of bytes as an argument. If the stack size cannot
2169 be set due to system constraints, an error is reported and the initial
2170 stack size is left unchanged. If undefined, the stack size is system
2173 @item @emph{Reference}:
2174 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 4.7
2180 @section @env{OMP_SCHEDULE} -- How threads are scheduled
2181 @cindex Environment Variable
2182 @cindex Implementation specific setting
2184 @item @emph{Description}:
2185 Allows to specify @code{schedule type} and @code{chunk size}.
2186 The value of the variable shall have the form: @code{type[,chunk]} where
2187 @code{type} is one of @code{static}, @code{dynamic}, @code{guided} or @code{auto}
2188 The optional @code{chunk} size shall be a positive integer. If undefined,
2189 dynamic scheduling and a chunk size of 1 is used.
2191 @item @emph{See also}:
2192 @ref{omp_set_schedule}
2194 @item @emph{Reference}:
2195 @uref{https://www.openmp.org, OpenMP specification v4.5}, Sections 2.7.1.1 and 4.1
2200 @node OMP_TARGET_OFFLOAD
2201 @section @env{OMP_TARGET_OFFLOAD} -- Controls offloading behaviour
2202 @cindex Environment Variable
2203 @cindex Implementation specific setting
2205 @item @emph{Description}:
2206 Specifies the behaviour with regard to offloading code to a device. This
2207 variable can be set to one of three values - @code{MANDATORY}, @code{DISABLED}
2210 If set to @code{MANDATORY}, the program will terminate with an error if
2211 the offload device is not present or is not supported. If set to
2212 @code{DISABLED}, then offloading is disabled and all code will run on the
2213 host. If set to @code{DEFAULT}, the program will try offloading to the
2214 device first, then fall back to running code on the host if it cannot.
2216 If undefined, then the program will behave as if @code{DEFAULT} was set.
2218 @item @emph{Reference}:
2219 @uref{https://www.openmp.org, OpenMP specification v5.0}, Section 6.17
2224 @node OMP_TEAMS_THREAD_LIMIT
2225 @section @env{OMP_TEAMS_THREAD_LIMIT} -- Set the maximum number of threads imposed by teams
2226 @cindex Environment Variable
2228 @item @emph{Description}:
2229 Specifies an upper bound for the number of threads to use by each contention
2230 group created by a teams construct without explicit @code{thread_limit}
2231 clause. The value of this variable shall be a positive integer. If undefined,
2232 the value of 0 is used which stands for an implementation defined upper
2235 @item @emph{See also}:
2236 @ref{OMP_THREAD_LIMIT}, @ref{omp_set_teams_thread_limit}
2238 @item @emph{Reference}:
2239 @uref{https://www.openmp.org, OpenMP specification v5.1}, Section 6.24
2244 @node OMP_THREAD_LIMIT
2245 @section @env{OMP_THREAD_LIMIT} -- Set the maximum number of threads
2246 @cindex Environment Variable
2248 @item @emph{Description}:
2249 Specifies the number of threads to use for the whole program. The
2250 value of this variable shall be a positive integer. If undefined,
2251 the number of threads is not limited.
2253 @item @emph{See also}:
2254 @ref{OMP_NUM_THREADS}, @ref{omp_get_thread_limit}
2256 @item @emph{Reference}:
2257 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 4.10
2262 @node OMP_WAIT_POLICY
2263 @section @env{OMP_WAIT_POLICY} -- How waiting threads are handled
2264 @cindex Environment Variable
2266 @item @emph{Description}:
2267 Specifies whether waiting threads should be active or passive. If
2268 the value is @code{PASSIVE}, waiting threads should not consume CPU
2269 power while waiting; while the value is @code{ACTIVE} specifies that
2270 they should. If undefined, threads wait actively for a short time
2271 before waiting passively.
2273 @item @emph{See also}:
2274 @ref{GOMP_SPINCOUNT}
2276 @item @emph{Reference}:
2277 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 4.8
2282 @node GOMP_CPU_AFFINITY
2283 @section @env{GOMP_CPU_AFFINITY} -- Bind threads to specific CPUs
2284 @cindex Environment Variable
2286 @item @emph{Description}:
2287 Binds threads to specific CPUs. The variable should contain a space-separated
2288 or comma-separated list of CPUs. This list may contain different kinds of
2289 entries: either single CPU numbers in any order, a range of CPUs (M-N)
2290 or a range with some stride (M-N:S). CPU numbers are zero based. For example,
2291 @code{GOMP_CPU_AFFINITY="0 3 1-2 4-15:2"} will bind the initial thread
2292 to CPU 0, the second to CPU 3, the third to CPU 1, the fourth to
2293 CPU 2, the fifth to CPU 4, the sixth through tenth to CPUs 6, 8, 10, 12,
2294 and 14 respectively and then start assigning back from the beginning of
2295 the list. @code{GOMP_CPU_AFFINITY=0} binds all threads to CPU 0.
2297 There is no libgomp library routine to determine whether a CPU affinity
2298 specification is in effect. As a workaround, language-specific library
2299 functions, e.g., @code{getenv} in C or @code{GET_ENVIRONMENT_VARIABLE} in
2300 Fortran, may be used to query the setting of the @code{GOMP_CPU_AFFINITY}
2301 environment variable. A defined CPU affinity on startup cannot be changed
2302 or disabled during the runtime of the application.
2304 If both @env{GOMP_CPU_AFFINITY} and @env{OMP_PROC_BIND} are set,
2305 @env{OMP_PROC_BIND} has a higher precedence. If neither has been set and
2306 @env{OMP_PROC_BIND} is unset, or when @env{OMP_PROC_BIND} is set to
2307 @code{FALSE}, the host system will handle the assignment of threads to CPUs.
2309 @item @emph{See also}:
2310 @ref{OMP_PLACES}, @ref{OMP_PROC_BIND}
2316 @section @env{GOMP_DEBUG} -- Enable debugging output
2317 @cindex Environment Variable
2319 @item @emph{Description}:
2320 Enable debugging output. The variable should be set to @code{0}
2321 (disabled, also the default if not set), or @code{1} (enabled).
2323 If enabled, some debugging output will be printed during execution.
2324 This is currently not specified in more detail, and subject to change.
2329 @node GOMP_STACKSIZE
2330 @section @env{GOMP_STACKSIZE} -- Set default thread stack size
2331 @cindex Environment Variable
2332 @cindex Implementation specific setting
2334 @item @emph{Description}:
2335 Set the default thread stack size in kilobytes. This is different from
2336 @code{pthread_attr_setstacksize} which gets the number of bytes as an
2337 argument. If the stack size cannot be set due to system constraints, an
2338 error is reported and the initial stack size is left unchanged. If undefined,
2339 the stack size is system dependent.
2341 @item @emph{See also}:
2344 @item @emph{Reference}:
2345 @uref{https://gcc.gnu.org/ml/gcc-patches/2006-06/msg00493.html,
2346 GCC Patches Mailinglist},
2347 @uref{https://gcc.gnu.org/ml/gcc-patches/2006-06/msg00496.html,
2348 GCC Patches Mailinglist}
2353 @node GOMP_SPINCOUNT
2354 @section @env{GOMP_SPINCOUNT} -- Set the busy-wait spin count
2355 @cindex Environment Variable
2356 @cindex Implementation specific setting
2358 @item @emph{Description}:
2359 Determines how long a threads waits actively with consuming CPU power
2360 before waiting passively without consuming CPU power. The value may be
2361 either @code{INFINITE}, @code{INFINITY} to always wait actively or an
2362 integer which gives the number of spins of the busy-wait loop. The
2363 integer may optionally be followed by the following suffixes acting
2364 as multiplication factors: @code{k} (kilo, thousand), @code{M} (mega,
2365 million), @code{G} (giga, billion), or @code{T} (tera, trillion).
2366 If undefined, 0 is used when @env{OMP_WAIT_POLICY} is @code{PASSIVE},
2367 300,000 is used when @env{OMP_WAIT_POLICY} is undefined and
2368 30 billion is used when @env{OMP_WAIT_POLICY} is @code{ACTIVE}.
2369 If there are more OpenMP threads than available CPUs, 1000 and 100
2370 spins are used for @env{OMP_WAIT_POLICY} being @code{ACTIVE} or
2371 undefined, respectively; unless the @env{GOMP_SPINCOUNT} is lower
2372 or @env{OMP_WAIT_POLICY} is @code{PASSIVE}.
2374 @item @emph{See also}:
2375 @ref{OMP_WAIT_POLICY}
2380 @node GOMP_RTEMS_THREAD_POOLS
2381 @section @env{GOMP_RTEMS_THREAD_POOLS} -- Set the RTEMS specific thread pools
2382 @cindex Environment Variable
2383 @cindex Implementation specific setting
2385 @item @emph{Description}:
2386 This environment variable is only used on the RTEMS real-time operating system.
2387 It determines the scheduler instance specific thread pools. The format for
2388 @env{GOMP_RTEMS_THREAD_POOLS} is a list of optional
2389 @code{<thread-pool-count>[$<priority>]@@<scheduler-name>} configurations
2390 separated by @code{:} where:
2392 @item @code{<thread-pool-count>} is the thread pool count for this scheduler
2394 @item @code{$<priority>} is an optional priority for the worker threads of a
2395 thread pool according to @code{pthread_setschedparam}. In case a priority
2396 value is omitted, then a worker thread will inherit the priority of the OpenMP
2397 primary thread that created it. The priority of the worker thread is not
2398 changed after creation, even if a new OpenMP primary thread using the worker has
2399 a different priority.
2400 @item @code{@@<scheduler-name>} is the scheduler instance name according to the
2401 RTEMS application configuration.
2403 In case no thread pool configuration is specified for a scheduler instance,
2404 then each OpenMP primary thread of this scheduler instance will use its own
2405 dynamically allocated thread pool. To limit the worker thread count of the
2406 thread pools, each OpenMP primary thread must call @code{omp_set_num_threads}.
2407 @item @emph{Example}:
2408 Lets suppose we have three scheduler instances @code{IO}, @code{WRK0}, and
2409 @code{WRK1} with @env{GOMP_RTEMS_THREAD_POOLS} set to
2410 @code{"1@@WRK0:3$4@@WRK1"}. Then there are no thread pool restrictions for
2411 scheduler instance @code{IO}. In the scheduler instance @code{WRK0} there is
2412 one thread pool available. Since no priority is specified for this scheduler
2413 instance, the worker thread inherits the priority of the OpenMP primary thread
2414 that created it. In the scheduler instance @code{WRK1} there are three thread
2415 pools available and their worker threads run at priority four.
2420 @c ---------------------------------------------------------------------
2422 @c ---------------------------------------------------------------------
2424 @node Enabling OpenACC
2425 @chapter Enabling OpenACC
2427 To activate the OpenACC extensions for C/C++ and Fortran, the compile-time
2428 flag @option{-fopenacc} must be specified. This enables the OpenACC directive
2429 @code{#pragma acc} in C/C++ and @code{!$acc} directives in free form,
2430 @code{c$acc}, @code{*$acc} and @code{!$acc} directives in fixed form,
2431 @code{!$} conditional compilation sentinels in free form and @code{c$},
2432 @code{*$} and @code{!$} sentinels in fixed form, for Fortran. The flag also
2433 arranges for automatic linking of the OpenACC runtime library
2434 (@ref{OpenACC Runtime Library Routines}).
2436 See @uref{https://gcc.gnu.org/wiki/OpenACC} for more information.
2438 A complete description of all OpenACC directives accepted may be found in
2439 the @uref{https://www.openacc.org, OpenACC} Application Programming
2440 Interface manual, version 2.6.
2444 @c ---------------------------------------------------------------------
2445 @c OpenACC Runtime Library Routines
2446 @c ---------------------------------------------------------------------
2448 @node OpenACC Runtime Library Routines
2449 @chapter OpenACC Runtime Library Routines
2451 The runtime routines described here are defined by section 3 of the OpenACC
2452 specifications in version 2.6.
2453 They have C linkage, and do not throw exceptions.
2454 Generally, they are available only for the host, with the exception of
2455 @code{acc_on_device}, which is available for both the host and the
2456 acceleration device.
2459 * acc_get_num_devices:: Get number of devices for the given device
2461 * acc_set_device_type:: Set type of device accelerator to use.
2462 * acc_get_device_type:: Get type of device accelerator to be used.
2463 * acc_set_device_num:: Set device number to use.
2464 * acc_get_device_num:: Get device number to be used.
2465 * acc_get_property:: Get device property.
2466 * acc_async_test:: Tests for completion of a specific asynchronous
2468 * acc_async_test_all:: Tests for completion of all asynchronous
2470 * acc_wait:: Wait for completion of a specific asynchronous
2472 * acc_wait_all:: Waits for completion of all asynchronous
2474 * acc_wait_all_async:: Wait for completion of all asynchronous
2476 * acc_wait_async:: Wait for completion of asynchronous operations.
2477 * acc_init:: Initialize runtime for a specific device type.
2478 * acc_shutdown:: Shuts down the runtime for a specific device
2480 * acc_on_device:: Whether executing on a particular device
2481 * acc_malloc:: Allocate device memory.
2482 * acc_free:: Free device memory.
2483 * acc_copyin:: Allocate device memory and copy host memory to
2485 * acc_present_or_copyin:: If the data is not present on the device,
2486 allocate device memory and copy from host
2488 * acc_create:: Allocate device memory and map it to host
2490 * acc_present_or_create:: If the data is not present on the device,
2491 allocate device memory and map it to host
2493 * acc_copyout:: Copy device memory to host memory.
2494 * acc_delete:: Free device memory.
2495 * acc_update_device:: Update device memory from mapped host memory.
2496 * acc_update_self:: Update host memory from mapped device memory.
2497 * acc_map_data:: Map previously allocated device memory to host
2499 * acc_unmap_data:: Unmap device memory from host memory.
2500 * acc_deviceptr:: Get device pointer associated with specific
2502 * acc_hostptr:: Get host pointer associated with specific
2504 * acc_is_present:: Indicate whether host variable / array is
2506 * acc_memcpy_to_device:: Copy host memory to device memory.
2507 * acc_memcpy_from_device:: Copy device memory to host memory.
2508 * acc_attach:: Let device pointer point to device-pointer target.
2509 * acc_detach:: Let device pointer point to host-pointer target.
2511 API routines for target platforms.
2513 * acc_get_current_cuda_device:: Get CUDA device handle.
2514 * acc_get_current_cuda_context::Get CUDA context handle.
2515 * acc_get_cuda_stream:: Get CUDA stream handle.
2516 * acc_set_cuda_stream:: Set CUDA stream handle.
2518 API routines for the OpenACC Profiling Interface.
2520 * acc_prof_register:: Register callbacks.
2521 * acc_prof_unregister:: Unregister callbacks.
2522 * acc_prof_lookup:: Obtain inquiry functions.
2523 * acc_register_library:: Library registration.
2528 @node acc_get_num_devices
2529 @section @code{acc_get_num_devices} -- Get number of devices for given device type
2531 @item @emph{Description}
2532 This function returns a value indicating the number of devices available
2533 for the device type specified in @var{devicetype}.
2536 @multitable @columnfractions .20 .80
2537 @item @emph{Prototype}: @tab @code{int acc_get_num_devices(acc_device_t devicetype);}
2540 @item @emph{Fortran}:
2541 @multitable @columnfractions .20 .80
2542 @item @emph{Interface}: @tab @code{integer function acc_get_num_devices(devicetype)}
2543 @item @tab @code{integer(kind=acc_device_kind) devicetype}
2546 @item @emph{Reference}:
2547 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
2553 @node acc_set_device_type
2554 @section @code{acc_set_device_type} -- Set type of device accelerator to use.
2556 @item @emph{Description}
2557 This function indicates to the runtime library which device type, specified
2558 in @var{devicetype}, to use when executing a parallel or kernels region.
2561 @multitable @columnfractions .20 .80
2562 @item @emph{Prototype}: @tab @code{acc_set_device_type(acc_device_t devicetype);}
2565 @item @emph{Fortran}:
2566 @multitable @columnfractions .20 .80
2567 @item @emph{Interface}: @tab @code{subroutine acc_set_device_type(devicetype)}
2568 @item @tab @code{integer(kind=acc_device_kind) devicetype}
2571 @item @emph{Reference}:
2572 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
2578 @node acc_get_device_type
2579 @section @code{acc_get_device_type} -- Get type of device accelerator to be used.
2581 @item @emph{Description}
2582 This function returns what device type will be used when executing a
2583 parallel or kernels region.
2585 This function returns @code{acc_device_none} if
2586 @code{acc_get_device_type} is called from
2587 @code{acc_ev_device_init_start}, @code{acc_ev_device_init_end}
2588 callbacks of the OpenACC Profiling Interface (@ref{OpenACC Profiling
2589 Interface}), that is, if the device is currently being initialized.
2592 @multitable @columnfractions .20 .80
2593 @item @emph{Prototype}: @tab @code{acc_device_t acc_get_device_type(void);}
2596 @item @emph{Fortran}:
2597 @multitable @columnfractions .20 .80
2598 @item @emph{Interface}: @tab @code{function acc_get_device_type(void)}
2599 @item @tab @code{integer(kind=acc_device_kind) acc_get_device_type}
2602 @item @emph{Reference}:
2603 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
2609 @node acc_set_device_num
2610 @section @code{acc_set_device_num} -- Set device number to use.
2612 @item @emph{Description}
2613 This function will indicate to the runtime which device number,
2614 specified by @var{devicenum}, associated with the specified device
2615 type @var{devicetype}.
2618 @multitable @columnfractions .20 .80
2619 @item @emph{Prototype}: @tab @code{acc_set_device_num(int devicenum, acc_device_t devicetype);}
2622 @item @emph{Fortran}:
2623 @multitable @columnfractions .20 .80
2624 @item @emph{Interface}: @tab @code{subroutine acc_set_device_num(devicenum, devicetype)}
2625 @item @tab @code{integer devicenum}
2626 @item @tab @code{integer(kind=acc_device_kind) devicetype}
2629 @item @emph{Reference}:
2630 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
2636 @node acc_get_device_num
2637 @section @code{acc_get_device_num} -- Get device number to be used.
2639 @item @emph{Description}
2640 This function returns which device number associated with the specified device
2641 type @var{devicetype}, will be used when executing a parallel or kernels
2645 @multitable @columnfractions .20 .80
2646 @item @emph{Prototype}: @tab @code{int acc_get_device_num(acc_device_t devicetype);}
2649 @item @emph{Fortran}:
2650 @multitable @columnfractions .20 .80
2651 @item @emph{Interface}: @tab @code{function acc_get_device_num(devicetype)}
2652 @item @tab @code{integer(kind=acc_device_kind) devicetype}
2653 @item @tab @code{integer acc_get_device_num}
2656 @item @emph{Reference}:
2657 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
2663 @node acc_get_property
2664 @section @code{acc_get_property} -- Get device property.
2665 @cindex acc_get_property
2666 @cindex acc_get_property_string
2668 @item @emph{Description}
2669 These routines return the value of the specified @var{property} for the
2670 device being queried according to @var{devicenum} and @var{devicetype}.
2671 Integer-valued and string-valued properties are returned by
2672 @code{acc_get_property} and @code{acc_get_property_string} respectively.
2673 The Fortran @code{acc_get_property_string} subroutine returns the string
2674 retrieved in its fourth argument while the remaining entry points are
2675 functions, which pass the return value as their result.
2677 Note for Fortran, only: the OpenACC technical committee corrected and, hence,
2678 modified the interface introduced in OpenACC 2.6. The kind-value parameter
2679 @code{acc_device_property} has been renamed to @code{acc_device_property_kind}
2680 for consistency and the return type of the @code{acc_get_property} function is
2681 now a @code{c_size_t} integer instead of a @code{acc_device_property} integer.
2682 The parameter @code{acc_device_property} will continue to be provided,
2683 but might be removed in a future version of GCC.
2686 @multitable @columnfractions .20 .80
2687 @item @emph{Prototype}: @tab @code{size_t acc_get_property(int devicenum, acc_device_t devicetype, acc_device_property_t property);}
2688 @item @emph{Prototype}: @tab @code{const char *acc_get_property_string(int devicenum, acc_device_t devicetype, acc_device_property_t property);}
2691 @item @emph{Fortran}:
2692 @multitable @columnfractions .20 .80
2693 @item @emph{Interface}: @tab @code{function acc_get_property(devicenum, devicetype, property)}
2694 @item @emph{Interface}: @tab @code{subroutine acc_get_property_string(devicenum, devicetype, property, string)}
2695 @item @tab @code{use ISO_C_Binding, only: c_size_t}
2696 @item @tab @code{integer devicenum}
2697 @item @tab @code{integer(kind=acc_device_kind) devicetype}
2698 @item @tab @code{integer(kind=acc_device_property_kind) property}
2699 @item @tab @code{integer(kind=c_size_t) acc_get_property}
2700 @item @tab @code{character(*) string}
2703 @item @emph{Reference}:
2704 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
2710 @node acc_async_test
2711 @section @code{acc_async_test} -- Test for completion of a specific asynchronous operation.
2713 @item @emph{Description}
2714 This function tests for completion of the asynchronous operation specified
2715 in @var{arg}. In C/C++, a non-zero value will be returned to indicate
2716 the specified asynchronous operation has completed. While Fortran will return
2717 a @code{true}. If the asynchronous operation has not completed, C/C++ returns
2718 a zero and Fortran returns a @code{false}.
2721 @multitable @columnfractions .20 .80
2722 @item @emph{Prototype}: @tab @code{int acc_async_test(int arg);}
2725 @item @emph{Fortran}:
2726 @multitable @columnfractions .20 .80
2727 @item @emph{Interface}: @tab @code{function acc_async_test(arg)}
2728 @item @tab @code{integer(kind=acc_handle_kind) arg}
2729 @item @tab @code{logical acc_async_test}
2732 @item @emph{Reference}:
2733 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
2739 @node acc_async_test_all
2740 @section @code{acc_async_test_all} -- Tests for completion of all asynchronous operations.
2742 @item @emph{Description}
2743 This function tests for completion of all asynchronous operations.
2744 In C/C++, a non-zero value will be returned to indicate all asynchronous
2745 operations have completed. While Fortran will return a @code{true}. If
2746 any asynchronous operation has not completed, C/C++ returns a zero and
2747 Fortran returns a @code{false}.
2750 @multitable @columnfractions .20 .80
2751 @item @emph{Prototype}: @tab @code{int acc_async_test_all(void);}
2754 @item @emph{Fortran}:
2755 @multitable @columnfractions .20 .80
2756 @item @emph{Interface}: @tab @code{function acc_async_test()}
2757 @item @tab @code{logical acc_get_device_num}
2760 @item @emph{Reference}:
2761 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
2768 @section @code{acc_wait} -- Wait for completion of a specific asynchronous operation.
2770 @item @emph{Description}
2771 This function waits for completion of the asynchronous operation
2772 specified in @var{arg}.
2775 @multitable @columnfractions .20 .80
2776 @item @emph{Prototype}: @tab @code{acc_wait(arg);}
2777 @item @emph{Prototype (OpenACC 1.0 compatibility)}: @tab @code{acc_async_wait(arg);}
2780 @item @emph{Fortran}:
2781 @multitable @columnfractions .20 .80
2782 @item @emph{Interface}: @tab @code{subroutine acc_wait(arg)}
2783 @item @tab @code{integer(acc_handle_kind) arg}
2784 @item @emph{Interface (OpenACC 1.0 compatibility)}: @tab @code{subroutine acc_async_wait(arg)}
2785 @item @tab @code{integer(acc_handle_kind) arg}
2788 @item @emph{Reference}:
2789 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
2796 @section @code{acc_wait_all} -- Waits for completion of all asynchronous operations.
2798 @item @emph{Description}
2799 This function waits for the completion of all asynchronous operations.
2802 @multitable @columnfractions .20 .80
2803 @item @emph{Prototype}: @tab @code{acc_wait_all(void);}
2804 @item @emph{Prototype (OpenACC 1.0 compatibility)}: @tab @code{acc_async_wait_all(void);}
2807 @item @emph{Fortran}:
2808 @multitable @columnfractions .20 .80
2809 @item @emph{Interface}: @tab @code{subroutine acc_wait_all()}
2810 @item @emph{Interface (OpenACC 1.0 compatibility)}: @tab @code{subroutine acc_async_wait_all()}
2813 @item @emph{Reference}:
2814 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
2820 @node acc_wait_all_async
2821 @section @code{acc_wait_all_async} -- Wait for completion of all asynchronous operations.
2823 @item @emph{Description}
2824 This function enqueues a wait operation on the queue @var{async} for any
2825 and all asynchronous operations that have been previously enqueued on
2829 @multitable @columnfractions .20 .80
2830 @item @emph{Prototype}: @tab @code{acc_wait_all_async(int async);}
2833 @item @emph{Fortran}:
2834 @multitable @columnfractions .20 .80
2835 @item @emph{Interface}: @tab @code{subroutine acc_wait_all_async(async)}
2836 @item @tab @code{integer(acc_handle_kind) async}
2839 @item @emph{Reference}:
2840 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
2846 @node acc_wait_async
2847 @section @code{acc_wait_async} -- Wait for completion of asynchronous operations.
2849 @item @emph{Description}
2850 This function enqueues a wait operation on queue @var{async} for any and all
2851 asynchronous operations enqueued on queue @var{arg}.
2854 @multitable @columnfractions .20 .80
2855 @item @emph{Prototype}: @tab @code{acc_wait_async(int arg, int async);}
2858 @item @emph{Fortran}:
2859 @multitable @columnfractions .20 .80
2860 @item @emph{Interface}: @tab @code{subroutine acc_wait_async(arg, async)}
2861 @item @tab @code{integer(acc_handle_kind) arg, async}
2864 @item @emph{Reference}:
2865 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
2872 @section @code{acc_init} -- Initialize runtime for a specific device type.
2874 @item @emph{Description}
2875 This function initializes the runtime for the device type specified in
2879 @multitable @columnfractions .20 .80
2880 @item @emph{Prototype}: @tab @code{acc_init(acc_device_t devicetype);}
2883 @item @emph{Fortran}:
2884 @multitable @columnfractions .20 .80
2885 @item @emph{Interface}: @tab @code{subroutine acc_init(devicetype)}
2886 @item @tab @code{integer(acc_device_kind) devicetype}
2889 @item @emph{Reference}:
2890 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
2897 @section @code{acc_shutdown} -- Shuts down the runtime for a specific device type.
2899 @item @emph{Description}
2900 This function shuts down the runtime for the device type specified in
2904 @multitable @columnfractions .20 .80
2905 @item @emph{Prototype}: @tab @code{acc_shutdown(acc_device_t devicetype);}
2908 @item @emph{Fortran}:
2909 @multitable @columnfractions .20 .80
2910 @item @emph{Interface}: @tab @code{subroutine acc_shutdown(devicetype)}
2911 @item @tab @code{integer(acc_device_kind) devicetype}
2914 @item @emph{Reference}:
2915 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
2922 @section @code{acc_on_device} -- Whether executing on a particular device
2924 @item @emph{Description}:
2925 This function returns whether the program is executing on a particular
2926 device specified in @var{devicetype}. In C/C++ a non-zero value is
2927 returned to indicate the device is executing on the specified device type.
2928 In Fortran, @code{true} will be returned. If the program is not executing
2929 on the specified device type C/C++ will return a zero, while Fortran will
2930 return @code{false}.
2933 @multitable @columnfractions .20 .80
2934 @item @emph{Prototype}: @tab @code{acc_on_device(acc_device_t devicetype);}
2937 @item @emph{Fortran}:
2938 @multitable @columnfractions .20 .80
2939 @item @emph{Interface}: @tab @code{function acc_on_device(devicetype)}
2940 @item @tab @code{integer(acc_device_kind) devicetype}
2941 @item @tab @code{logical acc_on_device}
2945 @item @emph{Reference}:
2946 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
2953 @section @code{acc_malloc} -- Allocate device memory.
2955 @item @emph{Description}
2956 This function allocates @var{len} bytes of device memory. It returns
2957 the device address of the allocated memory.
2960 @multitable @columnfractions .20 .80
2961 @item @emph{Prototype}: @tab @code{d_void* acc_malloc(size_t len);}
2964 @item @emph{Reference}:
2965 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
2972 @section @code{acc_free} -- Free device memory.
2974 @item @emph{Description}
2975 Free previously allocated device memory at the device address @code{a}.
2978 @multitable @columnfractions .20 .80
2979 @item @emph{Prototype}: @tab @code{acc_free(d_void *a);}
2982 @item @emph{Reference}:
2983 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
2990 @section @code{acc_copyin} -- Allocate device memory and copy host memory to it.
2992 @item @emph{Description}
2993 In C/C++, this function allocates @var{len} bytes of device memory
2994 and maps it to the specified host address in @var{a}. The device
2995 address of the newly allocated device memory is returned.
2997 In Fortran, two (2) forms are supported. In the first form, @var{a} specifies
2998 a contiguous array section. The second form @var{a} specifies a
2999 variable or array element and @var{len} specifies the length in bytes.
3002 @multitable @columnfractions .20 .80
3003 @item @emph{Prototype}: @tab @code{void *acc_copyin(h_void *a, size_t len);}
3004 @item @emph{Prototype}: @tab @code{void *acc_copyin_async(h_void *a, size_t len, int async);}
3007 @item @emph{Fortran}:
3008 @multitable @columnfractions .20 .80
3009 @item @emph{Interface}: @tab @code{subroutine acc_copyin(a)}
3010 @item @tab @code{type, dimension(:[,:]...) :: a}
3011 @item @emph{Interface}: @tab @code{subroutine acc_copyin(a, len)}
3012 @item @tab @code{type, dimension(:[,:]...) :: a}
3013 @item @tab @code{integer len}
3014 @item @emph{Interface}: @tab @code{subroutine acc_copyin_async(a, async)}
3015 @item @tab @code{type, dimension(:[,:]...) :: a}
3016 @item @tab @code{integer(acc_handle_kind) :: async}
3017 @item @emph{Interface}: @tab @code{subroutine acc_copyin_async(a, len, async)}
3018 @item @tab @code{type, dimension(:[,:]...) :: a}
3019 @item @tab @code{integer len}
3020 @item @tab @code{integer(acc_handle_kind) :: async}
3023 @item @emph{Reference}:
3024 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3030 @node acc_present_or_copyin
3031 @section @code{acc_present_or_copyin} -- If the data is not present on the device, allocate device memory and copy from host memory.
3033 @item @emph{Description}
3034 This function tests if the host data specified by @var{a} and of length
3035 @var{len} is present or not. If it is not present, then device memory
3036 will be allocated and the host memory copied. The device address of
3037 the newly allocated device memory is returned.
3039 In Fortran, two (2) forms are supported. In the first form, @var{a} specifies
3040 a contiguous array section. The second form @var{a} specifies a variable or
3041 array element and @var{len} specifies the length in bytes.
3043 Note that @code{acc_present_or_copyin} and @code{acc_pcopyin} exist for
3044 backward compatibility with OpenACC 2.0; use @ref{acc_copyin} instead.
3047 @multitable @columnfractions .20 .80
3048 @item @emph{Prototype}: @tab @code{void *acc_present_or_copyin(h_void *a, size_t len);}
3049 @item @emph{Prototype}: @tab @code{void *acc_pcopyin(h_void *a, size_t len);}
3052 @item @emph{Fortran}:
3053 @multitable @columnfractions .20 .80
3054 @item @emph{Interface}: @tab @code{subroutine acc_present_or_copyin(a)}
3055 @item @tab @code{type, dimension(:[,:]...) :: a}
3056 @item @emph{Interface}: @tab @code{subroutine acc_present_or_copyin(a, len)}
3057 @item @tab @code{type, dimension(:[,:]...) :: a}
3058 @item @tab @code{integer len}
3059 @item @emph{Interface}: @tab @code{subroutine acc_pcopyin(a)}
3060 @item @tab @code{type, dimension(:[,:]...) :: a}
3061 @item @emph{Interface}: @tab @code{subroutine acc_pcopyin(a, len)}
3062 @item @tab @code{type, dimension(:[,:]...) :: a}
3063 @item @tab @code{integer len}
3066 @item @emph{Reference}:
3067 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3074 @section @code{acc_create} -- Allocate device memory and map it to host memory.
3076 @item @emph{Description}
3077 This function allocates device memory and maps it to host memory specified
3078 by the host address @var{a} with a length of @var{len} bytes. In C/C++,
3079 the function returns the device address of the allocated device memory.
3081 In Fortran, two (2) forms are supported. In the first form, @var{a} specifies
3082 a contiguous array section. The second form @var{a} specifies a variable or
3083 array element and @var{len} specifies the length in bytes.
3086 @multitable @columnfractions .20 .80
3087 @item @emph{Prototype}: @tab @code{void *acc_create(h_void *a, size_t len);}
3088 @item @emph{Prototype}: @tab @code{void *acc_create_async(h_void *a, size_t len, int async);}
3091 @item @emph{Fortran}:
3092 @multitable @columnfractions .20 .80
3093 @item @emph{Interface}: @tab @code{subroutine acc_create(a)}
3094 @item @tab @code{type, dimension(:[,:]...) :: a}
3095 @item @emph{Interface}: @tab @code{subroutine acc_create(a, len)}
3096 @item @tab @code{type, dimension(:[,:]...) :: a}
3097 @item @tab @code{integer len}
3098 @item @emph{Interface}: @tab @code{subroutine acc_create_async(a, async)}
3099 @item @tab @code{type, dimension(:[,:]...) :: a}
3100 @item @tab @code{integer(acc_handle_kind) :: async}
3101 @item @emph{Interface}: @tab @code{subroutine acc_create_async(a, len, async)}
3102 @item @tab @code{type, dimension(:[,:]...) :: a}
3103 @item @tab @code{integer len}
3104 @item @tab @code{integer(acc_handle_kind) :: async}
3107 @item @emph{Reference}:
3108 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3114 @node acc_present_or_create
3115 @section @code{acc_present_or_create} -- If the data is not present on the device, allocate device memory and map it to host memory.
3117 @item @emph{Description}
3118 This function tests if the host data specified by @var{a} and of length
3119 @var{len} is present or not. If it is not present, then device memory
3120 will be allocated and mapped to host memory. In C/C++, the device address
3121 of the newly allocated device memory is returned.
3123 In Fortran, two (2) forms are supported. In the first form, @var{a} specifies
3124 a contiguous array section. The second form @var{a} specifies a variable or
3125 array element and @var{len} specifies the length in bytes.
3127 Note that @code{acc_present_or_create} and @code{acc_pcreate} exist for
3128 backward compatibility with OpenACC 2.0; use @ref{acc_create} instead.
3131 @multitable @columnfractions .20 .80
3132 @item @emph{Prototype}: @tab @code{void *acc_present_or_create(h_void *a, size_t len)}
3133 @item @emph{Prototype}: @tab @code{void *acc_pcreate(h_void *a, size_t len)}
3136 @item @emph{Fortran}:
3137 @multitable @columnfractions .20 .80
3138 @item @emph{Interface}: @tab @code{subroutine acc_present_or_create(a)}
3139 @item @tab @code{type, dimension(:[,:]...) :: a}
3140 @item @emph{Interface}: @tab @code{subroutine acc_present_or_create(a, len)}
3141 @item @tab @code{type, dimension(:[,:]...) :: a}
3142 @item @tab @code{integer len}
3143 @item @emph{Interface}: @tab @code{subroutine acc_pcreate(a)}
3144 @item @tab @code{type, dimension(:[,:]...) :: a}
3145 @item @emph{Interface}: @tab @code{subroutine acc_pcreate(a, len)}
3146 @item @tab @code{type, dimension(:[,:]...) :: a}
3147 @item @tab @code{integer len}
3150 @item @emph{Reference}:
3151 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3158 @section @code{acc_copyout} -- Copy device memory to host memory.
3160 @item @emph{Description}
3161 This function copies mapped device memory to host memory which is specified
3162 by host address @var{a} for a length @var{len} bytes in C/C++.
3164 In Fortran, two (2) forms are supported. In the first form, @var{a} specifies
3165 a contiguous array section. The second form @var{a} specifies a variable or
3166 array element and @var{len} specifies the length in bytes.
3169 @multitable @columnfractions .20 .80
3170 @item @emph{Prototype}: @tab @code{acc_copyout(h_void *a, size_t len);}
3171 @item @emph{Prototype}: @tab @code{acc_copyout_async(h_void *a, size_t len, int async);}
3172 @item @emph{Prototype}: @tab @code{acc_copyout_finalize(h_void *a, size_t len);}
3173 @item @emph{Prototype}: @tab @code{acc_copyout_finalize_async(h_void *a, size_t len, int async);}
3176 @item @emph{Fortran}:
3177 @multitable @columnfractions .20 .80
3178 @item @emph{Interface}: @tab @code{subroutine acc_copyout(a)}
3179 @item @tab @code{type, dimension(:[,:]...) :: a}
3180 @item @emph{Interface}: @tab @code{subroutine acc_copyout(a, len)}
3181 @item @tab @code{type, dimension(:[,:]...) :: a}
3182 @item @tab @code{integer len}
3183 @item @emph{Interface}: @tab @code{subroutine acc_copyout_async(a, async)}
3184 @item @tab @code{type, dimension(:[,:]...) :: a}
3185 @item @tab @code{integer(acc_handle_kind) :: async}
3186 @item @emph{Interface}: @tab @code{subroutine acc_copyout_async(a, len, async)}
3187 @item @tab @code{type, dimension(:[,:]...) :: a}
3188 @item @tab @code{integer len}
3189 @item @tab @code{integer(acc_handle_kind) :: async}
3190 @item @emph{Interface}: @tab @code{subroutine acc_copyout_finalize(a)}
3191 @item @tab @code{type, dimension(:[,:]...) :: a}
3192 @item @emph{Interface}: @tab @code{subroutine acc_copyout_finalize(a, len)}
3193 @item @tab @code{type, dimension(:[,:]...) :: a}
3194 @item @tab @code{integer len}
3195 @item @emph{Interface}: @tab @code{subroutine acc_copyout_finalize_async(a, async)}
3196 @item @tab @code{type, dimension(:[,:]...) :: a}
3197 @item @tab @code{integer(acc_handle_kind) :: async}
3198 @item @emph{Interface}: @tab @code{subroutine acc_copyout_finalize_async(a, len, async)}
3199 @item @tab @code{type, dimension(:[,:]...) :: a}
3200 @item @tab @code{integer len}
3201 @item @tab @code{integer(acc_handle_kind) :: async}
3204 @item @emph{Reference}:
3205 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3212 @section @code{acc_delete} -- Free device memory.
3214 @item @emph{Description}
3215 This function frees previously allocated device memory specified by
3216 the device address @var{a} and the length of @var{len} bytes.
3218 In Fortran, two (2) forms are supported. In the first form, @var{a} specifies
3219 a contiguous array section. The second form @var{a} specifies a variable or
3220 array element and @var{len} specifies the length in bytes.
3223 @multitable @columnfractions .20 .80
3224 @item @emph{Prototype}: @tab @code{acc_delete(h_void *a, size_t len);}
3225 @item @emph{Prototype}: @tab @code{acc_delete_async(h_void *a, size_t len, int async);}
3226 @item @emph{Prototype}: @tab @code{acc_delete_finalize(h_void *a, size_t len);}
3227 @item @emph{Prototype}: @tab @code{acc_delete_finalize_async(h_void *a, size_t len, int async);}
3230 @item @emph{Fortran}:
3231 @multitable @columnfractions .20 .80
3232 @item @emph{Interface}: @tab @code{subroutine acc_delete(a)}
3233 @item @tab @code{type, dimension(:[,:]...) :: a}
3234 @item @emph{Interface}: @tab @code{subroutine acc_delete(a, len)}
3235 @item @tab @code{type, dimension(:[,:]...) :: a}
3236 @item @tab @code{integer len}
3237 @item @emph{Interface}: @tab @code{subroutine acc_delete_async(a, async)}
3238 @item @tab @code{type, dimension(:[,:]...) :: a}
3239 @item @tab @code{integer(acc_handle_kind) :: async}
3240 @item @emph{Interface}: @tab @code{subroutine acc_delete_async(a, len, async)}
3241 @item @tab @code{type, dimension(:[,:]...) :: a}
3242 @item @tab @code{integer len}
3243 @item @tab @code{integer(acc_handle_kind) :: async}
3244 @item @emph{Interface}: @tab @code{subroutine acc_delete_finalize(a)}
3245 @item @tab @code{type, dimension(:[,:]...) :: a}
3246 @item @emph{Interface}: @tab @code{subroutine acc_delete_finalize(a, len)}
3247 @item @tab @code{type, dimension(:[,:]...) :: a}
3248 @item @tab @code{integer len}
3249 @item @emph{Interface}: @tab @code{subroutine acc_delete_async_finalize(a, async)}
3250 @item @tab @code{type, dimension(:[,:]...) :: a}
3251 @item @tab @code{integer(acc_handle_kind) :: async}
3252 @item @emph{Interface}: @tab @code{subroutine acc_delete_async_finalize(a, len, async)}
3253 @item @tab @code{type, dimension(:[,:]...) :: a}
3254 @item @tab @code{integer len}
3255 @item @tab @code{integer(acc_handle_kind) :: async}
3258 @item @emph{Reference}:
3259 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3265 @node acc_update_device
3266 @section @code{acc_update_device} -- Update device memory from mapped host memory.
3268 @item @emph{Description}
3269 This function updates the device copy from the previously mapped host memory.
3270 The host memory is specified with the host address @var{a} and a length of
3273 In Fortran, two (2) forms are supported. In the first form, @var{a} specifies
3274 a contiguous array section. The second form @var{a} specifies a variable or
3275 array element and @var{len} specifies the length in bytes.
3278 @multitable @columnfractions .20 .80
3279 @item @emph{Prototype}: @tab @code{acc_update_device(h_void *a, size_t len);}
3280 @item @emph{Prototype}: @tab @code{acc_update_device(h_void *a, size_t len, async);}
3283 @item @emph{Fortran}:
3284 @multitable @columnfractions .20 .80
3285 @item @emph{Interface}: @tab @code{subroutine acc_update_device(a)}
3286 @item @tab @code{type, dimension(:[,:]...) :: a}
3287 @item @emph{Interface}: @tab @code{subroutine acc_update_device(a, len)}
3288 @item @tab @code{type, dimension(:[,:]...) :: a}
3289 @item @tab @code{integer len}
3290 @item @emph{Interface}: @tab @code{subroutine acc_update_device_async(a, async)}
3291 @item @tab @code{type, dimension(:[,:]...) :: a}
3292 @item @tab @code{integer(acc_handle_kind) :: async}
3293 @item @emph{Interface}: @tab @code{subroutine acc_update_device_async(a, len, async)}
3294 @item @tab @code{type, dimension(:[,:]...) :: a}
3295 @item @tab @code{integer len}
3296 @item @tab @code{integer(acc_handle_kind) :: async}
3299 @item @emph{Reference}:
3300 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3306 @node acc_update_self
3307 @section @code{acc_update_self} -- Update host memory from mapped device memory.
3309 @item @emph{Description}
3310 This function updates the host copy from the previously mapped device memory.
3311 The host memory is specified with the host address @var{a} and a length of
3314 In Fortran, two (2) forms are supported. In the first form, @var{a} specifies
3315 a contiguous array section. The second form @var{a} specifies a variable or
3316 array element and @var{len} specifies the length in bytes.
3319 @multitable @columnfractions .20 .80
3320 @item @emph{Prototype}: @tab @code{acc_update_self(h_void *a, size_t len);}
3321 @item @emph{Prototype}: @tab @code{acc_update_self_async(h_void *a, size_t len, int async);}
3324 @item @emph{Fortran}:
3325 @multitable @columnfractions .20 .80
3326 @item @emph{Interface}: @tab @code{subroutine acc_update_self(a)}
3327 @item @tab @code{type, dimension(:[,:]...) :: a}
3328 @item @emph{Interface}: @tab @code{subroutine acc_update_self(a, len)}
3329 @item @tab @code{type, dimension(:[,:]...) :: a}
3330 @item @tab @code{integer len}
3331 @item @emph{Interface}: @tab @code{subroutine acc_update_self_async(a, async)}
3332 @item @tab @code{type, dimension(:[,:]...) :: a}
3333 @item @tab @code{integer(acc_handle_kind) :: async}
3334 @item @emph{Interface}: @tab @code{subroutine acc_update_self_async(a, len, async)}
3335 @item @tab @code{type, dimension(:[,:]...) :: a}
3336 @item @tab @code{integer len}
3337 @item @tab @code{integer(acc_handle_kind) :: async}
3340 @item @emph{Reference}:
3341 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3348 @section @code{acc_map_data} -- Map previously allocated device memory to host memory.
3350 @item @emph{Description}
3351 This function maps previously allocated device and host memory. The device
3352 memory is specified with the device address @var{d}. The host memory is
3353 specified with the host address @var{h} and a length of @var{len}.
3356 @multitable @columnfractions .20 .80
3357 @item @emph{Prototype}: @tab @code{acc_map_data(h_void *h, d_void *d, size_t len);}
3360 @item @emph{Reference}:
3361 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3367 @node acc_unmap_data
3368 @section @code{acc_unmap_data} -- Unmap device memory from host memory.
3370 @item @emph{Description}
3371 This function unmaps previously mapped device and host memory. The latter
3372 specified by @var{h}.
3375 @multitable @columnfractions .20 .80
3376 @item @emph{Prototype}: @tab @code{acc_unmap_data(h_void *h);}
3379 @item @emph{Reference}:
3380 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3387 @section @code{acc_deviceptr} -- Get device pointer associated with specific host address.
3389 @item @emph{Description}
3390 This function returns the device address that has been mapped to the
3391 host address specified by @var{h}.
3394 @multitable @columnfractions .20 .80
3395 @item @emph{Prototype}: @tab @code{void *acc_deviceptr(h_void *h);}
3398 @item @emph{Reference}:
3399 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3406 @section @code{acc_hostptr} -- Get host pointer associated with specific device address.
3408 @item @emph{Description}
3409 This function returns the host address that has been mapped to the
3410 device address specified by @var{d}.
3413 @multitable @columnfractions .20 .80
3414 @item @emph{Prototype}: @tab @code{void *acc_hostptr(d_void *d);}
3417 @item @emph{Reference}:
3418 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3424 @node acc_is_present
3425 @section @code{acc_is_present} -- Indicate whether host variable / array is present on device.
3427 @item @emph{Description}
3428 This function indicates whether the specified host address in @var{a} and a
3429 length of @var{len} bytes is present on the device. In C/C++, a non-zero
3430 value is returned to indicate the presence of the mapped memory on the
3431 device. A zero is returned to indicate the memory is not mapped on the
3434 In Fortran, two (2) forms are supported. In the first form, @var{a} specifies
3435 a contiguous array section. The second form @var{a} specifies a variable or
3436 array element and @var{len} specifies the length in bytes. If the host
3437 memory is mapped to device memory, then a @code{true} is returned. Otherwise,
3438 a @code{false} is return to indicate the mapped memory is not present.
3441 @multitable @columnfractions .20 .80
3442 @item @emph{Prototype}: @tab @code{int acc_is_present(h_void *a, size_t len);}
3445 @item @emph{Fortran}:
3446 @multitable @columnfractions .20 .80
3447 @item @emph{Interface}: @tab @code{function acc_is_present(a)}
3448 @item @tab @code{type, dimension(:[,:]...) :: a}
3449 @item @tab @code{logical acc_is_present}
3450 @item @emph{Interface}: @tab @code{function acc_is_present(a, len)}
3451 @item @tab @code{type, dimension(:[,:]...) :: a}
3452 @item @tab @code{integer len}
3453 @item @tab @code{logical acc_is_present}
3456 @item @emph{Reference}:
3457 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3463 @node acc_memcpy_to_device
3464 @section @code{acc_memcpy_to_device} -- Copy host memory to device memory.
3466 @item @emph{Description}
3467 This function copies host memory specified by host address of @var{src} to
3468 device memory specified by the device address @var{dest} for a length of
3472 @multitable @columnfractions .20 .80
3473 @item @emph{Prototype}: @tab @code{acc_memcpy_to_device(d_void *dest, h_void *src, size_t bytes);}
3476 @item @emph{Reference}:
3477 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3483 @node acc_memcpy_from_device
3484 @section @code{acc_memcpy_from_device} -- Copy device memory to host memory.
3486 @item @emph{Description}
3487 This function copies host memory specified by host address of @var{src} from
3488 device memory specified by the device address @var{dest} for a length of
3492 @multitable @columnfractions .20 .80
3493 @item @emph{Prototype}: @tab @code{acc_memcpy_from_device(d_void *dest, h_void *src, size_t bytes);}
3496 @item @emph{Reference}:
3497 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3504 @section @code{acc_attach} -- Let device pointer point to device-pointer target.
3506 @item @emph{Description}
3507 This function updates a pointer on the device from pointing to a host-pointer
3508 address to pointing to the corresponding device data.
3511 @multitable @columnfractions .20 .80
3512 @item @emph{Prototype}: @tab @code{acc_attach(h_void **ptr);}
3513 @item @emph{Prototype}: @tab @code{acc_attach_async(h_void **ptr, int async);}
3516 @item @emph{Reference}:
3517 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3524 @section @code{acc_detach} -- Let device pointer point to host-pointer target.
3526 @item @emph{Description}
3527 This function updates a pointer on the device from pointing to a device-pointer
3528 address to pointing to the corresponding host data.
3531 @multitable @columnfractions .20 .80
3532 @item @emph{Prototype}: @tab @code{acc_detach(h_void **ptr);}
3533 @item @emph{Prototype}: @tab @code{acc_detach_async(h_void **ptr, int async);}
3534 @item @emph{Prototype}: @tab @code{acc_detach_finalize(h_void **ptr);}
3535 @item @emph{Prototype}: @tab @code{acc_detach_finalize_async(h_void **ptr, int async);}
3538 @item @emph{Reference}:
3539 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3545 @node acc_get_current_cuda_device
3546 @section @code{acc_get_current_cuda_device} -- Get CUDA device handle.
3548 @item @emph{Description}
3549 This function returns the CUDA device handle. This handle is the same
3550 as used by the CUDA Runtime or Driver API's.
3553 @multitable @columnfractions .20 .80
3554 @item @emph{Prototype}: @tab @code{void *acc_get_current_cuda_device(void);}
3557 @item @emph{Reference}:
3558 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3564 @node acc_get_current_cuda_context
3565 @section @code{acc_get_current_cuda_context} -- Get CUDA context handle.
3567 @item @emph{Description}
3568 This function returns the CUDA context handle. This handle is the same
3569 as used by the CUDA Runtime or Driver API's.
3572 @multitable @columnfractions .20 .80
3573 @item @emph{Prototype}: @tab @code{void *acc_get_current_cuda_context(void);}
3576 @item @emph{Reference}:
3577 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3583 @node acc_get_cuda_stream
3584 @section @code{acc_get_cuda_stream} -- Get CUDA stream handle.
3586 @item @emph{Description}
3587 This function returns the CUDA stream handle for the queue @var{async}.
3588 This handle is the same as used by the CUDA Runtime or Driver API's.
3591 @multitable @columnfractions .20 .80
3592 @item @emph{Prototype}: @tab @code{void *acc_get_cuda_stream(int async);}
3595 @item @emph{Reference}:
3596 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3602 @node acc_set_cuda_stream
3603 @section @code{acc_set_cuda_stream} -- Set CUDA stream handle.
3605 @item @emph{Description}
3606 This function associates the stream handle specified by @var{stream} with
3607 the queue @var{async}.
3609 This cannot be used to change the stream handle associated with
3610 @code{acc_async_sync}.
3612 The return value is not specified.
3615 @multitable @columnfractions .20 .80
3616 @item @emph{Prototype}: @tab @code{int acc_set_cuda_stream(int async, void *stream);}
3619 @item @emph{Reference}:
3620 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3626 @node acc_prof_register
3627 @section @code{acc_prof_register} -- Register callbacks.
3629 @item @emph{Description}:
3630 This function registers callbacks.
3633 @multitable @columnfractions .20 .80
3634 @item @emph{Prototype}: @tab @code{void acc_prof_register (acc_event_t, acc_prof_callback, acc_register_t);}
3637 @item @emph{See also}:
3638 @ref{OpenACC Profiling Interface}
3640 @item @emph{Reference}:
3641 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3647 @node acc_prof_unregister
3648 @section @code{acc_prof_unregister} -- Unregister callbacks.
3650 @item @emph{Description}:
3651 This function unregisters callbacks.
3654 @multitable @columnfractions .20 .80
3655 @item @emph{Prototype}: @tab @code{void acc_prof_unregister (acc_event_t, acc_prof_callback, acc_register_t);}
3658 @item @emph{See also}:
3659 @ref{OpenACC Profiling Interface}
3661 @item @emph{Reference}:
3662 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3668 @node acc_prof_lookup
3669 @section @code{acc_prof_lookup} -- Obtain inquiry functions.
3671 @item @emph{Description}:
3672 Function to obtain inquiry functions.
3675 @multitable @columnfractions .20 .80
3676 @item @emph{Prototype}: @tab @code{acc_query_fn acc_prof_lookup (const char *);}
3679 @item @emph{See also}:
3680 @ref{OpenACC Profiling Interface}
3682 @item @emph{Reference}:
3683 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3689 @node acc_register_library
3690 @section @code{acc_register_library} -- Library registration.
3692 @item @emph{Description}:
3693 Function for library registration.
3696 @multitable @columnfractions .20 .80
3697 @item @emph{Prototype}: @tab @code{void acc_register_library (acc_prof_reg, acc_prof_reg, acc_prof_lookup_func);}
3700 @item @emph{See also}:
3701 @ref{OpenACC Profiling Interface}, @ref{ACC_PROFLIB}
3703 @item @emph{Reference}:
3704 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3710 @c ---------------------------------------------------------------------
3711 @c OpenACC Environment Variables
3712 @c ---------------------------------------------------------------------
3714 @node OpenACC Environment Variables
3715 @chapter OpenACC Environment Variables
3717 The variables @env{ACC_DEVICE_TYPE} and @env{ACC_DEVICE_NUM}
3718 are defined by section 4 of the OpenACC specification in version 2.0.
3719 The variable @env{ACC_PROFLIB}
3720 is defined by section 4 of the OpenACC specification in version 2.6.
3721 The variable @env{GCC_ACC_NOTIFY} is used for diagnostic purposes.
3732 @node ACC_DEVICE_TYPE
3733 @section @code{ACC_DEVICE_TYPE}
3735 @item @emph{Reference}:
3736 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3742 @node ACC_DEVICE_NUM
3743 @section @code{ACC_DEVICE_NUM}
3745 @item @emph{Reference}:
3746 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3753 @section @code{ACC_PROFLIB}
3755 @item @emph{See also}:
3756 @ref{acc_register_library}, @ref{OpenACC Profiling Interface}
3758 @item @emph{Reference}:
3759 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3765 @node GCC_ACC_NOTIFY
3766 @section @code{GCC_ACC_NOTIFY}
3768 @item @emph{Description}:
3769 Print debug information pertaining to the accelerator.
3774 @c ---------------------------------------------------------------------
3775 @c CUDA Streams Usage
3776 @c ---------------------------------------------------------------------
3778 @node CUDA Streams Usage
3779 @chapter CUDA Streams Usage
3781 This applies to the @code{nvptx} plugin only.
3783 The library provides elements that perform asynchronous movement of
3784 data and asynchronous operation of computing constructs. This
3785 asynchronous functionality is implemented by making use of CUDA
3786 streams@footnote{See "Stream Management" in "CUDA Driver API",
3787 TRM-06703-001, Version 5.5, for additional information}.
3789 The primary means by that the asynchronous functionality is accessed
3790 is through the use of those OpenACC directives which make use of the
3791 @code{async} and @code{wait} clauses. When the @code{async} clause is
3792 first used with a directive, it creates a CUDA stream. If an
3793 @code{async-argument} is used with the @code{async} clause, then the
3794 stream is associated with the specified @code{async-argument}.
3796 Following the creation of an association between a CUDA stream and the
3797 @code{async-argument} of an @code{async} clause, both the @code{wait}
3798 clause and the @code{wait} directive can be used. When either the
3799 clause or directive is used after stream creation, it creates a
3800 rendezvous point whereby execution waits until all operations
3801 associated with the @code{async-argument}, that is, stream, have
3804 Normally, the management of the streams that are created as a result of
3805 using the @code{async} clause, is done without any intervention by the
3806 caller. This implies the association between the @code{async-argument}
3807 and the CUDA stream will be maintained for the lifetime of the program.
3808 However, this association can be changed through the use of the library
3809 function @code{acc_set_cuda_stream}. When the function
3810 @code{acc_set_cuda_stream} is called, the CUDA stream that was
3811 originally associated with the @code{async} clause will be destroyed.
3812 Caution should be taken when changing the association as subsequent
3813 references to the @code{async-argument} refer to a different
3818 @c ---------------------------------------------------------------------
3819 @c OpenACC Library Interoperability
3820 @c ---------------------------------------------------------------------
3822 @node OpenACC Library Interoperability
3823 @chapter OpenACC Library Interoperability
3825 @section Introduction
3827 The OpenACC library uses the CUDA Driver API, and may interact with
3828 programs that use the Runtime library directly, or another library
3829 based on the Runtime library, e.g., CUBLAS@footnote{See section 2.26,
3830 "Interactions with the CUDA Driver API" in
3831 "CUDA Runtime API", Version 5.5, and section 2.27, "VDPAU
3832 Interoperability", in "CUDA Driver API", TRM-06703-001, Version 5.5,
3833 for additional information on library interoperability.}.
3834 This chapter describes the use cases and what changes are
3835 required in order to use both the OpenACC library and the CUBLAS and Runtime
3836 libraries within a program.
3838 @section First invocation: NVIDIA CUBLAS library API
3840 In this first use case (see below), a function in the CUBLAS library is called
3841 prior to any of the functions in the OpenACC library. More specifically, the
3842 function @code{cublasCreate()}.
3844 When invoked, the function initializes the library and allocates the
3845 hardware resources on the host and the device on behalf of the caller. Once
3846 the initialization and allocation has completed, a handle is returned to the
3847 caller. The OpenACC library also requires initialization and allocation of
3848 hardware resources. Since the CUBLAS library has already allocated the
3849 hardware resources for the device, all that is left to do is to initialize
3850 the OpenACC library and acquire the hardware resources on the host.
3852 Prior to calling the OpenACC function that initializes the library and
3853 allocate the host hardware resources, you need to acquire the device number
3854 that was allocated during the call to @code{cublasCreate()}. The invoking of the
3855 runtime library function @code{cudaGetDevice()} accomplishes this. Once
3856 acquired, the device number is passed along with the device type as
3857 parameters to the OpenACC library function @code{acc_set_device_num()}.
3859 Once the call to @code{acc_set_device_num()} has completed, the OpenACC
3860 library uses the context that was created during the call to
3861 @code{cublasCreate()}. In other words, both libraries will be sharing the
3865 /* Create the handle */
3866 s = cublasCreate(&h);
3867 if (s != CUBLAS_STATUS_SUCCESS)
3869 fprintf(stderr, "cublasCreate failed %d\n", s);
3873 /* Get the device number */
3874 e = cudaGetDevice(&dev);
3875 if (e != cudaSuccess)
3877 fprintf(stderr, "cudaGetDevice failed %d\n", e);
3881 /* Initialize OpenACC library and use device 'dev' */
3882 acc_set_device_num(dev, acc_device_nvidia);
3887 @section First invocation: OpenACC library API
3889 In this second use case (see below), a function in the OpenACC library is
3890 called prior to any of the functions in the CUBLAS library. More specificially,
3891 the function @code{acc_set_device_num()}.
3893 In the use case presented here, the function @code{acc_set_device_num()}
3894 is used to both initialize the OpenACC library and allocate the hardware
3895 resources on the host and the device. In the call to the function, the
3896 call parameters specify which device to use and what device
3897 type to use, i.e., @code{acc_device_nvidia}. It should be noted that this
3898 is but one method to initialize the OpenACC library and allocate the
3899 appropriate hardware resources. Other methods are available through the
3900 use of environment variables and these will be discussed in the next section.
3902 Once the call to @code{acc_set_device_num()} has completed, other OpenACC
3903 functions can be called as seen with multiple calls being made to
3904 @code{acc_copyin()}. In addition, calls can be made to functions in the
3905 CUBLAS library. In the use case a call to @code{cublasCreate()} is made
3906 subsequent to the calls to @code{acc_copyin()}.
3907 As seen in the previous use case, a call to @code{cublasCreate()}
3908 initializes the CUBLAS library and allocates the hardware resources on the
3909 host and the device. However, since the device has already been allocated,
3910 @code{cublasCreate()} will only initialize the CUBLAS library and allocate
3911 the appropriate hardware resources on the host. The context that was created
3912 as part of the OpenACC initialization is shared with the CUBLAS library,
3913 similarly to the first use case.
3918 acc_set_device_num(dev, acc_device_nvidia);
3920 /* Copy the first set to the device */
3921 d_X = acc_copyin(&h_X[0], N * sizeof (float));
3924 fprintf(stderr, "copyin error h_X\n");
3928 /* Copy the second set to the device */
3929 d_Y = acc_copyin(&h_Y1[0], N * sizeof (float));
3932 fprintf(stderr, "copyin error h_Y1\n");
3936 /* Create the handle */
3937 s = cublasCreate(&h);
3938 if (s != CUBLAS_STATUS_SUCCESS)
3940 fprintf(stderr, "cublasCreate failed %d\n", s);
3944 /* Perform saxpy using CUBLAS library function */
3945 s = cublasSaxpy(h, N, &alpha, d_X, 1, d_Y, 1);
3946 if (s != CUBLAS_STATUS_SUCCESS)
3948 fprintf(stderr, "cublasSaxpy failed %d\n", s);
3952 /* Copy the results from the device */
3953 acc_memcpy_from_device(&h_Y1[0], d_Y, N * sizeof (float));
3958 @section OpenACC library and environment variables
3960 There are two environment variables associated with the OpenACC library
3961 that may be used to control the device type and device number:
3962 @env{ACC_DEVICE_TYPE} and @env{ACC_DEVICE_NUM}, respectively. These two
3963 environment variables can be used as an alternative to calling
3964 @code{acc_set_device_num()}. As seen in the second use case, the device
3965 type and device number were specified using @code{acc_set_device_num()}.
3966 If however, the aforementioned environment variables were set, then the
3967 call to @code{acc_set_device_num()} would not be required.
3970 The use of the environment variables is only relevant when an OpenACC function
3971 is called prior to a call to @code{cudaCreate()}. If @code{cudaCreate()}
3972 is called prior to a call to an OpenACC function, then you must call
3973 @code{acc_set_device_num()}@footnote{More complete information
3974 about @env{ACC_DEVICE_TYPE} and @env{ACC_DEVICE_NUM} can be found in
3975 sections 4.1 and 4.2 of the @uref{https://www.openacc.org, OpenACC}
3976 Application Programming Interface”, Version 2.6.}
3980 @c ---------------------------------------------------------------------
3981 @c OpenACC Profiling Interface
3982 @c ---------------------------------------------------------------------
3984 @node OpenACC Profiling Interface
3985 @chapter OpenACC Profiling Interface
3987 @section Implementation Status and Implementation-Defined Behavior
3989 We're implementing the OpenACC Profiling Interface as defined by the
3990 OpenACC 2.6 specification. We're clarifying some aspects here as
3991 @emph{implementation-defined behavior}, while they're still under
3992 discussion within the OpenACC Technical Committee.
3994 This implementation is tuned to keep the performance impact as low as
3995 possible for the (very common) case that the Profiling Interface is
3996 not enabled. This is relevant, as the Profiling Interface affects all
3997 the @emph{hot} code paths (in the target code, not in the offloaded
3998 code). Users of the OpenACC Profiling Interface can be expected to
3999 understand that performance will be impacted to some degree once the
4000 Profiling Interface has gotten enabled: for example, because of the
4001 @emph{runtime} (libgomp) calling into a third-party @emph{library} for
4002 every event that has been registered.
4004 We're not yet accounting for the fact that @cite{OpenACC events may
4005 occur during event processing}.
4006 We just handle one case specially, as required by CUDA 9.0
4007 @command{nvprof}, that @code{acc_get_device_type}
4008 (@ref{acc_get_device_type})) may be called from
4009 @code{acc_ev_device_init_start}, @code{acc_ev_device_init_end}
4012 We're not yet implementing initialization via a
4013 @code{acc_register_library} function that is either statically linked
4014 in, or dynamically via @env{LD_PRELOAD}.
4015 Initialization via @code{acc_register_library} functions dynamically
4016 loaded via the @env{ACC_PROFLIB} environment variable does work, as
4017 does directly calling @code{acc_prof_register},
4018 @code{acc_prof_unregister}, @code{acc_prof_lookup}.
4020 As currently there are no inquiry functions defined, calls to
4021 @code{acc_prof_lookup} will always return @code{NULL}.
4023 There aren't separate @emph{start}, @emph{stop} events defined for the
4024 event types @code{acc_ev_create}, @code{acc_ev_delete},
4025 @code{acc_ev_alloc}, @code{acc_ev_free}. It's not clear if these
4026 should be triggered before or after the actual device-specific call is
4027 made. We trigger them after.
4029 Remarks about data provided to callbacks:
4033 @item @code{acc_prof_info.event_type}
4034 It's not clear if for @emph{nested} event callbacks (for example,
4035 @code{acc_ev_enqueue_launch_start} as part of a parent compute
4036 construct), this should be set for the nested event
4037 (@code{acc_ev_enqueue_launch_start}), or if the value of the parent
4038 construct should remain (@code{acc_ev_compute_construct_start}). In
4039 this implementation, the value will generally correspond to the
4040 innermost nested event type.
4042 @item @code{acc_prof_info.device_type}
4046 For @code{acc_ev_compute_construct_start}, and in presence of an
4047 @code{if} clause with @emph{false} argument, this will still refer to
4048 the offloading device type.
4049 It's not clear if that's the expected behavior.
4052 Complementary to the item before, for
4053 @code{acc_ev_compute_construct_end}, this is set to
4054 @code{acc_device_host} in presence of an @code{if} clause with
4055 @emph{false} argument.
4056 It's not clear if that's the expected behavior.
4060 @item @code{acc_prof_info.thread_id}
4061 Always @code{-1}; not yet implemented.
4063 @item @code{acc_prof_info.async}
4067 Not yet implemented correctly for
4068 @code{acc_ev_compute_construct_start}.
4071 In a compute construct, for host-fallback
4072 execution/@code{acc_device_host} it will always be
4073 @code{acc_async_sync}.
4074 It's not clear if that's the expected behavior.
4077 For @code{acc_ev_device_init_start} and @code{acc_ev_device_init_end},
4078 it will always be @code{acc_async_sync}.
4079 It's not clear if that's the expected behavior.
4083 @item @code{acc_prof_info.async_queue}
4084 There is no @cite{limited number of asynchronous queues} in libgomp.
4085 This will always have the same value as @code{acc_prof_info.async}.
4087 @item @code{acc_prof_info.src_file}
4088 Always @code{NULL}; not yet implemented.
4090 @item @code{acc_prof_info.func_name}
4091 Always @code{NULL}; not yet implemented.
4093 @item @code{acc_prof_info.line_no}
4094 Always @code{-1}; not yet implemented.
4096 @item @code{acc_prof_info.end_line_no}
4097 Always @code{-1}; not yet implemented.
4099 @item @code{acc_prof_info.func_line_no}
4100 Always @code{-1}; not yet implemented.
4102 @item @code{acc_prof_info.func_end_line_no}
4103 Always @code{-1}; not yet implemented.
4105 @item @code{acc_event_info.event_type}, @code{acc_event_info.*.event_type}
4106 Relating to @code{acc_prof_info.event_type} discussed above, in this
4107 implementation, this will always be the same value as
4108 @code{acc_prof_info.event_type}.
4110 @item @code{acc_event_info.*.parent_construct}
4114 Will be @code{acc_construct_parallel} for all OpenACC compute
4115 constructs as well as many OpenACC Runtime API calls; should be the
4116 one matching the actual construct, or
4117 @code{acc_construct_runtime_api}, respectively.
4120 Will be @code{acc_construct_enter_data} or
4121 @code{acc_construct_exit_data} when processing variable mappings
4122 specified in OpenACC @emph{declare} directives; should be
4123 @code{acc_construct_declare}.
4126 For implicit @code{acc_ev_device_init_start},
4127 @code{acc_ev_device_init_end}, and explicit as well as implicit
4128 @code{acc_ev_alloc}, @code{acc_ev_free},
4129 @code{acc_ev_enqueue_upload_start}, @code{acc_ev_enqueue_upload_end},
4130 @code{acc_ev_enqueue_download_start}, and
4131 @code{acc_ev_enqueue_download_end}, will be
4132 @code{acc_construct_parallel}; should reflect the real parent
4137 @item @code{acc_event_info.*.implicit}
4138 For @code{acc_ev_alloc}, @code{acc_ev_free},
4139 @code{acc_ev_enqueue_upload_start}, @code{acc_ev_enqueue_upload_end},
4140 @code{acc_ev_enqueue_download_start}, and
4141 @code{acc_ev_enqueue_download_end}, this currently will be @code{1}
4142 also for explicit usage.
4144 @item @code{acc_event_info.data_event.var_name}
4145 Always @code{NULL}; not yet implemented.
4147 @item @code{acc_event_info.data_event.host_ptr}
4148 For @code{acc_ev_alloc}, and @code{acc_ev_free}, this is always
4151 @item @code{typedef union acc_api_info}
4152 @dots{} as printed in @cite{5.2.3. Third Argument: API-Specific
4153 Information}. This should obviously be @code{typedef @emph{struct}
4156 @item @code{acc_api_info.device_api}
4157 Possibly not yet implemented correctly for
4158 @code{acc_ev_compute_construct_start},
4159 @code{acc_ev_device_init_start}, @code{acc_ev_device_init_end}:
4160 will always be @code{acc_device_api_none} for these event types.
4161 For @code{acc_ev_enter_data_start}, it will be
4162 @code{acc_device_api_none} in some cases.
4164 @item @code{acc_api_info.device_type}
4165 Always the same as @code{acc_prof_info.device_type}.
4167 @item @code{acc_api_info.vendor}
4168 Always @code{-1}; not yet implemented.
4170 @item @code{acc_api_info.device_handle}
4171 Always @code{NULL}; not yet implemented.
4173 @item @code{acc_api_info.context_handle}
4174 Always @code{NULL}; not yet implemented.
4176 @item @code{acc_api_info.async_handle}
4177 Always @code{NULL}; not yet implemented.
4181 Remarks about certain event types:
4185 @item @code{acc_ev_device_init_start}, @code{acc_ev_device_init_end}
4189 @c See 'DEVICE_INIT_INSIDE_COMPUTE_CONSTRUCT' in
4190 @c 'libgomp.oacc-c-c++-common/acc_prof-kernels-1.c',
4191 @c 'libgomp.oacc-c-c++-common/acc_prof-parallel-1.c'.
4192 When a compute construct triggers implicit
4193 @code{acc_ev_device_init_start} and @code{acc_ev_device_init_end}
4194 events, they currently aren't @emph{nested within} the corresponding
4195 @code{acc_ev_compute_construct_start} and
4196 @code{acc_ev_compute_construct_end}, but they're currently observed
4197 @emph{before} @code{acc_ev_compute_construct_start}.
4198 It's not clear what to do: the standard asks us provide a lot of
4199 details to the @code{acc_ev_compute_construct_start} callback, without
4200 (implicitly) initializing a device before?
4203 Callbacks for these event types will not be invoked for calls to the
4204 @code{acc_set_device_type} and @code{acc_set_device_num} functions.
4205 It's not clear if they should be.
4209 @item @code{acc_ev_enter_data_start}, @code{acc_ev_enter_data_end}, @code{acc_ev_exit_data_start}, @code{acc_ev_exit_data_end}
4213 Callbacks for these event types will also be invoked for OpenACC
4214 @emph{host_data} constructs.
4215 It's not clear if they should be.
4218 Callbacks for these event types will also be invoked when processing
4219 variable mappings specified in OpenACC @emph{declare} directives.
4220 It's not clear if they should be.
4226 Callbacks for the following event types will be invoked, but dispatch
4227 and information provided therein has not yet been thoroughly reviewed:
4230 @item @code{acc_ev_alloc}
4231 @item @code{acc_ev_free}
4232 @item @code{acc_ev_update_start}, @code{acc_ev_update_end}
4233 @item @code{acc_ev_enqueue_upload_start}, @code{acc_ev_enqueue_upload_end}
4234 @item @code{acc_ev_enqueue_download_start}, @code{acc_ev_enqueue_download_end}
4237 During device initialization, and finalization, respectively,
4238 callbacks for the following event types will not yet be invoked:
4241 @item @code{acc_ev_alloc}
4242 @item @code{acc_ev_free}
4245 Callbacks for the following event types have not yet been implemented,
4246 so currently won't be invoked:
4249 @item @code{acc_ev_device_shutdown_start}, @code{acc_ev_device_shutdown_end}
4250 @item @code{acc_ev_runtime_shutdown}
4251 @item @code{acc_ev_create}, @code{acc_ev_delete}
4252 @item @code{acc_ev_wait_start}, @code{acc_ev_wait_end}
4255 For the following runtime library functions, not all expected
4256 callbacks will be invoked (mostly concerning implicit device
4260 @item @code{acc_get_num_devices}
4261 @item @code{acc_set_device_type}
4262 @item @code{acc_get_device_type}
4263 @item @code{acc_set_device_num}
4264 @item @code{acc_get_device_num}
4265 @item @code{acc_init}
4266 @item @code{acc_shutdown}
4269 Aside from implicit device initialization, for the following runtime
4270 library functions, no callbacks will be invoked for shared-memory
4271 offloading devices (it's not clear if they should be):
4274 @item @code{acc_malloc}
4275 @item @code{acc_free}
4276 @item @code{acc_copyin}, @code{acc_present_or_copyin}, @code{acc_copyin_async}
4277 @item @code{acc_create}, @code{acc_present_or_create}, @code{acc_create_async}
4278 @item @code{acc_copyout}, @code{acc_copyout_async}, @code{acc_copyout_finalize}, @code{acc_copyout_finalize_async}
4279 @item @code{acc_delete}, @code{acc_delete_async}, @code{acc_delete_finalize}, @code{acc_delete_finalize_async}
4280 @item @code{acc_update_device}, @code{acc_update_device_async}
4281 @item @code{acc_update_self}, @code{acc_update_self_async}
4282 @item @code{acc_map_data}, @code{acc_unmap_data}
4283 @item @code{acc_memcpy_to_device}, @code{acc_memcpy_to_device_async}
4284 @item @code{acc_memcpy_from_device}, @code{acc_memcpy_from_device_async}
4287 @c ---------------------------------------------------------------------
4288 @c OpenMP-Implementation Specifics
4289 @c ---------------------------------------------------------------------
4291 @node OpenMP-Implementation Specifics
4292 @chapter OpenMP-Implementation Specifics
4295 * OpenMP Context Selectors::
4296 * Memory allocation with libmemkind::
4299 @node OpenMP Context Selectors
4300 @section OpenMP Context Selectors
4302 @code{vendor} is always @code{gnu}. References are to the GCC manual.
4304 @multitable @columnfractions .60 .10 .25
4305 @headitem @code{arch} @tab @code{kind} @tab @code{isa}
4306 @item @code{intel_mic}, @code{x86}, @code{x86_64}, @code{i386}, @code{i486},
4307 @code{i586}, @code{i686}, @code{ia32}
4309 @tab See @code{-m...} flags in ``x86 Options'' (without @code{-m})
4310 @item @code{amdgcn}, @code{gcn}
4312 @tab See @code{-march=} in ``AMD GCN Options''
4315 @tab See @code{-march=} in ``Nvidia PTX Options''
4318 @node Memory allocation with libmemkind
4319 @section Memory allocation with libmemkind
4321 On Linux systems, where the @uref{https://github.com/memkind/memkind, memkind
4322 library} (@code{libmemkind.so.0}) is available at runtime, it is used when
4323 creating memory allocators requesting
4326 @item the memory space @code{omp_high_bw_mem_space}
4327 @item the memory space @code{omp_large_cap_mem_space}
4328 @item the partition trait @code{omp_atv_interleaved}
4332 @c ---------------------------------------------------------------------
4333 @c Offload-Target Specifics
4334 @c ---------------------------------------------------------------------
4336 @node Offload-Target Specifics
4337 @chapter Offload-Target Specifics
4339 The following sections present notes on the offload-target specifics
4347 @section AMD Radeon (GCN)
4349 On the hardware side, there is the hierarchy (fine to coarse):
4351 @item work item (thread)
4354 @item compute unite (CU)
4357 All OpenMP and OpenACC levels are used, i.e.
4359 @item OpenMP's simd and OpenACC's vector map to work items (thread)
4360 @item OpenMP's threads (``parallel'') and OpenACC's workers map
4362 @item OpenMP's teams and OpenACC's gang use a threadpool with the
4363 size of the number of teams or gangs, respectively.
4368 @item Number of teams is the specified @code{num_teams} (OpenMP) or
4369 @code{num_gangs} (OpenACC) or otherwise the number of CU
4370 @item Number of wavefronts is 4 for gfx900 and 16 otherwise;
4371 @code{num_threads} (OpenMP) and @code{num_workers} (OpenACC)
4372 overrides this if smaller.
4373 @item The wavefront has 102 scalars and 64 vectors
4374 @item Number of workitems is always 64
4375 @item The hardware permits maximally 40 workgroups/CU and
4376 16 wavefronts/workgroup up to a limit of 40 wavefronts in total per CU.
4377 @item 80 scalars registers and 24 vector registers in non-kernel functions
4378 (the chosen procedure-calling API).
4379 @item For the kernel itself: as many as register pressure demands (number of
4380 teams and number of threads, scaled down if registers are exhausted)
4383 The implementation remark:
4385 @item I/O within OpenMP target regions and OpenACC parallel/kernels is supported
4386 using the C library @code{printf} functions and the Fortran
4387 @code{print}/@code{write} statements.
4395 On the hardware side, there is the hierarchy (fine to coarse):
4400 @item streaming multiprocessor
4403 All OpenMP and OpenACC levels are used, i.e.
4405 @item OpenMP's simd and OpenACC's vector map to threads
4406 @item OpenMP's threads (``parallel'') and OpenACC's workers map to warps
4407 @item OpenMP's teams and OpenACC's gang use a threadpool with the
4408 size of the number of teams or gangs, respectively.
4413 @item The @code{warp_size} is always 32
4414 @item CUDA kernel launched: @code{dim=@{#teams,1,1@}, blocks=@{#threads,warp_size,1@}}.
4417 Additional information can be obtained by setting the environment variable to
4418 @code{GOMP_DEBUG=1} (very verbose; grep for @code{kernel.*launch} for launch
4421 GCC generates generic PTX ISA code, which is just-in-time compiled by CUDA,
4422 which caches the JIT in the user's directory (see CUDA documentation; can be
4423 tuned by the environment variables @code{CUDA_CACHE_@{DISABLE,MAXSIZE,PATH@}}.
4425 Note: While PTX ISA is generic, the @code{-mptx=} and @code{-march=} commandline
4426 options still affect the used PTX ISA code and, thus, the requirments on
4427 CUDA version and hardware.
4429 The implementation remark:
4431 @item I/O within OpenMP target regions and OpenACC parallel/kernels is supported
4432 using the C library @code{printf} functions. Note that the Fortran
4433 @code{print}/@code{write} statements are not supported, yet.
4434 @item Compilation OpenMP code that contains @code{requires reverse_offload}
4435 requires at least @code{-march=sm_35}, compiling for @code{-march=sm_30}
4440 @c ---------------------------------------------------------------------
4442 @c ---------------------------------------------------------------------
4444 @node The libgomp ABI
4445 @chapter The libgomp ABI
4447 The following sections present notes on the external ABI as
4448 presented by libgomp. Only maintainers should need them.
4451 * Implementing MASTER construct::
4452 * Implementing CRITICAL construct::
4453 * Implementing ATOMIC construct::
4454 * Implementing FLUSH construct::
4455 * Implementing BARRIER construct::
4456 * Implementing THREADPRIVATE construct::
4457 * Implementing PRIVATE clause::
4458 * Implementing FIRSTPRIVATE LASTPRIVATE COPYIN and COPYPRIVATE clauses::
4459 * Implementing REDUCTION clause::
4460 * Implementing PARALLEL construct::
4461 * Implementing FOR construct::
4462 * Implementing ORDERED construct::
4463 * Implementing SECTIONS construct::
4464 * Implementing SINGLE construct::
4465 * Implementing OpenACC's PARALLEL construct::
4469 @node Implementing MASTER construct
4470 @section Implementing MASTER construct
4473 if (omp_get_thread_num () == 0)
4477 Alternately, we generate two copies of the parallel subfunction
4478 and only include this in the version run by the primary thread.
4479 Surely this is not worthwhile though...
4483 @node Implementing CRITICAL construct
4484 @section Implementing CRITICAL construct
4486 Without a specified name,
4489 void GOMP_critical_start (void);
4490 void GOMP_critical_end (void);
4493 so that we don't get COPY relocations from libgomp to the main
4496 With a specified name, use omp_set_lock and omp_unset_lock with
4497 name being transformed into a variable declared like
4500 omp_lock_t gomp_critical_user_<name> __attribute__((common))
4503 Ideally the ABI would specify that all zero is a valid unlocked
4504 state, and so we wouldn't need to initialize this at
4509 @node Implementing ATOMIC construct
4510 @section Implementing ATOMIC construct
4512 The target should implement the @code{__sync} builtins.
4514 Failing that we could add
4517 void GOMP_atomic_enter (void)
4518 void GOMP_atomic_exit (void)
4521 which reuses the regular lock code, but with yet another lock
4522 object private to the library.
4526 @node Implementing FLUSH construct
4527 @section Implementing FLUSH construct
4529 Expands to the @code{__sync_synchronize} builtin.
4533 @node Implementing BARRIER construct
4534 @section Implementing BARRIER construct
4537 void GOMP_barrier (void)
4541 @node Implementing THREADPRIVATE construct
4542 @section Implementing THREADPRIVATE construct
4544 In _most_ cases we can map this directly to @code{__thread}. Except
4545 that OMP allows constructors for C++ objects. We can either
4546 refuse to support this (how often is it used?) or we can
4547 implement something akin to .ctors.
4549 Even more ideally, this ctor feature is handled by extensions
4550 to the main pthreads library. Failing that, we can have a set
4551 of entry points to register ctor functions to be called.
4555 @node Implementing PRIVATE clause
4556 @section Implementing PRIVATE clause
4558 In association with a PARALLEL, or within the lexical extent
4559 of a PARALLEL block, the variable becomes a local variable in
4560 the parallel subfunction.
4562 In association with FOR or SECTIONS blocks, create a new
4563 automatic variable within the current function. This preserves
4564 the semantic of new variable creation.
4568 @node Implementing FIRSTPRIVATE LASTPRIVATE COPYIN and COPYPRIVATE clauses
4569 @section Implementing FIRSTPRIVATE LASTPRIVATE COPYIN and COPYPRIVATE clauses
4571 This seems simple enough for PARALLEL blocks. Create a private
4572 struct for communicating between the parent and subfunction.
4573 In the parent, copy in values for scalar and "small" structs;
4574 copy in addresses for others TREE_ADDRESSABLE types. In the
4575 subfunction, copy the value into the local variable.
4577 It is not clear what to do with bare FOR or SECTION blocks.
4578 The only thing I can figure is that we do something like:
4581 #pragma omp for firstprivate(x) lastprivate(y)
4582 for (int i = 0; i < n; ++i)
4599 where the "x=x" and "y=y" assignments actually have different
4600 uids for the two variables, i.e. not something you could write
4601 directly in C. Presumably this only makes sense if the "outer"
4602 x and y are global variables.
4604 COPYPRIVATE would work the same way, except the structure
4605 broadcast would have to happen via SINGLE machinery instead.
4609 @node Implementing REDUCTION clause
4610 @section Implementing REDUCTION clause
4612 The private struct mentioned in the previous section should have
4613 a pointer to an array of the type of the variable, indexed by the
4614 thread's @var{team_id}. The thread stores its final value into the
4615 array, and after the barrier, the primary thread iterates over the
4616 array to collect the values.
4619 @node Implementing PARALLEL construct
4620 @section Implementing PARALLEL construct
4623 #pragma omp parallel
4632 void subfunction (void *data)
4639 GOMP_parallel_start (subfunction, &data, num_threads);
4640 subfunction (&data);
4641 GOMP_parallel_end ();
4645 void GOMP_parallel_start (void (*fn)(void *), void *data, unsigned num_threads)
4648 The @var{FN} argument is the subfunction to be run in parallel.
4650 The @var{DATA} argument is a pointer to a structure used to
4651 communicate data in and out of the subfunction, as discussed
4652 above with respect to FIRSTPRIVATE et al.
4654 The @var{NUM_THREADS} argument is 1 if an IF clause is present
4655 and false, or the value of the NUM_THREADS clause, if
4658 The function needs to create the appropriate number of
4659 threads and/or launch them from the dock. It needs to
4660 create the team structure and assign team ids.
4663 void GOMP_parallel_end (void)
4666 Tears down the team and returns us to the previous @code{omp_in_parallel()} state.
4670 @node Implementing FOR construct
4671 @section Implementing FOR construct
4674 #pragma omp parallel for
4675 for (i = lb; i <= ub; i++)
4682 void subfunction (void *data)
4685 while (GOMP_loop_static_next (&_s0, &_e0))
4688 for (i = _s0; i < _e1; i++)
4691 GOMP_loop_end_nowait ();
4694 GOMP_parallel_loop_static (subfunction, NULL, 0, lb, ub+1, 1, 0);
4696 GOMP_parallel_end ();
4700 #pragma omp for schedule(runtime)
4701 for (i = 0; i < n; i++)
4710 if (GOMP_loop_runtime_start (0, n, 1, &_s0, &_e0))
4713 for (i = _s0, i < _e0; i++)
4715 @} while (GOMP_loop_runtime_next (&_s0, _&e0));
4720 Note that while it looks like there is trickiness to propagating
4721 a non-constant STEP, there isn't really. We're explicitly allowed
4722 to evaluate it as many times as we want, and any variables involved
4723 should automatically be handled as PRIVATE or SHARED like any other
4724 variables. So the expression should remain evaluable in the
4725 subfunction. We can also pull it into a local variable if we like,
4726 but since its supposed to remain unchanged, we can also not if we like.
4728 If we have SCHEDULE(STATIC), and no ORDERED, then we ought to be
4729 able to get away with no work-sharing context at all, since we can
4730 simply perform the arithmetic directly in each thread to divide up
4731 the iterations. Which would mean that we wouldn't need to call any
4734 There are separate routines for handling loops with an ORDERED
4735 clause. Bookkeeping for that is non-trivial...
4739 @node Implementing ORDERED construct
4740 @section Implementing ORDERED construct
4743 void GOMP_ordered_start (void)
4744 void GOMP_ordered_end (void)
4749 @node Implementing SECTIONS construct
4750 @section Implementing SECTIONS construct
4755 #pragma omp sections
4769 for (i = GOMP_sections_start (3); i != 0; i = GOMP_sections_next ())
4786 @node Implementing SINGLE construct
4787 @section Implementing SINGLE construct
4801 if (GOMP_single_start ())
4809 #pragma omp single copyprivate(x)
4816 datap = GOMP_single_copy_start ();
4821 GOMP_single_copy_end (&data);
4830 @node Implementing OpenACC's PARALLEL construct
4831 @section Implementing OpenACC's PARALLEL construct
4834 void GOACC_parallel ()
4839 @c ---------------------------------------------------------------------
4841 @c ---------------------------------------------------------------------
4843 @node Reporting Bugs
4844 @chapter Reporting Bugs
4846 Bugs in the GNU Offloading and Multi Processing Runtime Library should
4847 be reported via @uref{https://gcc.gnu.org/bugzilla/, Bugzilla}. Please add
4848 "openacc", or "openmp", or both to the keywords field in the bug
4849 report, as appropriate.
4853 @c ---------------------------------------------------------------------
4854 @c GNU General Public License
4855 @c ---------------------------------------------------------------------
4857 @include gpl_v3.texi
4861 @c ---------------------------------------------------------------------
4862 @c GNU Free Documentation License
4863 @c ---------------------------------------------------------------------
4869 @c ---------------------------------------------------------------------
4870 @c Funding Free Software
4871 @c ---------------------------------------------------------------------
4873 @include funding.texi
4875 @c ---------------------------------------------------------------------
4877 @c ---------------------------------------------------------------------
4880 @unnumbered Library Index