1 \input texinfo @c -*-texinfo-*-
4 @setfilename libgomp.info
10 Copyright @copyright{} 2006-2021 Free Software Foundation, Inc.
12 Permission is granted to copy, distribute and/or modify this document
13 under the terms of the GNU Free Documentation License, Version 1.3 or
14 any later version published by the Free Software Foundation; with the
15 Invariant Sections being ``Funding Free Software'', the Front-Cover
16 texts being (a) (see below), and with the Back-Cover Texts being (b)
17 (see below). A copy of the license is included in the section entitled
18 ``GNU Free Documentation License''.
20 (a) The FSF's Front-Cover Text is:
24 (b) The FSF's Back-Cover Text is:
26 You have freedom to copy and modify this GNU Manual, like GNU
27 software. Copies published by the Free Software Foundation raise
28 funds for GNU development.
32 @dircategory GNU Libraries
34 * libgomp: (libgomp). GNU Offloading and Multi Processing Runtime Library.
37 This manual documents libgomp, the GNU Offloading and Multi Processing
38 Runtime library. This is the GNU implementation of the OpenMP and
39 OpenACC APIs for parallel and accelerator programming in C/C++ and
42 Published by the Free Software Foundation
43 51 Franklin Street, Fifth Floor
44 Boston, MA 02110-1301 USA
50 @setchapternewpage odd
53 @title GNU Offloading and Multi Processing Runtime Library
54 @subtitle The GNU OpenMP and OpenACC Implementation
56 @vskip 0pt plus 1filll
57 @comment For the @value{version-GCC} Version*
59 Published by the Free Software Foundation @*
60 51 Franklin Street, Fifth Floor@*
61 Boston, MA 02110-1301, USA@*
71 @node Top, Enabling OpenMP
75 This manual documents the usage of libgomp, the GNU Offloading and
76 Multi Processing Runtime Library. This includes the GNU
77 implementation of the @uref{https://www.openmp.org, OpenMP} Application
78 Programming Interface (API) for multi-platform shared-memory parallel
79 programming in C/C++ and Fortran, and the GNU implementation of the
80 @uref{https://www.openacc.org, OpenACC} Application Programming
81 Interface (API) for offloading of code to accelerator devices in C/C++
84 Originally, libgomp implemented the GNU OpenMP Runtime Library. Based
85 on this, support for OpenACC and offloading (both OpenACC and OpenMP
86 4's target construct) has been added later on, and the library's name
87 changed to GNU Offloading and Multi Processing Runtime Library.
92 @comment When you add a new menu item, please keep the right hand
93 @comment aligned to the same column. Do not use tabs. This provides
94 @comment better formatting.
97 * Enabling OpenMP:: How to enable OpenMP for your applications.
98 * OpenMP Implementation Status:: List of implemented features by OpenMP version
99 * OpenMP Runtime Library Routines: Runtime Library Routines.
100 The OpenMP runtime application programming
102 * OpenMP Environment Variables: Environment Variables.
103 Influencing OpenMP runtime behavior with
104 environment variables.
105 * Enabling OpenACC:: How to enable OpenACC for your
107 * OpenACC Runtime Library Routines:: The OpenACC runtime application
108 programming interface.
109 * OpenACC Environment Variables:: Influencing OpenACC runtime behavior with
110 environment variables.
111 * CUDA Streams Usage:: Notes on the implementation of
112 asynchronous operations.
113 * OpenACC Library Interoperability:: OpenACC library interoperability with the
114 NVIDIA CUBLAS library.
115 * OpenACC Profiling Interface::
116 * The libgomp ABI:: Notes on the external ABI presented by libgomp.
117 * Reporting Bugs:: How to report bugs in the GNU Offloading and
118 Multi Processing Runtime Library.
119 * Copying:: GNU general public license says
120 how you can copy and share libgomp.
121 * GNU Free Documentation License::
122 How you can copy and share this manual.
123 * Funding:: How to help assure continued work for free
125 * Library Index:: Index of this documentation.
129 @c ---------------------------------------------------------------------
131 @c ---------------------------------------------------------------------
133 @node Enabling OpenMP
134 @chapter Enabling OpenMP
136 To activate the OpenMP extensions for C/C++ and Fortran, the compile-time
137 flag @command{-fopenmp} must be specified. This enables the OpenMP directive
138 @code{#pragma omp} in C/C++ and @code{!$omp} directives in free form,
139 @code{c$omp}, @code{*$omp} and @code{!$omp} directives in fixed form,
140 @code{!$} conditional compilation sentinels in free form and @code{c$},
141 @code{*$} and @code{!$} sentinels in fixed form, for Fortran. The flag also
142 arranges for automatic linking of the OpenMP runtime library
143 (@ref{Runtime Library Routines}).
145 A complete description of all OpenMP directives may be found in the
146 @uref{https://www.openmp.org, OpenMP Application Program Interface} manuals.
147 See also @ref{OpenMP Implementation Status}.
150 @c ---------------------------------------------------------------------
151 @c OpenMP Implementation Status
152 @c ---------------------------------------------------------------------
154 @node OpenMP Implementation Status
155 @chapter OpenMP Implementation Status
158 * OpenMP 4.5:: Feature completion status to 4.5 specification
159 * OpenMP 5.0:: Feature completion status to 5.0 specification
160 * OpenMP 5.1:: Feature completion status to 5.1 specification
163 The @code{_OPENMP} preprocessor macro and Fortran's @code{openmp_version}
164 parameter, provided by @code{omp_lib.h} and the @code{omp_lib} module, have
165 the value @code{201511} (i.e. OpenMP 4.5).
170 The OpenMP 4.5 specification is fully supported.
175 @unnumberedsubsec New features listed in Appendix B of the OpenMP specification
176 @c This list is sorted as in OpenMP 5.1's B.3 not as in OpenMP 5.0's B.2
178 @multitable @columnfractions .60 .10 .25
179 @headitem Description @tab Status @tab Comments
180 @item Array shaping @tab N @tab
181 @item Array sections with non-unit strides in C and C++ @tab N @tab
182 @item Iterators @tab Y @tab
183 @item @code{metadirective} directive @tab N @tab
184 @item @code{declare variant} directive
185 @tab P @tab simd traits not handled correctly
186 @item @emph{target-offload-var} ICV and @code{OMP_TARGET_OFFLOAD}
187 env variable @tab Y @tab
188 @item Nested-parallel changes to @emph{max-active-levels-var} ICV @tab Y @tab
189 @item @code{requires} directive @tab P
190 @tab Only fulfillable requirement is @code{atomic_default_mem_order}
191 @item @code{teams} construct outside an enclosing target region @tab Y @tab
192 @item Non-rectangular loop nests @tab P @tab Only C/C++
193 @item @code{!=} as relational-op in canonical loop form for C/C++ @tab Y @tab
194 @item @code{nonmonotonic} as default loop schedule modifier for worksharing-loop
195 constructs @tab Y @tab
196 @item Collapse of associated loops that are imperfectly nested loops @tab N @tab
197 @item Clauses @code{if}, @code{nontemporal} and @code{order(concurrent)} in
198 @code{simd} construct @tab Y @tab
199 @item @code{atomic} constructs in @code{simd} @tab Y @tab
200 @item @code{loop} construct @tab Y @tab
201 @item @code{order(concurrent)} clause @tab Y @tab
202 @item @code{scan} directive and @code{in_scan} modifier for the
203 @code{reduction} clause @tab Y @tab
204 @item @code{in_reduction} clause on @code{task} constructs @tab Y @tab
205 @item @code{in_reduction} clause on @code{target} constructs @tab P
206 @tab Only C/C++, @code{nowait} only stub
207 @item @code{task_reduction} clause with @code{taskgroup} @tab Y @tab
208 @item @code{task} modifier to @code{reduction} clause @tab Y @tab
209 @item @code{affinity} clause to @code{task} construct @tab Y @tab Stub only
210 @item @code{detach} clause to @code{task} construct @tab Y @tab
211 @item @code{omp_fulfill_event} runtime routine @tab Y @tab
212 @item @code{reduction} and @code{in_reduction} clauses on @code{taskloop}
213 and @code{taskloop simd} constructs @tab Y @tab
214 @item @code{taskloop} construct cancelable by @code{cancel} construct
216 @item @code{mutexinouset} @emph{dependence-type} for @code{depend} clause
218 @item Predefined memory spaces, memory allocators, allocator traits
219 @tab Y @tab Some are only stubs
220 @item Memory management routines @tab Y @tab
221 @item @code{allocate} directive @tab N @tab
222 @item @code{allocate} clause @tab P @tab initial support in C/C++ only
223 @item @code{use_device_addr} clause on @code{target data} @tab Y @tab
224 @item @code{ancestor} modifier on @code{device} clause
225 @tab P @tab Reverse offload unsupported
226 @item Implicit declare target directive @tab Y @tab
227 @item Discontiguous array section with @code{target update} construct
229 @item C/C++'s lvalue expressions in @code{to}, @code{from}
230 and @code{map} clauses @tab N @tab
231 @item C/C++'s lvalue expressions in @code{depend} clauses @tab Y @tab
232 @item Nested @code{declare target} directive @tab Y @tab
233 @item Combined @code{master} constructs @tab Y @tab
234 @item @code{depend} clause on @code{taskwait} @tab Y @tab
235 @item Weak memory ordering clauses on @code{atomic} and @code{flush} construct
237 @item @code{hint} clause on the @code{atomic} construct @tab Y @tab Stub only
238 @item @code{depobj} construct and depend objects @tab Y @tab
239 @item Lock hints were renamed to synchronization hints @tab Y @tab
240 @item @code{conditional} modifier to @code{lastprivate} clause @tab Y @tab
241 @item Map-order clarifications @tab P @tab
242 @item @code{close} @emph{map-type-modifier} @tab Y @tab
243 @item Mapping C/C++ pointer variables and to assign the address of
244 device memory mapped by an array section @tab P @tab
245 @item Mapping of Fortran pointer and allocatable variables, including pointer
246 and allocatable components of variables
247 @tab P @tab Mapping of vars with allocatable components unspported
248 @item @code{defaultmap} extensions @tab Y @tab
249 @item @code{declare mapper} directive @tab N @tab
250 @item @code{omp_get_supported_active_levels} routine @tab Y @tab
251 @item Runtime routines and environment variables to display runtime thread
252 affinity information @tab Y @tab
253 @item @code{omp_pause_resource} and @code{omp_pause_resource_all} runtime
255 @item @code{omp_get_device_num} runtime routine @tab Y @tab
256 @item OMPT interface @tab N @tab
257 @item OMPD interface @tab N @tab
260 @unnumberedsubsec Other new OpenMP 5.0 features
262 @multitable @columnfractions .60 .10 .25
263 @headitem Description @tab Status @tab Comments
264 @item Supporting C++'s range-based for loop @tab Y @tab
271 @unnumberedsubsec New features listed in Appendix B of the OpenMP specification
273 @multitable @columnfractions .60 .10 .25
274 @headitem Description @tab Status @tab Comments
275 @item OpenMP directive as C++ attribute specifiers @tab Y @tab
276 @item @code{omp_all_memory} reserved locator @tab N @tab
277 @item @emph{target_device trait} in OpenMP Context @tab N @tab
278 @item @code{target_device} selector set in context selectors @tab N @tab
279 @item C/C++'s @code{declare variant} directive: elision support of
280 preprocessed code @tab N @tab
281 @item @code{declare variant}: new clauses @code{adjust_args} and
282 @code{append_args} @tab N @tab
283 @item @code{dispatch} construct @tab N @tab
284 @item device-specific ICV settings the environment variables @tab N @tab
285 @item assume directive @tab N @tab
286 @item @code{nothing} directive @tab Y @tab
287 @item @code{error} directive @tab Y @tab
288 @item @code{masked} construct @tab Y @tab
289 @item @code{scope} directive @tab Y @tab
290 @item Loop transformation constructs @tab N @tab
291 @item @code{strict} modifier in the @code{grainsize} and @code{num_tasks}
292 clauses of the taskloop construct @tab Y @tab
293 @item @code{align} clause/modifier in @code{allocate} directive/clause
294 and @code{allocator} directive @tab P @tab C/C++ on clause only
295 @item @code{thread_limit} clause to @code{target} construct @tab Y @tab
296 @item @code{has_device_addr} clause to @code{target} construct @tab N @tab
297 @item iterators in @code{target update} motion clauses and @code{map}
299 @item indirect calls to the device version of a procedure or function in
300 @code{target} regions @tab N @tab
301 @item @code{interop} directive @tab N @tab
302 @item @code{omp_interop_t} object support in runtime routines @tab N @tab
303 @item @code{nowait} clause in @code{taskwait} directive @tab N @tab
304 @item Extensions to the @code{atomic} directive @tab P @tab C/C++ only
305 @item @code{seq_cst} clause on a @code{flush} construct @tab Y @tab
306 @item @code{inoutset} argument to the @code{depend} clause @tab N @tab
307 @item @code{private} and @code{firstprivate} argument to @code{default}
308 clause in C and C++ @tab Y @tab
309 @item @code{present} argument to @code{defaultmap} clause @tab N @tab
310 @item @code{omp_set_num_teams}, @code{omp_set_teams_thread_limit},
311 @code{omp_get_max_teams}, @code{omp_get_teams_thread_limit} runtime
313 @item @code{omp_target_is_accessible} runtime routine @tab N @tab
314 @item @code{omp_target_memcpy_async} and @code{omp_target_memcpy_rect_async}
315 runtime routines @tab N @tab
316 @item @code{omp_get_mapped_ptr} runtime routine @tab N @tab
317 @item @code{omp_calloc}, @code{omp_realloc}, @code{omp_aligned_alloc} and
318 @code{omp_aligned_calloc} runtime routines @tab Y @tab
319 @item @code{omp_alloctrait_key_t} enum: @code{omp_atv_serialized} added,
320 @code{omp_atv_default} changed @tab Y @tab
321 @item @code{omp_display_env} runtime routine @tab P
322 @tab Not inside @code{target} regions
323 @item @code{ompt_scope_endpoint_t} enum: @code{ompt_scope_beginend} @tab N @tab
324 @item @code{ompt_sync_region_t} enum additions @tab N @tab
325 @item @code{ompt_state_t} enum: @code{ompt_state_wait_barrier_implementation}
326 and @code{ompt_state_wait_barrier_teams} @tab N @tab
327 @item @code{ompt_callback_target_data_op_emi_t},
328 @code{ompt_callback_target_emi_t}, @code{ompt_callback_target_map_emi_t}
329 and @code{ompt_callback_target_submit_emi_t} @tab N @tab
330 @item @code{ompt_callback_error_t} type @tab N @tab
331 @item @code{OMP_PLACES} syntax extensions @tab Y @tab
332 @item @code{OMP_NUM_TEAMS} and @code{OMP_TEAMS_THREAD_LIMIT} environment
333 variables @tab Y @tab
336 @unnumberedsubsec Other new OpenMP 5.1 features
338 @multitable @columnfractions .60 .10 .25
339 @headitem Description @tab Status @tab Comments
340 @item Support of strictly structured blocks in Fortran @tab Y @tab
341 @item Support of structured block sequences in C/C++ @tab Y @tab
342 @item @code{unconstrained} and @code{reproducible} modifiers on @code{order}
347 @c ---------------------------------------------------------------------
348 @c OpenMP Runtime Library Routines
349 @c ---------------------------------------------------------------------
351 @node Runtime Library Routines
352 @chapter OpenMP Runtime Library Routines
354 The runtime routines described here are defined by Section 3 of the OpenMP
355 specification in version 4.5. The routines are structured in following
359 Control threads, processors and the parallel environment. They have C
360 linkage, and do not throw exceptions.
362 * omp_get_active_level:: Number of active parallel regions
363 * omp_get_ancestor_thread_num:: Ancestor thread ID
364 * omp_get_cancellation:: Whether cancellation support is enabled
365 * omp_get_default_device:: Get the default device for target regions
366 * omp_get_device_num:: Get device that current thread is running on
367 * omp_get_dynamic:: Dynamic teams setting
368 * omp_get_initial_device:: Device number of host device
369 * omp_get_level:: Number of parallel regions
370 * omp_get_max_active_levels:: Current maximum number of active regions
371 * omp_get_max_task_priority:: Maximum task priority value that can be set
372 * omp_get_max_teams:: Maximum number of teams for teams region
373 * omp_get_max_threads:: Maximum number of threads of parallel region
374 * omp_get_nested:: Nested parallel regions
375 * omp_get_num_devices:: Number of target devices
376 * omp_get_num_procs:: Number of processors online
377 * omp_get_num_teams:: Number of teams
378 * omp_get_num_threads:: Size of the active team
379 * omp_get_proc_bind:: Whether theads may be moved between CPUs
380 * omp_get_schedule:: Obtain the runtime scheduling method
381 * omp_get_supported_active_levels:: Maximum number of active regions supported
382 * omp_get_team_num:: Get team number
383 * omp_get_team_size:: Number of threads in a team
384 * omp_get_teams_thread_limit:: Maximum number of threads imposed by teams
385 * omp_get_thread_limit:: Maximum number of threads
386 * omp_get_thread_num:: Current thread ID
387 * omp_in_parallel:: Whether a parallel region is active
388 * omp_in_final:: Whether in final or included task region
389 * omp_is_initial_device:: Whether executing on the host device
390 * omp_set_default_device:: Set the default device for target regions
391 * omp_set_dynamic:: Enable/disable dynamic teams
392 * omp_set_max_active_levels:: Limits the number of active parallel regions
393 * omp_set_nested:: Enable/disable nested parallel regions
394 * omp_set_num_teams:: Set upper teams limit for teams region
395 * omp_set_num_threads:: Set upper team size limit
396 * omp_set_schedule:: Set the runtime scheduling method
397 * omp_set_teams_thread_limit:: Set upper thread limit for teams construct
399 Initialize, set, test, unset and destroy simple and nested locks.
401 * omp_init_lock:: Initialize simple lock
402 * omp_set_lock:: Wait for and set simple lock
403 * omp_test_lock:: Test and set simple lock if available
404 * omp_unset_lock:: Unset simple lock
405 * omp_destroy_lock:: Destroy simple lock
406 * omp_init_nest_lock:: Initialize nested lock
407 * omp_set_nest_lock:: Wait for and set simple lock
408 * omp_test_nest_lock:: Test and set nested lock if available
409 * omp_unset_nest_lock:: Unset nested lock
410 * omp_destroy_nest_lock:: Destroy nested lock
412 Portable, thread-based, wall clock timer.
414 * omp_get_wtick:: Get timer precision.
415 * omp_get_wtime:: Elapsed wall clock time.
417 Support for event objects.
419 * omp_fulfill_event:: Fulfill and destroy an OpenMP event.
424 @node omp_get_active_level
425 @section @code{omp_get_active_level} -- Number of parallel regions
427 @item @emph{Description}:
428 This function returns the nesting level for the active parallel blocks,
429 which enclose the calling call.
432 @multitable @columnfractions .20 .80
433 @item @emph{Prototype}: @tab @code{int omp_get_active_level(void);}
436 @item @emph{Fortran}:
437 @multitable @columnfractions .20 .80
438 @item @emph{Interface}: @tab @code{integer function omp_get_active_level()}
441 @item @emph{See also}:
442 @ref{omp_get_level}, @ref{omp_get_max_active_levels}, @ref{omp_set_max_active_levels}
444 @item @emph{Reference}:
445 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.20.
450 @node omp_get_ancestor_thread_num
451 @section @code{omp_get_ancestor_thread_num} -- Ancestor thread ID
453 @item @emph{Description}:
454 This function returns the thread identification number for the given
455 nesting level of the current thread. For values of @var{level} outside
456 zero to @code{omp_get_level} -1 is returned; if @var{level} is
457 @code{omp_get_level} the result is identical to @code{omp_get_thread_num}.
460 @multitable @columnfractions .20 .80
461 @item @emph{Prototype}: @tab @code{int omp_get_ancestor_thread_num(int level);}
464 @item @emph{Fortran}:
465 @multitable @columnfractions .20 .80
466 @item @emph{Interface}: @tab @code{integer function omp_get_ancestor_thread_num(level)}
467 @item @tab @code{integer level}
470 @item @emph{See also}:
471 @ref{omp_get_level}, @ref{omp_get_thread_num}, @ref{omp_get_team_size}
473 @item @emph{Reference}:
474 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.18.
479 @node omp_get_cancellation
480 @section @code{omp_get_cancellation} -- Whether cancellation support is enabled
482 @item @emph{Description}:
483 This function returns @code{true} if cancellation is activated, @code{false}
484 otherwise. Here, @code{true} and @code{false} represent their language-specific
485 counterparts. Unless @env{OMP_CANCELLATION} is set true, cancellations are
489 @multitable @columnfractions .20 .80
490 @item @emph{Prototype}: @tab @code{int omp_get_cancellation(void);}
493 @item @emph{Fortran}:
494 @multitable @columnfractions .20 .80
495 @item @emph{Interface}: @tab @code{logical function omp_get_cancellation()}
498 @item @emph{See also}:
499 @ref{OMP_CANCELLATION}
501 @item @emph{Reference}:
502 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.9.
507 @node omp_get_default_device
508 @section @code{omp_get_default_device} -- Get the default device for target regions
510 @item @emph{Description}:
511 Get the default device for target regions without device clause.
514 @multitable @columnfractions .20 .80
515 @item @emph{Prototype}: @tab @code{int omp_get_default_device(void);}
518 @item @emph{Fortran}:
519 @multitable @columnfractions .20 .80
520 @item @emph{Interface}: @tab @code{integer function omp_get_default_device()}
523 @item @emph{See also}:
524 @ref{OMP_DEFAULT_DEVICE}, @ref{omp_set_default_device}
526 @item @emph{Reference}:
527 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.30.
532 @node omp_get_device_num
533 @section @code{omp_get_device_num} -- Return device number of current device
535 @item @emph{Description}:
536 This function returns a device number that represents the device that the
537 current thread is executing on. For OpenMP 5.0, this must be equal to the
538 value returned by the @code{omp_get_initial_device} function when called
542 @multitable @columnfractions .20 .80
543 @item @emph{Prototype}: @tab @code{int omp_get_device_num(void);}
546 @item @emph{Fortran}:
547 @multitable @columnfractions .20 .80
548 @item @emph{Interface}: @tab @code{integer function omp_get_device_num()}
551 @item @emph{See also}:
552 @ref{omp_get_initial_device}
554 @item @emph{Reference}:
555 @uref{https://www.openmp.org, OpenMP specification v5.0}, Section 3.2.37.
560 @node omp_get_dynamic
561 @section @code{omp_get_dynamic} -- Dynamic teams setting
563 @item @emph{Description}:
564 This function returns @code{true} if enabled, @code{false} otherwise.
565 Here, @code{true} and @code{false} represent their language-specific
568 The dynamic team setting may be initialized at startup by the
569 @env{OMP_DYNAMIC} environment variable or at runtime using
570 @code{omp_set_dynamic}. If undefined, dynamic adjustment is
574 @multitable @columnfractions .20 .80
575 @item @emph{Prototype}: @tab @code{int omp_get_dynamic(void);}
578 @item @emph{Fortran}:
579 @multitable @columnfractions .20 .80
580 @item @emph{Interface}: @tab @code{logical function omp_get_dynamic()}
583 @item @emph{See also}:
584 @ref{omp_set_dynamic}, @ref{OMP_DYNAMIC}
586 @item @emph{Reference}:
587 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.8.
592 @node omp_get_initial_device
593 @section @code{omp_get_initial_device} -- Return device number of initial device
595 @item @emph{Description}:
596 This function returns a device number that represents the host device.
597 For OpenMP 5.1, this must be equal to the value returned by the
598 @code{omp_get_num_devices} function.
601 @multitable @columnfractions .20 .80
602 @item @emph{Prototype}: @tab @code{int omp_get_initial_device(void);}
605 @item @emph{Fortran}:
606 @multitable @columnfractions .20 .80
607 @item @emph{Interface}: @tab @code{integer function omp_get_initial_device()}
610 @item @emph{See also}:
611 @ref{omp_get_num_devices}
613 @item @emph{Reference}:
614 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.35.
620 @section @code{omp_get_level} -- Obtain the current nesting level
622 @item @emph{Description}:
623 This function returns the nesting level for the parallel blocks,
624 which enclose the calling call.
627 @multitable @columnfractions .20 .80
628 @item @emph{Prototype}: @tab @code{int omp_get_level(void);}
631 @item @emph{Fortran}:
632 @multitable @columnfractions .20 .80
633 @item @emph{Interface}: @tab @code{integer function omp_level()}
636 @item @emph{See also}:
637 @ref{omp_get_active_level}
639 @item @emph{Reference}:
640 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.17.
645 @node omp_get_max_active_levels
646 @section @code{omp_get_max_active_levels} -- Current maximum number of active regions
648 @item @emph{Description}:
649 This function obtains the maximum allowed number of nested, active parallel regions.
652 @multitable @columnfractions .20 .80
653 @item @emph{Prototype}: @tab @code{int omp_get_max_active_levels(void);}
656 @item @emph{Fortran}:
657 @multitable @columnfractions .20 .80
658 @item @emph{Interface}: @tab @code{integer function omp_get_max_active_levels()}
661 @item @emph{See also}:
662 @ref{omp_set_max_active_levels}, @ref{omp_get_active_level}
664 @item @emph{Reference}:
665 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.16.
669 @node omp_get_max_task_priority
670 @section @code{omp_get_max_task_priority} -- Maximum priority value
671 that can be set for tasks.
673 @item @emph{Description}:
674 This function obtains the maximum allowed priority number for tasks.
677 @multitable @columnfractions .20 .80
678 @item @emph{Prototype}: @tab @code{int omp_get_max_task_priority(void);}
681 @item @emph{Fortran}:
682 @multitable @columnfractions .20 .80
683 @item @emph{Interface}: @tab @code{integer function omp_get_max_task_priority()}
686 @item @emph{Reference}:
687 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.29.
691 @node omp_get_max_teams
692 @section @code{omp_get_max_teams} -- Maximum number of teams of teams region
694 @item @emph{Description}:
695 Return the maximum number of teams used for the teams region
696 that does not use the clause @code{num_teams}.
699 @multitable @columnfractions .20 .80
700 @item @emph{Prototype}: @tab @code{int omp_get_max_teams(void);}
703 @item @emph{Fortran}:
704 @multitable @columnfractions .20 .80
705 @item @emph{Interface}: @tab @code{integer function omp_get_max_teams()}
708 @item @emph{See also}:
709 @ref{omp_set_num_teams}, @ref{omp_get_num_teams}
711 @item @emph{Reference}:
712 @uref{https://www.openmp.org, OpenMP specification v5.1}, Section 3.4.4.
717 @node omp_get_max_threads
718 @section @code{omp_get_max_threads} -- Maximum number of threads of parallel region
720 @item @emph{Description}:
721 Return the maximum number of threads used for the current parallel region
722 that does not use the clause @code{num_threads}.
725 @multitable @columnfractions .20 .80
726 @item @emph{Prototype}: @tab @code{int omp_get_max_threads(void);}
729 @item @emph{Fortran}:
730 @multitable @columnfractions .20 .80
731 @item @emph{Interface}: @tab @code{integer function omp_get_max_threads()}
734 @item @emph{See also}:
735 @ref{omp_set_num_threads}, @ref{omp_set_dynamic}, @ref{omp_get_thread_limit}
737 @item @emph{Reference}:
738 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.3.
744 @section @code{omp_get_nested} -- Nested parallel regions
746 @item @emph{Description}:
747 This function returns @code{true} if nested parallel regions are
748 enabled, @code{false} otherwise. Here, @code{true} and @code{false}
749 represent their language-specific counterparts.
751 The state of nested parallel regions at startup depends on several
752 environment variables. If @env{OMP_MAX_ACTIVE_LEVELS} is defined
753 and is set to greater than one, then nested parallel regions will be
754 enabled. If not defined, then the value of the @env{OMP_NESTED}
755 environment variable will be followed if defined. If neither are
756 defined, then if either @env{OMP_NUM_THREADS} or @env{OMP_PROC_BIND}
757 are defined with a list of more than one value, then nested parallel
758 regions are enabled. If none of these are defined, then nested parallel
759 regions are disabled by default.
761 Nested parallel regions can be enabled or disabled at runtime using
762 @code{omp_set_nested}, or by setting the maximum number of nested
763 regions with @code{omp_set_max_active_levels} to one to disable, or
767 @multitable @columnfractions .20 .80
768 @item @emph{Prototype}: @tab @code{int omp_get_nested(void);}
771 @item @emph{Fortran}:
772 @multitable @columnfractions .20 .80
773 @item @emph{Interface}: @tab @code{logical function omp_get_nested()}
776 @item @emph{See also}:
777 @ref{omp_set_max_active_levels}, @ref{omp_set_nested},
778 @ref{OMP_MAX_ACTIVE_LEVELS}, @ref{OMP_NESTED}
780 @item @emph{Reference}:
781 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.11.
786 @node omp_get_num_devices
787 @section @code{omp_get_num_devices} -- Number of target devices
789 @item @emph{Description}:
790 Returns the number of target devices.
793 @multitable @columnfractions .20 .80
794 @item @emph{Prototype}: @tab @code{int omp_get_num_devices(void);}
797 @item @emph{Fortran}:
798 @multitable @columnfractions .20 .80
799 @item @emph{Interface}: @tab @code{integer function omp_get_num_devices()}
802 @item @emph{Reference}:
803 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.31.
808 @node omp_get_num_procs
809 @section @code{omp_get_num_procs} -- Number of processors online
811 @item @emph{Description}:
812 Returns the number of processors online on that device.
815 @multitable @columnfractions .20 .80
816 @item @emph{Prototype}: @tab @code{int omp_get_num_procs(void);}
819 @item @emph{Fortran}:
820 @multitable @columnfractions .20 .80
821 @item @emph{Interface}: @tab @code{integer function omp_get_num_procs()}
824 @item @emph{Reference}:
825 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.5.
830 @node omp_get_num_teams
831 @section @code{omp_get_num_teams} -- Number of teams
833 @item @emph{Description}:
834 Returns the number of teams in the current team region.
837 @multitable @columnfractions .20 .80
838 @item @emph{Prototype}: @tab @code{int omp_get_num_teams(void);}
841 @item @emph{Fortran}:
842 @multitable @columnfractions .20 .80
843 @item @emph{Interface}: @tab @code{integer function omp_get_num_teams()}
846 @item @emph{Reference}:
847 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.32.
852 @node omp_get_num_threads
853 @section @code{omp_get_num_threads} -- Size of the active team
855 @item @emph{Description}:
856 Returns the number of threads in the current team. In a sequential section of
857 the program @code{omp_get_num_threads} returns 1.
859 The default team size may be initialized at startup by the
860 @env{OMP_NUM_THREADS} environment variable. At runtime, the size
861 of the current team may be set either by the @code{NUM_THREADS}
862 clause or by @code{omp_set_num_threads}. If none of the above were
863 used to define a specific value and @env{OMP_DYNAMIC} is disabled,
864 one thread per CPU online is used.
867 @multitable @columnfractions .20 .80
868 @item @emph{Prototype}: @tab @code{int omp_get_num_threads(void);}
871 @item @emph{Fortran}:
872 @multitable @columnfractions .20 .80
873 @item @emph{Interface}: @tab @code{integer function omp_get_num_threads()}
876 @item @emph{See also}:
877 @ref{omp_get_max_threads}, @ref{omp_set_num_threads}, @ref{OMP_NUM_THREADS}
879 @item @emph{Reference}:
880 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.2.
885 @node omp_get_proc_bind
886 @section @code{omp_get_proc_bind} -- Whether theads may be moved between CPUs
888 @item @emph{Description}:
889 This functions returns the currently active thread affinity policy, which is
890 set via @env{OMP_PROC_BIND}. Possible values are @code{omp_proc_bind_false},
891 @code{omp_proc_bind_true}, @code{omp_proc_bind_primary},
892 @code{omp_proc_bind_master}, @code{omp_proc_bind_close} and @code{omp_proc_bind_spread},
893 where @code{omp_proc_bind_master} is an alias for @code{omp_proc_bind_primary}.
896 @multitable @columnfractions .20 .80
897 @item @emph{Prototype}: @tab @code{omp_proc_bind_t omp_get_proc_bind(void);}
900 @item @emph{Fortran}:
901 @multitable @columnfractions .20 .80
902 @item @emph{Interface}: @tab @code{integer(kind=omp_proc_bind_kind) function omp_get_proc_bind()}
905 @item @emph{See also}:
906 @ref{OMP_PROC_BIND}, @ref{OMP_PLACES}, @ref{GOMP_CPU_AFFINITY},
908 @item @emph{Reference}:
909 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.22.
914 @node omp_get_schedule
915 @section @code{omp_get_schedule} -- Obtain the runtime scheduling method
917 @item @emph{Description}:
918 Obtain the runtime scheduling method. The @var{kind} argument will be
919 set to the value @code{omp_sched_static}, @code{omp_sched_dynamic},
920 @code{omp_sched_guided} or @code{omp_sched_auto}. The second argument,
921 @var{chunk_size}, is set to the chunk size.
924 @multitable @columnfractions .20 .80
925 @item @emph{Prototype}: @tab @code{void omp_get_schedule(omp_sched_t *kind, int *chunk_size);}
928 @item @emph{Fortran}:
929 @multitable @columnfractions .20 .80
930 @item @emph{Interface}: @tab @code{subroutine omp_get_schedule(kind, chunk_size)}
931 @item @tab @code{integer(kind=omp_sched_kind) kind}
932 @item @tab @code{integer chunk_size}
935 @item @emph{See also}:
936 @ref{omp_set_schedule}, @ref{OMP_SCHEDULE}
938 @item @emph{Reference}:
939 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.13.
943 @node omp_get_supported_active_levels
944 @section @code{omp_get_supported_active_levels} -- Maximum number of active regions supported
946 @item @emph{Description}:
947 This function returns the maximum number of nested, active parallel regions
948 supported by this implementation.
951 @multitable @columnfractions .20 .80
952 @item @emph{Prototype}: @tab @code{int omp_get_supported_active_levels(void);}
955 @item @emph{Fortran}:
956 @multitable @columnfractions .20 .80
957 @item @emph{Interface}: @tab @code{integer function omp_get_supported_active_levels()}
960 @item @emph{See also}:
961 @ref{omp_get_max_active_levels}, @ref{omp_set_max_active_levels}
963 @item @emph{Reference}:
964 @uref{https://www.openmp.org, OpenMP specification v5.0}, Section 3.2.15.
969 @node omp_get_team_num
970 @section @code{omp_get_team_num} -- Get team number
972 @item @emph{Description}:
973 Returns the team number of the calling thread.
976 @multitable @columnfractions .20 .80
977 @item @emph{Prototype}: @tab @code{int omp_get_team_num(void);}
980 @item @emph{Fortran}:
981 @multitable @columnfractions .20 .80
982 @item @emph{Interface}: @tab @code{integer function omp_get_team_num()}
985 @item @emph{Reference}:
986 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.33.
991 @node omp_get_team_size
992 @section @code{omp_get_team_size} -- Number of threads in a team
994 @item @emph{Description}:
995 This function returns the number of threads in a thread team to which
996 either the current thread or its ancestor belongs. For values of @var{level}
997 outside zero to @code{omp_get_level}, -1 is returned; if @var{level} is zero,
998 1 is returned, and for @code{omp_get_level}, the result is identical
999 to @code{omp_get_num_threads}.
1002 @multitable @columnfractions .20 .80
1003 @item @emph{Prototype}: @tab @code{int omp_get_team_size(int level);}
1006 @item @emph{Fortran}:
1007 @multitable @columnfractions .20 .80
1008 @item @emph{Interface}: @tab @code{integer function omp_get_team_size(level)}
1009 @item @tab @code{integer level}
1012 @item @emph{See also}:
1013 @ref{omp_get_num_threads}, @ref{omp_get_level}, @ref{omp_get_ancestor_thread_num}
1015 @item @emph{Reference}:
1016 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.19.
1021 @node omp_get_teams_thread_limit
1022 @section @code{omp_get_teams_thread_limit} -- Maximum number of threads imposed by teams
1024 @item @emph{Description}:
1025 Return the maximum number of threads that will be able to participate in
1026 each team created by a teams construct.
1029 @multitable @columnfractions .20 .80
1030 @item @emph{Prototype}: @tab @code{int omp_get_teams_thread_limit(void);}
1033 @item @emph{Fortran}:
1034 @multitable @columnfractions .20 .80
1035 @item @emph{Interface}: @tab @code{integer function omp_get_teams_thread_limit()}
1038 @item @emph{See also}:
1039 @ref{omp_set_teams_thread_limit}, @ref{OMP_TEAMS_THREAD_LIMIT}
1041 @item @emph{Reference}:
1042 @uref{https://www.openmp.org, OpenMP specification v5.1}, Section 3.4.6.
1047 @node omp_get_thread_limit
1048 @section @code{omp_get_thread_limit} -- Maximum number of threads
1050 @item @emph{Description}:
1051 Return the maximum number of threads of the program.
1054 @multitable @columnfractions .20 .80
1055 @item @emph{Prototype}: @tab @code{int omp_get_thread_limit(void);}
1058 @item @emph{Fortran}:
1059 @multitable @columnfractions .20 .80
1060 @item @emph{Interface}: @tab @code{integer function omp_get_thread_limit()}
1063 @item @emph{See also}:
1064 @ref{omp_get_max_threads}, @ref{OMP_THREAD_LIMIT}
1066 @item @emph{Reference}:
1067 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.14.
1072 @node omp_get_thread_num
1073 @section @code{omp_get_thread_num} -- Current thread ID
1075 @item @emph{Description}:
1076 Returns a unique thread identification number within the current team.
1077 In a sequential parts of the program, @code{omp_get_thread_num}
1078 always returns 0. In parallel regions the return value varies
1079 from 0 to @code{omp_get_num_threads}-1 inclusive. The return
1080 value of the primary thread of a team is always 0.
1083 @multitable @columnfractions .20 .80
1084 @item @emph{Prototype}: @tab @code{int omp_get_thread_num(void);}
1087 @item @emph{Fortran}:
1088 @multitable @columnfractions .20 .80
1089 @item @emph{Interface}: @tab @code{integer function omp_get_thread_num()}
1092 @item @emph{See also}:
1093 @ref{omp_get_num_threads}, @ref{omp_get_ancestor_thread_num}
1095 @item @emph{Reference}:
1096 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.4.
1101 @node omp_in_parallel
1102 @section @code{omp_in_parallel} -- Whether a parallel region is active
1104 @item @emph{Description}:
1105 This function returns @code{true} if currently running in parallel,
1106 @code{false} otherwise. Here, @code{true} and @code{false} represent
1107 their language-specific counterparts.
1110 @multitable @columnfractions .20 .80
1111 @item @emph{Prototype}: @tab @code{int omp_in_parallel(void);}
1114 @item @emph{Fortran}:
1115 @multitable @columnfractions .20 .80
1116 @item @emph{Interface}: @tab @code{logical function omp_in_parallel()}
1119 @item @emph{Reference}:
1120 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.6.
1125 @section @code{omp_in_final} -- Whether in final or included task region
1127 @item @emph{Description}:
1128 This function returns @code{true} if currently running in a final
1129 or included task region, @code{false} otherwise. Here, @code{true}
1130 and @code{false} represent their language-specific counterparts.
1133 @multitable @columnfractions .20 .80
1134 @item @emph{Prototype}: @tab @code{int omp_in_final(void);}
1137 @item @emph{Fortran}:
1138 @multitable @columnfractions .20 .80
1139 @item @emph{Interface}: @tab @code{logical function omp_in_final()}
1142 @item @emph{Reference}:
1143 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.21.
1148 @node omp_is_initial_device
1149 @section @code{omp_is_initial_device} -- Whether executing on the host device
1151 @item @emph{Description}:
1152 This function returns @code{true} if currently running on the host device,
1153 @code{false} otherwise. Here, @code{true} and @code{false} represent
1154 their language-specific counterparts.
1157 @multitable @columnfractions .20 .80
1158 @item @emph{Prototype}: @tab @code{int omp_is_initial_device(void);}
1161 @item @emph{Fortran}:
1162 @multitable @columnfractions .20 .80
1163 @item @emph{Interface}: @tab @code{logical function omp_is_initial_device()}
1166 @item @emph{Reference}:
1167 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.34.
1172 @node omp_set_default_device
1173 @section @code{omp_set_default_device} -- Set the default device for target regions
1175 @item @emph{Description}:
1176 Set the default device for target regions without device clause. The argument
1177 shall be a nonnegative device number.
1180 @multitable @columnfractions .20 .80
1181 @item @emph{Prototype}: @tab @code{void omp_set_default_device(int device_num);}
1184 @item @emph{Fortran}:
1185 @multitable @columnfractions .20 .80
1186 @item @emph{Interface}: @tab @code{subroutine omp_set_default_device(device_num)}
1187 @item @tab @code{integer device_num}
1190 @item @emph{See also}:
1191 @ref{OMP_DEFAULT_DEVICE}, @ref{omp_get_default_device}
1193 @item @emph{Reference}:
1194 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.29.
1199 @node omp_set_dynamic
1200 @section @code{omp_set_dynamic} -- Enable/disable dynamic teams
1202 @item @emph{Description}:
1203 Enable or disable the dynamic adjustment of the number of threads
1204 within a team. The function takes the language-specific equivalent
1205 of @code{true} and @code{false}, where @code{true} enables dynamic
1206 adjustment of team sizes and @code{false} disables it.
1209 @multitable @columnfractions .20 .80
1210 @item @emph{Prototype}: @tab @code{void omp_set_dynamic(int dynamic_threads);}
1213 @item @emph{Fortran}:
1214 @multitable @columnfractions .20 .80
1215 @item @emph{Interface}: @tab @code{subroutine omp_set_dynamic(dynamic_threads)}
1216 @item @tab @code{logical, intent(in) :: dynamic_threads}
1219 @item @emph{See also}:
1220 @ref{OMP_DYNAMIC}, @ref{omp_get_dynamic}
1222 @item @emph{Reference}:
1223 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.7.
1228 @node omp_set_max_active_levels
1229 @section @code{omp_set_max_active_levels} -- Limits the number of active parallel regions
1231 @item @emph{Description}:
1232 This function limits the maximum allowed number of nested, active
1233 parallel regions. @var{max_levels} must be less or equal to
1234 the value returned by @code{omp_get_supported_active_levels}.
1237 @multitable @columnfractions .20 .80
1238 @item @emph{Prototype}: @tab @code{void omp_set_max_active_levels(int max_levels);}
1241 @item @emph{Fortran}:
1242 @multitable @columnfractions .20 .80
1243 @item @emph{Interface}: @tab @code{subroutine omp_set_max_active_levels(max_levels)}
1244 @item @tab @code{integer max_levels}
1247 @item @emph{See also}:
1248 @ref{omp_get_max_active_levels}, @ref{omp_get_active_level},
1249 @ref{omp_get_supported_active_levels}
1251 @item @emph{Reference}:
1252 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.15.
1257 @node omp_set_nested
1258 @section @code{omp_set_nested} -- Enable/disable nested parallel regions
1260 @item @emph{Description}:
1261 Enable or disable nested parallel regions, i.e., whether team members
1262 are allowed to create new teams. The function takes the language-specific
1263 equivalent of @code{true} and @code{false}, where @code{true} enables
1264 dynamic adjustment of team sizes and @code{false} disables it.
1266 Enabling nested parallel regions will also set the maximum number of
1267 active nested regions to the maximum supported. Disabling nested parallel
1268 regions will set the maximum number of active nested regions to one.
1271 @multitable @columnfractions .20 .80
1272 @item @emph{Prototype}: @tab @code{void omp_set_nested(int nested);}
1275 @item @emph{Fortran}:
1276 @multitable @columnfractions .20 .80
1277 @item @emph{Interface}: @tab @code{subroutine omp_set_nested(nested)}
1278 @item @tab @code{logical, intent(in) :: nested}
1281 @item @emph{See also}:
1282 @ref{omp_get_nested}, @ref{omp_set_max_active_levels},
1283 @ref{OMP_MAX_ACTIVE_LEVELS}, @ref{OMP_NESTED}
1285 @item @emph{Reference}:
1286 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.10.
1291 @node omp_set_num_teams
1292 @section @code{omp_set_num_teams} -- Set upper teams limit for teams construct
1294 @item @emph{Description}:
1295 Specifies the upper bound for number of teams created by the teams construct
1296 which does not specify a @code{num_teams} clause. The
1297 argument of @code{omp_set_num_teams} shall be a positive integer.
1300 @multitable @columnfractions .20 .80
1301 @item @emph{Prototype}: @tab @code{void omp_set_num_teams(int num_teams);}
1304 @item @emph{Fortran}:
1305 @multitable @columnfractions .20 .80
1306 @item @emph{Interface}: @tab @code{subroutine omp_set_num_teams(num_teams)}
1307 @item @tab @code{integer, intent(in) :: num_teams}
1310 @item @emph{See also}:
1311 @ref{OMP_NUM_TEAMS}, @ref{omp_get_num_teams}, @ref{omp_get_max_teams}
1313 @item @emph{Reference}:
1314 @uref{https://www.openmp.org, OpenMP specification v5.1}, Section 3.4.3.
1319 @node omp_set_num_threads
1320 @section @code{omp_set_num_threads} -- Set upper team size limit
1322 @item @emph{Description}:
1323 Specifies the number of threads used by default in subsequent parallel
1324 sections, if those do not specify a @code{num_threads} clause. The
1325 argument of @code{omp_set_num_threads} shall be a positive integer.
1328 @multitable @columnfractions .20 .80
1329 @item @emph{Prototype}: @tab @code{void omp_set_num_threads(int num_threads);}
1332 @item @emph{Fortran}:
1333 @multitable @columnfractions .20 .80
1334 @item @emph{Interface}: @tab @code{subroutine omp_set_num_threads(num_threads)}
1335 @item @tab @code{integer, intent(in) :: num_threads}
1338 @item @emph{See also}:
1339 @ref{OMP_NUM_THREADS}, @ref{omp_get_num_threads}, @ref{omp_get_max_threads}
1341 @item @emph{Reference}:
1342 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.1.
1347 @node omp_set_schedule
1348 @section @code{omp_set_schedule} -- Set the runtime scheduling method
1350 @item @emph{Description}:
1351 Sets the runtime scheduling method. The @var{kind} argument can have the
1352 value @code{omp_sched_static}, @code{omp_sched_dynamic},
1353 @code{omp_sched_guided} or @code{omp_sched_auto}. Except for
1354 @code{omp_sched_auto}, the chunk size is set to the value of
1355 @var{chunk_size} if positive, or to the default value if zero or negative.
1356 For @code{omp_sched_auto} the @var{chunk_size} argument is ignored.
1359 @multitable @columnfractions .20 .80
1360 @item @emph{Prototype}: @tab @code{void omp_set_schedule(omp_sched_t kind, int chunk_size);}
1363 @item @emph{Fortran}:
1364 @multitable @columnfractions .20 .80
1365 @item @emph{Interface}: @tab @code{subroutine omp_set_schedule(kind, chunk_size)}
1366 @item @tab @code{integer(kind=omp_sched_kind) kind}
1367 @item @tab @code{integer chunk_size}
1370 @item @emph{See also}:
1371 @ref{omp_get_schedule}
1374 @item @emph{Reference}:
1375 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.12.
1380 @node omp_set_teams_thread_limit
1381 @section @code{omp_set_teams_thread_limit} -- Set upper thread limit for teams construct
1383 @item @emph{Description}:
1384 Specifies the upper bound for number of threads that will be available
1385 for each team created by the teams construct which does not specify a
1386 @code{thread_limit} clause. The argument of
1387 @code{omp_set_teams_thread_limit} shall be a positive integer.
1390 @multitable @columnfractions .20 .80
1391 @item @emph{Prototype}: @tab @code{void omp_set_teams_thread_limit(int thread_limit);}
1394 @item @emph{Fortran}:
1395 @multitable @columnfractions .20 .80
1396 @item @emph{Interface}: @tab @code{subroutine omp_set_teams_thread_limit(thread_limit)}
1397 @item @tab @code{integer, intent(in) :: thread_limit}
1400 @item @emph{See also}:
1401 @ref{OMP_TEAMS_THREAD_LIMIT}, @ref{omp_get_teams_thread_limit}, @ref{omp_get_thread_limit}
1403 @item @emph{Reference}:
1404 @uref{https://www.openmp.org, OpenMP specification v5.1}, Section 3.4.5.
1410 @section @code{omp_init_lock} -- Initialize simple lock
1412 @item @emph{Description}:
1413 Initialize a simple lock. After initialization, the lock is in
1417 @multitable @columnfractions .20 .80
1418 @item @emph{Prototype}: @tab @code{void omp_init_lock(omp_lock_t *lock);}
1421 @item @emph{Fortran}:
1422 @multitable @columnfractions .20 .80
1423 @item @emph{Interface}: @tab @code{subroutine omp_init_lock(svar)}
1424 @item @tab @code{integer(omp_lock_kind), intent(out) :: svar}
1427 @item @emph{See also}:
1428 @ref{omp_destroy_lock}
1430 @item @emph{Reference}:
1431 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.3.1.
1437 @section @code{omp_set_lock} -- Wait for and set simple lock
1439 @item @emph{Description}:
1440 Before setting a simple lock, the lock variable must be initialized by
1441 @code{omp_init_lock}. The calling thread is blocked until the lock
1442 is available. If the lock is already held by the current thread,
1446 @multitable @columnfractions .20 .80
1447 @item @emph{Prototype}: @tab @code{void omp_set_lock(omp_lock_t *lock);}
1450 @item @emph{Fortran}:
1451 @multitable @columnfractions .20 .80
1452 @item @emph{Interface}: @tab @code{subroutine omp_set_lock(svar)}
1453 @item @tab @code{integer(omp_lock_kind), intent(inout) :: svar}
1456 @item @emph{See also}:
1457 @ref{omp_init_lock}, @ref{omp_test_lock}, @ref{omp_unset_lock}
1459 @item @emph{Reference}:
1460 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.3.4.
1466 @section @code{omp_test_lock} -- Test and set simple lock if available
1468 @item @emph{Description}:
1469 Before setting a simple lock, the lock variable must be initialized by
1470 @code{omp_init_lock}. Contrary to @code{omp_set_lock}, @code{omp_test_lock}
1471 does not block if the lock is not available. This function returns
1472 @code{true} upon success, @code{false} otherwise. Here, @code{true} and
1473 @code{false} represent their language-specific counterparts.
1476 @multitable @columnfractions .20 .80
1477 @item @emph{Prototype}: @tab @code{int omp_test_lock(omp_lock_t *lock);}
1480 @item @emph{Fortran}:
1481 @multitable @columnfractions .20 .80
1482 @item @emph{Interface}: @tab @code{logical function omp_test_lock(svar)}
1483 @item @tab @code{integer(omp_lock_kind), intent(inout) :: svar}
1486 @item @emph{See also}:
1487 @ref{omp_init_lock}, @ref{omp_set_lock}, @ref{omp_set_lock}
1489 @item @emph{Reference}:
1490 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.3.6.
1495 @node omp_unset_lock
1496 @section @code{omp_unset_lock} -- Unset simple lock
1498 @item @emph{Description}:
1499 A simple lock about to be unset must have been locked by @code{omp_set_lock}
1500 or @code{omp_test_lock} before. In addition, the lock must be held by the
1501 thread calling @code{omp_unset_lock}. Then, the lock becomes unlocked. If one
1502 or more threads attempted to set the lock before, one of them is chosen to,
1503 again, set the lock to itself.
1506 @multitable @columnfractions .20 .80
1507 @item @emph{Prototype}: @tab @code{void omp_unset_lock(omp_lock_t *lock);}
1510 @item @emph{Fortran}:
1511 @multitable @columnfractions .20 .80
1512 @item @emph{Interface}: @tab @code{subroutine omp_unset_lock(svar)}
1513 @item @tab @code{integer(omp_lock_kind), intent(inout) :: svar}
1516 @item @emph{See also}:
1517 @ref{omp_set_lock}, @ref{omp_test_lock}
1519 @item @emph{Reference}:
1520 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.3.5.
1525 @node omp_destroy_lock
1526 @section @code{omp_destroy_lock} -- Destroy simple lock
1528 @item @emph{Description}:
1529 Destroy a simple lock. In order to be destroyed, a simple lock must be
1530 in the unlocked state.
1533 @multitable @columnfractions .20 .80
1534 @item @emph{Prototype}: @tab @code{void omp_destroy_lock(omp_lock_t *lock);}
1537 @item @emph{Fortran}:
1538 @multitable @columnfractions .20 .80
1539 @item @emph{Interface}: @tab @code{subroutine omp_destroy_lock(svar)}
1540 @item @tab @code{integer(omp_lock_kind), intent(inout) :: svar}
1543 @item @emph{See also}:
1546 @item @emph{Reference}:
1547 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.3.3.
1552 @node omp_init_nest_lock
1553 @section @code{omp_init_nest_lock} -- Initialize nested lock
1555 @item @emph{Description}:
1556 Initialize a nested lock. After initialization, the lock is in
1557 an unlocked state and the nesting count is set to zero.
1560 @multitable @columnfractions .20 .80
1561 @item @emph{Prototype}: @tab @code{void omp_init_nest_lock(omp_nest_lock_t *lock);}
1564 @item @emph{Fortran}:
1565 @multitable @columnfractions .20 .80
1566 @item @emph{Interface}: @tab @code{subroutine omp_init_nest_lock(nvar)}
1567 @item @tab @code{integer(omp_nest_lock_kind), intent(out) :: nvar}
1570 @item @emph{See also}:
1571 @ref{omp_destroy_nest_lock}
1573 @item @emph{Reference}:
1574 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.3.1.
1578 @node omp_set_nest_lock
1579 @section @code{omp_set_nest_lock} -- Wait for and set nested lock
1581 @item @emph{Description}:
1582 Before setting a nested lock, the lock variable must be initialized by
1583 @code{omp_init_nest_lock}. The calling thread is blocked until the lock
1584 is available. If the lock is already held by the current thread, the
1585 nesting count for the lock is incremented.
1588 @multitable @columnfractions .20 .80
1589 @item @emph{Prototype}: @tab @code{void omp_set_nest_lock(omp_nest_lock_t *lock);}
1592 @item @emph{Fortran}:
1593 @multitable @columnfractions .20 .80
1594 @item @emph{Interface}: @tab @code{subroutine omp_set_nest_lock(nvar)}
1595 @item @tab @code{integer(omp_nest_lock_kind), intent(inout) :: nvar}
1598 @item @emph{See also}:
1599 @ref{omp_init_nest_lock}, @ref{omp_unset_nest_lock}
1601 @item @emph{Reference}:
1602 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.3.4.
1607 @node omp_test_nest_lock
1608 @section @code{omp_test_nest_lock} -- Test and set nested lock if available
1610 @item @emph{Description}:
1611 Before setting a nested lock, the lock variable must be initialized by
1612 @code{omp_init_nest_lock}. Contrary to @code{omp_set_nest_lock},
1613 @code{omp_test_nest_lock} does not block if the lock is not available.
1614 If the lock is already held by the current thread, the new nesting count
1615 is returned. Otherwise, the return value equals zero.
1618 @multitable @columnfractions .20 .80
1619 @item @emph{Prototype}: @tab @code{int omp_test_nest_lock(omp_nest_lock_t *lock);}
1622 @item @emph{Fortran}:
1623 @multitable @columnfractions .20 .80
1624 @item @emph{Interface}: @tab @code{logical function omp_test_nest_lock(nvar)}
1625 @item @tab @code{integer(omp_nest_lock_kind), intent(inout) :: nvar}
1629 @item @emph{See also}:
1630 @ref{omp_init_lock}, @ref{omp_set_lock}, @ref{omp_set_lock}
1632 @item @emph{Reference}:
1633 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.3.6.
1638 @node omp_unset_nest_lock
1639 @section @code{omp_unset_nest_lock} -- Unset nested lock
1641 @item @emph{Description}:
1642 A nested lock about to be unset must have been locked by @code{omp_set_nested_lock}
1643 or @code{omp_test_nested_lock} before. In addition, the lock must be held by the
1644 thread calling @code{omp_unset_nested_lock}. If the nesting count drops to zero, the
1645 lock becomes unlocked. If one ore more threads attempted to set the lock before,
1646 one of them is chosen to, again, set the lock to itself.
1649 @multitable @columnfractions .20 .80
1650 @item @emph{Prototype}: @tab @code{void omp_unset_nest_lock(omp_nest_lock_t *lock);}
1653 @item @emph{Fortran}:
1654 @multitable @columnfractions .20 .80
1655 @item @emph{Interface}: @tab @code{subroutine omp_unset_nest_lock(nvar)}
1656 @item @tab @code{integer(omp_nest_lock_kind), intent(inout) :: nvar}
1659 @item @emph{See also}:
1660 @ref{omp_set_nest_lock}
1662 @item @emph{Reference}:
1663 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.3.5.
1668 @node omp_destroy_nest_lock
1669 @section @code{omp_destroy_nest_lock} -- Destroy nested lock
1671 @item @emph{Description}:
1672 Destroy a nested lock. In order to be destroyed, a nested lock must be
1673 in the unlocked state and its nesting count must equal zero.
1676 @multitable @columnfractions .20 .80
1677 @item @emph{Prototype}: @tab @code{void omp_destroy_nest_lock(omp_nest_lock_t *);}
1680 @item @emph{Fortran}:
1681 @multitable @columnfractions .20 .80
1682 @item @emph{Interface}: @tab @code{subroutine omp_destroy_nest_lock(nvar)}
1683 @item @tab @code{integer(omp_nest_lock_kind), intent(inout) :: nvar}
1686 @item @emph{See also}:
1689 @item @emph{Reference}:
1690 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.3.3.
1696 @section @code{omp_get_wtick} -- Get timer precision
1698 @item @emph{Description}:
1699 Gets the timer precision, i.e., the number of seconds between two
1700 successive clock ticks.
1703 @multitable @columnfractions .20 .80
1704 @item @emph{Prototype}: @tab @code{double omp_get_wtick(void);}
1707 @item @emph{Fortran}:
1708 @multitable @columnfractions .20 .80
1709 @item @emph{Interface}: @tab @code{double precision function omp_get_wtick()}
1712 @item @emph{See also}:
1715 @item @emph{Reference}:
1716 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.4.2.
1722 @section @code{omp_get_wtime} -- Elapsed wall clock time
1724 @item @emph{Description}:
1725 Elapsed wall clock time in seconds. The time is measured per thread, no
1726 guarantee can be made that two distinct threads measure the same time.
1727 Time is measured from some "time in the past", which is an arbitrary time
1728 guaranteed not to change during the execution of the program.
1731 @multitable @columnfractions .20 .80
1732 @item @emph{Prototype}: @tab @code{double omp_get_wtime(void);}
1735 @item @emph{Fortran}:
1736 @multitable @columnfractions .20 .80
1737 @item @emph{Interface}: @tab @code{double precision function omp_get_wtime()}
1740 @item @emph{See also}:
1743 @item @emph{Reference}:
1744 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.4.1.
1749 @node omp_fulfill_event
1750 @section @code{omp_fulfill_event} -- Fulfill and destroy an OpenMP event
1752 @item @emph{Description}:
1753 Fulfill the event associated with the event handle argument. Currently, it
1754 is only used to fulfill events generated by detach clauses on task
1755 constructs - the effect of fulfilling the event is to allow the task to
1758 The result of calling @code{omp_fulfill_event} with an event handle other
1759 than that generated by a detach clause is undefined. Calling it with an
1760 event handle that has already been fulfilled is also undefined.
1763 @multitable @columnfractions .20 .80
1764 @item @emph{Prototype}: @tab @code{void omp_fulfill_event(omp_event_handle_t event);}
1767 @item @emph{Fortran}:
1768 @multitable @columnfractions .20 .80
1769 @item @emph{Interface}: @tab @code{subroutine omp_fulfill_event(event)}
1770 @item @tab @code{integer (kind=omp_event_handle_kind) :: event}
1773 @item @emph{Reference}:
1774 @uref{https://www.openmp.org, OpenMP specification v5.0}, Section 3.5.1.
1779 @c ---------------------------------------------------------------------
1780 @c OpenMP Environment Variables
1781 @c ---------------------------------------------------------------------
1783 @node Environment Variables
1784 @chapter OpenMP Environment Variables
1786 The environment variables which beginning with @env{OMP_} are defined by
1787 section 4 of the OpenMP specification in version 4.5, while those
1788 beginning with @env{GOMP_} are GNU extensions.
1791 * OMP_CANCELLATION:: Set whether cancellation is activated
1792 * OMP_DISPLAY_ENV:: Show OpenMP version and environment variables
1793 * OMP_DEFAULT_DEVICE:: Set the device used in target regions
1794 * OMP_DYNAMIC:: Dynamic adjustment of threads
1795 * OMP_MAX_ACTIVE_LEVELS:: Set the maximum number of nested parallel regions
1796 * OMP_MAX_TASK_PRIORITY:: Set the maximum task priority value
1797 * OMP_NESTED:: Nested parallel regions
1798 * OMP_NUM_TEAMS:: Specifies the number of teams to use by teams region
1799 * OMP_NUM_THREADS:: Specifies the number of threads to use
1800 * OMP_PROC_BIND:: Whether theads may be moved between CPUs
1801 * OMP_PLACES:: Specifies on which CPUs the theads should be placed
1802 * OMP_STACKSIZE:: Set default thread stack size
1803 * OMP_SCHEDULE:: How threads are scheduled
1804 * OMP_TARGET_OFFLOAD:: Controls offloading behaviour
1805 * OMP_TEAMS_THREAD_LIMIT:: Set the maximum number of threads imposed by teams
1806 * OMP_THREAD_LIMIT:: Set the maximum number of threads
1807 * OMP_WAIT_POLICY:: How waiting threads are handled
1808 * GOMP_CPU_AFFINITY:: Bind threads to specific CPUs
1809 * GOMP_DEBUG:: Enable debugging output
1810 * GOMP_STACKSIZE:: Set default thread stack size
1811 * GOMP_SPINCOUNT:: Set the busy-wait spin count
1812 * GOMP_RTEMS_THREAD_POOLS:: Set the RTEMS specific thread pools
1816 @node OMP_CANCELLATION
1817 @section @env{OMP_CANCELLATION} -- Set whether cancellation is activated
1818 @cindex Environment Variable
1820 @item @emph{Description}:
1821 If set to @code{TRUE}, the cancellation is activated. If set to @code{FALSE} or
1822 if unset, cancellation is disabled and the @code{cancel} construct is ignored.
1824 @item @emph{See also}:
1825 @ref{omp_get_cancellation}
1827 @item @emph{Reference}:
1828 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 4.11
1833 @node OMP_DISPLAY_ENV
1834 @section @env{OMP_DISPLAY_ENV} -- Show OpenMP version and environment variables
1835 @cindex Environment Variable
1837 @item @emph{Description}:
1838 If set to @code{TRUE}, the OpenMP version number and the values
1839 associated with the OpenMP environment variables are printed to @code{stderr}.
1840 If set to @code{VERBOSE}, it additionally shows the value of the environment
1841 variables which are GNU extensions. If undefined or set to @code{FALSE},
1842 this information will not be shown.
1845 @item @emph{Reference}:
1846 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 4.12
1851 @node OMP_DEFAULT_DEVICE
1852 @section @env{OMP_DEFAULT_DEVICE} -- Set the device used in target regions
1853 @cindex Environment Variable
1855 @item @emph{Description}:
1856 Set to choose the device which is used in a @code{target} region, unless the
1857 value is overridden by @code{omp_set_default_device} or by a @code{device}
1858 clause. The value shall be the nonnegative device number. If no device with
1859 the given device number exists, the code is executed on the host. If unset,
1860 device number 0 will be used.
1863 @item @emph{See also}:
1864 @ref{omp_get_default_device}, @ref{omp_set_default_device},
1866 @item @emph{Reference}:
1867 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 4.13
1873 @section @env{OMP_DYNAMIC} -- Dynamic adjustment of threads
1874 @cindex Environment Variable
1876 @item @emph{Description}:
1877 Enable or disable the dynamic adjustment of the number of threads
1878 within a team. The value of this environment variable shall be
1879 @code{TRUE} or @code{FALSE}. If undefined, dynamic adjustment is
1880 disabled by default.
1882 @item @emph{See also}:
1883 @ref{omp_set_dynamic}
1885 @item @emph{Reference}:
1886 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 4.3
1891 @node OMP_MAX_ACTIVE_LEVELS
1892 @section @env{OMP_MAX_ACTIVE_LEVELS} -- Set the maximum number of nested parallel regions
1893 @cindex Environment Variable
1895 @item @emph{Description}:
1896 Specifies the initial value for the maximum number of nested parallel
1897 regions. The value of this variable shall be a positive integer.
1898 If undefined, then if @env{OMP_NESTED} is defined and set to true, or
1899 if @env{OMP_NUM_THREADS} or @env{OMP_PROC_BIND} are defined and set to
1900 a list with more than one item, the maximum number of nested parallel
1901 regions will be initialized to the largest number supported, otherwise
1902 it will be set to one.
1904 @item @emph{See also}:
1905 @ref{omp_set_max_active_levels}, @ref{OMP_NESTED}
1907 @item @emph{Reference}:
1908 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 4.9
1913 @node OMP_MAX_TASK_PRIORITY
1914 @section @env{OMP_MAX_TASK_PRIORITY} -- Set the maximum priority
1915 number that can be set for a task.
1916 @cindex Environment Variable
1918 @item @emph{Description}:
1919 Specifies the initial value for the maximum priority value that can be
1920 set for a task. The value of this variable shall be a non-negative
1921 integer, and zero is allowed. If undefined, the default priority is
1924 @item @emph{See also}:
1925 @ref{omp_get_max_task_priority}
1927 @item @emph{Reference}:
1928 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 4.14
1934 @section @env{OMP_NESTED} -- Nested parallel regions
1935 @cindex Environment Variable
1936 @cindex Implementation specific setting
1938 @item @emph{Description}:
1939 Enable or disable nested parallel regions, i.e., whether team members
1940 are allowed to create new teams. The value of this environment variable
1941 shall be @code{TRUE} or @code{FALSE}. If set to @code{TRUE}, the number
1942 of maximum active nested regions supported will by default be set to the
1943 maximum supported, otherwise it will be set to one. If
1944 @env{OMP_MAX_ACTIVE_LEVELS} is defined, its setting will override this
1945 setting. If both are undefined, nested parallel regions are enabled if
1946 @env{OMP_NUM_THREADS} or @env{OMP_PROC_BINDS} are defined to a list with
1947 more than one item, otherwise they are disabled by default.
1949 @item @emph{See also}:
1950 @ref{omp_set_max_active_levels}, @ref{omp_set_nested}
1952 @item @emph{Reference}:
1953 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 4.6
1959 @section @env{OMP_NUM_TEAMS} -- Specifies the number of teams to use by teams region
1960 @cindex Environment Variable
1962 @item @emph{Description}:
1963 Specifies the upper bound for number of teams to use in teams regions
1964 without explicit @code{num_teams} clause. The value of this variable shall
1965 be a positive integer. If undefined it defaults to 0 which means
1966 implementation defined upper bound.
1968 @item @emph{See also}:
1969 @ref{omp_set_num_teams}
1971 @item @emph{Reference}:
1972 @uref{https://www.openmp.org, OpenMP specification v5.1}, Section 6.23
1977 @node OMP_NUM_THREADS
1978 @section @env{OMP_NUM_THREADS} -- Specifies the number of threads to use
1979 @cindex Environment Variable
1980 @cindex Implementation specific setting
1982 @item @emph{Description}:
1983 Specifies the default number of threads to use in parallel regions. The
1984 value of this variable shall be a comma-separated list of positive integers;
1985 the value specifies the number of threads to use for the corresponding nested
1986 level. Specifying more than one item in the list will automatically enable
1987 nesting by default. If undefined one thread per CPU is used.
1989 @item @emph{See also}:
1990 @ref{omp_set_num_threads}, @ref{OMP_NESTED}
1992 @item @emph{Reference}:
1993 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 4.2
1999 @section @env{OMP_PROC_BIND} -- Whether theads may be moved between CPUs
2000 @cindex Environment Variable
2002 @item @emph{Description}:
2003 Specifies whether threads may be moved between processors. If set to
2004 @code{TRUE}, OpenMP theads should not be moved; if set to @code{FALSE}
2005 they may be moved. Alternatively, a comma separated list with the
2006 values @code{PRIMARY}, @code{MASTER}, @code{CLOSE} and @code{SPREAD} can
2007 be used to specify the thread affinity policy for the corresponding nesting
2008 level. With @code{PRIMARY} and @code{MASTER} the worker threads are in the
2009 same place partition as the primary thread. With @code{CLOSE} those are
2010 kept close to the primary thread in contiguous place partitions. And
2011 with @code{SPREAD} a sparse distribution
2012 across the place partitions is used. Specifying more than one item in the
2013 list will automatically enable nesting by default.
2015 When undefined, @env{OMP_PROC_BIND} defaults to @code{TRUE} when
2016 @env{OMP_PLACES} or @env{GOMP_CPU_AFFINITY} is set and @code{FALSE} otherwise.
2018 @item @emph{See also}:
2019 @ref{omp_get_proc_bind}, @ref{GOMP_CPU_AFFINITY},
2020 @ref{OMP_NESTED}, @ref{OMP_PLACES}
2022 @item @emph{Reference}:
2023 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 4.4
2029 @section @env{OMP_PLACES} -- Specifies on which CPUs the theads should be placed
2030 @cindex Environment Variable
2032 @item @emph{Description}:
2033 The thread placement can be either specified using an abstract name or by an
2034 explicit list of the places. The abstract names @code{threads}, @code{cores}
2035 and @code{sockets} can be optionally followed by a positive number in
2036 parentheses, which denotes the how many places shall be created. With
2037 @code{threads} each place corresponds to a single hardware thread; @code{cores}
2038 to a single core with the corresponding number of hardware threads; and with
2039 @code{sockets} the place corresponds to a single socket. The resulting
2040 placement can be shown by setting the @env{OMP_DISPLAY_ENV} environment
2043 Alternatively, the placement can be specified explicitly as comma-separated
2044 list of places. A place is specified by set of nonnegative numbers in curly
2045 braces, denoting the denoting the hardware threads. The hardware threads
2046 belonging to a place can either be specified as comma-separated list of
2047 nonnegative thread numbers or using an interval. Multiple places can also be
2048 either specified by a comma-separated list of places or by an interval. To
2049 specify an interval, a colon followed by the count is placed after after
2050 the hardware thread number or the place. Optionally, the length can be
2051 followed by a colon and the stride number -- otherwise a unit stride is
2052 assumed. For instance, the following specifies the same places list:
2053 @code{"@{0,1,2@}, @{3,4,6@}, @{7,8,9@}, @{10,11,12@}"};
2054 @code{"@{0:3@}, @{3:3@}, @{7:3@}, @{10:3@}"}; and @code{"@{0:2@}:4:3"}.
2056 If @env{OMP_PLACES} and @env{GOMP_CPU_AFFINITY} are unset and
2057 @env{OMP_PROC_BIND} is either unset or @code{false}, threads may be moved
2058 between CPUs following no placement policy.
2060 @item @emph{See also}:
2061 @ref{OMP_PROC_BIND}, @ref{GOMP_CPU_AFFINITY}, @ref{omp_get_proc_bind},
2062 @ref{OMP_DISPLAY_ENV}
2064 @item @emph{Reference}:
2065 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 4.5
2071 @section @env{OMP_STACKSIZE} -- Set default thread stack size
2072 @cindex Environment Variable
2074 @item @emph{Description}:
2075 Set the default thread stack size in kilobytes, unless the number
2076 is suffixed by @code{B}, @code{K}, @code{M} or @code{G}, in which
2077 case the size is, respectively, in bytes, kilobytes, megabytes
2078 or gigabytes. This is different from @code{pthread_attr_setstacksize}
2079 which gets the number of bytes as an argument. If the stack size cannot
2080 be set due to system constraints, an error is reported and the initial
2081 stack size is left unchanged. If undefined, the stack size is system
2084 @item @emph{Reference}:
2085 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 4.7
2091 @section @env{OMP_SCHEDULE} -- How threads are scheduled
2092 @cindex Environment Variable
2093 @cindex Implementation specific setting
2095 @item @emph{Description}:
2096 Allows to specify @code{schedule type} and @code{chunk size}.
2097 The value of the variable shall have the form: @code{type[,chunk]} where
2098 @code{type} is one of @code{static}, @code{dynamic}, @code{guided} or @code{auto}
2099 The optional @code{chunk} size shall be a positive integer. If undefined,
2100 dynamic scheduling and a chunk size of 1 is used.
2102 @item @emph{See also}:
2103 @ref{omp_set_schedule}
2105 @item @emph{Reference}:
2106 @uref{https://www.openmp.org, OpenMP specification v4.5}, Sections 2.7.1.1 and 4.1
2111 @node OMP_TARGET_OFFLOAD
2112 @section @env{OMP_TARGET_OFFLOAD} -- Controls offloading behaviour
2113 @cindex Environment Variable
2114 @cindex Implementation specific setting
2116 @item @emph{Description}:
2117 Specifies the behaviour with regard to offloading code to a device. This
2118 variable can be set to one of three values - @code{MANDATORY}, @code{DISABLED}
2121 If set to @code{MANDATORY}, the program will terminate with an error if
2122 the offload device is not present or is not supported. If set to
2123 @code{DISABLED}, then offloading is disabled and all code will run on the
2124 host. If set to @code{DEFAULT}, the program will try offloading to the
2125 device first, then fall back to running code on the host if it cannot.
2127 If undefined, then the program will behave as if @code{DEFAULT} was set.
2129 @item @emph{Reference}:
2130 @uref{https://www.openmp.org, OpenMP specification v5.0}, Section 6.17
2135 @node OMP_TEAMS_THREAD_LIMIT
2136 @section @env{OMP_TEAMS_THREAD_LIMIT} -- Set the maximum number of threads imposed by teams
2137 @cindex Environment Variable
2139 @item @emph{Description}:
2140 Specifies an upper bound for the number of threads to use by each contention
2141 group created by a teams construct without explicit @code{thread_limit}
2142 clause. The value of this variable shall be a positive integer. If undefined,
2143 the value of 0 is used which stands for an implementation defined upper
2146 @item @emph{See also}:
2147 @ref{OMP_THREAD_LIMIT}, @ref{omp_set_teams_thread_limit}
2149 @item @emph{Reference}:
2150 @uref{https://www.openmp.org, OpenMP specification v5.1}, Section 6.24
2155 @node OMP_THREAD_LIMIT
2156 @section @env{OMP_THREAD_LIMIT} -- Set the maximum number of threads
2157 @cindex Environment Variable
2159 @item @emph{Description}:
2160 Specifies the number of threads to use for the whole program. The
2161 value of this variable shall be a positive integer. If undefined,
2162 the number of threads is not limited.
2164 @item @emph{See also}:
2165 @ref{OMP_NUM_THREADS}, @ref{omp_get_thread_limit}
2167 @item @emph{Reference}:
2168 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 4.10
2173 @node OMP_WAIT_POLICY
2174 @section @env{OMP_WAIT_POLICY} -- How waiting threads are handled
2175 @cindex Environment Variable
2177 @item @emph{Description}:
2178 Specifies whether waiting threads should be active or passive. If
2179 the value is @code{PASSIVE}, waiting threads should not consume CPU
2180 power while waiting; while the value is @code{ACTIVE} specifies that
2181 they should. If undefined, threads wait actively for a short time
2182 before waiting passively.
2184 @item @emph{See also}:
2185 @ref{GOMP_SPINCOUNT}
2187 @item @emph{Reference}:
2188 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 4.8
2193 @node GOMP_CPU_AFFINITY
2194 @section @env{GOMP_CPU_AFFINITY} -- Bind threads to specific CPUs
2195 @cindex Environment Variable
2197 @item @emph{Description}:
2198 Binds threads to specific CPUs. The variable should contain a space-separated
2199 or comma-separated list of CPUs. This list may contain different kinds of
2200 entries: either single CPU numbers in any order, a range of CPUs (M-N)
2201 or a range with some stride (M-N:S). CPU numbers are zero based. For example,
2202 @code{GOMP_CPU_AFFINITY="0 3 1-2 4-15:2"} will bind the initial thread
2203 to CPU 0, the second to CPU 3, the third to CPU 1, the fourth to
2204 CPU 2, the fifth to CPU 4, the sixth through tenth to CPUs 6, 8, 10, 12,
2205 and 14 respectively and then start assigning back from the beginning of
2206 the list. @code{GOMP_CPU_AFFINITY=0} binds all threads to CPU 0.
2208 There is no libgomp library routine to determine whether a CPU affinity
2209 specification is in effect. As a workaround, language-specific library
2210 functions, e.g., @code{getenv} in C or @code{GET_ENVIRONMENT_VARIABLE} in
2211 Fortran, may be used to query the setting of the @code{GOMP_CPU_AFFINITY}
2212 environment variable. A defined CPU affinity on startup cannot be changed
2213 or disabled during the runtime of the application.
2215 If both @env{GOMP_CPU_AFFINITY} and @env{OMP_PROC_BIND} are set,
2216 @env{OMP_PROC_BIND} has a higher precedence. If neither has been set and
2217 @env{OMP_PROC_BIND} is unset, or when @env{OMP_PROC_BIND} is set to
2218 @code{FALSE}, the host system will handle the assignment of threads to CPUs.
2220 @item @emph{See also}:
2221 @ref{OMP_PLACES}, @ref{OMP_PROC_BIND}
2227 @section @env{GOMP_DEBUG} -- Enable debugging output
2228 @cindex Environment Variable
2230 @item @emph{Description}:
2231 Enable debugging output. The variable should be set to @code{0}
2232 (disabled, also the default if not set), or @code{1} (enabled).
2234 If enabled, some debugging output will be printed during execution.
2235 This is currently not specified in more detail, and subject to change.
2240 @node GOMP_STACKSIZE
2241 @section @env{GOMP_STACKSIZE} -- Set default thread stack size
2242 @cindex Environment Variable
2243 @cindex Implementation specific setting
2245 @item @emph{Description}:
2246 Set the default thread stack size in kilobytes. This is different from
2247 @code{pthread_attr_setstacksize} which gets the number of bytes as an
2248 argument. If the stack size cannot be set due to system constraints, an
2249 error is reported and the initial stack size is left unchanged. If undefined,
2250 the stack size is system dependent.
2252 @item @emph{See also}:
2255 @item @emph{Reference}:
2256 @uref{https://gcc.gnu.org/ml/gcc-patches/2006-06/msg00493.html,
2257 GCC Patches Mailinglist},
2258 @uref{https://gcc.gnu.org/ml/gcc-patches/2006-06/msg00496.html,
2259 GCC Patches Mailinglist}
2264 @node GOMP_SPINCOUNT
2265 @section @env{GOMP_SPINCOUNT} -- Set the busy-wait spin count
2266 @cindex Environment Variable
2267 @cindex Implementation specific setting
2269 @item @emph{Description}:
2270 Determines how long a threads waits actively with consuming CPU power
2271 before waiting passively without consuming CPU power. The value may be
2272 either @code{INFINITE}, @code{INFINITY} to always wait actively or an
2273 integer which gives the number of spins of the busy-wait loop. The
2274 integer may optionally be followed by the following suffixes acting
2275 as multiplication factors: @code{k} (kilo, thousand), @code{M} (mega,
2276 million), @code{G} (giga, billion), or @code{T} (tera, trillion).
2277 If undefined, 0 is used when @env{OMP_WAIT_POLICY} is @code{PASSIVE},
2278 300,000 is used when @env{OMP_WAIT_POLICY} is undefined and
2279 30 billion is used when @env{OMP_WAIT_POLICY} is @code{ACTIVE}.
2280 If there are more OpenMP threads than available CPUs, 1000 and 100
2281 spins are used for @env{OMP_WAIT_POLICY} being @code{ACTIVE} or
2282 undefined, respectively; unless the @env{GOMP_SPINCOUNT} is lower
2283 or @env{OMP_WAIT_POLICY} is @code{PASSIVE}.
2285 @item @emph{See also}:
2286 @ref{OMP_WAIT_POLICY}
2291 @node GOMP_RTEMS_THREAD_POOLS
2292 @section @env{GOMP_RTEMS_THREAD_POOLS} -- Set the RTEMS specific thread pools
2293 @cindex Environment Variable
2294 @cindex Implementation specific setting
2296 @item @emph{Description}:
2297 This environment variable is only used on the RTEMS real-time operating system.
2298 It determines the scheduler instance specific thread pools. The format for
2299 @env{GOMP_RTEMS_THREAD_POOLS} is a list of optional
2300 @code{<thread-pool-count>[$<priority>]@@<scheduler-name>} configurations
2301 separated by @code{:} where:
2303 @item @code{<thread-pool-count>} is the thread pool count for this scheduler
2305 @item @code{$<priority>} is an optional priority for the worker threads of a
2306 thread pool according to @code{pthread_setschedparam}. In case a priority
2307 value is omitted, then a worker thread will inherit the priority of the OpenMP
2308 primary thread that created it. The priority of the worker thread is not
2309 changed after creation, even if a new OpenMP primary thread using the worker has
2310 a different priority.
2311 @item @code{@@<scheduler-name>} is the scheduler instance name according to the
2312 RTEMS application configuration.
2314 In case no thread pool configuration is specified for a scheduler instance,
2315 then each OpenMP primary thread of this scheduler instance will use its own
2316 dynamically allocated thread pool. To limit the worker thread count of the
2317 thread pools, each OpenMP primary thread must call @code{omp_set_num_threads}.
2318 @item @emph{Example}:
2319 Lets suppose we have three scheduler instances @code{IO}, @code{WRK0}, and
2320 @code{WRK1} with @env{GOMP_RTEMS_THREAD_POOLS} set to
2321 @code{"1@@WRK0:3$4@@WRK1"}. Then there are no thread pool restrictions for
2322 scheduler instance @code{IO}. In the scheduler instance @code{WRK0} there is
2323 one thread pool available. Since no priority is specified for this scheduler
2324 instance, the worker thread inherits the priority of the OpenMP primary thread
2325 that created it. In the scheduler instance @code{WRK1} there are three thread
2326 pools available and their worker threads run at priority four.
2331 @c ---------------------------------------------------------------------
2333 @c ---------------------------------------------------------------------
2335 @node Enabling OpenACC
2336 @chapter Enabling OpenACC
2338 To activate the OpenACC extensions for C/C++ and Fortran, the compile-time
2339 flag @option{-fopenacc} must be specified. This enables the OpenACC directive
2340 @code{#pragma acc} in C/C++ and @code{!$acc} directives in free form,
2341 @code{c$acc}, @code{*$acc} and @code{!$acc} directives in fixed form,
2342 @code{!$} conditional compilation sentinels in free form and @code{c$},
2343 @code{*$} and @code{!$} sentinels in fixed form, for Fortran. The flag also
2344 arranges for automatic linking of the OpenACC runtime library
2345 (@ref{OpenACC Runtime Library Routines}).
2347 See @uref{https://gcc.gnu.org/wiki/OpenACC} for more information.
2349 A complete description of all OpenACC directives accepted may be found in
2350 the @uref{https://www.openacc.org, OpenACC} Application Programming
2351 Interface manual, version 2.6.
2355 @c ---------------------------------------------------------------------
2356 @c OpenACC Runtime Library Routines
2357 @c ---------------------------------------------------------------------
2359 @node OpenACC Runtime Library Routines
2360 @chapter OpenACC Runtime Library Routines
2362 The runtime routines described here are defined by section 3 of the OpenACC
2363 specifications in version 2.6.
2364 They have C linkage, and do not throw exceptions.
2365 Generally, they are available only for the host, with the exception of
2366 @code{acc_on_device}, which is available for both the host and the
2367 acceleration device.
2370 * acc_get_num_devices:: Get number of devices for the given device
2372 * acc_set_device_type:: Set type of device accelerator to use.
2373 * acc_get_device_type:: Get type of device accelerator to be used.
2374 * acc_set_device_num:: Set device number to use.
2375 * acc_get_device_num:: Get device number to be used.
2376 * acc_get_property:: Get device property.
2377 * acc_async_test:: Tests for completion of a specific asynchronous
2379 * acc_async_test_all:: Tests for completion of all asynchronous
2381 * acc_wait:: Wait for completion of a specific asynchronous
2383 * acc_wait_all:: Waits for completion of all asynchronous
2385 * acc_wait_all_async:: Wait for completion of all asynchronous
2387 * acc_wait_async:: Wait for completion of asynchronous operations.
2388 * acc_init:: Initialize runtime for a specific device type.
2389 * acc_shutdown:: Shuts down the runtime for a specific device
2391 * acc_on_device:: Whether executing on a particular device
2392 * acc_malloc:: Allocate device memory.
2393 * acc_free:: Free device memory.
2394 * acc_copyin:: Allocate device memory and copy host memory to
2396 * acc_present_or_copyin:: If the data is not present on the device,
2397 allocate device memory and copy from host
2399 * acc_create:: Allocate device memory and map it to host
2401 * acc_present_or_create:: If the data is not present on the device,
2402 allocate device memory and map it to host
2404 * acc_copyout:: Copy device memory to host memory.
2405 * acc_delete:: Free device memory.
2406 * acc_update_device:: Update device memory from mapped host memory.
2407 * acc_update_self:: Update host memory from mapped device memory.
2408 * acc_map_data:: Map previously allocated device memory to host
2410 * acc_unmap_data:: Unmap device memory from host memory.
2411 * acc_deviceptr:: Get device pointer associated with specific
2413 * acc_hostptr:: Get host pointer associated with specific
2415 * acc_is_present:: Indicate whether host variable / array is
2417 * acc_memcpy_to_device:: Copy host memory to device memory.
2418 * acc_memcpy_from_device:: Copy device memory to host memory.
2419 * acc_attach:: Let device pointer point to device-pointer target.
2420 * acc_detach:: Let device pointer point to host-pointer target.
2422 API routines for target platforms.
2424 * acc_get_current_cuda_device:: Get CUDA device handle.
2425 * acc_get_current_cuda_context::Get CUDA context handle.
2426 * acc_get_cuda_stream:: Get CUDA stream handle.
2427 * acc_set_cuda_stream:: Set CUDA stream handle.
2429 API routines for the OpenACC Profiling Interface.
2431 * acc_prof_register:: Register callbacks.
2432 * acc_prof_unregister:: Unregister callbacks.
2433 * acc_prof_lookup:: Obtain inquiry functions.
2434 * acc_register_library:: Library registration.
2439 @node acc_get_num_devices
2440 @section @code{acc_get_num_devices} -- Get number of devices for given device type
2442 @item @emph{Description}
2443 This function returns a value indicating the number of devices available
2444 for the device type specified in @var{devicetype}.
2447 @multitable @columnfractions .20 .80
2448 @item @emph{Prototype}: @tab @code{int acc_get_num_devices(acc_device_t devicetype);}
2451 @item @emph{Fortran}:
2452 @multitable @columnfractions .20 .80
2453 @item @emph{Interface}: @tab @code{integer function acc_get_num_devices(devicetype)}
2454 @item @tab @code{integer(kind=acc_device_kind) devicetype}
2457 @item @emph{Reference}:
2458 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
2464 @node acc_set_device_type
2465 @section @code{acc_set_device_type} -- Set type of device accelerator to use.
2467 @item @emph{Description}
2468 This function indicates to the runtime library which device type, specified
2469 in @var{devicetype}, to use when executing a parallel or kernels region.
2472 @multitable @columnfractions .20 .80
2473 @item @emph{Prototype}: @tab @code{acc_set_device_type(acc_device_t devicetype);}
2476 @item @emph{Fortran}:
2477 @multitable @columnfractions .20 .80
2478 @item @emph{Interface}: @tab @code{subroutine acc_set_device_type(devicetype)}
2479 @item @tab @code{integer(kind=acc_device_kind) devicetype}
2482 @item @emph{Reference}:
2483 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
2489 @node acc_get_device_type
2490 @section @code{acc_get_device_type} -- Get type of device accelerator to be used.
2492 @item @emph{Description}
2493 This function returns what device type will be used when executing a
2494 parallel or kernels region.
2496 This function returns @code{acc_device_none} if
2497 @code{acc_get_device_type} is called from
2498 @code{acc_ev_device_init_start}, @code{acc_ev_device_init_end}
2499 callbacks of the OpenACC Profiling Interface (@ref{OpenACC Profiling
2500 Interface}), that is, if the device is currently being initialized.
2503 @multitable @columnfractions .20 .80
2504 @item @emph{Prototype}: @tab @code{acc_device_t acc_get_device_type(void);}
2507 @item @emph{Fortran}:
2508 @multitable @columnfractions .20 .80
2509 @item @emph{Interface}: @tab @code{function acc_get_device_type(void)}
2510 @item @tab @code{integer(kind=acc_device_kind) acc_get_device_type}
2513 @item @emph{Reference}:
2514 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
2520 @node acc_set_device_num
2521 @section @code{acc_set_device_num} -- Set device number to use.
2523 @item @emph{Description}
2524 This function will indicate to the runtime which device number,
2525 specified by @var{devicenum}, associated with the specified device
2526 type @var{devicetype}.
2529 @multitable @columnfractions .20 .80
2530 @item @emph{Prototype}: @tab @code{acc_set_device_num(int devicenum, acc_device_t devicetype);}
2533 @item @emph{Fortran}:
2534 @multitable @columnfractions .20 .80
2535 @item @emph{Interface}: @tab @code{subroutine acc_set_device_num(devicenum, devicetype)}
2536 @item @tab @code{integer devicenum}
2537 @item @tab @code{integer(kind=acc_device_kind) devicetype}
2540 @item @emph{Reference}:
2541 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
2547 @node acc_get_device_num
2548 @section @code{acc_get_device_num} -- Get device number to be used.
2550 @item @emph{Description}
2551 This function returns which device number associated with the specified device
2552 type @var{devicetype}, will be used when executing a parallel or kernels
2556 @multitable @columnfractions .20 .80
2557 @item @emph{Prototype}: @tab @code{int acc_get_device_num(acc_device_t devicetype);}
2560 @item @emph{Fortran}:
2561 @multitable @columnfractions .20 .80
2562 @item @emph{Interface}: @tab @code{function acc_get_device_num(devicetype)}
2563 @item @tab @code{integer(kind=acc_device_kind) devicetype}
2564 @item @tab @code{integer acc_get_device_num}
2567 @item @emph{Reference}:
2568 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
2574 @node acc_get_property
2575 @section @code{acc_get_property} -- Get device property.
2576 @cindex acc_get_property
2577 @cindex acc_get_property_string
2579 @item @emph{Description}
2580 These routines return the value of the specified @var{property} for the
2581 device being queried according to @var{devicenum} and @var{devicetype}.
2582 Integer-valued and string-valued properties are returned by
2583 @code{acc_get_property} and @code{acc_get_property_string} respectively.
2584 The Fortran @code{acc_get_property_string} subroutine returns the string
2585 retrieved in its fourth argument while the remaining entry points are
2586 functions, which pass the return value as their result.
2588 Note for Fortran, only: the OpenACC technical committee corrected and, hence,
2589 modified the interface introduced in OpenACC 2.6. The kind-value parameter
2590 @code{acc_device_property} has been renamed to @code{acc_device_property_kind}
2591 for consistency and the return type of the @code{acc_get_property} function is
2592 now a @code{c_size_t} integer instead of a @code{acc_device_property} integer.
2593 The parameter @code{acc_device_property} will continue to be provided,
2594 but might be removed in a future version of GCC.
2597 @multitable @columnfractions .20 .80
2598 @item @emph{Prototype}: @tab @code{size_t acc_get_property(int devicenum, acc_device_t devicetype, acc_device_property_t property);}
2599 @item @emph{Prototype}: @tab @code{const char *acc_get_property_string(int devicenum, acc_device_t devicetype, acc_device_property_t property);}
2602 @item @emph{Fortran}:
2603 @multitable @columnfractions .20 .80
2604 @item @emph{Interface}: @tab @code{function acc_get_property(devicenum, devicetype, property)}
2605 @item @emph{Interface}: @tab @code{subroutine acc_get_property_string(devicenum, devicetype, property, string)}
2606 @item @tab @code{use ISO_C_Binding, only: c_size_t}
2607 @item @tab @code{integer devicenum}
2608 @item @tab @code{integer(kind=acc_device_kind) devicetype}
2609 @item @tab @code{integer(kind=acc_device_property_kind) property}
2610 @item @tab @code{integer(kind=c_size_t) acc_get_property}
2611 @item @tab @code{character(*) string}
2614 @item @emph{Reference}:
2615 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
2621 @node acc_async_test
2622 @section @code{acc_async_test} -- Test for completion of a specific asynchronous operation.
2624 @item @emph{Description}
2625 This function tests for completion of the asynchronous operation specified
2626 in @var{arg}. In C/C++, a non-zero value will be returned to indicate
2627 the specified asynchronous operation has completed. While Fortran will return
2628 a @code{true}. If the asynchronous operation has not completed, C/C++ returns
2629 a zero and Fortran returns a @code{false}.
2632 @multitable @columnfractions .20 .80
2633 @item @emph{Prototype}: @tab @code{int acc_async_test(int arg);}
2636 @item @emph{Fortran}:
2637 @multitable @columnfractions .20 .80
2638 @item @emph{Interface}: @tab @code{function acc_async_test(arg)}
2639 @item @tab @code{integer(kind=acc_handle_kind) arg}
2640 @item @tab @code{logical acc_async_test}
2643 @item @emph{Reference}:
2644 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
2650 @node acc_async_test_all
2651 @section @code{acc_async_test_all} -- Tests for completion of all asynchronous operations.
2653 @item @emph{Description}
2654 This function tests for completion of all asynchronous operations.
2655 In C/C++, a non-zero value will be returned to indicate all asynchronous
2656 operations have completed. While Fortran will return a @code{true}. If
2657 any asynchronous operation has not completed, C/C++ returns a zero and
2658 Fortran returns a @code{false}.
2661 @multitable @columnfractions .20 .80
2662 @item @emph{Prototype}: @tab @code{int acc_async_test_all(void);}
2665 @item @emph{Fortran}:
2666 @multitable @columnfractions .20 .80
2667 @item @emph{Interface}: @tab @code{function acc_async_test()}
2668 @item @tab @code{logical acc_get_device_num}
2671 @item @emph{Reference}:
2672 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
2679 @section @code{acc_wait} -- Wait for completion of a specific asynchronous operation.
2681 @item @emph{Description}
2682 This function waits for completion of the asynchronous operation
2683 specified in @var{arg}.
2686 @multitable @columnfractions .20 .80
2687 @item @emph{Prototype}: @tab @code{acc_wait(arg);}
2688 @item @emph{Prototype (OpenACC 1.0 compatibility)}: @tab @code{acc_async_wait(arg);}
2691 @item @emph{Fortran}:
2692 @multitable @columnfractions .20 .80
2693 @item @emph{Interface}: @tab @code{subroutine acc_wait(arg)}
2694 @item @tab @code{integer(acc_handle_kind) arg}
2695 @item @emph{Interface (OpenACC 1.0 compatibility)}: @tab @code{subroutine acc_async_wait(arg)}
2696 @item @tab @code{integer(acc_handle_kind) arg}
2699 @item @emph{Reference}:
2700 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
2707 @section @code{acc_wait_all} -- Waits for completion of all asynchronous operations.
2709 @item @emph{Description}
2710 This function waits for the completion of all asynchronous operations.
2713 @multitable @columnfractions .20 .80
2714 @item @emph{Prototype}: @tab @code{acc_wait_all(void);}
2715 @item @emph{Prototype (OpenACC 1.0 compatibility)}: @tab @code{acc_async_wait_all(void);}
2718 @item @emph{Fortran}:
2719 @multitable @columnfractions .20 .80
2720 @item @emph{Interface}: @tab @code{subroutine acc_wait_all()}
2721 @item @emph{Interface (OpenACC 1.0 compatibility)}: @tab @code{subroutine acc_async_wait_all()}
2724 @item @emph{Reference}:
2725 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
2731 @node acc_wait_all_async
2732 @section @code{acc_wait_all_async} -- Wait for completion of all asynchronous operations.
2734 @item @emph{Description}
2735 This function enqueues a wait operation on the queue @var{async} for any
2736 and all asynchronous operations that have been previously enqueued on
2740 @multitable @columnfractions .20 .80
2741 @item @emph{Prototype}: @tab @code{acc_wait_all_async(int async);}
2744 @item @emph{Fortran}:
2745 @multitable @columnfractions .20 .80
2746 @item @emph{Interface}: @tab @code{subroutine acc_wait_all_async(async)}
2747 @item @tab @code{integer(acc_handle_kind) async}
2750 @item @emph{Reference}:
2751 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
2757 @node acc_wait_async
2758 @section @code{acc_wait_async} -- Wait for completion of asynchronous operations.
2760 @item @emph{Description}
2761 This function enqueues a wait operation on queue @var{async} for any and all
2762 asynchronous operations enqueued on queue @var{arg}.
2765 @multitable @columnfractions .20 .80
2766 @item @emph{Prototype}: @tab @code{acc_wait_async(int arg, int async);}
2769 @item @emph{Fortran}:
2770 @multitable @columnfractions .20 .80
2771 @item @emph{Interface}: @tab @code{subroutine acc_wait_async(arg, async)}
2772 @item @tab @code{integer(acc_handle_kind) arg, async}
2775 @item @emph{Reference}:
2776 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
2783 @section @code{acc_init} -- Initialize runtime for a specific device type.
2785 @item @emph{Description}
2786 This function initializes the runtime for the device type specified in
2790 @multitable @columnfractions .20 .80
2791 @item @emph{Prototype}: @tab @code{acc_init(acc_device_t devicetype);}
2794 @item @emph{Fortran}:
2795 @multitable @columnfractions .20 .80
2796 @item @emph{Interface}: @tab @code{subroutine acc_init(devicetype)}
2797 @item @tab @code{integer(acc_device_kind) devicetype}
2800 @item @emph{Reference}:
2801 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
2808 @section @code{acc_shutdown} -- Shuts down the runtime for a specific device type.
2810 @item @emph{Description}
2811 This function shuts down the runtime for the device type specified in
2815 @multitable @columnfractions .20 .80
2816 @item @emph{Prototype}: @tab @code{acc_shutdown(acc_device_t devicetype);}
2819 @item @emph{Fortran}:
2820 @multitable @columnfractions .20 .80
2821 @item @emph{Interface}: @tab @code{subroutine acc_shutdown(devicetype)}
2822 @item @tab @code{integer(acc_device_kind) devicetype}
2825 @item @emph{Reference}:
2826 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
2833 @section @code{acc_on_device} -- Whether executing on a particular device
2835 @item @emph{Description}:
2836 This function returns whether the program is executing on a particular
2837 device specified in @var{devicetype}. In C/C++ a non-zero value is
2838 returned to indicate the device is executing on the specified device type.
2839 In Fortran, @code{true} will be returned. If the program is not executing
2840 on the specified device type C/C++ will return a zero, while Fortran will
2841 return @code{false}.
2844 @multitable @columnfractions .20 .80
2845 @item @emph{Prototype}: @tab @code{acc_on_device(acc_device_t devicetype);}
2848 @item @emph{Fortran}:
2849 @multitable @columnfractions .20 .80
2850 @item @emph{Interface}: @tab @code{function acc_on_device(devicetype)}
2851 @item @tab @code{integer(acc_device_kind) devicetype}
2852 @item @tab @code{logical acc_on_device}
2856 @item @emph{Reference}:
2857 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
2864 @section @code{acc_malloc} -- Allocate device memory.
2866 @item @emph{Description}
2867 This function allocates @var{len} bytes of device memory. It returns
2868 the device address of the allocated memory.
2871 @multitable @columnfractions .20 .80
2872 @item @emph{Prototype}: @tab @code{d_void* acc_malloc(size_t len);}
2875 @item @emph{Reference}:
2876 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
2883 @section @code{acc_free} -- Free device memory.
2885 @item @emph{Description}
2886 Free previously allocated device memory at the device address @code{a}.
2889 @multitable @columnfractions .20 .80
2890 @item @emph{Prototype}: @tab @code{acc_free(d_void *a);}
2893 @item @emph{Reference}:
2894 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
2901 @section @code{acc_copyin} -- Allocate device memory and copy host memory to it.
2903 @item @emph{Description}
2904 In C/C++, this function allocates @var{len} bytes of device memory
2905 and maps it to the specified host address in @var{a}. The device
2906 address of the newly allocated device memory is returned.
2908 In Fortran, two (2) forms are supported. In the first form, @var{a} specifies
2909 a contiguous array section. The second form @var{a} specifies a
2910 variable or array element and @var{len} specifies the length in bytes.
2913 @multitable @columnfractions .20 .80
2914 @item @emph{Prototype}: @tab @code{void *acc_copyin(h_void *a, size_t len);}
2915 @item @emph{Prototype}: @tab @code{void *acc_copyin_async(h_void *a, size_t len, int async);}
2918 @item @emph{Fortran}:
2919 @multitable @columnfractions .20 .80
2920 @item @emph{Interface}: @tab @code{subroutine acc_copyin(a)}
2921 @item @tab @code{type, dimension(:[,:]...) :: a}
2922 @item @emph{Interface}: @tab @code{subroutine acc_copyin(a, len)}
2923 @item @tab @code{type, dimension(:[,:]...) :: a}
2924 @item @tab @code{integer len}
2925 @item @emph{Interface}: @tab @code{subroutine acc_copyin_async(a, async)}
2926 @item @tab @code{type, dimension(:[,:]...) :: a}
2927 @item @tab @code{integer(acc_handle_kind) :: async}
2928 @item @emph{Interface}: @tab @code{subroutine acc_copyin_async(a, len, async)}
2929 @item @tab @code{type, dimension(:[,:]...) :: a}
2930 @item @tab @code{integer len}
2931 @item @tab @code{integer(acc_handle_kind) :: async}
2934 @item @emph{Reference}:
2935 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
2941 @node acc_present_or_copyin
2942 @section @code{acc_present_or_copyin} -- If the data is not present on the device, allocate device memory and copy from host memory.
2944 @item @emph{Description}
2945 This function tests if the host data specified by @var{a} and of length
2946 @var{len} is present or not. If it is not present, then device memory
2947 will be allocated and the host memory copied. The device address of
2948 the newly allocated device memory is returned.
2950 In Fortran, two (2) forms are supported. In the first form, @var{a} specifies
2951 a contiguous array section. The second form @var{a} specifies a variable or
2952 array element and @var{len} specifies the length in bytes.
2954 Note that @code{acc_present_or_copyin} and @code{acc_pcopyin} exist for
2955 backward compatibility with OpenACC 2.0; use @ref{acc_copyin} instead.
2958 @multitable @columnfractions .20 .80
2959 @item @emph{Prototype}: @tab @code{void *acc_present_or_copyin(h_void *a, size_t len);}
2960 @item @emph{Prototype}: @tab @code{void *acc_pcopyin(h_void *a, size_t len);}
2963 @item @emph{Fortran}:
2964 @multitable @columnfractions .20 .80
2965 @item @emph{Interface}: @tab @code{subroutine acc_present_or_copyin(a)}
2966 @item @tab @code{type, dimension(:[,:]...) :: a}
2967 @item @emph{Interface}: @tab @code{subroutine acc_present_or_copyin(a, len)}
2968 @item @tab @code{type, dimension(:[,:]...) :: a}
2969 @item @tab @code{integer len}
2970 @item @emph{Interface}: @tab @code{subroutine acc_pcopyin(a)}
2971 @item @tab @code{type, dimension(:[,:]...) :: a}
2972 @item @emph{Interface}: @tab @code{subroutine acc_pcopyin(a, len)}
2973 @item @tab @code{type, dimension(:[,:]...) :: a}
2974 @item @tab @code{integer len}
2977 @item @emph{Reference}:
2978 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
2985 @section @code{acc_create} -- Allocate device memory and map it to host memory.
2987 @item @emph{Description}
2988 This function allocates device memory and maps it to host memory specified
2989 by the host address @var{a} with a length of @var{len} bytes. In C/C++,
2990 the function returns the device address of the allocated device memory.
2992 In Fortran, two (2) forms are supported. In the first form, @var{a} specifies
2993 a contiguous array section. The second form @var{a} specifies a variable or
2994 array element and @var{len} specifies the length in bytes.
2997 @multitable @columnfractions .20 .80
2998 @item @emph{Prototype}: @tab @code{void *acc_create(h_void *a, size_t len);}
2999 @item @emph{Prototype}: @tab @code{void *acc_create_async(h_void *a, size_t len, int async);}
3002 @item @emph{Fortran}:
3003 @multitable @columnfractions .20 .80
3004 @item @emph{Interface}: @tab @code{subroutine acc_create(a)}
3005 @item @tab @code{type, dimension(:[,:]...) :: a}
3006 @item @emph{Interface}: @tab @code{subroutine acc_create(a, len)}
3007 @item @tab @code{type, dimension(:[,:]...) :: a}
3008 @item @tab @code{integer len}
3009 @item @emph{Interface}: @tab @code{subroutine acc_create_async(a, async)}
3010 @item @tab @code{type, dimension(:[,:]...) :: a}
3011 @item @tab @code{integer(acc_handle_kind) :: async}
3012 @item @emph{Interface}: @tab @code{subroutine acc_create_async(a, len, async)}
3013 @item @tab @code{type, dimension(:[,:]...) :: a}
3014 @item @tab @code{integer len}
3015 @item @tab @code{integer(acc_handle_kind) :: async}
3018 @item @emph{Reference}:
3019 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3025 @node acc_present_or_create
3026 @section @code{acc_present_or_create} -- If the data is not present on the device, allocate device memory and map it to host memory.
3028 @item @emph{Description}
3029 This function tests if the host data specified by @var{a} and of length
3030 @var{len} is present or not. If it is not present, then device memory
3031 will be allocated and mapped to host memory. In C/C++, the device address
3032 of the newly allocated device memory is returned.
3034 In Fortran, two (2) forms are supported. In the first form, @var{a} specifies
3035 a contiguous array section. The second form @var{a} specifies a variable or
3036 array element and @var{len} specifies the length in bytes.
3038 Note that @code{acc_present_or_create} and @code{acc_pcreate} exist for
3039 backward compatibility with OpenACC 2.0; use @ref{acc_create} instead.
3042 @multitable @columnfractions .20 .80
3043 @item @emph{Prototype}: @tab @code{void *acc_present_or_create(h_void *a, size_t len)}
3044 @item @emph{Prototype}: @tab @code{void *acc_pcreate(h_void *a, size_t len)}
3047 @item @emph{Fortran}:
3048 @multitable @columnfractions .20 .80
3049 @item @emph{Interface}: @tab @code{subroutine acc_present_or_create(a)}
3050 @item @tab @code{type, dimension(:[,:]...) :: a}
3051 @item @emph{Interface}: @tab @code{subroutine acc_present_or_create(a, len)}
3052 @item @tab @code{type, dimension(:[,:]...) :: a}
3053 @item @tab @code{integer len}
3054 @item @emph{Interface}: @tab @code{subroutine acc_pcreate(a)}
3055 @item @tab @code{type, dimension(:[,:]...) :: a}
3056 @item @emph{Interface}: @tab @code{subroutine acc_pcreate(a, len)}
3057 @item @tab @code{type, dimension(:[,:]...) :: a}
3058 @item @tab @code{integer len}
3061 @item @emph{Reference}:
3062 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3069 @section @code{acc_copyout} -- Copy device memory to host memory.
3071 @item @emph{Description}
3072 This function copies mapped device memory to host memory which is specified
3073 by host address @var{a} for a length @var{len} bytes in C/C++.
3075 In Fortran, two (2) forms are supported. In the first form, @var{a} specifies
3076 a contiguous array section. The second form @var{a} specifies a variable or
3077 array element and @var{len} specifies the length in bytes.
3080 @multitable @columnfractions .20 .80
3081 @item @emph{Prototype}: @tab @code{acc_copyout(h_void *a, size_t len);}
3082 @item @emph{Prototype}: @tab @code{acc_copyout_async(h_void *a, size_t len, int async);}
3083 @item @emph{Prototype}: @tab @code{acc_copyout_finalize(h_void *a, size_t len);}
3084 @item @emph{Prototype}: @tab @code{acc_copyout_finalize_async(h_void *a, size_t len, int async);}
3087 @item @emph{Fortran}:
3088 @multitable @columnfractions .20 .80
3089 @item @emph{Interface}: @tab @code{subroutine acc_copyout(a)}
3090 @item @tab @code{type, dimension(:[,:]...) :: a}
3091 @item @emph{Interface}: @tab @code{subroutine acc_copyout(a, len)}
3092 @item @tab @code{type, dimension(:[,:]...) :: a}
3093 @item @tab @code{integer len}
3094 @item @emph{Interface}: @tab @code{subroutine acc_copyout_async(a, async)}
3095 @item @tab @code{type, dimension(:[,:]...) :: a}
3096 @item @tab @code{integer(acc_handle_kind) :: async}
3097 @item @emph{Interface}: @tab @code{subroutine acc_copyout_async(a, len, async)}
3098 @item @tab @code{type, dimension(:[,:]...) :: a}
3099 @item @tab @code{integer len}
3100 @item @tab @code{integer(acc_handle_kind) :: async}
3101 @item @emph{Interface}: @tab @code{subroutine acc_copyout_finalize(a)}
3102 @item @tab @code{type, dimension(:[,:]...) :: a}
3103 @item @emph{Interface}: @tab @code{subroutine acc_copyout_finalize(a, len)}
3104 @item @tab @code{type, dimension(:[,:]...) :: a}
3105 @item @tab @code{integer len}
3106 @item @emph{Interface}: @tab @code{subroutine acc_copyout_finalize_async(a, async)}
3107 @item @tab @code{type, dimension(:[,:]...) :: a}
3108 @item @tab @code{integer(acc_handle_kind) :: async}
3109 @item @emph{Interface}: @tab @code{subroutine acc_copyout_finalize_async(a, len, async)}
3110 @item @tab @code{type, dimension(:[,:]...) :: a}
3111 @item @tab @code{integer len}
3112 @item @tab @code{integer(acc_handle_kind) :: async}
3115 @item @emph{Reference}:
3116 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3123 @section @code{acc_delete} -- Free device memory.
3125 @item @emph{Description}
3126 This function frees previously allocated device memory specified by
3127 the device address @var{a} and the length of @var{len} bytes.
3129 In Fortran, two (2) forms are supported. In the first form, @var{a} specifies
3130 a contiguous array section. The second form @var{a} specifies a variable or
3131 array element and @var{len} specifies the length in bytes.
3134 @multitable @columnfractions .20 .80
3135 @item @emph{Prototype}: @tab @code{acc_delete(h_void *a, size_t len);}
3136 @item @emph{Prototype}: @tab @code{acc_delete_async(h_void *a, size_t len, int async);}
3137 @item @emph{Prototype}: @tab @code{acc_delete_finalize(h_void *a, size_t len);}
3138 @item @emph{Prototype}: @tab @code{acc_delete_finalize_async(h_void *a, size_t len, int async);}
3141 @item @emph{Fortran}:
3142 @multitable @columnfractions .20 .80
3143 @item @emph{Interface}: @tab @code{subroutine acc_delete(a)}
3144 @item @tab @code{type, dimension(:[,:]...) :: a}
3145 @item @emph{Interface}: @tab @code{subroutine acc_delete(a, len)}
3146 @item @tab @code{type, dimension(:[,:]...) :: a}
3147 @item @tab @code{integer len}
3148 @item @emph{Interface}: @tab @code{subroutine acc_delete_async(a, async)}
3149 @item @tab @code{type, dimension(:[,:]...) :: a}
3150 @item @tab @code{integer(acc_handle_kind) :: async}
3151 @item @emph{Interface}: @tab @code{subroutine acc_delete_async(a, len, async)}
3152 @item @tab @code{type, dimension(:[,:]...) :: a}
3153 @item @tab @code{integer len}
3154 @item @tab @code{integer(acc_handle_kind) :: async}
3155 @item @emph{Interface}: @tab @code{subroutine acc_delete_finalize(a)}
3156 @item @tab @code{type, dimension(:[,:]...) :: a}
3157 @item @emph{Interface}: @tab @code{subroutine acc_delete_finalize(a, len)}
3158 @item @tab @code{type, dimension(:[,:]...) :: a}
3159 @item @tab @code{integer len}
3160 @item @emph{Interface}: @tab @code{subroutine acc_delete_async_finalize(a, async)}
3161 @item @tab @code{type, dimension(:[,:]...) :: a}
3162 @item @tab @code{integer(acc_handle_kind) :: async}
3163 @item @emph{Interface}: @tab @code{subroutine acc_delete_async_finalize(a, len, async)}
3164 @item @tab @code{type, dimension(:[,:]...) :: a}
3165 @item @tab @code{integer len}
3166 @item @tab @code{integer(acc_handle_kind) :: async}
3169 @item @emph{Reference}:
3170 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3176 @node acc_update_device
3177 @section @code{acc_update_device} -- Update device memory from mapped host memory.
3179 @item @emph{Description}
3180 This function updates the device copy from the previously mapped host memory.
3181 The host memory is specified with the host address @var{a} and a length of
3184 In Fortran, two (2) forms are supported. In the first form, @var{a} specifies
3185 a contiguous array section. The second form @var{a} specifies a variable or
3186 array element and @var{len} specifies the length in bytes.
3189 @multitable @columnfractions .20 .80
3190 @item @emph{Prototype}: @tab @code{acc_update_device(h_void *a, size_t len);}
3191 @item @emph{Prototype}: @tab @code{acc_update_device(h_void *a, size_t len, async);}
3194 @item @emph{Fortran}:
3195 @multitable @columnfractions .20 .80
3196 @item @emph{Interface}: @tab @code{subroutine acc_update_device(a)}
3197 @item @tab @code{type, dimension(:[,:]...) :: a}
3198 @item @emph{Interface}: @tab @code{subroutine acc_update_device(a, len)}
3199 @item @tab @code{type, dimension(:[,:]...) :: a}
3200 @item @tab @code{integer len}
3201 @item @emph{Interface}: @tab @code{subroutine acc_update_device_async(a, async)}
3202 @item @tab @code{type, dimension(:[,:]...) :: a}
3203 @item @tab @code{integer(acc_handle_kind) :: async}
3204 @item @emph{Interface}: @tab @code{subroutine acc_update_device_async(a, len, async)}
3205 @item @tab @code{type, dimension(:[,:]...) :: a}
3206 @item @tab @code{integer len}
3207 @item @tab @code{integer(acc_handle_kind) :: async}
3210 @item @emph{Reference}:
3211 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3217 @node acc_update_self
3218 @section @code{acc_update_self} -- Update host memory from mapped device memory.
3220 @item @emph{Description}
3221 This function updates the host copy from the previously mapped device memory.
3222 The host memory is specified with the host address @var{a} and a length of
3225 In Fortran, two (2) forms are supported. In the first form, @var{a} specifies
3226 a contiguous array section. The second form @var{a} specifies a variable or
3227 array element and @var{len} specifies the length in bytes.
3230 @multitable @columnfractions .20 .80
3231 @item @emph{Prototype}: @tab @code{acc_update_self(h_void *a, size_t len);}
3232 @item @emph{Prototype}: @tab @code{acc_update_self_async(h_void *a, size_t len, int async);}
3235 @item @emph{Fortran}:
3236 @multitable @columnfractions .20 .80
3237 @item @emph{Interface}: @tab @code{subroutine acc_update_self(a)}
3238 @item @tab @code{type, dimension(:[,:]...) :: a}
3239 @item @emph{Interface}: @tab @code{subroutine acc_update_self(a, len)}
3240 @item @tab @code{type, dimension(:[,:]...) :: a}
3241 @item @tab @code{integer len}
3242 @item @emph{Interface}: @tab @code{subroutine acc_update_self_async(a, async)}
3243 @item @tab @code{type, dimension(:[,:]...) :: a}
3244 @item @tab @code{integer(acc_handle_kind) :: async}
3245 @item @emph{Interface}: @tab @code{subroutine acc_update_self_async(a, len, async)}
3246 @item @tab @code{type, dimension(:[,:]...) :: a}
3247 @item @tab @code{integer len}
3248 @item @tab @code{integer(acc_handle_kind) :: async}
3251 @item @emph{Reference}:
3252 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3259 @section @code{acc_map_data} -- Map previously allocated device memory to host memory.
3261 @item @emph{Description}
3262 This function maps previously allocated device and host memory. The device
3263 memory is specified with the device address @var{d}. The host memory is
3264 specified with the host address @var{h} and a length of @var{len}.
3267 @multitable @columnfractions .20 .80
3268 @item @emph{Prototype}: @tab @code{acc_map_data(h_void *h, d_void *d, size_t len);}
3271 @item @emph{Reference}:
3272 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3278 @node acc_unmap_data
3279 @section @code{acc_unmap_data} -- Unmap device memory from host memory.
3281 @item @emph{Description}
3282 This function unmaps previously mapped device and host memory. The latter
3283 specified by @var{h}.
3286 @multitable @columnfractions .20 .80
3287 @item @emph{Prototype}: @tab @code{acc_unmap_data(h_void *h);}
3290 @item @emph{Reference}:
3291 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3298 @section @code{acc_deviceptr} -- Get device pointer associated with specific host address.
3300 @item @emph{Description}
3301 This function returns the device address that has been mapped to the
3302 host address specified by @var{h}.
3305 @multitable @columnfractions .20 .80
3306 @item @emph{Prototype}: @tab @code{void *acc_deviceptr(h_void *h);}
3309 @item @emph{Reference}:
3310 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3317 @section @code{acc_hostptr} -- Get host pointer associated with specific device address.
3319 @item @emph{Description}
3320 This function returns the host address that has been mapped to the
3321 device address specified by @var{d}.
3324 @multitable @columnfractions .20 .80
3325 @item @emph{Prototype}: @tab @code{void *acc_hostptr(d_void *d);}
3328 @item @emph{Reference}:
3329 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3335 @node acc_is_present
3336 @section @code{acc_is_present} -- Indicate whether host variable / array is present on device.
3338 @item @emph{Description}
3339 This function indicates whether the specified host address in @var{a} and a
3340 length of @var{len} bytes is present on the device. In C/C++, a non-zero
3341 value is returned to indicate the presence of the mapped memory on the
3342 device. A zero is returned to indicate the memory is not mapped on the
3345 In Fortran, two (2) forms are supported. In the first form, @var{a} specifies
3346 a contiguous array section. The second form @var{a} specifies a variable or
3347 array element and @var{len} specifies the length in bytes. If the host
3348 memory is mapped to device memory, then a @code{true} is returned. Otherwise,
3349 a @code{false} is return to indicate the mapped memory is not present.
3352 @multitable @columnfractions .20 .80
3353 @item @emph{Prototype}: @tab @code{int acc_is_present(h_void *a, size_t len);}
3356 @item @emph{Fortran}:
3357 @multitable @columnfractions .20 .80
3358 @item @emph{Interface}: @tab @code{function acc_is_present(a)}
3359 @item @tab @code{type, dimension(:[,:]...) :: a}
3360 @item @tab @code{logical acc_is_present}
3361 @item @emph{Interface}: @tab @code{function acc_is_present(a, len)}
3362 @item @tab @code{type, dimension(:[,:]...) :: a}
3363 @item @tab @code{integer len}
3364 @item @tab @code{logical acc_is_present}
3367 @item @emph{Reference}:
3368 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3374 @node acc_memcpy_to_device
3375 @section @code{acc_memcpy_to_device} -- Copy host memory to device memory.
3377 @item @emph{Description}
3378 This function copies host memory specified by host address of @var{src} to
3379 device memory specified by the device address @var{dest} for a length of
3383 @multitable @columnfractions .20 .80
3384 @item @emph{Prototype}: @tab @code{acc_memcpy_to_device(d_void *dest, h_void *src, size_t bytes);}
3387 @item @emph{Reference}:
3388 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3394 @node acc_memcpy_from_device
3395 @section @code{acc_memcpy_from_device} -- Copy device memory to host memory.
3397 @item @emph{Description}
3398 This function copies host memory specified by host address of @var{src} from
3399 device memory specified by the device address @var{dest} for a length of
3403 @multitable @columnfractions .20 .80
3404 @item @emph{Prototype}: @tab @code{acc_memcpy_from_device(d_void *dest, h_void *src, size_t bytes);}
3407 @item @emph{Reference}:
3408 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3415 @section @code{acc_attach} -- Let device pointer point to device-pointer target.
3417 @item @emph{Description}
3418 This function updates a pointer on the device from pointing to a host-pointer
3419 address to pointing to the corresponding device data.
3422 @multitable @columnfractions .20 .80
3423 @item @emph{Prototype}: @tab @code{acc_attach(h_void **ptr);}
3424 @item @emph{Prototype}: @tab @code{acc_attach_async(h_void **ptr, int async);}
3427 @item @emph{Reference}:
3428 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3435 @section @code{acc_detach} -- Let device pointer point to host-pointer target.
3437 @item @emph{Description}
3438 This function updates a pointer on the device from pointing to a device-pointer
3439 address to pointing to the corresponding host data.
3442 @multitable @columnfractions .20 .80
3443 @item @emph{Prototype}: @tab @code{acc_detach(h_void **ptr);}
3444 @item @emph{Prototype}: @tab @code{acc_detach_async(h_void **ptr, int async);}
3445 @item @emph{Prototype}: @tab @code{acc_detach_finalize(h_void **ptr);}
3446 @item @emph{Prototype}: @tab @code{acc_detach_finalize_async(h_void **ptr, int async);}
3449 @item @emph{Reference}:
3450 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3456 @node acc_get_current_cuda_device
3457 @section @code{acc_get_current_cuda_device} -- Get CUDA device handle.
3459 @item @emph{Description}
3460 This function returns the CUDA device handle. This handle is the same
3461 as used by the CUDA Runtime or Driver API's.
3464 @multitable @columnfractions .20 .80
3465 @item @emph{Prototype}: @tab @code{void *acc_get_current_cuda_device(void);}
3468 @item @emph{Reference}:
3469 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3475 @node acc_get_current_cuda_context
3476 @section @code{acc_get_current_cuda_context} -- Get CUDA context handle.
3478 @item @emph{Description}
3479 This function returns the CUDA context handle. This handle is the same
3480 as used by the CUDA Runtime or Driver API's.
3483 @multitable @columnfractions .20 .80
3484 @item @emph{Prototype}: @tab @code{void *acc_get_current_cuda_context(void);}
3487 @item @emph{Reference}:
3488 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3494 @node acc_get_cuda_stream
3495 @section @code{acc_get_cuda_stream} -- Get CUDA stream handle.
3497 @item @emph{Description}
3498 This function returns the CUDA stream handle for the queue @var{async}.
3499 This handle is the same as used by the CUDA Runtime or Driver API's.
3502 @multitable @columnfractions .20 .80
3503 @item @emph{Prototype}: @tab @code{void *acc_get_cuda_stream(int async);}
3506 @item @emph{Reference}:
3507 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3513 @node acc_set_cuda_stream
3514 @section @code{acc_set_cuda_stream} -- Set CUDA stream handle.
3516 @item @emph{Description}
3517 This function associates the stream handle specified by @var{stream} with
3518 the queue @var{async}.
3520 This cannot be used to change the stream handle associated with
3521 @code{acc_async_sync}.
3523 The return value is not specified.
3526 @multitable @columnfractions .20 .80
3527 @item @emph{Prototype}: @tab @code{int acc_set_cuda_stream(int async, void *stream);}
3530 @item @emph{Reference}:
3531 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3537 @node acc_prof_register
3538 @section @code{acc_prof_register} -- Register callbacks.
3540 @item @emph{Description}:
3541 This function registers callbacks.
3544 @multitable @columnfractions .20 .80
3545 @item @emph{Prototype}: @tab @code{void acc_prof_register (acc_event_t, acc_prof_callback, acc_register_t);}
3548 @item @emph{See also}:
3549 @ref{OpenACC Profiling Interface}
3551 @item @emph{Reference}:
3552 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3558 @node acc_prof_unregister
3559 @section @code{acc_prof_unregister} -- Unregister callbacks.
3561 @item @emph{Description}:
3562 This function unregisters callbacks.
3565 @multitable @columnfractions .20 .80
3566 @item @emph{Prototype}: @tab @code{void acc_prof_unregister (acc_event_t, acc_prof_callback, acc_register_t);}
3569 @item @emph{See also}:
3570 @ref{OpenACC Profiling Interface}
3572 @item @emph{Reference}:
3573 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3579 @node acc_prof_lookup
3580 @section @code{acc_prof_lookup} -- Obtain inquiry functions.
3582 @item @emph{Description}:
3583 Function to obtain inquiry functions.
3586 @multitable @columnfractions .20 .80
3587 @item @emph{Prototype}: @tab @code{acc_query_fn acc_prof_lookup (const char *);}
3590 @item @emph{See also}:
3591 @ref{OpenACC Profiling Interface}
3593 @item @emph{Reference}:
3594 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3600 @node acc_register_library
3601 @section @code{acc_register_library} -- Library registration.
3603 @item @emph{Description}:
3604 Function for library registration.
3607 @multitable @columnfractions .20 .80
3608 @item @emph{Prototype}: @tab @code{void acc_register_library (acc_prof_reg, acc_prof_reg, acc_prof_lookup_func);}
3611 @item @emph{See also}:
3612 @ref{OpenACC Profiling Interface}, @ref{ACC_PROFLIB}
3614 @item @emph{Reference}:
3615 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3621 @c ---------------------------------------------------------------------
3622 @c OpenACC Environment Variables
3623 @c ---------------------------------------------------------------------
3625 @node OpenACC Environment Variables
3626 @chapter OpenACC Environment Variables
3628 The variables @env{ACC_DEVICE_TYPE} and @env{ACC_DEVICE_NUM}
3629 are defined by section 4 of the OpenACC specification in version 2.0.
3630 The variable @env{ACC_PROFLIB}
3631 is defined by section 4 of the OpenACC specification in version 2.6.
3632 The variable @env{GCC_ACC_NOTIFY} is used for diagnostic purposes.
3643 @node ACC_DEVICE_TYPE
3644 @section @code{ACC_DEVICE_TYPE}
3646 @item @emph{Reference}:
3647 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3653 @node ACC_DEVICE_NUM
3654 @section @code{ACC_DEVICE_NUM}
3656 @item @emph{Reference}:
3657 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3664 @section @code{ACC_PROFLIB}
3666 @item @emph{See also}:
3667 @ref{acc_register_library}, @ref{OpenACC Profiling Interface}
3669 @item @emph{Reference}:
3670 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3676 @node GCC_ACC_NOTIFY
3677 @section @code{GCC_ACC_NOTIFY}
3679 @item @emph{Description}:
3680 Print debug information pertaining to the accelerator.
3685 @c ---------------------------------------------------------------------
3686 @c CUDA Streams Usage
3687 @c ---------------------------------------------------------------------
3689 @node CUDA Streams Usage
3690 @chapter CUDA Streams Usage
3692 This applies to the @code{nvptx} plugin only.
3694 The library provides elements that perform asynchronous movement of
3695 data and asynchronous operation of computing constructs. This
3696 asynchronous functionality is implemented by making use of CUDA
3697 streams@footnote{See "Stream Management" in "CUDA Driver API",
3698 TRM-06703-001, Version 5.5, for additional information}.
3700 The primary means by that the asynchronous functionality is accessed
3701 is through the use of those OpenACC directives which make use of the
3702 @code{async} and @code{wait} clauses. When the @code{async} clause is
3703 first used with a directive, it creates a CUDA stream. If an
3704 @code{async-argument} is used with the @code{async} clause, then the
3705 stream is associated with the specified @code{async-argument}.
3707 Following the creation of an association between a CUDA stream and the
3708 @code{async-argument} of an @code{async} clause, both the @code{wait}
3709 clause and the @code{wait} directive can be used. When either the
3710 clause or directive is used after stream creation, it creates a
3711 rendezvous point whereby execution waits until all operations
3712 associated with the @code{async-argument}, that is, stream, have
3715 Normally, the management of the streams that are created as a result of
3716 using the @code{async} clause, is done without any intervention by the
3717 caller. This implies the association between the @code{async-argument}
3718 and the CUDA stream will be maintained for the lifetime of the program.
3719 However, this association can be changed through the use of the library
3720 function @code{acc_set_cuda_stream}. When the function
3721 @code{acc_set_cuda_stream} is called, the CUDA stream that was
3722 originally associated with the @code{async} clause will be destroyed.
3723 Caution should be taken when changing the association as subsequent
3724 references to the @code{async-argument} refer to a different
3729 @c ---------------------------------------------------------------------
3730 @c OpenACC Library Interoperability
3731 @c ---------------------------------------------------------------------
3733 @node OpenACC Library Interoperability
3734 @chapter OpenACC Library Interoperability
3736 @section Introduction
3738 The OpenACC library uses the CUDA Driver API, and may interact with
3739 programs that use the Runtime library directly, or another library
3740 based on the Runtime library, e.g., CUBLAS@footnote{See section 2.26,
3741 "Interactions with the CUDA Driver API" in
3742 "CUDA Runtime API", Version 5.5, and section 2.27, "VDPAU
3743 Interoperability", in "CUDA Driver API", TRM-06703-001, Version 5.5,
3744 for additional information on library interoperability.}.
3745 This chapter describes the use cases and what changes are
3746 required in order to use both the OpenACC library and the CUBLAS and Runtime
3747 libraries within a program.
3749 @section First invocation: NVIDIA CUBLAS library API
3751 In this first use case (see below), a function in the CUBLAS library is called
3752 prior to any of the functions in the OpenACC library. More specifically, the
3753 function @code{cublasCreate()}.
3755 When invoked, the function initializes the library and allocates the
3756 hardware resources on the host and the device on behalf of the caller. Once
3757 the initialization and allocation has completed, a handle is returned to the
3758 caller. The OpenACC library also requires initialization and allocation of
3759 hardware resources. Since the CUBLAS library has already allocated the
3760 hardware resources for the device, all that is left to do is to initialize
3761 the OpenACC library and acquire the hardware resources on the host.
3763 Prior to calling the OpenACC function that initializes the library and
3764 allocate the host hardware resources, you need to acquire the device number
3765 that was allocated during the call to @code{cublasCreate()}. The invoking of the
3766 runtime library function @code{cudaGetDevice()} accomplishes this. Once
3767 acquired, the device number is passed along with the device type as
3768 parameters to the OpenACC library function @code{acc_set_device_num()}.
3770 Once the call to @code{acc_set_device_num()} has completed, the OpenACC
3771 library uses the context that was created during the call to
3772 @code{cublasCreate()}. In other words, both libraries will be sharing the
3776 /* Create the handle */
3777 s = cublasCreate(&h);
3778 if (s != CUBLAS_STATUS_SUCCESS)
3780 fprintf(stderr, "cublasCreate failed %d\n", s);
3784 /* Get the device number */
3785 e = cudaGetDevice(&dev);
3786 if (e != cudaSuccess)
3788 fprintf(stderr, "cudaGetDevice failed %d\n", e);
3792 /* Initialize OpenACC library and use device 'dev' */
3793 acc_set_device_num(dev, acc_device_nvidia);
3798 @section First invocation: OpenACC library API
3800 In this second use case (see below), a function in the OpenACC library is
3801 called prior to any of the functions in the CUBLAS library. More specificially,
3802 the function @code{acc_set_device_num()}.
3804 In the use case presented here, the function @code{acc_set_device_num()}
3805 is used to both initialize the OpenACC library and allocate the hardware
3806 resources on the host and the device. In the call to the function, the
3807 call parameters specify which device to use and what device
3808 type to use, i.e., @code{acc_device_nvidia}. It should be noted that this
3809 is but one method to initialize the OpenACC library and allocate the
3810 appropriate hardware resources. Other methods are available through the
3811 use of environment variables and these will be discussed in the next section.
3813 Once the call to @code{acc_set_device_num()} has completed, other OpenACC
3814 functions can be called as seen with multiple calls being made to
3815 @code{acc_copyin()}. In addition, calls can be made to functions in the
3816 CUBLAS library. In the use case a call to @code{cublasCreate()} is made
3817 subsequent to the calls to @code{acc_copyin()}.
3818 As seen in the previous use case, a call to @code{cublasCreate()}
3819 initializes the CUBLAS library and allocates the hardware resources on the
3820 host and the device. However, since the device has already been allocated,
3821 @code{cublasCreate()} will only initialize the CUBLAS library and allocate
3822 the appropriate hardware resources on the host. The context that was created
3823 as part of the OpenACC initialization is shared with the CUBLAS library,
3824 similarly to the first use case.
3829 acc_set_device_num(dev, acc_device_nvidia);
3831 /* Copy the first set to the device */
3832 d_X = acc_copyin(&h_X[0], N * sizeof (float));
3835 fprintf(stderr, "copyin error h_X\n");
3839 /* Copy the second set to the device */
3840 d_Y = acc_copyin(&h_Y1[0], N * sizeof (float));
3843 fprintf(stderr, "copyin error h_Y1\n");
3847 /* Create the handle */
3848 s = cublasCreate(&h);
3849 if (s != CUBLAS_STATUS_SUCCESS)
3851 fprintf(stderr, "cublasCreate failed %d\n", s);
3855 /* Perform saxpy using CUBLAS library function */
3856 s = cublasSaxpy(h, N, &alpha, d_X, 1, d_Y, 1);
3857 if (s != CUBLAS_STATUS_SUCCESS)
3859 fprintf(stderr, "cublasSaxpy failed %d\n", s);
3863 /* Copy the results from the device */
3864 acc_memcpy_from_device(&h_Y1[0], d_Y, N * sizeof (float));
3869 @section OpenACC library and environment variables
3871 There are two environment variables associated with the OpenACC library
3872 that may be used to control the device type and device number:
3873 @env{ACC_DEVICE_TYPE} and @env{ACC_DEVICE_NUM}, respectively. These two
3874 environment variables can be used as an alternative to calling
3875 @code{acc_set_device_num()}. As seen in the second use case, the device
3876 type and device number were specified using @code{acc_set_device_num()}.
3877 If however, the aforementioned environment variables were set, then the
3878 call to @code{acc_set_device_num()} would not be required.
3881 The use of the environment variables is only relevant when an OpenACC function
3882 is called prior to a call to @code{cudaCreate()}. If @code{cudaCreate()}
3883 is called prior to a call to an OpenACC function, then you must call
3884 @code{acc_set_device_num()}@footnote{More complete information
3885 about @env{ACC_DEVICE_TYPE} and @env{ACC_DEVICE_NUM} can be found in
3886 sections 4.1 and 4.2 of the @uref{https://www.openacc.org, OpenACC}
3887 Application Programming Interface”, Version 2.6.}
3891 @c ---------------------------------------------------------------------
3892 @c OpenACC Profiling Interface
3893 @c ---------------------------------------------------------------------
3895 @node OpenACC Profiling Interface
3896 @chapter OpenACC Profiling Interface
3898 @section Implementation Status and Implementation-Defined Behavior
3900 We're implementing the OpenACC Profiling Interface as defined by the
3901 OpenACC 2.6 specification. We're clarifying some aspects here as
3902 @emph{implementation-defined behavior}, while they're still under
3903 discussion within the OpenACC Technical Committee.
3905 This implementation is tuned to keep the performance impact as low as
3906 possible for the (very common) case that the Profiling Interface is
3907 not enabled. This is relevant, as the Profiling Interface affects all
3908 the @emph{hot} code paths (in the target code, not in the offloaded
3909 code). Users of the OpenACC Profiling Interface can be expected to
3910 understand that performance will be impacted to some degree once the
3911 Profiling Interface has gotten enabled: for example, because of the
3912 @emph{runtime} (libgomp) calling into a third-party @emph{library} for
3913 every event that has been registered.
3915 We're not yet accounting for the fact that @cite{OpenACC events may
3916 occur during event processing}.
3917 We just handle one case specially, as required by CUDA 9.0
3918 @command{nvprof}, that @code{acc_get_device_type}
3919 (@ref{acc_get_device_type})) may be called from
3920 @code{acc_ev_device_init_start}, @code{acc_ev_device_init_end}
3923 We're not yet implementing initialization via a
3924 @code{acc_register_library} function that is either statically linked
3925 in, or dynamically via @env{LD_PRELOAD}.
3926 Initialization via @code{acc_register_library} functions dynamically
3927 loaded via the @env{ACC_PROFLIB} environment variable does work, as
3928 does directly calling @code{acc_prof_register},
3929 @code{acc_prof_unregister}, @code{acc_prof_lookup}.
3931 As currently there are no inquiry functions defined, calls to
3932 @code{acc_prof_lookup} will always return @code{NULL}.
3934 There aren't separate @emph{start}, @emph{stop} events defined for the
3935 event types @code{acc_ev_create}, @code{acc_ev_delete},
3936 @code{acc_ev_alloc}, @code{acc_ev_free}. It's not clear if these
3937 should be triggered before or after the actual device-specific call is
3938 made. We trigger them after.
3940 Remarks about data provided to callbacks:
3944 @item @code{acc_prof_info.event_type}
3945 It's not clear if for @emph{nested} event callbacks (for example,
3946 @code{acc_ev_enqueue_launch_start} as part of a parent compute
3947 construct), this should be set for the nested event
3948 (@code{acc_ev_enqueue_launch_start}), or if the value of the parent
3949 construct should remain (@code{acc_ev_compute_construct_start}). In
3950 this implementation, the value will generally correspond to the
3951 innermost nested event type.
3953 @item @code{acc_prof_info.device_type}
3957 For @code{acc_ev_compute_construct_start}, and in presence of an
3958 @code{if} clause with @emph{false} argument, this will still refer to
3959 the offloading device type.
3960 It's not clear if that's the expected behavior.
3963 Complementary to the item before, for
3964 @code{acc_ev_compute_construct_end}, this is set to
3965 @code{acc_device_host} in presence of an @code{if} clause with
3966 @emph{false} argument.
3967 It's not clear if that's the expected behavior.
3971 @item @code{acc_prof_info.thread_id}
3972 Always @code{-1}; not yet implemented.
3974 @item @code{acc_prof_info.async}
3978 Not yet implemented correctly for
3979 @code{acc_ev_compute_construct_start}.
3982 In a compute construct, for host-fallback
3983 execution/@code{acc_device_host} it will always be
3984 @code{acc_async_sync}.
3985 It's not clear if that's the expected behavior.
3988 For @code{acc_ev_device_init_start} and @code{acc_ev_device_init_end},
3989 it will always be @code{acc_async_sync}.
3990 It's not clear if that's the expected behavior.
3994 @item @code{acc_prof_info.async_queue}
3995 There is no @cite{limited number of asynchronous queues} in libgomp.
3996 This will always have the same value as @code{acc_prof_info.async}.
3998 @item @code{acc_prof_info.src_file}
3999 Always @code{NULL}; not yet implemented.
4001 @item @code{acc_prof_info.func_name}
4002 Always @code{NULL}; not yet implemented.
4004 @item @code{acc_prof_info.line_no}
4005 Always @code{-1}; not yet implemented.
4007 @item @code{acc_prof_info.end_line_no}
4008 Always @code{-1}; not yet implemented.
4010 @item @code{acc_prof_info.func_line_no}
4011 Always @code{-1}; not yet implemented.
4013 @item @code{acc_prof_info.func_end_line_no}
4014 Always @code{-1}; not yet implemented.
4016 @item @code{acc_event_info.event_type}, @code{acc_event_info.*.event_type}
4017 Relating to @code{acc_prof_info.event_type} discussed above, in this
4018 implementation, this will always be the same value as
4019 @code{acc_prof_info.event_type}.
4021 @item @code{acc_event_info.*.parent_construct}
4025 Will be @code{acc_construct_parallel} for all OpenACC compute
4026 constructs as well as many OpenACC Runtime API calls; should be the
4027 one matching the actual construct, or
4028 @code{acc_construct_runtime_api}, respectively.
4031 Will be @code{acc_construct_enter_data} or
4032 @code{acc_construct_exit_data} when processing variable mappings
4033 specified in OpenACC @emph{declare} directives; should be
4034 @code{acc_construct_declare}.
4037 For implicit @code{acc_ev_device_init_start},
4038 @code{acc_ev_device_init_end}, and explicit as well as implicit
4039 @code{acc_ev_alloc}, @code{acc_ev_free},
4040 @code{acc_ev_enqueue_upload_start}, @code{acc_ev_enqueue_upload_end},
4041 @code{acc_ev_enqueue_download_start}, and
4042 @code{acc_ev_enqueue_download_end}, will be
4043 @code{acc_construct_parallel}; should reflect the real parent
4048 @item @code{acc_event_info.*.implicit}
4049 For @code{acc_ev_alloc}, @code{acc_ev_free},
4050 @code{acc_ev_enqueue_upload_start}, @code{acc_ev_enqueue_upload_end},
4051 @code{acc_ev_enqueue_download_start}, and
4052 @code{acc_ev_enqueue_download_end}, this currently will be @code{1}
4053 also for explicit usage.
4055 @item @code{acc_event_info.data_event.var_name}
4056 Always @code{NULL}; not yet implemented.
4058 @item @code{acc_event_info.data_event.host_ptr}
4059 For @code{acc_ev_alloc}, and @code{acc_ev_free}, this is always
4062 @item @code{typedef union acc_api_info}
4063 @dots{} as printed in @cite{5.2.3. Third Argument: API-Specific
4064 Information}. This should obviously be @code{typedef @emph{struct}
4067 @item @code{acc_api_info.device_api}
4068 Possibly not yet implemented correctly for
4069 @code{acc_ev_compute_construct_start},
4070 @code{acc_ev_device_init_start}, @code{acc_ev_device_init_end}:
4071 will always be @code{acc_device_api_none} for these event types.
4072 For @code{acc_ev_enter_data_start}, it will be
4073 @code{acc_device_api_none} in some cases.
4075 @item @code{acc_api_info.device_type}
4076 Always the same as @code{acc_prof_info.device_type}.
4078 @item @code{acc_api_info.vendor}
4079 Always @code{-1}; not yet implemented.
4081 @item @code{acc_api_info.device_handle}
4082 Always @code{NULL}; not yet implemented.
4084 @item @code{acc_api_info.context_handle}
4085 Always @code{NULL}; not yet implemented.
4087 @item @code{acc_api_info.async_handle}
4088 Always @code{NULL}; not yet implemented.
4092 Remarks about certain event types:
4096 @item @code{acc_ev_device_init_start}, @code{acc_ev_device_init_end}
4100 @c See 'DEVICE_INIT_INSIDE_COMPUTE_CONSTRUCT' in
4101 @c 'libgomp.oacc-c-c++-common/acc_prof-kernels-1.c',
4102 @c 'libgomp.oacc-c-c++-common/acc_prof-parallel-1.c'.
4103 When a compute construct triggers implicit
4104 @code{acc_ev_device_init_start} and @code{acc_ev_device_init_end}
4105 events, they currently aren't @emph{nested within} the corresponding
4106 @code{acc_ev_compute_construct_start} and
4107 @code{acc_ev_compute_construct_end}, but they're currently observed
4108 @emph{before} @code{acc_ev_compute_construct_start}.
4109 It's not clear what to do: the standard asks us provide a lot of
4110 details to the @code{acc_ev_compute_construct_start} callback, without
4111 (implicitly) initializing a device before?
4114 Callbacks for these event types will not be invoked for calls to the
4115 @code{acc_set_device_type} and @code{acc_set_device_num} functions.
4116 It's not clear if they should be.
4120 @item @code{acc_ev_enter_data_start}, @code{acc_ev_enter_data_end}, @code{acc_ev_exit_data_start}, @code{acc_ev_exit_data_end}
4124 Callbacks for these event types will also be invoked for OpenACC
4125 @emph{host_data} constructs.
4126 It's not clear if they should be.
4129 Callbacks for these event types will also be invoked when processing
4130 variable mappings specified in OpenACC @emph{declare} directives.
4131 It's not clear if they should be.
4137 Callbacks for the following event types will be invoked, but dispatch
4138 and information provided therein has not yet been thoroughly reviewed:
4141 @item @code{acc_ev_alloc}
4142 @item @code{acc_ev_free}
4143 @item @code{acc_ev_update_start}, @code{acc_ev_update_end}
4144 @item @code{acc_ev_enqueue_upload_start}, @code{acc_ev_enqueue_upload_end}
4145 @item @code{acc_ev_enqueue_download_start}, @code{acc_ev_enqueue_download_end}
4148 During device initialization, and finalization, respectively,
4149 callbacks for the following event types will not yet be invoked:
4152 @item @code{acc_ev_alloc}
4153 @item @code{acc_ev_free}
4156 Callbacks for the following event types have not yet been implemented,
4157 so currently won't be invoked:
4160 @item @code{acc_ev_device_shutdown_start}, @code{acc_ev_device_shutdown_end}
4161 @item @code{acc_ev_runtime_shutdown}
4162 @item @code{acc_ev_create}, @code{acc_ev_delete}
4163 @item @code{acc_ev_wait_start}, @code{acc_ev_wait_end}
4166 For the following runtime library functions, not all expected
4167 callbacks will be invoked (mostly concerning implicit device
4171 @item @code{acc_get_num_devices}
4172 @item @code{acc_set_device_type}
4173 @item @code{acc_get_device_type}
4174 @item @code{acc_set_device_num}
4175 @item @code{acc_get_device_num}
4176 @item @code{acc_init}
4177 @item @code{acc_shutdown}
4180 Aside from implicit device initialization, for the following runtime
4181 library functions, no callbacks will be invoked for shared-memory
4182 offloading devices (it's not clear if they should be):
4185 @item @code{acc_malloc}
4186 @item @code{acc_free}
4187 @item @code{acc_copyin}, @code{acc_present_or_copyin}, @code{acc_copyin_async}
4188 @item @code{acc_create}, @code{acc_present_or_create}, @code{acc_create_async}
4189 @item @code{acc_copyout}, @code{acc_copyout_async}, @code{acc_copyout_finalize}, @code{acc_copyout_finalize_async}
4190 @item @code{acc_delete}, @code{acc_delete_async}, @code{acc_delete_finalize}, @code{acc_delete_finalize_async}
4191 @item @code{acc_update_device}, @code{acc_update_device_async}
4192 @item @code{acc_update_self}, @code{acc_update_self_async}
4193 @item @code{acc_map_data}, @code{acc_unmap_data}
4194 @item @code{acc_memcpy_to_device}, @code{acc_memcpy_to_device_async}
4195 @item @code{acc_memcpy_from_device}, @code{acc_memcpy_from_device_async}
4200 @c ---------------------------------------------------------------------
4202 @c ---------------------------------------------------------------------
4204 @node The libgomp ABI
4205 @chapter The libgomp ABI
4207 The following sections present notes on the external ABI as
4208 presented by libgomp. Only maintainers should need them.
4211 * Implementing MASTER construct::
4212 * Implementing CRITICAL construct::
4213 * Implementing ATOMIC construct::
4214 * Implementing FLUSH construct::
4215 * Implementing BARRIER construct::
4216 * Implementing THREADPRIVATE construct::
4217 * Implementing PRIVATE clause::
4218 * Implementing FIRSTPRIVATE LASTPRIVATE COPYIN and COPYPRIVATE clauses::
4219 * Implementing REDUCTION clause::
4220 * Implementing PARALLEL construct::
4221 * Implementing FOR construct::
4222 * Implementing ORDERED construct::
4223 * Implementing SECTIONS construct::
4224 * Implementing SINGLE construct::
4225 * Implementing OpenACC's PARALLEL construct::
4229 @node Implementing MASTER construct
4230 @section Implementing MASTER construct
4233 if (omp_get_thread_num () == 0)
4237 Alternately, we generate two copies of the parallel subfunction
4238 and only include this in the version run by the primary thread.
4239 Surely this is not worthwhile though...
4243 @node Implementing CRITICAL construct
4244 @section Implementing CRITICAL construct
4246 Without a specified name,
4249 void GOMP_critical_start (void);
4250 void GOMP_critical_end (void);
4253 so that we don't get COPY relocations from libgomp to the main
4256 With a specified name, use omp_set_lock and omp_unset_lock with
4257 name being transformed into a variable declared like
4260 omp_lock_t gomp_critical_user_<name> __attribute__((common))
4263 Ideally the ABI would specify that all zero is a valid unlocked
4264 state, and so we wouldn't need to initialize this at
4269 @node Implementing ATOMIC construct
4270 @section Implementing ATOMIC construct
4272 The target should implement the @code{__sync} builtins.
4274 Failing that we could add
4277 void GOMP_atomic_enter (void)
4278 void GOMP_atomic_exit (void)
4281 which reuses the regular lock code, but with yet another lock
4282 object private to the library.
4286 @node Implementing FLUSH construct
4287 @section Implementing FLUSH construct
4289 Expands to the @code{__sync_synchronize} builtin.
4293 @node Implementing BARRIER construct
4294 @section Implementing BARRIER construct
4297 void GOMP_barrier (void)
4301 @node Implementing THREADPRIVATE construct
4302 @section Implementing THREADPRIVATE construct
4304 In _most_ cases we can map this directly to @code{__thread}. Except
4305 that OMP allows constructors for C++ objects. We can either
4306 refuse to support this (how often is it used?) or we can
4307 implement something akin to .ctors.
4309 Even more ideally, this ctor feature is handled by extensions
4310 to the main pthreads library. Failing that, we can have a set
4311 of entry points to register ctor functions to be called.
4315 @node Implementing PRIVATE clause
4316 @section Implementing PRIVATE clause
4318 In association with a PARALLEL, or within the lexical extent
4319 of a PARALLEL block, the variable becomes a local variable in
4320 the parallel subfunction.
4322 In association with FOR or SECTIONS blocks, create a new
4323 automatic variable within the current function. This preserves
4324 the semantic of new variable creation.
4328 @node Implementing FIRSTPRIVATE LASTPRIVATE COPYIN and COPYPRIVATE clauses
4329 @section Implementing FIRSTPRIVATE LASTPRIVATE COPYIN and COPYPRIVATE clauses
4331 This seems simple enough for PARALLEL blocks. Create a private
4332 struct for communicating between the parent and subfunction.
4333 In the parent, copy in values for scalar and "small" structs;
4334 copy in addresses for others TREE_ADDRESSABLE types. In the
4335 subfunction, copy the value into the local variable.
4337 It is not clear what to do with bare FOR or SECTION blocks.
4338 The only thing I can figure is that we do something like:
4341 #pragma omp for firstprivate(x) lastprivate(y)
4342 for (int i = 0; i < n; ++i)
4359 where the "x=x" and "y=y" assignments actually have different
4360 uids for the two variables, i.e. not something you could write
4361 directly in C. Presumably this only makes sense if the "outer"
4362 x and y are global variables.
4364 COPYPRIVATE would work the same way, except the structure
4365 broadcast would have to happen via SINGLE machinery instead.
4369 @node Implementing REDUCTION clause
4370 @section Implementing REDUCTION clause
4372 The private struct mentioned in the previous section should have
4373 a pointer to an array of the type of the variable, indexed by the
4374 thread's @var{team_id}. The thread stores its final value into the
4375 array, and after the barrier, the primary thread iterates over the
4376 array to collect the values.
4379 @node Implementing PARALLEL construct
4380 @section Implementing PARALLEL construct
4383 #pragma omp parallel
4392 void subfunction (void *data)
4399 GOMP_parallel_start (subfunction, &data, num_threads);
4400 subfunction (&data);
4401 GOMP_parallel_end ();
4405 void GOMP_parallel_start (void (*fn)(void *), void *data, unsigned num_threads)
4408 The @var{FN} argument is the subfunction to be run in parallel.
4410 The @var{DATA} argument is a pointer to a structure used to
4411 communicate data in and out of the subfunction, as discussed
4412 above with respect to FIRSTPRIVATE et al.
4414 The @var{NUM_THREADS} argument is 1 if an IF clause is present
4415 and false, or the value of the NUM_THREADS clause, if
4418 The function needs to create the appropriate number of
4419 threads and/or launch them from the dock. It needs to
4420 create the team structure and assign team ids.
4423 void GOMP_parallel_end (void)
4426 Tears down the team and returns us to the previous @code{omp_in_parallel()} state.
4430 @node Implementing FOR construct
4431 @section Implementing FOR construct
4434 #pragma omp parallel for
4435 for (i = lb; i <= ub; i++)
4442 void subfunction (void *data)
4445 while (GOMP_loop_static_next (&_s0, &_e0))
4448 for (i = _s0; i < _e1; i++)
4451 GOMP_loop_end_nowait ();
4454 GOMP_parallel_loop_static (subfunction, NULL, 0, lb, ub+1, 1, 0);
4456 GOMP_parallel_end ();
4460 #pragma omp for schedule(runtime)
4461 for (i = 0; i < n; i++)
4470 if (GOMP_loop_runtime_start (0, n, 1, &_s0, &_e0))
4473 for (i = _s0, i < _e0; i++)
4475 @} while (GOMP_loop_runtime_next (&_s0, _&e0));
4480 Note that while it looks like there is trickiness to propagating
4481 a non-constant STEP, there isn't really. We're explicitly allowed
4482 to evaluate it as many times as we want, and any variables involved
4483 should automatically be handled as PRIVATE or SHARED like any other
4484 variables. So the expression should remain evaluable in the
4485 subfunction. We can also pull it into a local variable if we like,
4486 but since its supposed to remain unchanged, we can also not if we like.
4488 If we have SCHEDULE(STATIC), and no ORDERED, then we ought to be
4489 able to get away with no work-sharing context at all, since we can
4490 simply perform the arithmetic directly in each thread to divide up
4491 the iterations. Which would mean that we wouldn't need to call any
4494 There are separate routines for handling loops with an ORDERED
4495 clause. Bookkeeping for that is non-trivial...
4499 @node Implementing ORDERED construct
4500 @section Implementing ORDERED construct
4503 void GOMP_ordered_start (void)
4504 void GOMP_ordered_end (void)
4509 @node Implementing SECTIONS construct
4510 @section Implementing SECTIONS construct
4515 #pragma omp sections
4529 for (i = GOMP_sections_start (3); i != 0; i = GOMP_sections_next ())
4546 @node Implementing SINGLE construct
4547 @section Implementing SINGLE construct
4561 if (GOMP_single_start ())
4569 #pragma omp single copyprivate(x)
4576 datap = GOMP_single_copy_start ();
4581 GOMP_single_copy_end (&data);
4590 @node Implementing OpenACC's PARALLEL construct
4591 @section Implementing OpenACC's PARALLEL construct
4594 void GOACC_parallel ()
4599 @c ---------------------------------------------------------------------
4601 @c ---------------------------------------------------------------------
4603 @node Reporting Bugs
4604 @chapter Reporting Bugs
4606 Bugs in the GNU Offloading and Multi Processing Runtime Library should
4607 be reported via @uref{https://gcc.gnu.org/bugzilla/, Bugzilla}. Please add
4608 "openacc", or "openmp", or both to the keywords field in the bug
4609 report, as appropriate.
4613 @c ---------------------------------------------------------------------
4614 @c GNU General Public License
4615 @c ---------------------------------------------------------------------
4617 @include gpl_v3.texi
4621 @c ---------------------------------------------------------------------
4622 @c GNU Free Documentation License
4623 @c ---------------------------------------------------------------------
4629 @c ---------------------------------------------------------------------
4630 @c Funding Free Software
4631 @c ---------------------------------------------------------------------
4633 @include funding.texi
4635 @c ---------------------------------------------------------------------
4637 @c ---------------------------------------------------------------------
4640 @unnumbered Library Index