1 \input texinfo @c -*-texinfo-*-
4 @setfilename libgomp.info
10 Copyright @copyright{} 2006-2023 Free Software Foundation, Inc.
12 Permission is granted to copy, distribute and/or modify this document
13 under the terms of the GNU Free Documentation License, Version 1.3 or
14 any later version published by the Free Software Foundation; with the
15 Invariant Sections being ``Funding Free Software'', the Front-Cover
16 texts being (a) (see below), and with the Back-Cover Texts being (b)
17 (see below). A copy of the license is included in the section entitled
18 ``GNU Free Documentation License''.
20 (a) The FSF's Front-Cover Text is:
24 (b) The FSF's Back-Cover Text is:
26 You have freedom to copy and modify this GNU Manual, like GNU
27 software. Copies published by the Free Software Foundation raise
28 funds for GNU development.
32 @dircategory GNU Libraries
34 * libgomp: (libgomp). GNU Offloading and Multi Processing Runtime Library.
37 This manual documents libgomp, the GNU Offloading and Multi Processing
38 Runtime library. This is the GNU implementation of the OpenMP and
39 OpenACC APIs for parallel and accelerator programming in C/C++ and
42 Published by the Free Software Foundation
43 51 Franklin Street, Fifth Floor
44 Boston, MA 02110-1301 USA
50 @setchapternewpage odd
53 @title GNU Offloading and Multi Processing Runtime Library
54 @subtitle The GNU OpenMP and OpenACC Implementation
56 @vskip 0pt plus 1filll
57 @comment For the @value{version-GCC} Version*
59 Published by the Free Software Foundation @*
60 51 Franklin Street, Fifth Floor@*
61 Boston, MA 02110-1301, USA@*
71 @node Top, Enabling OpenMP
75 This manual documents the usage of libgomp, the GNU Offloading and
76 Multi Processing Runtime Library. This includes the GNU
77 implementation of the @uref{https://www.openmp.org, OpenMP} Application
78 Programming Interface (API) for multi-platform shared-memory parallel
79 programming in C/C++ and Fortran, and the GNU implementation of the
80 @uref{https://www.openacc.org, OpenACC} Application Programming
81 Interface (API) for offloading of code to accelerator devices in C/C++
84 Originally, libgomp implemented the GNU OpenMP Runtime Library. Based
85 on this, support for OpenACC and offloading (both OpenACC and OpenMP
86 4's target construct) has been added later on, and the library's name
87 changed to GNU Offloading and Multi Processing Runtime Library.
92 @comment When you add a new menu item, please keep the right hand
93 @comment aligned to the same column. Do not use tabs. This provides
94 @comment better formatting.
97 * Enabling OpenMP:: How to enable OpenMP for your applications.
98 * OpenMP Implementation Status:: List of implemented features by OpenMP version
99 * OpenMP Runtime Library Routines: Runtime Library Routines.
100 The OpenMP runtime application programming
102 * OpenMP Environment Variables: Environment Variables.
103 Influencing OpenMP runtime behavior with
104 environment variables.
105 * Enabling OpenACC:: How to enable OpenACC for your
107 * OpenACC Runtime Library Routines:: The OpenACC runtime application
108 programming interface.
109 * OpenACC Environment Variables:: Influencing OpenACC runtime behavior with
110 environment variables.
111 * CUDA Streams Usage:: Notes on the implementation of
112 asynchronous operations.
113 * OpenACC Library Interoperability:: OpenACC library interoperability with the
114 NVIDIA CUBLAS library.
115 * OpenACC Profiling Interface::
116 * OpenMP-Implementation Specifics:: Notes specifics of this OpenMP
118 * Offload-Target Specifics:: Notes on offload-target specific internals
119 * The libgomp ABI:: Notes on the external ABI presented by libgomp.
120 * Reporting Bugs:: How to report bugs in the GNU Offloading and
121 Multi Processing Runtime Library.
122 * Copying:: GNU general public license says
123 how you can copy and share libgomp.
124 * GNU Free Documentation License::
125 How you can copy and share this manual.
126 * Funding:: How to help assure continued work for free
128 * Library Index:: Index of this documentation.
132 @c ---------------------------------------------------------------------
134 @c ---------------------------------------------------------------------
136 @node Enabling OpenMP
137 @chapter Enabling OpenMP
139 To activate the OpenMP extensions for C/C++ and Fortran, the compile-time
140 flag @command{-fopenmp} must be specified. This enables the OpenMP directive
141 @code{#pragma omp} in C/C++ and @code{!$omp} directives in free form,
142 @code{c$omp}, @code{*$omp} and @code{!$omp} directives in fixed form,
143 @code{!$} conditional compilation sentinels in free form and @code{c$},
144 @code{*$} and @code{!$} sentinels in fixed form, for Fortran. The flag also
145 arranges for automatic linking of the OpenMP runtime library
146 (@ref{Runtime Library Routines}).
148 A complete description of all OpenMP directives may be found in the
149 @uref{https://www.openmp.org, OpenMP Application Program Interface} manuals.
150 See also @ref{OpenMP Implementation Status}.
153 @c ---------------------------------------------------------------------
154 @c OpenMP Implementation Status
155 @c ---------------------------------------------------------------------
157 @node OpenMP Implementation Status
158 @chapter OpenMP Implementation Status
161 * OpenMP 4.5:: Feature completion status to 4.5 specification
162 * OpenMP 5.0:: Feature completion status to 5.0 specification
163 * OpenMP 5.1:: Feature completion status to 5.1 specification
164 * OpenMP 5.2:: Feature completion status to 5.2 specification
165 * OpenMP Technical Report 11:: Feature completion status to first 6.0 preview
168 The @code{_OPENMP} preprocessor macro and Fortran's @code{openmp_version}
169 parameter, provided by @code{omp_lib.h} and the @code{omp_lib} module, have
170 the value @code{201511} (i.e. OpenMP 4.5).
175 The OpenMP 4.5 specification is fully supported.
180 @unnumberedsubsec New features listed in Appendix B of the OpenMP specification
181 @c This list is sorted as in OpenMP 5.1's B.3 not as in OpenMP 5.0's B.2
183 @multitable @columnfractions .60 .10 .25
184 @headitem Description @tab Status @tab Comments
185 @item Array shaping @tab N @tab
186 @item Array sections with non-unit strides in C and C++ @tab N @tab
187 @item Iterators @tab Y @tab
188 @item @code{metadirective} directive @tab N @tab
189 @item @code{declare variant} directive
190 @tab P @tab @emph{simd} traits not handled correctly
191 @item @var{target-offload-var} ICV and @code{OMP_TARGET_OFFLOAD}
192 env variable @tab Y @tab
193 @item Nested-parallel changes to @var{max-active-levels-var} ICV @tab Y @tab
194 @item @code{requires} directive @tab P
195 @tab complete but no non-host device provides @code{unified_shared_memory}
196 @item @code{teams} construct outside an enclosing target region @tab Y @tab
197 @item Non-rectangular loop nests @tab P
198 @tab Full support for C/C++, partial for Fortran
199 (@uref{https://gcc.gnu.org/PR110735,PR110735})
200 @item @code{!=} as relational-op in canonical loop form for C/C++ @tab Y @tab
201 @item @code{nonmonotonic} as default loop schedule modifier for worksharing-loop
202 constructs @tab Y @tab
203 @item Collapse of associated loops that are imperfectly nested loops @tab Y @tab
204 @item Clauses @code{if}, @code{nontemporal} and @code{order(concurrent)} in
205 @code{simd} construct @tab Y @tab
206 @item @code{atomic} constructs in @code{simd} @tab Y @tab
207 @item @code{loop} construct @tab Y @tab
208 @item @code{order(concurrent)} clause @tab Y @tab
209 @item @code{scan} directive and @code{in_scan} modifier for the
210 @code{reduction} clause @tab Y @tab
211 @item @code{in_reduction} clause on @code{task} constructs @tab Y @tab
212 @item @code{in_reduction} clause on @code{target} constructs @tab P
213 @tab @code{nowait} only stub
214 @item @code{task_reduction} clause with @code{taskgroup} @tab Y @tab
215 @item @code{task} modifier to @code{reduction} clause @tab Y @tab
216 @item @code{affinity} clause to @code{task} construct @tab Y @tab Stub only
217 @item @code{detach} clause to @code{task} construct @tab Y @tab
218 @item @code{omp_fulfill_event} runtime routine @tab Y @tab
219 @item @code{reduction} and @code{in_reduction} clauses on @code{taskloop}
220 and @code{taskloop simd} constructs @tab Y @tab
221 @item @code{taskloop} construct cancelable by @code{cancel} construct
223 @item @code{mutexinoutset} @emph{dependence-type} for @code{depend} clause
225 @item Predefined memory spaces, memory allocators, allocator traits
226 @tab Y @tab See also @ref{Memory allocation}
227 @item Memory management routines @tab Y @tab
228 @item @code{allocate} directive @tab P @tab Only C, only stack variables
229 @item @code{allocate} clause @tab P @tab Initial support
230 @item @code{use_device_addr} clause on @code{target data} @tab Y @tab
231 @item @code{ancestor} modifier on @code{device} clause @tab Y @tab
232 @item Implicit declare target directive @tab Y @tab
233 @item Discontiguous array section with @code{target update} construct
235 @item C/C++'s lvalue expressions in @code{to}, @code{from}
236 and @code{map} clauses @tab N @tab
237 @item C/C++'s lvalue expressions in @code{depend} clauses @tab Y @tab
238 @item Nested @code{declare target} directive @tab Y @tab
239 @item Combined @code{master} constructs @tab Y @tab
240 @item @code{depend} clause on @code{taskwait} @tab Y @tab
241 @item Weak memory ordering clauses on @code{atomic} and @code{flush} construct
243 @item @code{hint} clause on the @code{atomic} construct @tab Y @tab Stub only
244 @item @code{depobj} construct and depend objects @tab Y @tab
245 @item Lock hints were renamed to synchronization hints @tab Y @tab
246 @item @code{conditional} modifier to @code{lastprivate} clause @tab Y @tab
247 @item Map-order clarifications @tab P @tab
248 @item @code{close} @emph{map-type-modifier} @tab Y @tab
249 @item Mapping C/C++ pointer variables and to assign the address of
250 device memory mapped by an array section @tab P @tab
251 @item Mapping of Fortran pointer and allocatable variables, including pointer
252 and allocatable components of variables
253 @tab P @tab Mapping of vars with allocatable components unsupported
254 @item @code{defaultmap} extensions @tab Y @tab
255 @item @code{declare mapper} directive @tab N @tab
256 @item @code{omp_get_supported_active_levels} routine @tab Y @tab
257 @item Runtime routines and environment variables to display runtime thread
258 affinity information @tab Y @tab
259 @item @code{omp_pause_resource} and @code{omp_pause_resource_all} runtime
261 @item @code{omp_get_device_num} runtime routine @tab Y @tab
262 @item OMPT interface @tab N @tab
263 @item OMPD interface @tab N @tab
266 @unnumberedsubsec Other new OpenMP 5.0 features
268 @multitable @columnfractions .60 .10 .25
269 @headitem Description @tab Status @tab Comments
270 @item Supporting C++'s range-based for loop @tab Y @tab
277 @unnumberedsubsec New features listed in Appendix B of the OpenMP specification
279 @multitable @columnfractions .60 .10 .25
280 @headitem Description @tab Status @tab Comments
281 @item OpenMP directive as C++ attribute specifiers @tab Y @tab
282 @item @code{omp_all_memory} reserved locator @tab Y @tab
283 @item @emph{target_device trait} in OpenMP Context @tab N @tab
284 @item @code{target_device} selector set in context selectors @tab N @tab
285 @item C/C++'s @code{declare variant} directive: elision support of
286 preprocessed code @tab N @tab
287 @item @code{declare variant}: new clauses @code{adjust_args} and
288 @code{append_args} @tab N @tab
289 @item @code{dispatch} construct @tab N @tab
290 @item device-specific ICV settings with environment variables @tab Y @tab
291 @item @code{assume} and @code{assumes} directives @tab Y @tab
292 @item @code{nothing} directive @tab Y @tab
293 @item @code{error} directive @tab Y @tab
294 @item @code{masked} construct @tab Y @tab
295 @item @code{scope} directive @tab Y @tab
296 @item Loop transformation constructs @tab N @tab
297 @item @code{strict} modifier in the @code{grainsize} and @code{num_tasks}
298 clauses of the @code{taskloop} construct @tab Y @tab
299 @item @code{align} clause in @code{allocate} directive @tab P
300 @tab Only C (and only stack variables)
301 @item @code{align} modifier in @code{allocate} clause @tab Y @tab
302 @item @code{thread_limit} clause to @code{target} construct @tab Y @tab
303 @item @code{has_device_addr} clause to @code{target} construct @tab Y @tab
304 @item Iterators in @code{target update} motion clauses and @code{map}
306 @item Indirect calls to the device version of a procedure or function in
307 @code{target} regions @tab N @tab
308 @item @code{interop} directive @tab N @tab
309 @item @code{omp_interop_t} object support in runtime routines @tab N @tab
310 @item @code{nowait} clause in @code{taskwait} directive @tab Y @tab
311 @item Extensions to the @code{atomic} directive @tab Y @tab
312 @item @code{seq_cst} clause on a @code{flush} construct @tab Y @tab
313 @item @code{inoutset} argument to the @code{depend} clause @tab Y @tab
314 @item @code{private} and @code{firstprivate} argument to @code{default}
315 clause in C and C++ @tab Y @tab
316 @item @code{present} argument to @code{defaultmap} clause @tab Y @tab
317 @item @code{omp_set_num_teams}, @code{omp_set_teams_thread_limit},
318 @code{omp_get_max_teams}, @code{omp_get_teams_thread_limit} runtime
320 @item @code{omp_target_is_accessible} runtime routine @tab Y @tab
321 @item @code{omp_target_memcpy_async} and @code{omp_target_memcpy_rect_async}
322 runtime routines @tab Y @tab
323 @item @code{omp_get_mapped_ptr} runtime routine @tab Y @tab
324 @item @code{omp_calloc}, @code{omp_realloc}, @code{omp_aligned_alloc} and
325 @code{omp_aligned_calloc} runtime routines @tab Y @tab
326 @item @code{omp_alloctrait_key_t} enum: @code{omp_atv_serialized} added,
327 @code{omp_atv_default} changed @tab Y @tab
328 @item @code{omp_display_env} runtime routine @tab Y @tab
329 @item @code{ompt_scope_endpoint_t} enum: @code{ompt_scope_beginend} @tab N @tab
330 @item @code{ompt_sync_region_t} enum additions @tab N @tab
331 @item @code{ompt_state_t} enum: @code{ompt_state_wait_barrier_implementation}
332 and @code{ompt_state_wait_barrier_teams} @tab N @tab
333 @item @code{ompt_callback_target_data_op_emi_t},
334 @code{ompt_callback_target_emi_t}, @code{ompt_callback_target_map_emi_t}
335 and @code{ompt_callback_target_submit_emi_t} @tab N @tab
336 @item @code{ompt_callback_error_t} type @tab N @tab
337 @item @code{OMP_PLACES} syntax extensions @tab Y @tab
338 @item @code{OMP_NUM_TEAMS} and @code{OMP_TEAMS_THREAD_LIMIT} environment
339 variables @tab Y @tab
342 @unnumberedsubsec Other new OpenMP 5.1 features
344 @multitable @columnfractions .60 .10 .25
345 @headitem Description @tab Status @tab Comments
346 @item Support of strictly structured blocks in Fortran @tab Y @tab
347 @item Support of structured block sequences in C/C++ @tab Y @tab
348 @item @code{unconstrained} and @code{reproducible} modifiers on @code{order}
350 @item Support @code{begin/end declare target} syntax in C/C++ @tab Y @tab
351 @item Pointer predetermined firstprivate getting initialized
352 to address of matching mapped list item per 5.1, Sect. 2.21.7.2 @tab N @tab
353 @item For Fortran, diagnose placing declarative before/between @code{USE},
354 @code{IMPORT}, and @code{IMPLICIT} as invalid @tab N @tab
355 @item Optional comma between directive and clause in the @code{#pragma} form @tab Y @tab
356 @item @code{indirect} clause in @code{declare target} @tab N @tab
357 @item @code{device_type(nohost)}/@code{device_type(host)} for variables @tab N @tab
358 @item @code{present} modifier to the @code{map}, @code{to} and @code{from}
366 @unnumberedsubsec New features listed in Appendix B of the OpenMP specification
368 @multitable @columnfractions .60 .10 .25
369 @headitem Description @tab Status @tab Comments
370 @item @code{omp_in_explicit_task} routine and @var{explicit-task-var} ICV
372 @item @code{omp}/@code{ompx}/@code{omx} sentinels and @code{omp_}/@code{ompx_}
374 @tab warning for @code{ompx/omx} sentinels@footnote{The @code{ompx}
375 sentinel as C/C++ pragma and C++ attributes are warned for with
376 @code{-Wunknown-pragmas} (implied by @code{-Wall}) and @code{-Wattributes}
377 (enabled by default), respectively; for Fortran free-source code, there is
378 a warning enabled by default and, for fixed-source code, the @code{omx}
379 sentinel is warned for with with @code{-Wsurprising} (enabled by
380 @code{-Wall}). Unknown clauses are always rejected with an error.}
381 @item Clauses on @code{end} directive can be on directive @tab Y @tab
382 @item @code{destroy} clause with destroy-var argument on @code{depobj}
384 @item Deprecation of no-argument @code{destroy} clause on @code{depobj}
386 @item @code{linear} clause syntax changes and @code{step} modifier @tab Y @tab
387 @item Deprecation of minus operator for reductions @tab N @tab
388 @item Deprecation of separating @code{map} modifiers without comma @tab N @tab
389 @item @code{declare mapper} with iterator and @code{present} modifiers
391 @item If a matching mapped list item is not found in the data environment, the
392 pointer retains its original value @tab Y @tab
393 @item New @code{enter} clause as alias for @code{to} on declare target directive
395 @item Deprecation of @code{to} clause on declare target directive @tab N @tab
396 @item Extended list of directives permitted in Fortran pure procedures
398 @item New @code{allocators} directive for Fortran @tab N @tab
399 @item Deprecation of @code{allocate} directive for Fortran
400 allocatables/pointers @tab N @tab
401 @item Optional paired @code{end} directive with @code{dispatch} @tab N @tab
402 @item New @code{memspace} and @code{traits} modifiers for @code{uses_allocators}
404 @item Deprecation of traits array following the allocator_handle expression in
405 @code{uses_allocators} @tab N @tab
406 @item New @code{otherwise} clause as alias for @code{default} on metadirectives
408 @item Deprecation of @code{default} clause on metadirectives @tab N @tab
409 @item Deprecation of delimited form of @code{declare target} @tab N @tab
410 @item Reproducible semantics changed for @code{order(concurrent)} @tab N @tab
411 @item @code{allocate} and @code{firstprivate} clauses on @code{scope}
413 @item @code{ompt_callback_work} @tab N @tab
414 @item Default map-type for the @code{map} clause in @code{target enter/exit data}
416 @item New @code{doacross} clause as alias for @code{depend} with
417 @code{source}/@code{sink} modifier @tab Y @tab
418 @item Deprecation of @code{depend} with @code{source}/@code{sink} modifier
420 @item @code{omp_cur_iteration} keyword @tab Y @tab
423 @unnumberedsubsec Other new OpenMP 5.2 features
425 @multitable @columnfractions .60 .10 .25
426 @headitem Description @tab Status @tab Comments
427 @item For Fortran, optional comma between directive and clause @tab N @tab
428 @item Conforming device numbers and @code{omp_initial_device} and
429 @code{omp_invalid_device} enum/PARAMETER @tab Y @tab
430 @item Initial value of @var{default-device-var} ICV with
431 @code{OMP_TARGET_OFFLOAD=mandatory} @tab Y @tab
432 @item @code{all} as @emph{implicit-behavior} for @code{defaultmap} @tab Y @tab
433 @item @emph{interop_types} in any position of the modifier list for the @code{init} clause
434 of the @code{interop} construct @tab N @tab
438 @node OpenMP Technical Report 11
439 @section OpenMP Technical Report 11
441 Technical Report (TR) 11 is the first preview for OpenMP 6.0.
443 @unnumberedsubsec New features listed in Appendix B of the OpenMP specification
444 @multitable @columnfractions .60 .10 .25
445 @item Features deprecated in versions 5.2, 5.1 and 5.0 were removed
446 @tab N/A @tab Backward compatibility
447 @item The @code{decl} attribute was added to the C++ attribute syntax
449 @item @code{_ALL} suffix to the device-scope environment variables
450 @tab P @tab Host device number wrongly accepted
451 @item For Fortran, @emph{locator list} can be also function reference with
452 data pointer result @tab N @tab
453 @item Ref-count change for @code{use_device_ptr}/@code{use_device_addr}
455 @item Implicit reduction identifiers of C++ classes
457 @item Change of the @emph{map-type} property from @emph{ultimate} to
458 @emph{default} @tab N @tab
459 @item Concept of @emph{assumed-size arrays} in C and C++
461 @item Mapping of @emph{assumed-size arrays} in C, C++ and Fortran
463 @item @code{groupprivate} directive @tab N @tab
464 @item @code{local} clause to declare target directive @tab N @tab
465 @item @code{part_size} allocator trait @tab N @tab
466 @item @code{pin_device}, @code{preferred_device} and @code{target_access}
469 @item @code{access} allocator trait changes @tab N @tab
470 @item Extension of @code{interop} operation of @code{append_args}, allowing all
471 modifiers of the @code{init} clause
473 @item @code{interop} clause to @code{dispatch} @tab N @tab
474 @item @code{apply} code to loop-transforming constructs @tab N @tab
475 @item @code{omp_curr_progress_width} identifier @tab N @tab
476 @item @code{safesync} clause to the @code{parallel} construct @tab N @tab
477 @item @code{omp_get_max_progress_width} runtime routine @tab N @tab
478 @item @code{strict} modifier keyword to @code{num_threads} @tab N @tab
479 @item @code{memscope} clause to @code{atomic} and @code{flush} @tab N @tab
480 @item Routines for obtaining memory spaces/allocators for shared/device memory
482 @item @code{omp_get_memspace_num_resources} routine @tab N @tab
483 @item @code{omp_get_submemspace} routine @tab N @tab
484 @item @code{ompt_get_buffer_limits} OMPT routine @tab N @tab
485 @item Extension of @code{OMP_DEFAULT_DEVICE} and new
486 @code{OMP_AVAILABLE_DEVICES} environment vars @tab N @tab
487 @item Supporting increments with abstract names in @code{OMP_PLACES} @tab N @tab
490 @unnumberedsubsec Other new TR 11 features
491 @multitable @columnfractions .60 .10 .25
492 @item Relaxed Fortran restrictions to the @code{aligned} clause @tab N @tab
493 @item Mapping lambda captures @tab N @tab
494 @item For Fortran, atomic compare with storing the comparison result
500 @c ---------------------------------------------------------------------
501 @c OpenMP Runtime Library Routines
502 @c ---------------------------------------------------------------------
504 @node Runtime Library Routines
505 @chapter OpenMP Runtime Library Routines
507 The runtime routines described here are defined by Section 18 of the OpenMP
508 specification in version 5.2.
511 * Thread Team Routines::
512 * Thread Affinity Routines::
513 * Teams Region Routines::
515 @c * Resource Relinquishing Routines::
516 * Device Information Routines::
517 @c * Device Memory Routines::
521 @c * Interoperability Routines::
522 * Memory Management Routines::
523 @c * Tool Control Routine::
524 @c * Environment Display Routine::
529 @node Thread Team Routines
530 @section Thread Team Routines
532 Routines controlling threads in the current contention group.
533 They have C linkage and do not throw exceptions.
536 * omp_set_num_threads:: Set upper team size limit
537 * omp_get_num_threads:: Size of the active team
538 * omp_get_max_threads:: Maximum number of threads of parallel region
539 * omp_get_thread_num:: Current thread ID
540 * omp_in_parallel:: Whether a parallel region is active
541 * omp_set_dynamic:: Enable/disable dynamic teams
542 * omp_get_dynamic:: Dynamic teams setting
543 * omp_get_cancellation:: Whether cancellation support is enabled
544 * omp_set_nested:: Enable/disable nested parallel regions
545 * omp_get_nested:: Nested parallel regions
546 * omp_set_schedule:: Set the runtime scheduling method
547 * omp_get_schedule:: Obtain the runtime scheduling method
548 * omp_get_teams_thread_limit:: Maximum number of threads imposed by teams
549 * omp_get_supported_active_levels:: Maximum number of active regions supported
550 * omp_set_max_active_levels:: Limits the number of active parallel regions
551 * omp_get_max_active_levels:: Current maximum number of active regions
552 * omp_get_level:: Number of parallel regions
553 * omp_get_ancestor_thread_num:: Ancestor thread ID
554 * omp_get_team_size:: Number of threads in a team
555 * omp_get_active_level:: Number of active parallel regions
560 @node omp_set_num_threads
561 @subsection @code{omp_set_num_threads} -- Set upper team size limit
563 @item @emph{Description}:
564 Specifies the number of threads used by default in subsequent parallel
565 sections, if those do not specify a @code{num_threads} clause. The
566 argument of @code{omp_set_num_threads} shall be a positive integer.
569 @multitable @columnfractions .20 .80
570 @item @emph{Prototype}: @tab @code{void omp_set_num_threads(int num_threads);}
573 @item @emph{Fortran}:
574 @multitable @columnfractions .20 .80
575 @item @emph{Interface}: @tab @code{subroutine omp_set_num_threads(num_threads)}
576 @item @tab @code{integer, intent(in) :: num_threads}
579 @item @emph{See also}:
580 @ref{OMP_NUM_THREADS}, @ref{omp_get_num_threads}, @ref{omp_get_max_threads}
582 @item @emph{Reference}:
583 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.1.
588 @node omp_get_num_threads
589 @subsection @code{omp_get_num_threads} -- Size of the active team
591 @item @emph{Description}:
592 Returns the number of threads in the current team. In a sequential section of
593 the program @code{omp_get_num_threads} returns 1.
595 The default team size may be initialized at startup by the
596 @env{OMP_NUM_THREADS} environment variable. At runtime, the size
597 of the current team may be set either by the @code{NUM_THREADS}
598 clause or by @code{omp_set_num_threads}. If none of the above were
599 used to define a specific value and @env{OMP_DYNAMIC} is disabled,
600 one thread per CPU online is used.
603 @multitable @columnfractions .20 .80
604 @item @emph{Prototype}: @tab @code{int omp_get_num_threads(void);}
607 @item @emph{Fortran}:
608 @multitable @columnfractions .20 .80
609 @item @emph{Interface}: @tab @code{integer function omp_get_num_threads()}
612 @item @emph{See also}:
613 @ref{omp_get_max_threads}, @ref{omp_set_num_threads}, @ref{OMP_NUM_THREADS}
615 @item @emph{Reference}:
616 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.2.
621 @node omp_get_max_threads
622 @subsection @code{omp_get_max_threads} -- Maximum number of threads of parallel region
624 @item @emph{Description}:
625 Return the maximum number of threads used for the current parallel region
626 that does not use the clause @code{num_threads}.
629 @multitable @columnfractions .20 .80
630 @item @emph{Prototype}: @tab @code{int omp_get_max_threads(void);}
633 @item @emph{Fortran}:
634 @multitable @columnfractions .20 .80
635 @item @emph{Interface}: @tab @code{integer function omp_get_max_threads()}
638 @item @emph{See also}:
639 @ref{omp_set_num_threads}, @ref{omp_set_dynamic}, @ref{omp_get_thread_limit}
641 @item @emph{Reference}:
642 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.3.
647 @node omp_get_thread_num
648 @subsection @code{omp_get_thread_num} -- Current thread ID
650 @item @emph{Description}:
651 Returns a unique thread identification number within the current team.
652 In a sequential parts of the program, @code{omp_get_thread_num}
653 always returns 0. In parallel regions the return value varies
654 from 0 to @code{omp_get_num_threads}-1 inclusive. The return
655 value of the primary thread of a team is always 0.
658 @multitable @columnfractions .20 .80
659 @item @emph{Prototype}: @tab @code{int omp_get_thread_num(void);}
662 @item @emph{Fortran}:
663 @multitable @columnfractions .20 .80
664 @item @emph{Interface}: @tab @code{integer function omp_get_thread_num()}
667 @item @emph{See also}:
668 @ref{omp_get_num_threads}, @ref{omp_get_ancestor_thread_num}
670 @item @emph{Reference}:
671 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.4.
676 @node omp_in_parallel
677 @subsection @code{omp_in_parallel} -- Whether a parallel region is active
679 @item @emph{Description}:
680 This function returns @code{true} if currently running in parallel,
681 @code{false} otherwise. Here, @code{true} and @code{false} represent
682 their language-specific counterparts.
685 @multitable @columnfractions .20 .80
686 @item @emph{Prototype}: @tab @code{int omp_in_parallel(void);}
689 @item @emph{Fortran}:
690 @multitable @columnfractions .20 .80
691 @item @emph{Interface}: @tab @code{logical function omp_in_parallel()}
694 @item @emph{Reference}:
695 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.6.
699 @node omp_set_dynamic
700 @subsection @code{omp_set_dynamic} -- Enable/disable dynamic teams
702 @item @emph{Description}:
703 Enable or disable the dynamic adjustment of the number of threads
704 within a team. The function takes the language-specific equivalent
705 of @code{true} and @code{false}, where @code{true} enables dynamic
706 adjustment of team sizes and @code{false} disables it.
709 @multitable @columnfractions .20 .80
710 @item @emph{Prototype}: @tab @code{void omp_set_dynamic(int dynamic_threads);}
713 @item @emph{Fortran}:
714 @multitable @columnfractions .20 .80
715 @item @emph{Interface}: @tab @code{subroutine omp_set_dynamic(dynamic_threads)}
716 @item @tab @code{logical, intent(in) :: dynamic_threads}
719 @item @emph{See also}:
720 @ref{OMP_DYNAMIC}, @ref{omp_get_dynamic}
722 @item @emph{Reference}:
723 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.7.
728 @node omp_get_dynamic
729 @subsection @code{omp_get_dynamic} -- Dynamic teams setting
731 @item @emph{Description}:
732 This function returns @code{true} if enabled, @code{false} otherwise.
733 Here, @code{true} and @code{false} represent their language-specific
736 The dynamic team setting may be initialized at startup by the
737 @env{OMP_DYNAMIC} environment variable or at runtime using
738 @code{omp_set_dynamic}. If undefined, dynamic adjustment is
742 @multitable @columnfractions .20 .80
743 @item @emph{Prototype}: @tab @code{int omp_get_dynamic(void);}
746 @item @emph{Fortran}:
747 @multitable @columnfractions .20 .80
748 @item @emph{Interface}: @tab @code{logical function omp_get_dynamic()}
751 @item @emph{See also}:
752 @ref{omp_set_dynamic}, @ref{OMP_DYNAMIC}
754 @item @emph{Reference}:
755 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.8.
760 @node omp_get_cancellation
761 @subsection @code{omp_get_cancellation} -- Whether cancellation support is enabled
763 @item @emph{Description}:
764 This function returns @code{true} if cancellation is activated, @code{false}
765 otherwise. Here, @code{true} and @code{false} represent their language-specific
766 counterparts. Unless @env{OMP_CANCELLATION} is set true, cancellations are
770 @multitable @columnfractions .20 .80
771 @item @emph{Prototype}: @tab @code{int omp_get_cancellation(void);}
774 @item @emph{Fortran}:
775 @multitable @columnfractions .20 .80
776 @item @emph{Interface}: @tab @code{logical function omp_get_cancellation()}
779 @item @emph{See also}:
780 @ref{OMP_CANCELLATION}
782 @item @emph{Reference}:
783 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.9.
789 @subsection @code{omp_set_nested} -- Enable/disable nested parallel regions
791 @item @emph{Description}:
792 Enable or disable nested parallel regions, i.e., whether team members
793 are allowed to create new teams. The function takes the language-specific
794 equivalent of @code{true} and @code{false}, where @code{true} enables
795 dynamic adjustment of team sizes and @code{false} disables it.
797 Enabling nested parallel regions will also set the maximum number of
798 active nested regions to the maximum supported. Disabling nested parallel
799 regions will set the maximum number of active nested regions to one.
801 Note that the @code{omp_set_nested} API routine was deprecated
802 in the OpenMP specification 5.2 in favor of @code{omp_set_max_active_levels}.
805 @multitable @columnfractions .20 .80
806 @item @emph{Prototype}: @tab @code{void omp_set_nested(int nested);}
809 @item @emph{Fortran}:
810 @multitable @columnfractions .20 .80
811 @item @emph{Interface}: @tab @code{subroutine omp_set_nested(nested)}
812 @item @tab @code{logical, intent(in) :: nested}
815 @item @emph{See also}:
816 @ref{omp_get_nested}, @ref{omp_set_max_active_levels},
817 @ref{OMP_MAX_ACTIVE_LEVELS}, @ref{OMP_NESTED}
819 @item @emph{Reference}:
820 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.10.
826 @subsection @code{omp_get_nested} -- Nested parallel regions
828 @item @emph{Description}:
829 This function returns @code{true} if nested parallel regions are
830 enabled, @code{false} otherwise. Here, @code{true} and @code{false}
831 represent their language-specific counterparts.
833 The state of nested parallel regions at startup depends on several
834 environment variables. If @env{OMP_MAX_ACTIVE_LEVELS} is defined
835 and is set to greater than one, then nested parallel regions will be
836 enabled. If not defined, then the value of the @env{OMP_NESTED}
837 environment variable will be followed if defined. If neither are
838 defined, then if either @env{OMP_NUM_THREADS} or @env{OMP_PROC_BIND}
839 are defined with a list of more than one value, then nested parallel
840 regions are enabled. If none of these are defined, then nested parallel
841 regions are disabled by default.
843 Nested parallel regions can be enabled or disabled at runtime using
844 @code{omp_set_nested}, or by setting the maximum number of nested
845 regions with @code{omp_set_max_active_levels} to one to disable, or
848 Note that the @code{omp_get_nested} API routine was deprecated
849 in the OpenMP specification 5.2 in favor of @code{omp_get_max_active_levels}.
852 @multitable @columnfractions .20 .80
853 @item @emph{Prototype}: @tab @code{int omp_get_nested(void);}
856 @item @emph{Fortran}:
857 @multitable @columnfractions .20 .80
858 @item @emph{Interface}: @tab @code{logical function omp_get_nested()}
861 @item @emph{See also}:
862 @ref{omp_get_max_active_levels}, @ref{omp_set_nested},
863 @ref{OMP_MAX_ACTIVE_LEVELS}, @ref{OMP_NESTED}
865 @item @emph{Reference}:
866 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.11.
871 @node omp_set_schedule
872 @subsection @code{omp_set_schedule} -- Set the runtime scheduling method
874 @item @emph{Description}:
875 Sets the runtime scheduling method. The @var{kind} argument can have the
876 value @code{omp_sched_static}, @code{omp_sched_dynamic},
877 @code{omp_sched_guided} or @code{omp_sched_auto}. Except for
878 @code{omp_sched_auto}, the chunk size is set to the value of
879 @var{chunk_size} if positive, or to the default value if zero or negative.
880 For @code{omp_sched_auto} the @var{chunk_size} argument is ignored.
883 @multitable @columnfractions .20 .80
884 @item @emph{Prototype}: @tab @code{void omp_set_schedule(omp_sched_t kind, int chunk_size);}
887 @item @emph{Fortran}:
888 @multitable @columnfractions .20 .80
889 @item @emph{Interface}: @tab @code{subroutine omp_set_schedule(kind, chunk_size)}
890 @item @tab @code{integer(kind=omp_sched_kind) kind}
891 @item @tab @code{integer chunk_size}
894 @item @emph{See also}:
895 @ref{omp_get_schedule}
898 @item @emph{Reference}:
899 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.12.
904 @node omp_get_schedule
905 @subsection @code{omp_get_schedule} -- Obtain the runtime scheduling method
907 @item @emph{Description}:
908 Obtain the runtime scheduling method. The @var{kind} argument will be
909 set to the value @code{omp_sched_static}, @code{omp_sched_dynamic},
910 @code{omp_sched_guided} or @code{omp_sched_auto}. The second argument,
911 @var{chunk_size}, is set to the chunk size.
914 @multitable @columnfractions .20 .80
915 @item @emph{Prototype}: @tab @code{void omp_get_schedule(omp_sched_t *kind, int *chunk_size);}
918 @item @emph{Fortran}:
919 @multitable @columnfractions .20 .80
920 @item @emph{Interface}: @tab @code{subroutine omp_get_schedule(kind, chunk_size)}
921 @item @tab @code{integer(kind=omp_sched_kind) kind}
922 @item @tab @code{integer chunk_size}
925 @item @emph{See also}:
926 @ref{omp_set_schedule}, @ref{OMP_SCHEDULE}
928 @item @emph{Reference}:
929 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.13.
933 @node omp_get_teams_thread_limit
934 @subsection @code{omp_get_teams_thread_limit} -- Maximum number of threads imposed by teams
936 @item @emph{Description}:
937 Return the maximum number of threads that will be able to participate in
938 each team created by a teams construct.
941 @multitable @columnfractions .20 .80
942 @item @emph{Prototype}: @tab @code{int omp_get_teams_thread_limit(void);}
945 @item @emph{Fortran}:
946 @multitable @columnfractions .20 .80
947 @item @emph{Interface}: @tab @code{integer function omp_get_teams_thread_limit()}
950 @item @emph{See also}:
951 @ref{omp_set_teams_thread_limit}, @ref{OMP_TEAMS_THREAD_LIMIT}
953 @item @emph{Reference}:
954 @uref{https://www.openmp.org, OpenMP specification v5.1}, Section 3.4.6.
959 @node omp_get_supported_active_levels
960 @subsection @code{omp_get_supported_active_levels} -- Maximum number of active regions supported
962 @item @emph{Description}:
963 This function returns the maximum number of nested, active parallel regions
964 supported by this implementation.
967 @multitable @columnfractions .20 .80
968 @item @emph{Prototype}: @tab @code{int omp_get_supported_active_levels(void);}
971 @item @emph{Fortran}:
972 @multitable @columnfractions .20 .80
973 @item @emph{Interface}: @tab @code{integer function omp_get_supported_active_levels()}
976 @item @emph{See also}:
977 @ref{omp_get_max_active_levels}, @ref{omp_set_max_active_levels}
979 @item @emph{Reference}:
980 @uref{https://www.openmp.org, OpenMP specification v5.0}, Section 3.2.15.
985 @node omp_set_max_active_levels
986 @subsection @code{omp_set_max_active_levels} -- Limits the number of active parallel regions
988 @item @emph{Description}:
989 This function limits the maximum allowed number of nested, active
990 parallel regions. @var{max_levels} must be less or equal to
991 the value returned by @code{omp_get_supported_active_levels}.
994 @multitable @columnfractions .20 .80
995 @item @emph{Prototype}: @tab @code{void omp_set_max_active_levels(int max_levels);}
998 @item @emph{Fortran}:
999 @multitable @columnfractions .20 .80
1000 @item @emph{Interface}: @tab @code{subroutine omp_set_max_active_levels(max_levels)}
1001 @item @tab @code{integer max_levels}
1004 @item @emph{See also}:
1005 @ref{omp_get_max_active_levels}, @ref{omp_get_active_level},
1006 @ref{omp_get_supported_active_levels}
1008 @item @emph{Reference}:
1009 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.15.
1014 @node omp_get_max_active_levels
1015 @subsection @code{omp_get_max_active_levels} -- Current maximum number of active regions
1017 @item @emph{Description}:
1018 This function obtains the maximum allowed number of nested, active parallel regions.
1021 @multitable @columnfractions .20 .80
1022 @item @emph{Prototype}: @tab @code{int omp_get_max_active_levels(void);}
1025 @item @emph{Fortran}:
1026 @multitable @columnfractions .20 .80
1027 @item @emph{Interface}: @tab @code{integer function omp_get_max_active_levels()}
1030 @item @emph{See also}:
1031 @ref{omp_set_max_active_levels}, @ref{omp_get_active_level}
1033 @item @emph{Reference}:
1034 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.16.
1039 @subsection @code{omp_get_level} -- Obtain the current nesting level
1041 @item @emph{Description}:
1042 This function returns the nesting level for the parallel blocks,
1043 which enclose the calling call.
1046 @multitable @columnfractions .20 .80
1047 @item @emph{Prototype}: @tab @code{int omp_get_level(void);}
1050 @item @emph{Fortran}:
1051 @multitable @columnfractions .20 .80
1052 @item @emph{Interface}: @tab @code{integer function omp_level()}
1055 @item @emph{See also}:
1056 @ref{omp_get_active_level}
1058 @item @emph{Reference}:
1059 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.17.
1064 @node omp_get_ancestor_thread_num
1065 @subsection @code{omp_get_ancestor_thread_num} -- Ancestor thread ID
1067 @item @emph{Description}:
1068 This function returns the thread identification number for the given
1069 nesting level of the current thread. For values of @var{level} outside
1070 zero to @code{omp_get_level} -1 is returned; if @var{level} is
1071 @code{omp_get_level} the result is identical to @code{omp_get_thread_num}.
1074 @multitable @columnfractions .20 .80
1075 @item @emph{Prototype}: @tab @code{int omp_get_ancestor_thread_num(int level);}
1078 @item @emph{Fortran}:
1079 @multitable @columnfractions .20 .80
1080 @item @emph{Interface}: @tab @code{integer function omp_get_ancestor_thread_num(level)}
1081 @item @tab @code{integer level}
1084 @item @emph{See also}:
1085 @ref{omp_get_level}, @ref{omp_get_thread_num}, @ref{omp_get_team_size}
1087 @item @emph{Reference}:
1088 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.18.
1093 @node omp_get_team_size
1094 @subsection @code{omp_get_team_size} -- Number of threads in a team
1096 @item @emph{Description}:
1097 This function returns the number of threads in a thread team to which
1098 either the current thread or its ancestor belongs. For values of @var{level}
1099 outside zero to @code{omp_get_level}, -1 is returned; if @var{level} is zero,
1100 1 is returned, and for @code{omp_get_level}, the result is identical
1101 to @code{omp_get_num_threads}.
1104 @multitable @columnfractions .20 .80
1105 @item @emph{Prototype}: @tab @code{int omp_get_team_size(int level);}
1108 @item @emph{Fortran}:
1109 @multitable @columnfractions .20 .80
1110 @item @emph{Interface}: @tab @code{integer function omp_get_team_size(level)}
1111 @item @tab @code{integer level}
1114 @item @emph{See also}:
1115 @ref{omp_get_num_threads}, @ref{omp_get_level}, @ref{omp_get_ancestor_thread_num}
1117 @item @emph{Reference}:
1118 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.19.
1123 @node omp_get_active_level
1124 @subsection @code{omp_get_active_level} -- Number of parallel regions
1126 @item @emph{Description}:
1127 This function returns the nesting level for the active parallel blocks,
1128 which enclose the calling call.
1131 @multitable @columnfractions .20 .80
1132 @item @emph{Prototype}: @tab @code{int omp_get_active_level(void);}
1135 @item @emph{Fortran}:
1136 @multitable @columnfractions .20 .80
1137 @item @emph{Interface}: @tab @code{integer function omp_get_active_level()}
1140 @item @emph{See also}:
1141 @ref{omp_get_level}, @ref{omp_get_max_active_levels}, @ref{omp_set_max_active_levels}
1143 @item @emph{Reference}:
1144 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.20.
1149 @node Thread Affinity Routines
1150 @section Thread Affinity Routines
1152 Routines controlling and accessing thread-affinity policies.
1153 They have C linkage and do not throw exceptions.
1156 * omp_get_proc_bind:: Whether threads may be moved between CPUs
1157 @c * omp_get_num_places:: <fixme>
1158 @c * omp_get_place_num_procs:: <fixme>
1159 @c * omp_get_place_proc_ids:: <fixme>
1160 @c * omp_get_place_num:: <fixme>
1161 @c * omp_get_partition_num_places:: <fixme>
1162 @c * omp_get_partition_place_nums:: <fixme>
1163 @c * omp_set_affinity_format:: <fixme>
1164 @c * omp_get_affinity_format:: <fixme>
1165 @c * omp_display_affinity:: <fixme>
1166 @c * omp_capture_affinity:: <fixme>
1171 @node omp_get_proc_bind
1172 @subsection @code{omp_get_proc_bind} -- Whether threads may be moved between CPUs
1174 @item @emph{Description}:
1175 This functions returns the currently active thread affinity policy, which is
1176 set via @env{OMP_PROC_BIND}. Possible values are @code{omp_proc_bind_false},
1177 @code{omp_proc_bind_true}, @code{omp_proc_bind_primary},
1178 @code{omp_proc_bind_master}, @code{omp_proc_bind_close} and @code{omp_proc_bind_spread},
1179 where @code{omp_proc_bind_master} is an alias for @code{omp_proc_bind_primary}.
1182 @multitable @columnfractions .20 .80
1183 @item @emph{Prototype}: @tab @code{omp_proc_bind_t omp_get_proc_bind(void);}
1186 @item @emph{Fortran}:
1187 @multitable @columnfractions .20 .80
1188 @item @emph{Interface}: @tab @code{integer(kind=omp_proc_bind_kind) function omp_get_proc_bind()}
1191 @item @emph{See also}:
1192 @ref{OMP_PROC_BIND}, @ref{OMP_PLACES}, @ref{GOMP_CPU_AFFINITY},
1194 @item @emph{Reference}:
1195 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.22.
1200 @node Teams Region Routines
1201 @section Teams Region Routines
1203 Routines controlling the league of teams that are executed in a @code{teams}
1204 region. They have C linkage and do not throw exceptions.
1207 * omp_get_num_teams:: Number of teams
1208 * omp_get_team_num:: Get team number
1209 * omp_set_num_teams:: Set upper teams limit for teams region
1210 * omp_get_max_teams:: Maximum number of teams for teams region
1211 * omp_set_teams_thread_limit:: Set upper thread limit for teams construct
1212 * omp_get_thread_limit:: Maximum number of threads
1217 @node omp_get_num_teams
1218 @subsection @code{omp_get_num_teams} -- Number of teams
1220 @item @emph{Description}:
1221 Returns the number of teams in the current team region.
1224 @multitable @columnfractions .20 .80
1225 @item @emph{Prototype}: @tab @code{int omp_get_num_teams(void);}
1228 @item @emph{Fortran}:
1229 @multitable @columnfractions .20 .80
1230 @item @emph{Interface}: @tab @code{integer function omp_get_num_teams()}
1233 @item @emph{Reference}:
1234 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.32.
1239 @node omp_get_team_num
1240 @subsection @code{omp_get_team_num} -- Get team number
1242 @item @emph{Description}:
1243 Returns the team number of the calling thread.
1246 @multitable @columnfractions .20 .80
1247 @item @emph{Prototype}: @tab @code{int omp_get_team_num(void);}
1250 @item @emph{Fortran}:
1251 @multitable @columnfractions .20 .80
1252 @item @emph{Interface}: @tab @code{integer function omp_get_team_num()}
1255 @item @emph{Reference}:
1256 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.33.
1261 @node omp_set_num_teams
1262 @subsection @code{omp_set_num_teams} -- Set upper teams limit for teams construct
1264 @item @emph{Description}:
1265 Specifies the upper bound for number of teams created by the teams construct
1266 which does not specify a @code{num_teams} clause. The
1267 argument of @code{omp_set_num_teams} shall be a positive integer.
1270 @multitable @columnfractions .20 .80
1271 @item @emph{Prototype}: @tab @code{void omp_set_num_teams(int num_teams);}
1274 @item @emph{Fortran}:
1275 @multitable @columnfractions .20 .80
1276 @item @emph{Interface}: @tab @code{subroutine omp_set_num_teams(num_teams)}
1277 @item @tab @code{integer, intent(in) :: num_teams}
1280 @item @emph{See also}:
1281 @ref{OMP_NUM_TEAMS}, @ref{omp_get_num_teams}, @ref{omp_get_max_teams}
1283 @item @emph{Reference}:
1284 @uref{https://www.openmp.org, OpenMP specification v5.1}, Section 3.4.3.
1289 @node omp_get_max_teams
1290 @subsection @code{omp_get_max_teams} -- Maximum number of teams of teams region
1292 @item @emph{Description}:
1293 Return the maximum number of teams used for the teams region
1294 that does not use the clause @code{num_teams}.
1297 @multitable @columnfractions .20 .80
1298 @item @emph{Prototype}: @tab @code{int omp_get_max_teams(void);}
1301 @item @emph{Fortran}:
1302 @multitable @columnfractions .20 .80
1303 @item @emph{Interface}: @tab @code{integer function omp_get_max_teams()}
1306 @item @emph{See also}:
1307 @ref{omp_set_num_teams}, @ref{omp_get_num_teams}
1309 @item @emph{Reference}:
1310 @uref{https://www.openmp.org, OpenMP specification v5.1}, Section 3.4.4.
1315 @node omp_set_teams_thread_limit
1316 @subsection @code{omp_set_teams_thread_limit} -- Set upper thread limit for teams construct
1318 @item @emph{Description}:
1319 Specifies the upper bound for number of threads that will be available
1320 for each team created by the teams construct which does not specify a
1321 @code{thread_limit} clause. The argument of
1322 @code{omp_set_teams_thread_limit} shall be a positive integer.
1325 @multitable @columnfractions .20 .80
1326 @item @emph{Prototype}: @tab @code{void omp_set_teams_thread_limit(int thread_limit);}
1329 @item @emph{Fortran}:
1330 @multitable @columnfractions .20 .80
1331 @item @emph{Interface}: @tab @code{subroutine omp_set_teams_thread_limit(thread_limit)}
1332 @item @tab @code{integer, intent(in) :: thread_limit}
1335 @item @emph{See also}:
1336 @ref{OMP_TEAMS_THREAD_LIMIT}, @ref{omp_get_teams_thread_limit}, @ref{omp_get_thread_limit}
1338 @item @emph{Reference}:
1339 @uref{https://www.openmp.org, OpenMP specification v5.1}, Section 3.4.5.
1344 @node omp_get_thread_limit
1345 @subsection @code{omp_get_thread_limit} -- Maximum number of threads
1347 @item @emph{Description}:
1348 Return the maximum number of threads of the program.
1351 @multitable @columnfractions .20 .80
1352 @item @emph{Prototype}: @tab @code{int omp_get_thread_limit(void);}
1355 @item @emph{Fortran}:
1356 @multitable @columnfractions .20 .80
1357 @item @emph{Interface}: @tab @code{integer function omp_get_thread_limit()}
1360 @item @emph{See also}:
1361 @ref{omp_get_max_threads}, @ref{OMP_THREAD_LIMIT}
1363 @item @emph{Reference}:
1364 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.14.
1369 @node Tasking Routines
1370 @section Tasking Routines
1372 Routines relating to explicit tasks.
1373 They have C linkage and do not throw exceptions.
1376 * omp_get_max_task_priority:: Maximum task priority value that can be set
1377 * omp_in_explicit_task:: Whether a given task is an explicit task
1378 * omp_in_final:: Whether in final or included task region
1383 @node omp_get_max_task_priority
1384 @subsection @code{omp_get_max_task_priority} -- Maximum priority value
1385 that can be set for tasks.
1387 @item @emph{Description}:
1388 This function obtains the maximum allowed priority number for tasks.
1391 @multitable @columnfractions .20 .80
1392 @item @emph{Prototype}: @tab @code{int omp_get_max_task_priority(void);}
1395 @item @emph{Fortran}:
1396 @multitable @columnfractions .20 .80
1397 @item @emph{Interface}: @tab @code{integer function omp_get_max_task_priority()}
1400 @item @emph{Reference}:
1401 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.29.
1406 @node omp_in_explicit_task
1407 @subsection @code{omp_in_explicit_task} -- Whether a given task is an explicit task
1409 @item @emph{Description}:
1410 The function returns the @var{explicit-task-var} ICV; it returns true when the
1411 encountering task was generated by a task-generating construct such as
1412 @code{target}, @code{task} or @code{taskloop}. Otherwise, the encountering task
1413 is in an implicit task region such as generated by the implicit or explicit
1414 @code{parallel} region and @code{omp_in_explicit_task} returns false.
1417 @multitable @columnfractions .20 .80
1418 @item @emph{Prototype}: @tab @code{int omp_in_explicit_task(void);}
1421 @item @emph{Fortran}:
1422 @multitable @columnfractions .20 .80
1423 @item @emph{Interface}: @tab @code{logical function omp_in_explicit_task()}
1426 @item @emph{Reference}:
1427 @uref{https://www.openmp.org, OpenMP specification v5.2}, Section 18.5.2.
1433 @subsection @code{omp_in_final} -- Whether in final or included task region
1435 @item @emph{Description}:
1436 This function returns @code{true} if currently running in a final
1437 or included task region, @code{false} otherwise. Here, @code{true}
1438 and @code{false} represent their language-specific counterparts.
1441 @multitable @columnfractions .20 .80
1442 @item @emph{Prototype}: @tab @code{int omp_in_final(void);}
1445 @item @emph{Fortran}:
1446 @multitable @columnfractions .20 .80
1447 @item @emph{Interface}: @tab @code{logical function omp_in_final()}
1450 @item @emph{Reference}:
1451 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.21.
1456 @c @node Resource Relinquishing Routines
1457 @c @section Resource Relinquishing Routines
1459 @c Routines releasing resources used by the OpenMP runtime.
1460 @c They have C linkage and do not throw exceptions.
1463 @c * omp_pause_resource:: <fixme>
1464 @c * omp_pause_resource_all:: <fixme>
1467 @node Device Information Routines
1468 @section Device Information Routines
1470 Routines related to devices available to an OpenMP program.
1471 They have C linkage and do not throw exceptions.
1474 * omp_get_num_procs:: Number of processors online
1475 @c * omp_get_max_progress_width:: <fixme>/TR11
1476 * omp_set_default_device:: Set the default device for target regions
1477 * omp_get_default_device:: Get the default device for target regions
1478 * omp_get_num_devices:: Number of target devices
1479 * omp_get_device_num:: Get device that current thread is running on
1480 * omp_is_initial_device:: Whether executing on the host device
1481 * omp_get_initial_device:: Device number of host device
1486 @node omp_get_num_procs
1487 @subsection @code{omp_get_num_procs} -- Number of processors online
1489 @item @emph{Description}:
1490 Returns the number of processors online on that device.
1493 @multitable @columnfractions .20 .80
1494 @item @emph{Prototype}: @tab @code{int omp_get_num_procs(void);}
1497 @item @emph{Fortran}:
1498 @multitable @columnfractions .20 .80
1499 @item @emph{Interface}: @tab @code{integer function omp_get_num_procs()}
1502 @item @emph{Reference}:
1503 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.5.
1508 @node omp_set_default_device
1509 @subsection @code{omp_set_default_device} -- Set the default device for target regions
1511 @item @emph{Description}:
1512 Set the default device for target regions without device clause. The argument
1513 shall be a nonnegative device number.
1516 @multitable @columnfractions .20 .80
1517 @item @emph{Prototype}: @tab @code{void omp_set_default_device(int device_num);}
1520 @item @emph{Fortran}:
1521 @multitable @columnfractions .20 .80
1522 @item @emph{Interface}: @tab @code{subroutine omp_set_default_device(device_num)}
1523 @item @tab @code{integer device_num}
1526 @item @emph{See also}:
1527 @ref{OMP_DEFAULT_DEVICE}, @ref{omp_get_default_device}
1529 @item @emph{Reference}:
1530 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.29.
1535 @node omp_get_default_device
1536 @subsection @code{omp_get_default_device} -- Get the default device for target regions
1538 @item @emph{Description}:
1539 Get the default device for target regions without device clause.
1542 @multitable @columnfractions .20 .80
1543 @item @emph{Prototype}: @tab @code{int omp_get_default_device(void);}
1546 @item @emph{Fortran}:
1547 @multitable @columnfractions .20 .80
1548 @item @emph{Interface}: @tab @code{integer function omp_get_default_device()}
1551 @item @emph{See also}:
1552 @ref{OMP_DEFAULT_DEVICE}, @ref{omp_set_default_device}
1554 @item @emph{Reference}:
1555 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.30.
1560 @node omp_get_num_devices
1561 @subsection @code{omp_get_num_devices} -- Number of target devices
1563 @item @emph{Description}:
1564 Returns the number of target devices.
1567 @multitable @columnfractions .20 .80
1568 @item @emph{Prototype}: @tab @code{int omp_get_num_devices(void);}
1571 @item @emph{Fortran}:
1572 @multitable @columnfractions .20 .80
1573 @item @emph{Interface}: @tab @code{integer function omp_get_num_devices()}
1576 @item @emph{Reference}:
1577 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.31.
1582 @node omp_get_device_num
1583 @subsection @code{omp_get_device_num} -- Return device number of current device
1585 @item @emph{Description}:
1586 This function returns a device number that represents the device that the
1587 current thread is executing on. For OpenMP 5.0, this must be equal to the
1588 value returned by the @code{omp_get_initial_device} function when called
1592 @multitable @columnfractions .20 .80
1593 @item @emph{Prototype}: @tab @code{int omp_get_device_num(void);}
1596 @item @emph{Fortran}:
1597 @multitable @columnfractions .20 .80
1598 @item @emph{Interface}: @tab @code{integer function omp_get_device_num()}
1601 @item @emph{See also}:
1602 @ref{omp_get_initial_device}
1604 @item @emph{Reference}:
1605 @uref{https://www.openmp.org, OpenMP specification v5.0}, Section 3.2.37.
1610 @node omp_is_initial_device
1611 @subsection @code{omp_is_initial_device} -- Whether executing on the host device
1613 @item @emph{Description}:
1614 This function returns @code{true} if currently running on the host device,
1615 @code{false} otherwise. Here, @code{true} and @code{false} represent
1616 their language-specific counterparts.
1619 @multitable @columnfractions .20 .80
1620 @item @emph{Prototype}: @tab @code{int omp_is_initial_device(void);}
1623 @item @emph{Fortran}:
1624 @multitable @columnfractions .20 .80
1625 @item @emph{Interface}: @tab @code{logical function omp_is_initial_device()}
1628 @item @emph{Reference}:
1629 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.34.
1634 @node omp_get_initial_device
1635 @subsection @code{omp_get_initial_device} -- Return device number of initial device
1637 @item @emph{Description}:
1638 This function returns a device number that represents the host device.
1639 For OpenMP 5.1, this must be equal to the value returned by the
1640 @code{omp_get_num_devices} function.
1643 @multitable @columnfractions .20 .80
1644 @item @emph{Prototype}: @tab @code{int omp_get_initial_device(void);}
1647 @item @emph{Fortran}:
1648 @multitable @columnfractions .20 .80
1649 @item @emph{Interface}: @tab @code{integer function omp_get_initial_device()}
1652 @item @emph{See also}:
1653 @ref{omp_get_num_devices}
1655 @item @emph{Reference}:
1656 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.2.35.
1661 @c @node Device Memory Routines
1662 @c @section Device Memory Routines
1664 @c Routines related to memory allocation and managing corresponding
1665 @c pointers on devices. They have C linkage and do not throw exceptions.
1668 @c * omp_target_alloc:: <fixme>
1669 @c * omp_target_free:: <fixme>
1670 @c * omp_target_is_present:: <fixme>
1671 @c * omp_target_is_accessible:: <fixme>
1672 @c * omp_target_memcpy:: <fixme>
1673 @c * omp_target_memcpy_rect:: <fixme>
1674 @c * omp_target_memcpy_async:: <fixme>
1675 @c * omp_target_memcpy_rect_async:: <fixme>
1676 @c * omp_target_associate_ptr:: <fixme>
1677 @c * omp_target_disassociate_ptr:: <fixme>
1678 @c * omp_get_mapped_ptr:: <fixme>
1682 @section Lock Routines
1684 Initialize, set, test, unset and destroy simple and nested locks.
1685 The routines have C linkage and do not throw exceptions.
1688 * omp_init_lock:: Initialize simple lock
1689 * omp_init_nest_lock:: Initialize nested lock
1690 @c * omp_init_lock_with_hint:: <fixme>
1691 @c * omp_init_nest_lock_with_hint:: <fixme>
1692 * omp_destroy_lock:: Destroy simple lock
1693 * omp_destroy_nest_lock:: Destroy nested lock
1694 * omp_set_lock:: Wait for and set simple lock
1695 * omp_set_nest_lock:: Wait for and set simple lock
1696 * omp_unset_lock:: Unset simple lock
1697 * omp_unset_nest_lock:: Unset nested lock
1698 * omp_test_lock:: Test and set simple lock if available
1699 * omp_test_nest_lock:: Test and set nested lock if available
1705 @subsection @code{omp_init_lock} -- Initialize simple lock
1707 @item @emph{Description}:
1708 Initialize a simple lock. After initialization, the lock is in
1712 @multitable @columnfractions .20 .80
1713 @item @emph{Prototype}: @tab @code{void omp_init_lock(omp_lock_t *lock);}
1716 @item @emph{Fortran}:
1717 @multitable @columnfractions .20 .80
1718 @item @emph{Interface}: @tab @code{subroutine omp_init_lock(svar)}
1719 @item @tab @code{integer(omp_lock_kind), intent(out) :: svar}
1722 @item @emph{See also}:
1723 @ref{omp_destroy_lock}
1725 @item @emph{Reference}:
1726 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.3.1.
1731 @node omp_init_nest_lock
1732 @subsection @code{omp_init_nest_lock} -- Initialize nested lock
1734 @item @emph{Description}:
1735 Initialize a nested lock. After initialization, the lock is in
1736 an unlocked state and the nesting count is set to zero.
1739 @multitable @columnfractions .20 .80
1740 @item @emph{Prototype}: @tab @code{void omp_init_nest_lock(omp_nest_lock_t *lock);}
1743 @item @emph{Fortran}:
1744 @multitable @columnfractions .20 .80
1745 @item @emph{Interface}: @tab @code{subroutine omp_init_nest_lock(nvar)}
1746 @item @tab @code{integer(omp_nest_lock_kind), intent(out) :: nvar}
1749 @item @emph{See also}:
1750 @ref{omp_destroy_nest_lock}
1752 @item @emph{Reference}:
1753 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.3.1.
1758 @node omp_destroy_lock
1759 @subsection @code{omp_destroy_lock} -- Destroy simple lock
1761 @item @emph{Description}:
1762 Destroy a simple lock. In order to be destroyed, a simple lock must be
1763 in the unlocked state.
1766 @multitable @columnfractions .20 .80
1767 @item @emph{Prototype}: @tab @code{void omp_destroy_lock(omp_lock_t *lock);}
1770 @item @emph{Fortran}:
1771 @multitable @columnfractions .20 .80
1772 @item @emph{Interface}: @tab @code{subroutine omp_destroy_lock(svar)}
1773 @item @tab @code{integer(omp_lock_kind), intent(inout) :: svar}
1776 @item @emph{See also}:
1779 @item @emph{Reference}:
1780 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.3.3.
1785 @node omp_destroy_nest_lock
1786 @subsection @code{omp_destroy_nest_lock} -- Destroy nested lock
1788 @item @emph{Description}:
1789 Destroy a nested lock. In order to be destroyed, a nested lock must be
1790 in the unlocked state and its nesting count must equal zero.
1793 @multitable @columnfractions .20 .80
1794 @item @emph{Prototype}: @tab @code{void omp_destroy_nest_lock(omp_nest_lock_t *);}
1797 @item @emph{Fortran}:
1798 @multitable @columnfractions .20 .80
1799 @item @emph{Interface}: @tab @code{subroutine omp_destroy_nest_lock(nvar)}
1800 @item @tab @code{integer(omp_nest_lock_kind), intent(inout) :: nvar}
1803 @item @emph{See also}:
1806 @item @emph{Reference}:
1807 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.3.3.
1813 @subsection @code{omp_set_lock} -- Wait for and set simple lock
1815 @item @emph{Description}:
1816 Before setting a simple lock, the lock variable must be initialized by
1817 @code{omp_init_lock}. The calling thread is blocked until the lock
1818 is available. If the lock is already held by the current thread,
1822 @multitable @columnfractions .20 .80
1823 @item @emph{Prototype}: @tab @code{void omp_set_lock(omp_lock_t *lock);}
1826 @item @emph{Fortran}:
1827 @multitable @columnfractions .20 .80
1828 @item @emph{Interface}: @tab @code{subroutine omp_set_lock(svar)}
1829 @item @tab @code{integer(omp_lock_kind), intent(inout) :: svar}
1832 @item @emph{See also}:
1833 @ref{omp_init_lock}, @ref{omp_test_lock}, @ref{omp_unset_lock}
1835 @item @emph{Reference}:
1836 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.3.4.
1841 @node omp_set_nest_lock
1842 @subsection @code{omp_set_nest_lock} -- Wait for and set nested lock
1844 @item @emph{Description}:
1845 Before setting a nested lock, the lock variable must be initialized by
1846 @code{omp_init_nest_lock}. The calling thread is blocked until the lock
1847 is available. If the lock is already held by the current thread, the
1848 nesting count for the lock is incremented.
1851 @multitable @columnfractions .20 .80
1852 @item @emph{Prototype}: @tab @code{void omp_set_nest_lock(omp_nest_lock_t *lock);}
1855 @item @emph{Fortran}:
1856 @multitable @columnfractions .20 .80
1857 @item @emph{Interface}: @tab @code{subroutine omp_set_nest_lock(nvar)}
1858 @item @tab @code{integer(omp_nest_lock_kind), intent(inout) :: nvar}
1861 @item @emph{See also}:
1862 @ref{omp_init_nest_lock}, @ref{omp_unset_nest_lock}
1864 @item @emph{Reference}:
1865 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.3.4.
1870 @node omp_unset_lock
1871 @subsection @code{omp_unset_lock} -- Unset simple lock
1873 @item @emph{Description}:
1874 A simple lock about to be unset must have been locked by @code{omp_set_lock}
1875 or @code{omp_test_lock} before. In addition, the lock must be held by the
1876 thread calling @code{omp_unset_lock}. Then, the lock becomes unlocked. If one
1877 or more threads attempted to set the lock before, one of them is chosen to,
1878 again, set the lock to itself.
1881 @multitable @columnfractions .20 .80
1882 @item @emph{Prototype}: @tab @code{void omp_unset_lock(omp_lock_t *lock);}
1885 @item @emph{Fortran}:
1886 @multitable @columnfractions .20 .80
1887 @item @emph{Interface}: @tab @code{subroutine omp_unset_lock(svar)}
1888 @item @tab @code{integer(omp_lock_kind), intent(inout) :: svar}
1891 @item @emph{See also}:
1892 @ref{omp_set_lock}, @ref{omp_test_lock}
1894 @item @emph{Reference}:
1895 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.3.5.
1900 @node omp_unset_nest_lock
1901 @subsection @code{omp_unset_nest_lock} -- Unset nested lock
1903 @item @emph{Description}:
1904 A nested lock about to be unset must have been locked by @code{omp_set_nested_lock}
1905 or @code{omp_test_nested_lock} before. In addition, the lock must be held by the
1906 thread calling @code{omp_unset_nested_lock}. If the nesting count drops to zero, the
1907 lock becomes unlocked. If one ore more threads attempted to set the lock before,
1908 one of them is chosen to, again, set the lock to itself.
1911 @multitable @columnfractions .20 .80
1912 @item @emph{Prototype}: @tab @code{void omp_unset_nest_lock(omp_nest_lock_t *lock);}
1915 @item @emph{Fortran}:
1916 @multitable @columnfractions .20 .80
1917 @item @emph{Interface}: @tab @code{subroutine omp_unset_nest_lock(nvar)}
1918 @item @tab @code{integer(omp_nest_lock_kind), intent(inout) :: nvar}
1921 @item @emph{See also}:
1922 @ref{omp_set_nest_lock}
1924 @item @emph{Reference}:
1925 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.3.5.
1931 @subsection @code{omp_test_lock} -- Test and set simple lock if available
1933 @item @emph{Description}:
1934 Before setting a simple lock, the lock variable must be initialized by
1935 @code{omp_init_lock}. Contrary to @code{omp_set_lock}, @code{omp_test_lock}
1936 does not block if the lock is not available. This function returns
1937 @code{true} upon success, @code{false} otherwise. Here, @code{true} and
1938 @code{false} represent their language-specific counterparts.
1941 @multitable @columnfractions .20 .80
1942 @item @emph{Prototype}: @tab @code{int omp_test_lock(omp_lock_t *lock);}
1945 @item @emph{Fortran}:
1946 @multitable @columnfractions .20 .80
1947 @item @emph{Interface}: @tab @code{logical function omp_test_lock(svar)}
1948 @item @tab @code{integer(omp_lock_kind), intent(inout) :: svar}
1951 @item @emph{See also}:
1952 @ref{omp_init_lock}, @ref{omp_set_lock}, @ref{omp_set_lock}
1954 @item @emph{Reference}:
1955 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.3.6.
1960 @node omp_test_nest_lock
1961 @subsection @code{omp_test_nest_lock} -- Test and set nested lock if available
1963 @item @emph{Description}:
1964 Before setting a nested lock, the lock variable must be initialized by
1965 @code{omp_init_nest_lock}. Contrary to @code{omp_set_nest_lock},
1966 @code{omp_test_nest_lock} does not block if the lock is not available.
1967 If the lock is already held by the current thread, the new nesting count
1968 is returned. Otherwise, the return value equals zero.
1971 @multitable @columnfractions .20 .80
1972 @item @emph{Prototype}: @tab @code{int omp_test_nest_lock(omp_nest_lock_t *lock);}
1975 @item @emph{Fortran}:
1976 @multitable @columnfractions .20 .80
1977 @item @emph{Interface}: @tab @code{logical function omp_test_nest_lock(nvar)}
1978 @item @tab @code{integer(omp_nest_lock_kind), intent(inout) :: nvar}
1982 @item @emph{See also}:
1983 @ref{omp_init_lock}, @ref{omp_set_lock}, @ref{omp_set_lock}
1985 @item @emph{Reference}:
1986 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.3.6.
1991 @node Timing Routines
1992 @section Timing Routines
1994 Portable, thread-based, wall clock timer.
1995 The routines have C linkage and do not throw exceptions.
1998 * omp_get_wtick:: Get timer precision.
1999 * omp_get_wtime:: Elapsed wall clock time.
2005 @subsection @code{omp_get_wtick} -- Get timer precision
2007 @item @emph{Description}:
2008 Gets the timer precision, i.e., the number of seconds between two
2009 successive clock ticks.
2012 @multitable @columnfractions .20 .80
2013 @item @emph{Prototype}: @tab @code{double omp_get_wtick(void);}
2016 @item @emph{Fortran}:
2017 @multitable @columnfractions .20 .80
2018 @item @emph{Interface}: @tab @code{double precision function omp_get_wtick()}
2021 @item @emph{See also}:
2024 @item @emph{Reference}:
2025 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.4.2.
2031 @subsection @code{omp_get_wtime} -- Elapsed wall clock time
2033 @item @emph{Description}:
2034 Elapsed wall clock time in seconds. The time is measured per thread, no
2035 guarantee can be made that two distinct threads measure the same time.
2036 Time is measured from some "time in the past", which is an arbitrary time
2037 guaranteed not to change during the execution of the program.
2040 @multitable @columnfractions .20 .80
2041 @item @emph{Prototype}: @tab @code{double omp_get_wtime(void);}
2044 @item @emph{Fortran}:
2045 @multitable @columnfractions .20 .80
2046 @item @emph{Interface}: @tab @code{double precision function omp_get_wtime()}
2049 @item @emph{See also}:
2052 @item @emph{Reference}:
2053 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 3.4.1.
2059 @section Event Routine
2061 Support for event objects.
2062 The routine has C linkage and do not throw exceptions.
2065 * omp_fulfill_event:: Fulfill and destroy an OpenMP event.
2070 @node omp_fulfill_event
2071 @subsection @code{omp_fulfill_event} -- Fulfill and destroy an OpenMP event
2073 @item @emph{Description}:
2074 Fulfill the event associated with the event handle argument. Currently, it
2075 is only used to fulfill events generated by detach clauses on task
2076 constructs - the effect of fulfilling the event is to allow the task to
2079 The result of calling @code{omp_fulfill_event} with an event handle other
2080 than that generated by a detach clause is undefined. Calling it with an
2081 event handle that has already been fulfilled is also undefined.
2084 @multitable @columnfractions .20 .80
2085 @item @emph{Prototype}: @tab @code{void omp_fulfill_event(omp_event_handle_t event);}
2088 @item @emph{Fortran}:
2089 @multitable @columnfractions .20 .80
2090 @item @emph{Interface}: @tab @code{subroutine omp_fulfill_event(event)}
2091 @item @tab @code{integer (kind=omp_event_handle_kind) :: event}
2094 @item @emph{Reference}:
2095 @uref{https://www.openmp.org, OpenMP specification v5.0}, Section 3.5.1.
2100 @c @node Interoperability Routines
2101 @c @section Interoperability Routines
2103 @c Routines to obtain properties from an @code{omp_interop_t} object.
2104 @c They have C linkage and do not throw exceptions.
2107 @c * omp_get_num_interop_properties:: <fixme>
2108 @c * omp_get_interop_int:: <fixme>
2109 @c * omp_get_interop_ptr:: <fixme>
2110 @c * omp_get_interop_str:: <fixme>
2111 @c * omp_get_interop_name:: <fixme>
2112 @c * omp_get_interop_type_desc:: <fixme>
2113 @c * omp_get_interop_rc_desc:: <fixme>
2116 @node Memory Management Routines
2117 @section Memory Management Routines
2119 Routines to manage and allocate memory on the current device.
2120 They have C linkage and do not throw exceptions.
2123 * omp_init_allocator:: Create an allocator
2124 * omp_destroy_allocator:: Destroy an allocator
2125 * omp_set_default_allocator:: Set the default allocator
2126 * omp_get_default_allocator:: Get the default allocator
2127 @c * omp_alloc:: <fixme>
2128 @c * omp_aligned_alloc:: <fixme>
2129 @c * omp_free:: <fixme>
2130 @c * omp_calloc:: <fixme>
2131 @c * omp_aligned_calloc:: <fixme>
2132 @c * omp_realloc:: <fixme>
2133 @c * omp_get_memspace_num_resources:: <fixme>/TR11
2134 @c * omp_get_submemspace:: <fixme>/TR11
2139 @node omp_init_allocator
2140 @subsection @code{omp_init_allocator} -- Create an allocator
2142 @item @emph{Description}:
2143 Create an allocator that uses the specified memory space and has the specified
2144 traits; if an allocator that fulfills the requirements cannot be created,
2145 @code{omp_null_allocator} is returned.
2147 The predefined memory spaces and available traits can be found at
2148 @ref{OMP_ALLOCATOR}, where the trait names have to be be prefixed by
2149 @code{omp_atk_} (e.g. @code{omp_atk_pinned}) and the named trait values by
2150 @code{omp_atv_} (e.g. @code{omp_atv_true}); additionally, @code{omp_atv_default}
2151 may be used as trait value to specify that the default value should be used.
2154 @multitable @columnfractions .20 .80
2155 @item @emph{Prototype}: @tab @code{omp_allocator_handle_t omp_init_allocator(}
2156 @item @tab @code{ omp_memspace_handle_t memspace,}
2157 @item @tab @code{ int ntraits,}
2158 @item @tab @code{ const omp_alloctrait_t traits[]);}
2161 @item @emph{Fortran}:
2162 @multitable @columnfractions .20 .80
2163 @item @emph{Interface}: @tab @code{function omp_init_allocator(memspace, ntraits, traits)}
2164 @item @tab @code{integer (kind=omp_allocator_handle_kind) :: omp_init_allocator}
2165 @item @tab @code{integer (kind=omp_memspace_handle_kind), intent(in) :: memspace}
2166 @item @tab @code{integer, intent(in) :: ntraits}
2167 @item @tab @code{type (omp_alloctrait), intent(in) :: traits(*)}
2170 @item @emph{See also}:
2171 @ref{OMP_ALLOCATOR}, @ref{Memory allocation}, @ref{omp_destroy_allocator}
2173 @item @emph{Reference}:
2174 @uref{https://www.openmp.org, OpenMP specification v5.0}, Section 3.7.2
2179 @node omp_destroy_allocator
2180 @subsection @code{omp_destroy_allocator} -- Destroy an allocator
2182 @item @emph{Description}:
2183 Releases all resources used by a memory allocator, which must not represent
2184 a predefined memory allocator. Accessing memory after its allocator has been
2185 destroyed has unspecified behavior. Passing @code{omp_null_allocator} to the
2186 routine is permitted but will have no effect.
2190 @multitable @columnfractions .20 .80
2191 @item @emph{Prototype}: @tab @code{void omp_destroy_allocator (omp_allocator_handle_t allocator);}
2194 @item @emph{Fortran}:
2195 @multitable @columnfractions .20 .80
2196 @item @emph{Interface}: @tab @code{subroutine omp_destroy_allocator(allocator)}
2197 @item @tab @code{integer (kind=omp_allocator_handle_kind), intent(in) :: allocator}
2200 @item @emph{See also}:
2201 @ref{omp_init_allocator}
2203 @item @emph{Reference}:
2204 @uref{https://www.openmp.org, OpenMP specification v5.0}, Section 3.7.3
2209 @node omp_set_default_allocator
2210 @subsection @code{omp_set_default_allocator} -- Set the default allocator
2212 @item @emph{Description}:
2213 Sets the default allocator that is used when no allocator has been specified
2214 in the @code{allocate} or @code{allocator} clause or if an OpenMP memory
2215 routine is invoked with the @code{omp_null_allocator} allocator.
2218 @multitable @columnfractions .20 .80
2219 @item @emph{Prototype}: @tab @code{void omp_set_default_allocator(omp_allocator_handle_t allocator);}
2222 @item @emph{Fortran}:
2223 @multitable @columnfractions .20 .80
2224 @item @emph{Interface}: @tab @code{subroutine omp_set_default_allocator(allocator)}
2225 @item @tab @code{integer (kind=omp_allocator_handle_kind), intent(in) :: allocator}
2228 @item @emph{See also}:
2229 @ref{omp_get_default_allocator}, @ref{omp_init_allocator}, @ref{OMP_ALLOCATOR},
2230 @ref{Memory allocation}
2232 @item @emph{Reference}:
2233 @uref{https://www.openmp.org, OpenMP specification v5.0}, Section 3.7.4
2238 @node omp_get_default_allocator
2239 @subsection @code{omp_get_default_allocator} -- Get the default allocator
2241 @item @emph{Description}:
2242 The routine returns the default allocator that is used when no allocator has
2243 been specified in the @code{allocate} or @code{allocator} clause or if an
2244 OpenMP memory routine is invoked with the @code{omp_null_allocator} allocator.
2247 @multitable @columnfractions .20 .80
2248 @item @emph{Prototype}: @tab @code{omp_allocator_handle_t omp_get_default_allocator();}
2251 @item @emph{Fortran}:
2252 @multitable @columnfractions .20 .80
2253 @item @emph{Interface}: @tab @code{function omp_get_default_allocator()}
2254 @item @tab @code{integer (kind=omp_allocator_handle_kind) :: omp_get_default_allocator}
2257 @item @emph{See also}:
2258 @ref{omp_set_default_allocator}, @ref{OMP_ALLOCATOR}
2260 @item @emph{Reference}:
2261 @uref{https://www.openmp.org, OpenMP specification v5.0}, Section 3.7.5
2266 @c @node Tool Control Routine
2270 @c @node Environment Display Routine
2271 @c @section Environment Display Routine
2273 @c Routine to display the OpenMP number and the initial value of ICVs.
2274 @c It has C linkage and do not throw exceptions.
2277 @c * omp_display_env:: <fixme>
2280 @c ---------------------------------------------------------------------
2281 @c OpenMP Environment Variables
2282 @c ---------------------------------------------------------------------
2284 @node Environment Variables
2285 @chapter OpenMP Environment Variables
2287 The environment variables which beginning with @env{OMP_} are defined by
2288 section 4 of the OpenMP specification in version 4.5 or in a later version
2289 of the specification, while those beginning with @env{GOMP_} are GNU extensions.
2290 Most @env{OMP_} environment variables have an associated internal control
2293 For any OpenMP environment variable that sets an ICV and is neither
2294 @code{OMP_DEFAULT_DEVICE} nor has global ICV scope, associated
2295 device-specific environment variables exist. For them, the environment
2296 variable without suffix affects the host. The suffix @code{_DEV_} followed
2297 by a non-negative device number less that the number of available devices sets
2298 the ICV for the corresponding device. The suffix @code{_DEV} sets the ICV
2299 of all non-host devices for which a device-specific corresponding environment
2300 variable has not been set while the @code{_ALL} suffix sets the ICV of all
2301 host and non-host devices for which a more specific corresponding environment
2302 variable is not set.
2305 * OMP_ALLOCATOR:: Set the default allocator
2306 * OMP_AFFINITY_FORMAT:: Set the format string used for affinity display
2307 * OMP_CANCELLATION:: Set whether cancellation is activated
2308 * OMP_DISPLAY_AFFINITY:: Display thread affinity information
2309 * OMP_DISPLAY_ENV:: Show OpenMP version and environment variables
2310 * OMP_DEFAULT_DEVICE:: Set the device used in target regions
2311 * OMP_DYNAMIC:: Dynamic adjustment of threads
2312 * OMP_MAX_ACTIVE_LEVELS:: Set the maximum number of nested parallel regions
2313 * OMP_MAX_TASK_PRIORITY:: Set the maximum task priority value
2314 * OMP_NESTED:: Nested parallel regions
2315 * OMP_NUM_TEAMS:: Specifies the number of teams to use by teams region
2316 * OMP_NUM_THREADS:: Specifies the number of threads to use
2317 * OMP_PROC_BIND:: Whether threads may be moved between CPUs
2318 * OMP_PLACES:: Specifies on which CPUs the threads should be placed
2319 * OMP_STACKSIZE:: Set default thread stack size
2320 * OMP_SCHEDULE:: How threads are scheduled
2321 * OMP_TARGET_OFFLOAD:: Controls offloading behaviour
2322 * OMP_TEAMS_THREAD_LIMIT:: Set the maximum number of threads imposed by teams
2323 * OMP_THREAD_LIMIT:: Set the maximum number of threads
2324 * OMP_WAIT_POLICY:: How waiting threads are handled
2325 * GOMP_CPU_AFFINITY:: Bind threads to specific CPUs
2326 * GOMP_DEBUG:: Enable debugging output
2327 * GOMP_STACKSIZE:: Set default thread stack size
2328 * GOMP_SPINCOUNT:: Set the busy-wait spin count
2329 * GOMP_RTEMS_THREAD_POOLS:: Set the RTEMS specific thread pools
2334 @section @env{OMP_ALLOCATOR} -- Set the default allocator
2335 @cindex Environment Variable
2337 @item @emph{ICV:} @var{def-allocator-var}
2338 @item @emph{Scope:} data environment
2339 @item @emph{Description}:
2340 Sets the default allocator that is used when no allocator has been specified
2341 in the @code{allocate} or @code{allocator} clause or if an OpenMP memory
2342 routine is invoked with the @code{omp_null_allocator} allocator.
2343 If unset, @code{omp_default_mem_alloc} is used.
2345 The value can either be a predefined allocator or a predefined memory space
2346 or a predefined memory space followed by a colon and a comma-separated list
2347 of memory trait and value pairs, separated by @code{=}.
2349 Note: The corresponding device environment variables are currently not
2350 supported. Therefore, the non-host @var{def-allocator-var} ICVs are always
2351 initialized to @code{omp_default_mem_alloc}. However, on all devices,
2352 the @code{omp_set_default_allocator} API routine can be used to change
2355 @multitable @columnfractions .45 .45
2356 @headitem Predefined allocators @tab Associated predefined memory spaces
2357 @item omp_default_mem_alloc @tab omp_default_mem_space
2358 @item omp_large_cap_mem_alloc @tab omp_large_cap_mem_space
2359 @item omp_const_mem_alloc @tab omp_const_mem_space
2360 @item omp_high_bw_mem_alloc @tab omp_high_bw_mem_space
2361 @item omp_low_lat_mem_alloc @tab omp_low_lat_mem_space
2362 @item omp_cgroup_mem_alloc @tab --
2363 @item omp_pteam_mem_alloc @tab --
2364 @item omp_thread_mem_alloc @tab --
2367 The predefined allocators use the default values for the traits,
2368 as listed below. Except that the last three allocators have the
2369 @code{access} trait set to @code{cgroup}, @code{pteam}, and
2370 @code{thread}, respectively.
2372 @multitable @columnfractions .25 .40 .25
2373 @headitem Trait @tab Allowed values @tab Default value
2374 @item @code{sync_hint} @tab @code{contended}, @code{uncontended},
2375 @code{serialized}, @code{private}
2376 @tab @code{contended}
2377 @item @code{alignment} @tab Positive integer being a power of two
2379 @item @code{access} @tab @code{all}, @code{cgroup},
2380 @code{pteam}, @code{thread}
2382 @item @code{pool_size} @tab Positive integer
2383 @tab See @ref{Memory allocation}
2384 @item @code{fallback} @tab @code{default_mem_fb}, @code{null_fb},
2385 @code{abort_fb}, @code{allocator_fb}
2387 @item @code{fb_data} @tab @emph{unsupported as it needs an allocator handle}
2389 @item @code{pinned} @tab @code{true}, @code{false}
2391 @item @code{partition} @tab @code{environment}, @code{nearest},
2392 @code{blocked}, @code{interleaved}
2393 @tab @code{environment}
2396 For the @code{fallback} trait, the default value is @code{null_fb} for the
2397 @code{omp_default_mem_alloc} allocator and any allocator that is associated
2398 with device memory; for all other other allocators, it is @code{default_mem_fb}
2403 OMP_ALLOCATOR=omp_high_bw_mem_alloc
2404 OMP_ALLOCATOR=omp_large_cap_mem_space
2405 OMP_ALLOCATOR=omp_low_lat_mem_space:pinned=true,partition=nearest
2408 @item @emph{See also}:
2409 @ref{Memory allocation}, @ref{omp_get_default_allocator},
2410 @ref{omp_set_default_allocator}
2412 @item @emph{Reference}:
2413 @uref{https://www.openmp.org, OpenMP specification v5.0}, Section 6.21
2418 @node OMP_AFFINITY_FORMAT
2419 @section @env{OMP_AFFINITY_FORMAT} -- Set the format string used for affinity display
2420 @cindex Environment Variable
2422 @item @emph{ICV:} @var{affinity-format-var}
2423 @item @emph{Scope:} device
2424 @item @emph{Description}:
2425 Sets the format string used when displaying OpenMP thread affinity information.
2426 Special values are output using @code{%} followed by an optional size
2427 specification and then either the single-character field type or its long
2428 name enclosed in curly braces; using @code{%%} will display a literal percent.
2429 The size specification consists of an optional @code{0.} or @code{.} followed
2430 by a positive integer, specifying the minimal width of the output. With
2431 @code{0.} and numerical values, the output is padded with zeros on the left;
2432 with @code{.}, the output is padded by spaces on the left; otherwise, the
2433 output is padded by spaces on the right. If unset, the value is
2434 ``@code{level %L thread %i affinity %A}''.
2436 Supported field types are:
2438 @multitable @columnfractions .10 .25 .60
2439 @item t @tab team_num @tab value returned by @code{omp_get_team_num}
2440 @item T @tab num_teams @tab value returned by @code{omp_get_num_teams}
2441 @item L @tab nesting_level @tab value returned by @code{omp_get_level}
2442 @item n @tab thread_num @tab value returned by @code{omp_get_thread_num}
2443 @item N @tab num_threads @tab value returned by @code{omp_get_num_threads}
2444 @item a @tab ancestor_tnum
2445 @tab value returned by
2446 @code{omp_get_ancestor_thread_num(omp_get_level()-1)}
2447 @item H @tab host @tab name of the host that executes the thread
2448 @item P @tab process_id @tab process identifier
2449 @item i @tab native_thread_id @tab native thread identifier
2450 @item A @tab thread_affinity
2451 @tab comma separated list of integer values or ranges, representing the
2452 processors on which a process might execute, subject to affinity
2456 For instance, after setting
2459 OMP_AFFINITY_FORMAT="%0.2a!%n!%.4L!%N;%.2t;%0.2T;%@{team_num@};%@{num_teams@};%A"
2462 with either @code{OMP_DISPLAY_AFFINITY} being set or when calling
2463 @code{omp_display_affinity} with @code{NULL} or an empty string, the program
2464 might display the following:
2467 00!0! 1!4; 0;01;0;1;0-11
2468 00!3! 1!4; 0;01;0;1;0-11
2469 00!2! 1!4; 0;01;0;1;0-11
2470 00!1! 1!4; 0;01;0;1;0-11
2473 @item @emph{See also}:
2474 @ref{OMP_DISPLAY_AFFINITY}
2476 @item @emph{Reference}:
2477 @uref{https://www.openmp.org, OpenMP specification v5.0}, Section 6.14
2482 @node OMP_CANCELLATION
2483 @section @env{OMP_CANCELLATION} -- Set whether cancellation is activated
2484 @cindex Environment Variable
2486 @item @emph{ICV:} @var{cancel-var}
2487 @item @emph{Scope:} global
2488 @item @emph{Description}:
2489 If set to @code{TRUE}, the cancellation is activated. If set to @code{FALSE} or
2490 if unset, cancellation is disabled and the @code{cancel} construct is ignored.
2492 @item @emph{See also}:
2493 @ref{omp_get_cancellation}
2495 @item @emph{Reference}:
2496 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 4.11
2501 @node OMP_DISPLAY_AFFINITY
2502 @section @env{OMP_DISPLAY_AFFINITY} -- Display thread affinity information
2503 @cindex Environment Variable
2505 @item @emph{ICV:} @var{display-affinity-var}
2506 @item @emph{Scope:} global
2507 @item @emph{Description}:
2508 If set to @code{FALSE} or if unset, affinity displaying is disabled.
2509 If set to @code{TRUE}, the runtime will display affinity information about
2510 OpenMP threads in a parallel region upon entering the region and every time
2513 @item @emph{See also}:
2514 @ref{OMP_AFFINITY_FORMAT}
2516 @item @emph{Reference}:
2517 @uref{https://www.openmp.org, OpenMP specification v5.0}, Section 6.13
2523 @node OMP_DISPLAY_ENV
2524 @section @env{OMP_DISPLAY_ENV} -- Show OpenMP version and environment variables
2525 @cindex Environment Variable
2527 @item @emph{ICV:} none
2528 @item @emph{Scope:} not applicable
2529 @item @emph{Description}:
2530 If set to @code{TRUE}, the OpenMP version number and the values
2531 associated with the OpenMP environment variables are printed to @code{stderr}.
2532 If set to @code{VERBOSE}, it additionally shows the value of the environment
2533 variables which are GNU extensions. If undefined or set to @code{FALSE},
2534 this information will not be shown.
2537 @item @emph{Reference}:
2538 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 4.12
2543 @node OMP_DEFAULT_DEVICE
2544 @section @env{OMP_DEFAULT_DEVICE} -- Set the device used in target regions
2545 @cindex Environment Variable
2547 @item @emph{ICV:} @var{default-device-var}
2548 @item @emph{Scope:} data environment
2549 @item @emph{Description}:
2550 Set to choose the device which is used in a @code{target} region, unless the
2551 value is overridden by @code{omp_set_default_device} or by a @code{device}
2552 clause. The value shall be the nonnegative device number. If no device with
2553 the given device number exists, the code is executed on the host. If unset,
2554 @env{OMP_TARGET_OFFLOAD} is @code{mandatory} and no non-host devices are
2555 available, it is set to @code{omp_invalid_device}. Otherwise, if unset,
2556 device number 0 will be used.
2559 @item @emph{See also}:
2560 @ref{omp_get_default_device}, @ref{omp_set_default_device},
2562 @item @emph{Reference}:
2563 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 4.13
2569 @section @env{OMP_DYNAMIC} -- Dynamic adjustment of threads
2570 @cindex Environment Variable
2572 @item @emph{ICV:} @var{dyn-var}
2573 @item @emph{Scope:} global
2574 @item @emph{Description}:
2575 Enable or disable the dynamic adjustment of the number of threads
2576 within a team. The value of this environment variable shall be
2577 @code{TRUE} or @code{FALSE}. If undefined, dynamic adjustment is
2578 disabled by default.
2580 @item @emph{See also}:
2581 @ref{omp_set_dynamic}
2583 @item @emph{Reference}:
2584 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 4.3
2589 @node OMP_MAX_ACTIVE_LEVELS
2590 @section @env{OMP_MAX_ACTIVE_LEVELS} -- Set the maximum number of nested parallel regions
2591 @cindex Environment Variable
2593 @item @emph{ICV:} @var{max-active-levels-var}
2594 @item @emph{Scope:} data environment
2595 @item @emph{Description}:
2596 Specifies the initial value for the maximum number of nested parallel
2597 regions. The value of this variable shall be a positive integer.
2598 If undefined, then if @env{OMP_NESTED} is defined and set to true, or
2599 if @env{OMP_NUM_THREADS} or @env{OMP_PROC_BIND} are defined and set to
2600 a list with more than one item, the maximum number of nested parallel
2601 regions will be initialized to the largest number supported, otherwise
2602 it will be set to one.
2604 @item @emph{See also}:
2605 @ref{omp_set_max_active_levels}, @ref{OMP_NESTED}, @ref{OMP_PROC_BIND},
2606 @ref{OMP_NUM_THREADS}
2609 @item @emph{Reference}:
2610 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 4.9
2615 @node OMP_MAX_TASK_PRIORITY
2616 @section @env{OMP_MAX_TASK_PRIORITY} -- Set the maximum priority
2617 number that can be set for a task.
2618 @cindex Environment Variable
2620 @item @emph{ICV:} @var{max-task-priority-var}
2621 @item @emph{Scope:} global
2622 @item @emph{Description}:
2623 Specifies the initial value for the maximum priority value that can be
2624 set for a task. The value of this variable shall be a non-negative
2625 integer, and zero is allowed. If undefined, the default priority is
2628 @item @emph{See also}:
2629 @ref{omp_get_max_task_priority}
2631 @item @emph{Reference}:
2632 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 4.14
2638 @section @env{OMP_NESTED} -- Nested parallel regions
2639 @cindex Environment Variable
2640 @cindex Implementation specific setting
2642 @item @emph{ICV:} @var{max-active-levels-var}
2643 @item @emph{Scope:} data environment
2644 @item @emph{Description}:
2645 Enable or disable nested parallel regions, i.e., whether team members
2646 are allowed to create new teams. The value of this environment variable
2647 shall be @code{TRUE} or @code{FALSE}. If set to @code{TRUE}, the number
2648 of maximum active nested regions supported will by default be set to the
2649 maximum supported, otherwise it will be set to one. If
2650 @env{OMP_MAX_ACTIVE_LEVELS} is defined, its setting will override this
2651 setting. If both are undefined, nested parallel regions are enabled if
2652 @env{OMP_NUM_THREADS} or @env{OMP_PROC_BINDS} are defined to a list with
2653 more than one item, otherwise they are disabled by default.
2655 Note that the @code{OMP_NESTED} environment variable was deprecated in
2656 the OpenMP specification 5.2 in favor of @code{OMP_MAX_ACTIVE_LEVELS}.
2658 @item @emph{See also}:
2659 @ref{omp_set_max_active_levels}, @ref{omp_set_nested},
2660 @ref{OMP_MAX_ACTIVE_LEVELS}
2662 @item @emph{Reference}:
2663 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 4.6
2669 @section @env{OMP_NUM_TEAMS} -- Specifies the number of teams to use by teams region
2670 @cindex Environment Variable
2672 @item @emph{ICV:} @var{nteams-var}
2673 @item @emph{Scope:} device
2674 @item @emph{Description}:
2675 Specifies the upper bound for number of teams to use in teams regions
2676 without explicit @code{num_teams} clause. The value of this variable shall
2677 be a positive integer. If undefined it defaults to 0 which means
2678 implementation defined upper bound.
2680 @item @emph{See also}:
2681 @ref{omp_set_num_teams}
2683 @item @emph{Reference}:
2684 @uref{https://www.openmp.org, OpenMP specification v5.1}, Section 6.23
2689 @node OMP_NUM_THREADS
2690 @section @env{OMP_NUM_THREADS} -- Specifies the number of threads to use
2691 @cindex Environment Variable
2692 @cindex Implementation specific setting
2694 @item @emph{ICV:} @var{nthreads-var}
2695 @item @emph{Scope:} data environment
2696 @item @emph{Description}:
2697 Specifies the default number of threads to use in parallel regions. The
2698 value of this variable shall be a comma-separated list of positive integers;
2699 the value specifies the number of threads to use for the corresponding nested
2700 level. Specifying more than one item in the list will automatically enable
2701 nesting by default. If undefined one thread per CPU is used.
2703 When a list with more than value is specified, it also affects the
2704 @var{max-active-levels-var} ICV as described in @ref{OMP_MAX_ACTIVE_LEVELS}.
2706 @item @emph{See also}:
2707 @ref{omp_set_num_threads}, @ref{OMP_MAX_ACTIVE_LEVELS}
2709 @item @emph{Reference}:
2710 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 4.2
2716 @section @env{OMP_PROC_BIND} -- Whether threads may be moved between CPUs
2717 @cindex Environment Variable
2719 @item @emph{ICV:} @var{bind-var}
2720 @item @emph{Scope:} data environment
2721 @item @emph{Description}:
2722 Specifies whether threads may be moved between processors. If set to
2723 @code{TRUE}, OpenMP threads should not be moved; if set to @code{FALSE}
2724 they may be moved. Alternatively, a comma separated list with the
2725 values @code{PRIMARY}, @code{MASTER}, @code{CLOSE} and @code{SPREAD} can
2726 be used to specify the thread affinity policy for the corresponding nesting
2727 level. With @code{PRIMARY} and @code{MASTER} the worker threads are in the
2728 same place partition as the primary thread. With @code{CLOSE} those are
2729 kept close to the primary thread in contiguous place partitions. And
2730 with @code{SPREAD} a sparse distribution
2731 across the place partitions is used. Specifying more than one item in the
2732 list will automatically enable nesting by default.
2734 When a list is specified, it also affects the @var{max-active-levels-var} ICV
2735 as described in @ref{OMP_MAX_ACTIVE_LEVELS}.
2737 When undefined, @env{OMP_PROC_BIND} defaults to @code{TRUE} when
2738 @env{OMP_PLACES} or @env{GOMP_CPU_AFFINITY} is set and @code{FALSE} otherwise.
2740 @item @emph{See also}:
2741 @ref{omp_get_proc_bind}, @ref{GOMP_CPU_AFFINITY}, @ref{OMP_PLACES},
2742 @ref{OMP_MAX_ACTIVE_LEVELS}
2744 @item @emph{Reference}:
2745 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 4.4
2751 @section @env{OMP_PLACES} -- Specifies on which CPUs the threads should be placed
2752 @cindex Environment Variable
2754 @item @emph{ICV:} @var{place-partition-var}
2755 @item @emph{Scope:} implicit tasks
2756 @item @emph{Description}:
2757 The thread placement can be either specified using an abstract name or by an
2758 explicit list of the places. The abstract names @code{threads}, @code{cores},
2759 @code{sockets}, @code{ll_caches} and @code{numa_domains} can be optionally
2760 followed by a positive number in parentheses, which denotes the how many places
2761 shall be created. With @code{threads} each place corresponds to a single
2762 hardware thread; @code{cores} to a single core with the corresponding number of
2763 hardware threads; with @code{sockets} the place corresponds to a single
2764 socket; with @code{ll_caches} to a set of cores that shares the last level
2765 cache on the device; and @code{numa_domains} to a set of cores for which their
2766 closest memory on the device is the same memory and at a similar distance from
2767 the cores. The resulting placement can be shown by setting the
2768 @env{OMP_DISPLAY_ENV} environment variable.
2770 Alternatively, the placement can be specified explicitly as comma-separated
2771 list of places. A place is specified by set of nonnegative numbers in curly
2772 braces, denoting the hardware threads. The curly braces can be omitted
2773 when only a single number has been specified. The hardware threads
2774 belonging to a place can either be specified as comma-separated list of
2775 nonnegative thread numbers or using an interval. Multiple places can also be
2776 either specified by a comma-separated list of places or by an interval. To
2777 specify an interval, a colon followed by the count is placed after
2778 the hardware thread number or the place. Optionally, the length can be
2779 followed by a colon and the stride number -- otherwise a unit stride is
2780 assumed. Placing an exclamation mark (@code{!}) directly before a curly
2781 brace or numbers inside the curly braces (excluding intervals) will
2782 exclude those hardware threads.
2784 For instance, the following specifies the same places list:
2785 @code{"@{0,1,2@}, @{3,4,6@}, @{7,8,9@}, @{10,11,12@}"};
2786 @code{"@{0:3@}, @{3:3@}, @{7:3@}, @{10:3@}"}; and @code{"@{0:2@}:4:3"}.
2788 If @env{OMP_PLACES} and @env{GOMP_CPU_AFFINITY} are unset and
2789 @env{OMP_PROC_BIND} is either unset or @code{false}, threads may be moved
2790 between CPUs following no placement policy.
2792 @item @emph{See also}:
2793 @ref{OMP_PROC_BIND}, @ref{GOMP_CPU_AFFINITY}, @ref{omp_get_proc_bind},
2794 @ref{OMP_DISPLAY_ENV}
2796 @item @emph{Reference}:
2797 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 4.5
2803 @section @env{OMP_STACKSIZE} -- Set default thread stack size
2804 @cindex Environment Variable
2806 @item @emph{ICV:} @var{stacksize-var}
2807 @item @emph{Scope:} device
2808 @item @emph{Description}:
2809 Set the default thread stack size in kilobytes, unless the number
2810 is suffixed by @code{B}, @code{K}, @code{M} or @code{G}, in which
2811 case the size is, respectively, in bytes, kilobytes, megabytes
2812 or gigabytes. This is different from @code{pthread_attr_setstacksize}
2813 which gets the number of bytes as an argument. If the stack size cannot
2814 be set due to system constraints, an error is reported and the initial
2815 stack size is left unchanged. If undefined, the stack size is system
2818 @item @emph{See also}:
2819 @ref{GOMP_STACKSIZE}
2821 @item @emph{Reference}:
2822 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 4.7
2828 @section @env{OMP_SCHEDULE} -- How threads are scheduled
2829 @cindex Environment Variable
2830 @cindex Implementation specific setting
2832 @item @emph{ICV:} @var{run-sched-var}
2833 @item @emph{Scope:} data environment
2834 @item @emph{Description}:
2835 Allows to specify @code{schedule type} and @code{chunk size}.
2836 The value of the variable shall have the form: @code{type[,chunk]} where
2837 @code{type} is one of @code{static}, @code{dynamic}, @code{guided} or @code{auto}
2838 The optional @code{chunk} size shall be a positive integer. If undefined,
2839 dynamic scheduling and a chunk size of 1 is used.
2841 @item @emph{See also}:
2842 @ref{omp_set_schedule}
2844 @item @emph{Reference}:
2845 @uref{https://www.openmp.org, OpenMP specification v4.5}, Sections 2.7.1.1 and 4.1
2850 @node OMP_TARGET_OFFLOAD
2851 @section @env{OMP_TARGET_OFFLOAD} -- Controls offloading behaviour
2852 @cindex Environment Variable
2853 @cindex Implementation specific setting
2855 @item @emph{ICV:} @var{target-offload-var}
2856 @item @emph{Scope:} global
2857 @item @emph{Description}:
2858 Specifies the behaviour with regard to offloading code to a device. This
2859 variable can be set to one of three values - @code{MANDATORY}, @code{DISABLED}
2862 If set to @code{MANDATORY}, the program will terminate with an error if
2863 the offload device is not present or is not supported. If set to
2864 @code{DISABLED}, then offloading is disabled and all code will run on the
2865 host. If set to @code{DEFAULT}, the program will try offloading to the
2866 device first, then fall back to running code on the host if it cannot.
2868 If undefined, then the program will behave as if @code{DEFAULT} was set.
2870 @item @emph{Reference}:
2871 @uref{https://www.openmp.org, OpenMP specification v5.0}, Section 6.17
2876 @node OMP_TEAMS_THREAD_LIMIT
2877 @section @env{OMP_TEAMS_THREAD_LIMIT} -- Set the maximum number of threads imposed by teams
2878 @cindex Environment Variable
2880 @item @emph{ICV:} @var{teams-thread-limit-var}
2881 @item @emph{Scope:} device
2882 @item @emph{Description}:
2883 Specifies an upper bound for the number of threads to use by each contention
2884 group created by a teams construct without explicit @code{thread_limit}
2885 clause. The value of this variable shall be a positive integer. If undefined,
2886 the value of 0 is used which stands for an implementation defined upper
2889 @item @emph{See also}:
2890 @ref{OMP_THREAD_LIMIT}, @ref{omp_set_teams_thread_limit}
2892 @item @emph{Reference}:
2893 @uref{https://www.openmp.org, OpenMP specification v5.1}, Section 6.24
2898 @node OMP_THREAD_LIMIT
2899 @section @env{OMP_THREAD_LIMIT} -- Set the maximum number of threads
2900 @cindex Environment Variable
2902 @item @emph{ICV:} @var{thread-limit-var}
2903 @item @emph{Scope:} data environment
2904 @item @emph{Description}:
2905 Specifies the number of threads to use for the whole program. The
2906 value of this variable shall be a positive integer. If undefined,
2907 the number of threads is not limited.
2909 @item @emph{See also}:
2910 @ref{OMP_NUM_THREADS}, @ref{omp_get_thread_limit}
2912 @item @emph{Reference}:
2913 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 4.10
2918 @node OMP_WAIT_POLICY
2919 @section @env{OMP_WAIT_POLICY} -- How waiting threads are handled
2920 @cindex Environment Variable
2922 @item @emph{Description}:
2923 Specifies whether waiting threads should be active or passive. If
2924 the value is @code{PASSIVE}, waiting threads should not consume CPU
2925 power while waiting; while the value is @code{ACTIVE} specifies that
2926 they should. If undefined, threads wait actively for a short time
2927 before waiting passively.
2929 @item @emph{See also}:
2930 @ref{GOMP_SPINCOUNT}
2932 @item @emph{Reference}:
2933 @uref{https://www.openmp.org, OpenMP specification v4.5}, Section 4.8
2938 @node GOMP_CPU_AFFINITY
2939 @section @env{GOMP_CPU_AFFINITY} -- Bind threads to specific CPUs
2940 @cindex Environment Variable
2942 @item @emph{Description}:
2943 Binds threads to specific CPUs. The variable should contain a space-separated
2944 or comma-separated list of CPUs. This list may contain different kinds of
2945 entries: either single CPU numbers in any order, a range of CPUs (M-N)
2946 or a range with some stride (M-N:S). CPU numbers are zero based. For example,
2947 @code{GOMP_CPU_AFFINITY="0 3 1-2 4-15:2"} will bind the initial thread
2948 to CPU 0, the second to CPU 3, the third to CPU 1, the fourth to
2949 CPU 2, the fifth to CPU 4, the sixth through tenth to CPUs 6, 8, 10, 12,
2950 and 14 respectively and then start assigning back from the beginning of
2951 the list. @code{GOMP_CPU_AFFINITY=0} binds all threads to CPU 0.
2953 There is no libgomp library routine to determine whether a CPU affinity
2954 specification is in effect. As a workaround, language-specific library
2955 functions, e.g., @code{getenv} in C or @code{GET_ENVIRONMENT_VARIABLE} in
2956 Fortran, may be used to query the setting of the @code{GOMP_CPU_AFFINITY}
2957 environment variable. A defined CPU affinity on startup cannot be changed
2958 or disabled during the runtime of the application.
2960 If both @env{GOMP_CPU_AFFINITY} and @env{OMP_PROC_BIND} are set,
2961 @env{OMP_PROC_BIND} has a higher precedence. If neither has been set and
2962 @env{OMP_PROC_BIND} is unset, or when @env{OMP_PROC_BIND} is set to
2963 @code{FALSE}, the host system will handle the assignment of threads to CPUs.
2965 @item @emph{See also}:
2966 @ref{OMP_PLACES}, @ref{OMP_PROC_BIND}
2972 @section @env{GOMP_DEBUG} -- Enable debugging output
2973 @cindex Environment Variable
2975 @item @emph{Description}:
2976 Enable debugging output. The variable should be set to @code{0}
2977 (disabled, also the default if not set), or @code{1} (enabled).
2979 If enabled, some debugging output will be printed during execution.
2980 This is currently not specified in more detail, and subject to change.
2985 @node GOMP_STACKSIZE
2986 @section @env{GOMP_STACKSIZE} -- Set default thread stack size
2987 @cindex Environment Variable
2988 @cindex Implementation specific setting
2990 @item @emph{Description}:
2991 Set the default thread stack size in kilobytes. This is different from
2992 @code{pthread_attr_setstacksize} which gets the number of bytes as an
2993 argument. If the stack size cannot be set due to system constraints, an
2994 error is reported and the initial stack size is left unchanged. If undefined,
2995 the stack size is system dependent.
2997 @item @emph{See also}:
3000 @item @emph{Reference}:
3001 @uref{https://gcc.gnu.org/ml/gcc-patches/2006-06/msg00493.html,
3002 GCC Patches Mailinglist},
3003 @uref{https://gcc.gnu.org/ml/gcc-patches/2006-06/msg00496.html,
3004 GCC Patches Mailinglist}
3009 @node GOMP_SPINCOUNT
3010 @section @env{GOMP_SPINCOUNT} -- Set the busy-wait spin count
3011 @cindex Environment Variable
3012 @cindex Implementation specific setting
3014 @item @emph{Description}:
3015 Determines how long a threads waits actively with consuming CPU power
3016 before waiting passively without consuming CPU power. The value may be
3017 either @code{INFINITE}, @code{INFINITY} to always wait actively or an
3018 integer which gives the number of spins of the busy-wait loop. The
3019 integer may optionally be followed by the following suffixes acting
3020 as multiplication factors: @code{k} (kilo, thousand), @code{M} (mega,
3021 million), @code{G} (giga, billion), or @code{T} (tera, trillion).
3022 If undefined, 0 is used when @env{OMP_WAIT_POLICY} is @code{PASSIVE},
3023 300,000 is used when @env{OMP_WAIT_POLICY} is undefined and
3024 30 billion is used when @env{OMP_WAIT_POLICY} is @code{ACTIVE}.
3025 If there are more OpenMP threads than available CPUs, 1000 and 100
3026 spins are used for @env{OMP_WAIT_POLICY} being @code{ACTIVE} or
3027 undefined, respectively; unless the @env{GOMP_SPINCOUNT} is lower
3028 or @env{OMP_WAIT_POLICY} is @code{PASSIVE}.
3030 @item @emph{See also}:
3031 @ref{OMP_WAIT_POLICY}
3036 @node GOMP_RTEMS_THREAD_POOLS
3037 @section @env{GOMP_RTEMS_THREAD_POOLS} -- Set the RTEMS specific thread pools
3038 @cindex Environment Variable
3039 @cindex Implementation specific setting
3041 @item @emph{Description}:
3042 This environment variable is only used on the RTEMS real-time operating system.
3043 It determines the scheduler instance specific thread pools. The format for
3044 @env{GOMP_RTEMS_THREAD_POOLS} is a list of optional
3045 @code{<thread-pool-count>[$<priority>]@@<scheduler-name>} configurations
3046 separated by @code{:} where:
3048 @item @code{<thread-pool-count>} is the thread pool count for this scheduler
3050 @item @code{$<priority>} is an optional priority for the worker threads of a
3051 thread pool according to @code{pthread_setschedparam}. In case a priority
3052 value is omitted, then a worker thread will inherit the priority of the OpenMP
3053 primary thread that created it. The priority of the worker thread is not
3054 changed after creation, even if a new OpenMP primary thread using the worker has
3055 a different priority.
3056 @item @code{@@<scheduler-name>} is the scheduler instance name according to the
3057 RTEMS application configuration.
3059 In case no thread pool configuration is specified for a scheduler instance,
3060 then each OpenMP primary thread of this scheduler instance will use its own
3061 dynamically allocated thread pool. To limit the worker thread count of the
3062 thread pools, each OpenMP primary thread must call @code{omp_set_num_threads}.
3063 @item @emph{Example}:
3064 Lets suppose we have three scheduler instances @code{IO}, @code{WRK0}, and
3065 @code{WRK1} with @env{GOMP_RTEMS_THREAD_POOLS} set to
3066 @code{"1@@WRK0:3$4@@WRK1"}. Then there are no thread pool restrictions for
3067 scheduler instance @code{IO}. In the scheduler instance @code{WRK0} there is
3068 one thread pool available. Since no priority is specified for this scheduler
3069 instance, the worker thread inherits the priority of the OpenMP primary thread
3070 that created it. In the scheduler instance @code{WRK1} there are three thread
3071 pools available and their worker threads run at priority four.
3076 @c ---------------------------------------------------------------------
3078 @c ---------------------------------------------------------------------
3080 @node Enabling OpenACC
3081 @chapter Enabling OpenACC
3083 To activate the OpenACC extensions for C/C++ and Fortran, the compile-time
3084 flag @option{-fopenacc} must be specified. This enables the OpenACC directive
3085 @code{#pragma acc} in C/C++ and @code{!$acc} directives in free form,
3086 @code{c$acc}, @code{*$acc} and @code{!$acc} directives in fixed form,
3087 @code{!$} conditional compilation sentinels in free form and @code{c$},
3088 @code{*$} and @code{!$} sentinels in fixed form, for Fortran. The flag also
3089 arranges for automatic linking of the OpenACC runtime library
3090 (@ref{OpenACC Runtime Library Routines}).
3092 See @uref{https://gcc.gnu.org/wiki/OpenACC} for more information.
3094 A complete description of all OpenACC directives accepted may be found in
3095 the @uref{https://www.openacc.org, OpenACC} Application Programming
3096 Interface manual, version 2.6.
3100 @c ---------------------------------------------------------------------
3101 @c OpenACC Runtime Library Routines
3102 @c ---------------------------------------------------------------------
3104 @node OpenACC Runtime Library Routines
3105 @chapter OpenACC Runtime Library Routines
3107 The runtime routines described here are defined by section 3 of the OpenACC
3108 specifications in version 2.6.
3109 They have C linkage, and do not throw exceptions.
3110 Generally, they are available only for the host, with the exception of
3111 @code{acc_on_device}, which is available for both the host and the
3112 acceleration device.
3115 * acc_get_num_devices:: Get number of devices for the given device
3117 * acc_set_device_type:: Set type of device accelerator to use.
3118 * acc_get_device_type:: Get type of device accelerator to be used.
3119 * acc_set_device_num:: Set device number to use.
3120 * acc_get_device_num:: Get device number to be used.
3121 * acc_get_property:: Get device property.
3122 * acc_async_test:: Tests for completion of a specific asynchronous
3124 * acc_async_test_all:: Tests for completion of all asynchronous
3126 * acc_wait:: Wait for completion of a specific asynchronous
3128 * acc_wait_all:: Waits for completion of all asynchronous
3130 * acc_wait_all_async:: Wait for completion of all asynchronous
3132 * acc_wait_async:: Wait for completion of asynchronous operations.
3133 * acc_init:: Initialize runtime for a specific device type.
3134 * acc_shutdown:: Shuts down the runtime for a specific device
3136 * acc_on_device:: Whether executing on a particular device
3137 * acc_malloc:: Allocate device memory.
3138 * acc_free:: Free device memory.
3139 * acc_copyin:: Allocate device memory and copy host memory to
3141 * acc_present_or_copyin:: If the data is not present on the device,
3142 allocate device memory and copy from host
3144 * acc_create:: Allocate device memory and map it to host
3146 * acc_present_or_create:: If the data is not present on the device,
3147 allocate device memory and map it to host
3149 * acc_copyout:: Copy device memory to host memory.
3150 * acc_delete:: Free device memory.
3151 * acc_update_device:: Update device memory from mapped host memory.
3152 * acc_update_self:: Update host memory from mapped device memory.
3153 * acc_map_data:: Map previously allocated device memory to host
3155 * acc_unmap_data:: Unmap device memory from host memory.
3156 * acc_deviceptr:: Get device pointer associated with specific
3158 * acc_hostptr:: Get host pointer associated with specific
3160 * acc_is_present:: Indicate whether host variable / array is
3162 * acc_memcpy_to_device:: Copy host memory to device memory.
3163 * acc_memcpy_from_device:: Copy device memory to host memory.
3164 * acc_attach:: Let device pointer point to device-pointer target.
3165 * acc_detach:: Let device pointer point to host-pointer target.
3167 API routines for target platforms.
3169 * acc_get_current_cuda_device:: Get CUDA device handle.
3170 * acc_get_current_cuda_context::Get CUDA context handle.
3171 * acc_get_cuda_stream:: Get CUDA stream handle.
3172 * acc_set_cuda_stream:: Set CUDA stream handle.
3174 API routines for the OpenACC Profiling Interface.
3176 * acc_prof_register:: Register callbacks.
3177 * acc_prof_unregister:: Unregister callbacks.
3178 * acc_prof_lookup:: Obtain inquiry functions.
3179 * acc_register_library:: Library registration.
3184 @node acc_get_num_devices
3185 @section @code{acc_get_num_devices} -- Get number of devices for given device type
3187 @item @emph{Description}
3188 This function returns a value indicating the number of devices available
3189 for the device type specified in @var{devicetype}.
3192 @multitable @columnfractions .20 .80
3193 @item @emph{Prototype}: @tab @code{int acc_get_num_devices(acc_device_t devicetype);}
3196 @item @emph{Fortran}:
3197 @multitable @columnfractions .20 .80
3198 @item @emph{Interface}: @tab @code{integer function acc_get_num_devices(devicetype)}
3199 @item @tab @code{integer(kind=acc_device_kind) devicetype}
3202 @item @emph{Reference}:
3203 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3209 @node acc_set_device_type
3210 @section @code{acc_set_device_type} -- Set type of device accelerator to use.
3212 @item @emph{Description}
3213 This function indicates to the runtime library which device type, specified
3214 in @var{devicetype}, to use when executing a parallel or kernels region.
3217 @multitable @columnfractions .20 .80
3218 @item @emph{Prototype}: @tab @code{acc_set_device_type(acc_device_t devicetype);}
3221 @item @emph{Fortran}:
3222 @multitable @columnfractions .20 .80
3223 @item @emph{Interface}: @tab @code{subroutine acc_set_device_type(devicetype)}
3224 @item @tab @code{integer(kind=acc_device_kind) devicetype}
3227 @item @emph{Reference}:
3228 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3234 @node acc_get_device_type
3235 @section @code{acc_get_device_type} -- Get type of device accelerator to be used.
3237 @item @emph{Description}
3238 This function returns what device type will be used when executing a
3239 parallel or kernels region.
3241 This function returns @code{acc_device_none} if
3242 @code{acc_get_device_type} is called from
3243 @code{acc_ev_device_init_start}, @code{acc_ev_device_init_end}
3244 callbacks of the OpenACC Profiling Interface (@ref{OpenACC Profiling
3245 Interface}), that is, if the device is currently being initialized.
3248 @multitable @columnfractions .20 .80
3249 @item @emph{Prototype}: @tab @code{acc_device_t acc_get_device_type(void);}
3252 @item @emph{Fortran}:
3253 @multitable @columnfractions .20 .80
3254 @item @emph{Interface}: @tab @code{function acc_get_device_type(void)}
3255 @item @tab @code{integer(kind=acc_device_kind) acc_get_device_type}
3258 @item @emph{Reference}:
3259 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3265 @node acc_set_device_num
3266 @section @code{acc_set_device_num} -- Set device number to use.
3268 @item @emph{Description}
3269 This function will indicate to the runtime which device number,
3270 specified by @var{devicenum}, associated with the specified device
3271 type @var{devicetype}.
3274 @multitable @columnfractions .20 .80
3275 @item @emph{Prototype}: @tab @code{acc_set_device_num(int devicenum, acc_device_t devicetype);}
3278 @item @emph{Fortran}:
3279 @multitable @columnfractions .20 .80
3280 @item @emph{Interface}: @tab @code{subroutine acc_set_device_num(devicenum, devicetype)}
3281 @item @tab @code{integer devicenum}
3282 @item @tab @code{integer(kind=acc_device_kind) devicetype}
3285 @item @emph{Reference}:
3286 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3292 @node acc_get_device_num
3293 @section @code{acc_get_device_num} -- Get device number to be used.
3295 @item @emph{Description}
3296 This function returns which device number associated with the specified device
3297 type @var{devicetype}, will be used when executing a parallel or kernels
3301 @multitable @columnfractions .20 .80
3302 @item @emph{Prototype}: @tab @code{int acc_get_device_num(acc_device_t devicetype);}
3305 @item @emph{Fortran}:
3306 @multitable @columnfractions .20 .80
3307 @item @emph{Interface}: @tab @code{function acc_get_device_num(devicetype)}
3308 @item @tab @code{integer(kind=acc_device_kind) devicetype}
3309 @item @tab @code{integer acc_get_device_num}
3312 @item @emph{Reference}:
3313 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3319 @node acc_get_property
3320 @section @code{acc_get_property} -- Get device property.
3321 @cindex acc_get_property
3322 @cindex acc_get_property_string
3324 @item @emph{Description}
3325 These routines return the value of the specified @var{property} for the
3326 device being queried according to @var{devicenum} and @var{devicetype}.
3327 Integer-valued and string-valued properties are returned by
3328 @code{acc_get_property} and @code{acc_get_property_string} respectively.
3329 The Fortran @code{acc_get_property_string} subroutine returns the string
3330 retrieved in its fourth argument while the remaining entry points are
3331 functions, which pass the return value as their result.
3333 Note for Fortran, only: the OpenACC technical committee corrected and, hence,
3334 modified the interface introduced in OpenACC 2.6. The kind-value parameter
3335 @code{acc_device_property} has been renamed to @code{acc_device_property_kind}
3336 for consistency and the return type of the @code{acc_get_property} function is
3337 now a @code{c_size_t} integer instead of a @code{acc_device_property} integer.
3338 The parameter @code{acc_device_property} will continue to be provided,
3339 but might be removed in a future version of GCC.
3342 @multitable @columnfractions .20 .80
3343 @item @emph{Prototype}: @tab @code{size_t acc_get_property(int devicenum, acc_device_t devicetype, acc_device_property_t property);}
3344 @item @emph{Prototype}: @tab @code{const char *acc_get_property_string(int devicenum, acc_device_t devicetype, acc_device_property_t property);}
3347 @item @emph{Fortran}:
3348 @multitable @columnfractions .20 .80
3349 @item @emph{Interface}: @tab @code{function acc_get_property(devicenum, devicetype, property)}
3350 @item @emph{Interface}: @tab @code{subroutine acc_get_property_string(devicenum, devicetype, property, string)}
3351 @item @tab @code{use ISO_C_Binding, only: c_size_t}
3352 @item @tab @code{integer devicenum}
3353 @item @tab @code{integer(kind=acc_device_kind) devicetype}
3354 @item @tab @code{integer(kind=acc_device_property_kind) property}
3355 @item @tab @code{integer(kind=c_size_t) acc_get_property}
3356 @item @tab @code{character(*) string}
3359 @item @emph{Reference}:
3360 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3366 @node acc_async_test
3367 @section @code{acc_async_test} -- Test for completion of a specific asynchronous operation.
3369 @item @emph{Description}
3370 This function tests for completion of the asynchronous operation specified
3371 in @var{arg}. In C/C++, a non-zero value will be returned to indicate
3372 the specified asynchronous operation has completed. While Fortran will return
3373 a @code{true}. If the asynchronous operation has not completed, C/C++ returns
3374 a zero and Fortran returns a @code{false}.
3377 @multitable @columnfractions .20 .80
3378 @item @emph{Prototype}: @tab @code{int acc_async_test(int arg);}
3381 @item @emph{Fortran}:
3382 @multitable @columnfractions .20 .80
3383 @item @emph{Interface}: @tab @code{function acc_async_test(arg)}
3384 @item @tab @code{integer(kind=acc_handle_kind) arg}
3385 @item @tab @code{logical acc_async_test}
3388 @item @emph{Reference}:
3389 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3395 @node acc_async_test_all
3396 @section @code{acc_async_test_all} -- Tests for completion of all asynchronous operations.
3398 @item @emph{Description}
3399 This function tests for completion of all asynchronous operations.
3400 In C/C++, a non-zero value will be returned to indicate all asynchronous
3401 operations have completed. While Fortran will return a @code{true}. If
3402 any asynchronous operation has not completed, C/C++ returns a zero and
3403 Fortran returns a @code{false}.
3406 @multitable @columnfractions .20 .80
3407 @item @emph{Prototype}: @tab @code{int acc_async_test_all(void);}
3410 @item @emph{Fortran}:
3411 @multitable @columnfractions .20 .80
3412 @item @emph{Interface}: @tab @code{function acc_async_test()}
3413 @item @tab @code{logical acc_get_device_num}
3416 @item @emph{Reference}:
3417 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3424 @section @code{acc_wait} -- Wait for completion of a specific asynchronous operation.
3426 @item @emph{Description}
3427 This function waits for completion of the asynchronous operation
3428 specified in @var{arg}.
3431 @multitable @columnfractions .20 .80
3432 @item @emph{Prototype}: @tab @code{acc_wait(arg);}
3433 @item @emph{Prototype (OpenACC 1.0 compatibility)}: @tab @code{acc_async_wait(arg);}
3436 @item @emph{Fortran}:
3437 @multitable @columnfractions .20 .80
3438 @item @emph{Interface}: @tab @code{subroutine acc_wait(arg)}
3439 @item @tab @code{integer(acc_handle_kind) arg}
3440 @item @emph{Interface (OpenACC 1.0 compatibility)}: @tab @code{subroutine acc_async_wait(arg)}
3441 @item @tab @code{integer(acc_handle_kind) arg}
3444 @item @emph{Reference}:
3445 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3452 @section @code{acc_wait_all} -- Waits for completion of all asynchronous operations.
3454 @item @emph{Description}
3455 This function waits for the completion of all asynchronous operations.
3458 @multitable @columnfractions .20 .80
3459 @item @emph{Prototype}: @tab @code{acc_wait_all(void);}
3460 @item @emph{Prototype (OpenACC 1.0 compatibility)}: @tab @code{acc_async_wait_all(void);}
3463 @item @emph{Fortran}:
3464 @multitable @columnfractions .20 .80
3465 @item @emph{Interface}: @tab @code{subroutine acc_wait_all()}
3466 @item @emph{Interface (OpenACC 1.0 compatibility)}: @tab @code{subroutine acc_async_wait_all()}
3469 @item @emph{Reference}:
3470 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3476 @node acc_wait_all_async
3477 @section @code{acc_wait_all_async} -- Wait for completion of all asynchronous operations.
3479 @item @emph{Description}
3480 This function enqueues a wait operation on the queue @var{async} for any
3481 and all asynchronous operations that have been previously enqueued on
3485 @multitable @columnfractions .20 .80
3486 @item @emph{Prototype}: @tab @code{acc_wait_all_async(int async);}
3489 @item @emph{Fortran}:
3490 @multitable @columnfractions .20 .80
3491 @item @emph{Interface}: @tab @code{subroutine acc_wait_all_async(async)}
3492 @item @tab @code{integer(acc_handle_kind) async}
3495 @item @emph{Reference}:
3496 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3502 @node acc_wait_async
3503 @section @code{acc_wait_async} -- Wait for completion of asynchronous operations.
3505 @item @emph{Description}
3506 This function enqueues a wait operation on queue @var{async} for any and all
3507 asynchronous operations enqueued on queue @var{arg}.
3510 @multitable @columnfractions .20 .80
3511 @item @emph{Prototype}: @tab @code{acc_wait_async(int arg, int async);}
3514 @item @emph{Fortran}:
3515 @multitable @columnfractions .20 .80
3516 @item @emph{Interface}: @tab @code{subroutine acc_wait_async(arg, async)}
3517 @item @tab @code{integer(acc_handle_kind) arg, async}
3520 @item @emph{Reference}:
3521 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3528 @section @code{acc_init} -- Initialize runtime for a specific device type.
3530 @item @emph{Description}
3531 This function initializes the runtime for the device type specified in
3535 @multitable @columnfractions .20 .80
3536 @item @emph{Prototype}: @tab @code{acc_init(acc_device_t devicetype);}
3539 @item @emph{Fortran}:
3540 @multitable @columnfractions .20 .80
3541 @item @emph{Interface}: @tab @code{subroutine acc_init(devicetype)}
3542 @item @tab @code{integer(acc_device_kind) devicetype}
3545 @item @emph{Reference}:
3546 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3553 @section @code{acc_shutdown} -- Shuts down the runtime for a specific device type.
3555 @item @emph{Description}
3556 This function shuts down the runtime for the device type specified in
3560 @multitable @columnfractions .20 .80
3561 @item @emph{Prototype}: @tab @code{acc_shutdown(acc_device_t devicetype);}
3564 @item @emph{Fortran}:
3565 @multitable @columnfractions .20 .80
3566 @item @emph{Interface}: @tab @code{subroutine acc_shutdown(devicetype)}
3567 @item @tab @code{integer(acc_device_kind) devicetype}
3570 @item @emph{Reference}:
3571 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3578 @section @code{acc_on_device} -- Whether executing on a particular device
3580 @item @emph{Description}:
3581 This function returns whether the program is executing on a particular
3582 device specified in @var{devicetype}. In C/C++ a non-zero value is
3583 returned to indicate the device is executing on the specified device type.
3584 In Fortran, @code{true} will be returned. If the program is not executing
3585 on the specified device type C/C++ will return a zero, while Fortran will
3586 return @code{false}.
3589 @multitable @columnfractions .20 .80
3590 @item @emph{Prototype}: @tab @code{acc_on_device(acc_device_t devicetype);}
3593 @item @emph{Fortran}:
3594 @multitable @columnfractions .20 .80
3595 @item @emph{Interface}: @tab @code{function acc_on_device(devicetype)}
3596 @item @tab @code{integer(acc_device_kind) devicetype}
3597 @item @tab @code{logical acc_on_device}
3601 @item @emph{Reference}:
3602 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3609 @section @code{acc_malloc} -- Allocate device memory.
3611 @item @emph{Description}
3612 This function allocates @var{len} bytes of device memory. It returns
3613 the device address of the allocated memory.
3616 @multitable @columnfractions .20 .80
3617 @item @emph{Prototype}: @tab @code{d_void* acc_malloc(size_t len);}
3620 @item @emph{Reference}:
3621 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3628 @section @code{acc_free} -- Free device memory.
3630 @item @emph{Description}
3631 Free previously allocated device memory at the device address @code{a}.
3634 @multitable @columnfractions .20 .80
3635 @item @emph{Prototype}: @tab @code{acc_free(d_void *a);}
3638 @item @emph{Reference}:
3639 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3646 @section @code{acc_copyin} -- Allocate device memory and copy host memory to it.
3648 @item @emph{Description}
3649 In C/C++, this function allocates @var{len} bytes of device memory
3650 and maps it to the specified host address in @var{a}. The device
3651 address of the newly allocated device memory is returned.
3653 In Fortran, two (2) forms are supported. In the first form, @var{a} specifies
3654 a contiguous array section. The second form @var{a} specifies a
3655 variable or array element and @var{len} specifies the length in bytes.
3658 @multitable @columnfractions .20 .80
3659 @item @emph{Prototype}: @tab @code{void *acc_copyin(h_void *a, size_t len);}
3660 @item @emph{Prototype}: @tab @code{void *acc_copyin_async(h_void *a, size_t len, int async);}
3663 @item @emph{Fortran}:
3664 @multitable @columnfractions .20 .80
3665 @item @emph{Interface}: @tab @code{subroutine acc_copyin(a)}
3666 @item @tab @code{type, dimension(:[,:]...) :: a}
3667 @item @emph{Interface}: @tab @code{subroutine acc_copyin(a, len)}
3668 @item @tab @code{type, dimension(:[,:]...) :: a}
3669 @item @tab @code{integer len}
3670 @item @emph{Interface}: @tab @code{subroutine acc_copyin_async(a, async)}
3671 @item @tab @code{type, dimension(:[,:]...) :: a}
3672 @item @tab @code{integer(acc_handle_kind) :: async}
3673 @item @emph{Interface}: @tab @code{subroutine acc_copyin_async(a, len, async)}
3674 @item @tab @code{type, dimension(:[,:]...) :: a}
3675 @item @tab @code{integer len}
3676 @item @tab @code{integer(acc_handle_kind) :: async}
3679 @item @emph{Reference}:
3680 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3686 @node acc_present_or_copyin
3687 @section @code{acc_present_or_copyin} -- If the data is not present on the device, allocate device memory and copy from host memory.
3689 @item @emph{Description}
3690 This function tests if the host data specified by @var{a} and of length
3691 @var{len} is present or not. If it is not present, then device memory
3692 will be allocated and the host memory copied. The device address of
3693 the newly allocated device memory is returned.
3695 In Fortran, two (2) forms are supported. In the first form, @var{a} specifies
3696 a contiguous array section. The second form @var{a} specifies a variable or
3697 array element and @var{len} specifies the length in bytes.
3699 Note that @code{acc_present_or_copyin} and @code{acc_pcopyin} exist for
3700 backward compatibility with OpenACC 2.0; use @ref{acc_copyin} instead.
3703 @multitable @columnfractions .20 .80
3704 @item @emph{Prototype}: @tab @code{void *acc_present_or_copyin(h_void *a, size_t len);}
3705 @item @emph{Prototype}: @tab @code{void *acc_pcopyin(h_void *a, size_t len);}
3708 @item @emph{Fortran}:
3709 @multitable @columnfractions .20 .80
3710 @item @emph{Interface}: @tab @code{subroutine acc_present_or_copyin(a)}
3711 @item @tab @code{type, dimension(:[,:]...) :: a}
3712 @item @emph{Interface}: @tab @code{subroutine acc_present_or_copyin(a, len)}
3713 @item @tab @code{type, dimension(:[,:]...) :: a}
3714 @item @tab @code{integer len}
3715 @item @emph{Interface}: @tab @code{subroutine acc_pcopyin(a)}
3716 @item @tab @code{type, dimension(:[,:]...) :: a}
3717 @item @emph{Interface}: @tab @code{subroutine acc_pcopyin(a, len)}
3718 @item @tab @code{type, dimension(:[,:]...) :: a}
3719 @item @tab @code{integer len}
3722 @item @emph{Reference}:
3723 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3730 @section @code{acc_create} -- Allocate device memory and map it to host memory.
3732 @item @emph{Description}
3733 This function allocates device memory and maps it to host memory specified
3734 by the host address @var{a} with a length of @var{len} bytes. In C/C++,
3735 the function returns the device address of the allocated device memory.
3737 In Fortran, two (2) forms are supported. In the first form, @var{a} specifies
3738 a contiguous array section. The second form @var{a} specifies a variable or
3739 array element and @var{len} specifies the length in bytes.
3742 @multitable @columnfractions .20 .80
3743 @item @emph{Prototype}: @tab @code{void *acc_create(h_void *a, size_t len);}
3744 @item @emph{Prototype}: @tab @code{void *acc_create_async(h_void *a, size_t len, int async);}
3747 @item @emph{Fortran}:
3748 @multitable @columnfractions .20 .80
3749 @item @emph{Interface}: @tab @code{subroutine acc_create(a)}
3750 @item @tab @code{type, dimension(:[,:]...) :: a}
3751 @item @emph{Interface}: @tab @code{subroutine acc_create(a, len)}
3752 @item @tab @code{type, dimension(:[,:]...) :: a}
3753 @item @tab @code{integer len}
3754 @item @emph{Interface}: @tab @code{subroutine acc_create_async(a, async)}
3755 @item @tab @code{type, dimension(:[,:]...) :: a}
3756 @item @tab @code{integer(acc_handle_kind) :: async}
3757 @item @emph{Interface}: @tab @code{subroutine acc_create_async(a, len, async)}
3758 @item @tab @code{type, dimension(:[,:]...) :: a}
3759 @item @tab @code{integer len}
3760 @item @tab @code{integer(acc_handle_kind) :: async}
3763 @item @emph{Reference}:
3764 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3770 @node acc_present_or_create
3771 @section @code{acc_present_or_create} -- If the data is not present on the device, allocate device memory and map it to host memory.
3773 @item @emph{Description}
3774 This function tests if the host data specified by @var{a} and of length
3775 @var{len} is present or not. If it is not present, then device memory
3776 will be allocated and mapped to host memory. In C/C++, the device address
3777 of the newly allocated device memory is returned.
3779 In Fortran, two (2) forms are supported. In the first form, @var{a} specifies
3780 a contiguous array section. The second form @var{a} specifies a variable or
3781 array element and @var{len} specifies the length in bytes.
3783 Note that @code{acc_present_or_create} and @code{acc_pcreate} exist for
3784 backward compatibility with OpenACC 2.0; use @ref{acc_create} instead.
3787 @multitable @columnfractions .20 .80
3788 @item @emph{Prototype}: @tab @code{void *acc_present_or_create(h_void *a, size_t len)}
3789 @item @emph{Prototype}: @tab @code{void *acc_pcreate(h_void *a, size_t len)}
3792 @item @emph{Fortran}:
3793 @multitable @columnfractions .20 .80
3794 @item @emph{Interface}: @tab @code{subroutine acc_present_or_create(a)}
3795 @item @tab @code{type, dimension(:[,:]...) :: a}
3796 @item @emph{Interface}: @tab @code{subroutine acc_present_or_create(a, len)}
3797 @item @tab @code{type, dimension(:[,:]...) :: a}
3798 @item @tab @code{integer len}
3799 @item @emph{Interface}: @tab @code{subroutine acc_pcreate(a)}
3800 @item @tab @code{type, dimension(:[,:]...) :: a}
3801 @item @emph{Interface}: @tab @code{subroutine acc_pcreate(a, len)}
3802 @item @tab @code{type, dimension(:[,:]...) :: a}
3803 @item @tab @code{integer len}
3806 @item @emph{Reference}:
3807 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3814 @section @code{acc_copyout} -- Copy device memory to host memory.
3816 @item @emph{Description}
3817 This function copies mapped device memory to host memory which is specified
3818 by host address @var{a} for a length @var{len} bytes in C/C++.
3820 In Fortran, two (2) forms are supported. In the first form, @var{a} specifies
3821 a contiguous array section. The second form @var{a} specifies a variable or
3822 array element and @var{len} specifies the length in bytes.
3825 @multitable @columnfractions .20 .80
3826 @item @emph{Prototype}: @tab @code{acc_copyout(h_void *a, size_t len);}
3827 @item @emph{Prototype}: @tab @code{acc_copyout_async(h_void *a, size_t len, int async);}
3828 @item @emph{Prototype}: @tab @code{acc_copyout_finalize(h_void *a, size_t len);}
3829 @item @emph{Prototype}: @tab @code{acc_copyout_finalize_async(h_void *a, size_t len, int async);}
3832 @item @emph{Fortran}:
3833 @multitable @columnfractions .20 .80
3834 @item @emph{Interface}: @tab @code{subroutine acc_copyout(a)}
3835 @item @tab @code{type, dimension(:[,:]...) :: a}
3836 @item @emph{Interface}: @tab @code{subroutine acc_copyout(a, len)}
3837 @item @tab @code{type, dimension(:[,:]...) :: a}
3838 @item @tab @code{integer len}
3839 @item @emph{Interface}: @tab @code{subroutine acc_copyout_async(a, async)}
3840 @item @tab @code{type, dimension(:[,:]...) :: a}
3841 @item @tab @code{integer(acc_handle_kind) :: async}
3842 @item @emph{Interface}: @tab @code{subroutine acc_copyout_async(a, len, async)}
3843 @item @tab @code{type, dimension(:[,:]...) :: a}
3844 @item @tab @code{integer len}
3845 @item @tab @code{integer(acc_handle_kind) :: async}
3846 @item @emph{Interface}: @tab @code{subroutine acc_copyout_finalize(a)}
3847 @item @tab @code{type, dimension(:[,:]...) :: a}
3848 @item @emph{Interface}: @tab @code{subroutine acc_copyout_finalize(a, len)}
3849 @item @tab @code{type, dimension(:[,:]...) :: a}
3850 @item @tab @code{integer len}
3851 @item @emph{Interface}: @tab @code{subroutine acc_copyout_finalize_async(a, async)}
3852 @item @tab @code{type, dimension(:[,:]...) :: a}
3853 @item @tab @code{integer(acc_handle_kind) :: async}
3854 @item @emph{Interface}: @tab @code{subroutine acc_copyout_finalize_async(a, len, async)}
3855 @item @tab @code{type, dimension(:[,:]...) :: a}
3856 @item @tab @code{integer len}
3857 @item @tab @code{integer(acc_handle_kind) :: async}
3860 @item @emph{Reference}:
3861 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3868 @section @code{acc_delete} -- Free device memory.
3870 @item @emph{Description}
3871 This function frees previously allocated device memory specified by
3872 the device address @var{a} and the length of @var{len} bytes.
3874 In Fortran, two (2) forms are supported. In the first form, @var{a} specifies
3875 a contiguous array section. The second form @var{a} specifies a variable or
3876 array element and @var{len} specifies the length in bytes.
3879 @multitable @columnfractions .20 .80
3880 @item @emph{Prototype}: @tab @code{acc_delete(h_void *a, size_t len);}
3881 @item @emph{Prototype}: @tab @code{acc_delete_async(h_void *a, size_t len, int async);}
3882 @item @emph{Prototype}: @tab @code{acc_delete_finalize(h_void *a, size_t len);}
3883 @item @emph{Prototype}: @tab @code{acc_delete_finalize_async(h_void *a, size_t len, int async);}
3886 @item @emph{Fortran}:
3887 @multitable @columnfractions .20 .80
3888 @item @emph{Interface}: @tab @code{subroutine acc_delete(a)}
3889 @item @tab @code{type, dimension(:[,:]...) :: a}
3890 @item @emph{Interface}: @tab @code{subroutine acc_delete(a, len)}
3891 @item @tab @code{type, dimension(:[,:]...) :: a}
3892 @item @tab @code{integer len}
3893 @item @emph{Interface}: @tab @code{subroutine acc_delete_async(a, async)}
3894 @item @tab @code{type, dimension(:[,:]...) :: a}
3895 @item @tab @code{integer(acc_handle_kind) :: async}
3896 @item @emph{Interface}: @tab @code{subroutine acc_delete_async(a, len, async)}
3897 @item @tab @code{type, dimension(:[,:]...) :: a}
3898 @item @tab @code{integer len}
3899 @item @tab @code{integer(acc_handle_kind) :: async}
3900 @item @emph{Interface}: @tab @code{subroutine acc_delete_finalize(a)}
3901 @item @tab @code{type, dimension(:[,:]...) :: a}
3902 @item @emph{Interface}: @tab @code{subroutine acc_delete_finalize(a, len)}
3903 @item @tab @code{type, dimension(:[,:]...) :: a}
3904 @item @tab @code{integer len}
3905 @item @emph{Interface}: @tab @code{subroutine acc_delete_async_finalize(a, async)}
3906 @item @tab @code{type, dimension(:[,:]...) :: a}
3907 @item @tab @code{integer(acc_handle_kind) :: async}
3908 @item @emph{Interface}: @tab @code{subroutine acc_delete_async_finalize(a, len, async)}
3909 @item @tab @code{type, dimension(:[,:]...) :: a}
3910 @item @tab @code{integer len}
3911 @item @tab @code{integer(acc_handle_kind) :: async}
3914 @item @emph{Reference}:
3915 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3921 @node acc_update_device
3922 @section @code{acc_update_device} -- Update device memory from mapped host memory.
3924 @item @emph{Description}
3925 This function updates the device copy from the previously mapped host memory.
3926 The host memory is specified with the host address @var{a} and a length of
3929 In Fortran, two (2) forms are supported. In the first form, @var{a} specifies
3930 a contiguous array section. The second form @var{a} specifies a variable or
3931 array element and @var{len} specifies the length in bytes.
3934 @multitable @columnfractions .20 .80
3935 @item @emph{Prototype}: @tab @code{acc_update_device(h_void *a, size_t len);}
3936 @item @emph{Prototype}: @tab @code{acc_update_device(h_void *a, size_t len, async);}
3939 @item @emph{Fortran}:
3940 @multitable @columnfractions .20 .80
3941 @item @emph{Interface}: @tab @code{subroutine acc_update_device(a)}
3942 @item @tab @code{type, dimension(:[,:]...) :: a}
3943 @item @emph{Interface}: @tab @code{subroutine acc_update_device(a, len)}
3944 @item @tab @code{type, dimension(:[,:]...) :: a}
3945 @item @tab @code{integer len}
3946 @item @emph{Interface}: @tab @code{subroutine acc_update_device_async(a, async)}
3947 @item @tab @code{type, dimension(:[,:]...) :: a}
3948 @item @tab @code{integer(acc_handle_kind) :: async}
3949 @item @emph{Interface}: @tab @code{subroutine acc_update_device_async(a, len, async)}
3950 @item @tab @code{type, dimension(:[,:]...) :: a}
3951 @item @tab @code{integer len}
3952 @item @tab @code{integer(acc_handle_kind) :: async}
3955 @item @emph{Reference}:
3956 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
3962 @node acc_update_self
3963 @section @code{acc_update_self} -- Update host memory from mapped device memory.
3965 @item @emph{Description}
3966 This function updates the host copy from the previously mapped device memory.
3967 The host memory is specified with the host address @var{a} and a length of
3970 In Fortran, two (2) forms are supported. In the first form, @var{a} specifies
3971 a contiguous array section. The second form @var{a} specifies a variable or
3972 array element and @var{len} specifies the length in bytes.
3975 @multitable @columnfractions .20 .80
3976 @item @emph{Prototype}: @tab @code{acc_update_self(h_void *a, size_t len);}
3977 @item @emph{Prototype}: @tab @code{acc_update_self_async(h_void *a, size_t len, int async);}
3980 @item @emph{Fortran}:
3981 @multitable @columnfractions .20 .80
3982 @item @emph{Interface}: @tab @code{subroutine acc_update_self(a)}
3983 @item @tab @code{type, dimension(:[,:]...) :: a}
3984 @item @emph{Interface}: @tab @code{subroutine acc_update_self(a, len)}
3985 @item @tab @code{type, dimension(:[,:]...) :: a}
3986 @item @tab @code{integer len}
3987 @item @emph{Interface}: @tab @code{subroutine acc_update_self_async(a, async)}
3988 @item @tab @code{type, dimension(:[,:]...) :: a}
3989 @item @tab @code{integer(acc_handle_kind) :: async}
3990 @item @emph{Interface}: @tab @code{subroutine acc_update_self_async(a, len, async)}
3991 @item @tab @code{type, dimension(:[,:]...) :: a}
3992 @item @tab @code{integer len}
3993 @item @tab @code{integer(acc_handle_kind) :: async}
3996 @item @emph{Reference}:
3997 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
4004 @section @code{acc_map_data} -- Map previously allocated device memory to host memory.
4006 @item @emph{Description}
4007 This function maps previously allocated device and host memory. The device
4008 memory is specified with the device address @var{d}. The host memory is
4009 specified with the host address @var{h} and a length of @var{len}.
4012 @multitable @columnfractions .20 .80
4013 @item @emph{Prototype}: @tab @code{acc_map_data(h_void *h, d_void *d, size_t len);}
4016 @item @emph{Reference}:
4017 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
4023 @node acc_unmap_data
4024 @section @code{acc_unmap_data} -- Unmap device memory from host memory.
4026 @item @emph{Description}
4027 This function unmaps previously mapped device and host memory. The latter
4028 specified by @var{h}.
4031 @multitable @columnfractions .20 .80
4032 @item @emph{Prototype}: @tab @code{acc_unmap_data(h_void *h);}
4035 @item @emph{Reference}:
4036 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
4043 @section @code{acc_deviceptr} -- Get device pointer associated with specific host address.
4045 @item @emph{Description}
4046 This function returns the device address that has been mapped to the
4047 host address specified by @var{h}.
4050 @multitable @columnfractions .20 .80
4051 @item @emph{Prototype}: @tab @code{void *acc_deviceptr(h_void *h);}
4054 @item @emph{Reference}:
4055 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
4062 @section @code{acc_hostptr} -- Get host pointer associated with specific device address.
4064 @item @emph{Description}
4065 This function returns the host address that has been mapped to the
4066 device address specified by @var{d}.
4069 @multitable @columnfractions .20 .80
4070 @item @emph{Prototype}: @tab @code{void *acc_hostptr(d_void *d);}
4073 @item @emph{Reference}:
4074 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
4080 @node acc_is_present
4081 @section @code{acc_is_present} -- Indicate whether host variable / array is present on device.
4083 @item @emph{Description}
4084 This function indicates whether the specified host address in @var{a} and a
4085 length of @var{len} bytes is present on the device. In C/C++, a non-zero
4086 value is returned to indicate the presence of the mapped memory on the
4087 device. A zero is returned to indicate the memory is not mapped on the
4090 In Fortran, two (2) forms are supported. In the first form, @var{a} specifies
4091 a contiguous array section. The second form @var{a} specifies a variable or
4092 array element and @var{len} specifies the length in bytes. If the host
4093 memory is mapped to device memory, then a @code{true} is returned. Otherwise,
4094 a @code{false} is return to indicate the mapped memory is not present.
4097 @multitable @columnfractions .20 .80
4098 @item @emph{Prototype}: @tab @code{int acc_is_present(h_void *a, size_t len);}
4101 @item @emph{Fortran}:
4102 @multitable @columnfractions .20 .80
4103 @item @emph{Interface}: @tab @code{function acc_is_present(a)}
4104 @item @tab @code{type, dimension(:[,:]...) :: a}
4105 @item @tab @code{logical acc_is_present}
4106 @item @emph{Interface}: @tab @code{function acc_is_present(a, len)}
4107 @item @tab @code{type, dimension(:[,:]...) :: a}
4108 @item @tab @code{integer len}
4109 @item @tab @code{logical acc_is_present}
4112 @item @emph{Reference}:
4113 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
4119 @node acc_memcpy_to_device
4120 @section @code{acc_memcpy_to_device} -- Copy host memory to device memory.
4122 @item @emph{Description}
4123 This function copies host memory specified by host address of @var{src} to
4124 device memory specified by the device address @var{dest} for a length of
4128 @multitable @columnfractions .20 .80
4129 @item @emph{Prototype}: @tab @code{acc_memcpy_to_device(d_void *dest, h_void *src, size_t bytes);}
4132 @item @emph{Reference}:
4133 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
4139 @node acc_memcpy_from_device
4140 @section @code{acc_memcpy_from_device} -- Copy device memory to host memory.
4142 @item @emph{Description}
4143 This function copies host memory specified by host address of @var{src} from
4144 device memory specified by the device address @var{dest} for a length of
4148 @multitable @columnfractions .20 .80
4149 @item @emph{Prototype}: @tab @code{acc_memcpy_from_device(d_void *dest, h_void *src, size_t bytes);}
4152 @item @emph{Reference}:
4153 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
4160 @section @code{acc_attach} -- Let device pointer point to device-pointer target.
4162 @item @emph{Description}
4163 This function updates a pointer on the device from pointing to a host-pointer
4164 address to pointing to the corresponding device data.
4167 @multitable @columnfractions .20 .80
4168 @item @emph{Prototype}: @tab @code{acc_attach(h_void **ptr);}
4169 @item @emph{Prototype}: @tab @code{acc_attach_async(h_void **ptr, int async);}
4172 @item @emph{Reference}:
4173 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
4180 @section @code{acc_detach} -- Let device pointer point to host-pointer target.
4182 @item @emph{Description}
4183 This function updates a pointer on the device from pointing to a device-pointer
4184 address to pointing to the corresponding host data.
4187 @multitable @columnfractions .20 .80
4188 @item @emph{Prototype}: @tab @code{acc_detach(h_void **ptr);}
4189 @item @emph{Prototype}: @tab @code{acc_detach_async(h_void **ptr, int async);}
4190 @item @emph{Prototype}: @tab @code{acc_detach_finalize(h_void **ptr);}
4191 @item @emph{Prototype}: @tab @code{acc_detach_finalize_async(h_void **ptr, int async);}
4194 @item @emph{Reference}:
4195 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
4201 @node acc_get_current_cuda_device
4202 @section @code{acc_get_current_cuda_device} -- Get CUDA device handle.
4204 @item @emph{Description}
4205 This function returns the CUDA device handle. This handle is the same
4206 as used by the CUDA Runtime or Driver API's.
4209 @multitable @columnfractions .20 .80
4210 @item @emph{Prototype}: @tab @code{void *acc_get_current_cuda_device(void);}
4213 @item @emph{Reference}:
4214 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
4220 @node acc_get_current_cuda_context
4221 @section @code{acc_get_current_cuda_context} -- Get CUDA context handle.
4223 @item @emph{Description}
4224 This function returns the CUDA context handle. This handle is the same
4225 as used by the CUDA Runtime or Driver API's.
4228 @multitable @columnfractions .20 .80
4229 @item @emph{Prototype}: @tab @code{void *acc_get_current_cuda_context(void);}
4232 @item @emph{Reference}:
4233 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
4239 @node acc_get_cuda_stream
4240 @section @code{acc_get_cuda_stream} -- Get CUDA stream handle.
4242 @item @emph{Description}
4243 This function returns the CUDA stream handle for the queue @var{async}.
4244 This handle is the same as used by the CUDA Runtime or Driver API's.
4247 @multitable @columnfractions .20 .80
4248 @item @emph{Prototype}: @tab @code{void *acc_get_cuda_stream(int async);}
4251 @item @emph{Reference}:
4252 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
4258 @node acc_set_cuda_stream
4259 @section @code{acc_set_cuda_stream} -- Set CUDA stream handle.
4261 @item @emph{Description}
4262 This function associates the stream handle specified by @var{stream} with
4263 the queue @var{async}.
4265 This cannot be used to change the stream handle associated with
4266 @code{acc_async_sync}.
4268 The return value is not specified.
4271 @multitable @columnfractions .20 .80
4272 @item @emph{Prototype}: @tab @code{int acc_set_cuda_stream(int async, void *stream);}
4275 @item @emph{Reference}:
4276 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
4282 @node acc_prof_register
4283 @section @code{acc_prof_register} -- Register callbacks.
4285 @item @emph{Description}:
4286 This function registers callbacks.
4289 @multitable @columnfractions .20 .80
4290 @item @emph{Prototype}: @tab @code{void acc_prof_register (acc_event_t, acc_prof_callback, acc_register_t);}
4293 @item @emph{See also}:
4294 @ref{OpenACC Profiling Interface}
4296 @item @emph{Reference}:
4297 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
4303 @node acc_prof_unregister
4304 @section @code{acc_prof_unregister} -- Unregister callbacks.
4306 @item @emph{Description}:
4307 This function unregisters callbacks.
4310 @multitable @columnfractions .20 .80
4311 @item @emph{Prototype}: @tab @code{void acc_prof_unregister (acc_event_t, acc_prof_callback, acc_register_t);}
4314 @item @emph{See also}:
4315 @ref{OpenACC Profiling Interface}
4317 @item @emph{Reference}:
4318 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
4324 @node acc_prof_lookup
4325 @section @code{acc_prof_lookup} -- Obtain inquiry functions.
4327 @item @emph{Description}:
4328 Function to obtain inquiry functions.
4331 @multitable @columnfractions .20 .80
4332 @item @emph{Prototype}: @tab @code{acc_query_fn acc_prof_lookup (const char *);}
4335 @item @emph{See also}:
4336 @ref{OpenACC Profiling Interface}
4338 @item @emph{Reference}:
4339 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
4345 @node acc_register_library
4346 @section @code{acc_register_library} -- Library registration.
4348 @item @emph{Description}:
4349 Function for library registration.
4352 @multitable @columnfractions .20 .80
4353 @item @emph{Prototype}: @tab @code{void acc_register_library (acc_prof_reg, acc_prof_reg, acc_prof_lookup_func);}
4356 @item @emph{See also}:
4357 @ref{OpenACC Profiling Interface}, @ref{ACC_PROFLIB}
4359 @item @emph{Reference}:
4360 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
4366 @c ---------------------------------------------------------------------
4367 @c OpenACC Environment Variables
4368 @c ---------------------------------------------------------------------
4370 @node OpenACC Environment Variables
4371 @chapter OpenACC Environment Variables
4373 The variables @env{ACC_DEVICE_TYPE} and @env{ACC_DEVICE_NUM}
4374 are defined by section 4 of the OpenACC specification in version 2.0.
4375 The variable @env{ACC_PROFLIB}
4376 is defined by section 4 of the OpenACC specification in version 2.6.
4377 The variable @env{GCC_ACC_NOTIFY} is used for diagnostic purposes.
4388 @node ACC_DEVICE_TYPE
4389 @section @code{ACC_DEVICE_TYPE}
4391 @item @emph{Reference}:
4392 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
4398 @node ACC_DEVICE_NUM
4399 @section @code{ACC_DEVICE_NUM}
4401 @item @emph{Reference}:
4402 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
4409 @section @code{ACC_PROFLIB}
4411 @item @emph{See also}:
4412 @ref{acc_register_library}, @ref{OpenACC Profiling Interface}
4414 @item @emph{Reference}:
4415 @uref{https://www.openacc.org, OpenACC specification v2.6}, section
4421 @node GCC_ACC_NOTIFY
4422 @section @code{GCC_ACC_NOTIFY}
4424 @item @emph{Description}:
4425 Print debug information pertaining to the accelerator.
4430 @c ---------------------------------------------------------------------
4431 @c CUDA Streams Usage
4432 @c ---------------------------------------------------------------------
4434 @node CUDA Streams Usage
4435 @chapter CUDA Streams Usage
4437 This applies to the @code{nvptx} plugin only.
4439 The library provides elements that perform asynchronous movement of
4440 data and asynchronous operation of computing constructs. This
4441 asynchronous functionality is implemented by making use of CUDA
4442 streams@footnote{See "Stream Management" in "CUDA Driver API",
4443 TRM-06703-001, Version 5.5, for additional information}.
4445 The primary means by that the asynchronous functionality is accessed
4446 is through the use of those OpenACC directives which make use of the
4447 @code{async} and @code{wait} clauses. When the @code{async} clause is
4448 first used with a directive, it creates a CUDA stream. If an
4449 @code{async-argument} is used with the @code{async} clause, then the
4450 stream is associated with the specified @code{async-argument}.
4452 Following the creation of an association between a CUDA stream and the
4453 @code{async-argument} of an @code{async} clause, both the @code{wait}
4454 clause and the @code{wait} directive can be used. When either the
4455 clause or directive is used after stream creation, it creates a
4456 rendezvous point whereby execution waits until all operations
4457 associated with the @code{async-argument}, that is, stream, have
4460 Normally, the management of the streams that are created as a result of
4461 using the @code{async} clause, is done without any intervention by the
4462 caller. This implies the association between the @code{async-argument}
4463 and the CUDA stream will be maintained for the lifetime of the program.
4464 However, this association can be changed through the use of the library
4465 function @code{acc_set_cuda_stream}. When the function
4466 @code{acc_set_cuda_stream} is called, the CUDA stream that was
4467 originally associated with the @code{async} clause will be destroyed.
4468 Caution should be taken when changing the association as subsequent
4469 references to the @code{async-argument} refer to a different
4474 @c ---------------------------------------------------------------------
4475 @c OpenACC Library Interoperability
4476 @c ---------------------------------------------------------------------
4478 @node OpenACC Library Interoperability
4479 @chapter OpenACC Library Interoperability
4481 @section Introduction
4483 The OpenACC library uses the CUDA Driver API, and may interact with
4484 programs that use the Runtime library directly, or another library
4485 based on the Runtime library, e.g., CUBLAS@footnote{See section 2.26,
4486 "Interactions with the CUDA Driver API" in
4487 "CUDA Runtime API", Version 5.5, and section 2.27, "VDPAU
4488 Interoperability", in "CUDA Driver API", TRM-06703-001, Version 5.5,
4489 for additional information on library interoperability.}.
4490 This chapter describes the use cases and what changes are
4491 required in order to use both the OpenACC library and the CUBLAS and Runtime
4492 libraries within a program.
4494 @section First invocation: NVIDIA CUBLAS library API
4496 In this first use case (see below), a function in the CUBLAS library is called
4497 prior to any of the functions in the OpenACC library. More specifically, the
4498 function @code{cublasCreate()}.
4500 When invoked, the function initializes the library and allocates the
4501 hardware resources on the host and the device on behalf of the caller. Once
4502 the initialization and allocation has completed, a handle is returned to the
4503 caller. The OpenACC library also requires initialization and allocation of
4504 hardware resources. Since the CUBLAS library has already allocated the
4505 hardware resources for the device, all that is left to do is to initialize
4506 the OpenACC library and acquire the hardware resources on the host.
4508 Prior to calling the OpenACC function that initializes the library and
4509 allocate the host hardware resources, you need to acquire the device number
4510 that was allocated during the call to @code{cublasCreate()}. The invoking of the
4511 runtime library function @code{cudaGetDevice()} accomplishes this. Once
4512 acquired, the device number is passed along with the device type as
4513 parameters to the OpenACC library function @code{acc_set_device_num()}.
4515 Once the call to @code{acc_set_device_num()} has completed, the OpenACC
4516 library uses the context that was created during the call to
4517 @code{cublasCreate()}. In other words, both libraries will be sharing the
4521 /* Create the handle */
4522 s = cublasCreate(&h);
4523 if (s != CUBLAS_STATUS_SUCCESS)
4525 fprintf(stderr, "cublasCreate failed %d\n", s);
4529 /* Get the device number */
4530 e = cudaGetDevice(&dev);
4531 if (e != cudaSuccess)
4533 fprintf(stderr, "cudaGetDevice failed %d\n", e);
4537 /* Initialize OpenACC library and use device 'dev' */
4538 acc_set_device_num(dev, acc_device_nvidia);
4543 @section First invocation: OpenACC library API
4545 In this second use case (see below), a function in the OpenACC library is
4546 called prior to any of the functions in the CUBLAS library. More specifically,
4547 the function @code{acc_set_device_num()}.
4549 In the use case presented here, the function @code{acc_set_device_num()}
4550 is used to both initialize the OpenACC library and allocate the hardware
4551 resources on the host and the device. In the call to the function, the
4552 call parameters specify which device to use and what device
4553 type to use, i.e., @code{acc_device_nvidia}. It should be noted that this
4554 is but one method to initialize the OpenACC library and allocate the
4555 appropriate hardware resources. Other methods are available through the
4556 use of environment variables and these will be discussed in the next section.
4558 Once the call to @code{acc_set_device_num()} has completed, other OpenACC
4559 functions can be called as seen with multiple calls being made to
4560 @code{acc_copyin()}. In addition, calls can be made to functions in the
4561 CUBLAS library. In the use case a call to @code{cublasCreate()} is made
4562 subsequent to the calls to @code{acc_copyin()}.
4563 As seen in the previous use case, a call to @code{cublasCreate()}
4564 initializes the CUBLAS library and allocates the hardware resources on the
4565 host and the device. However, since the device has already been allocated,
4566 @code{cublasCreate()} will only initialize the CUBLAS library and allocate
4567 the appropriate hardware resources on the host. The context that was created
4568 as part of the OpenACC initialization is shared with the CUBLAS library,
4569 similarly to the first use case.
4574 acc_set_device_num(dev, acc_device_nvidia);
4576 /* Copy the first set to the device */
4577 d_X = acc_copyin(&h_X[0], N * sizeof (float));
4580 fprintf(stderr, "copyin error h_X\n");
4584 /* Copy the second set to the device */
4585 d_Y = acc_copyin(&h_Y1[0], N * sizeof (float));
4588 fprintf(stderr, "copyin error h_Y1\n");
4592 /* Create the handle */
4593 s = cublasCreate(&h);
4594 if (s != CUBLAS_STATUS_SUCCESS)
4596 fprintf(stderr, "cublasCreate failed %d\n", s);
4600 /* Perform saxpy using CUBLAS library function */
4601 s = cublasSaxpy(h, N, &alpha, d_X, 1, d_Y, 1);
4602 if (s != CUBLAS_STATUS_SUCCESS)
4604 fprintf(stderr, "cublasSaxpy failed %d\n", s);
4608 /* Copy the results from the device */
4609 acc_memcpy_from_device(&h_Y1[0], d_Y, N * sizeof (float));
4614 @section OpenACC library and environment variables
4616 There are two environment variables associated with the OpenACC library
4617 that may be used to control the device type and device number:
4618 @env{ACC_DEVICE_TYPE} and @env{ACC_DEVICE_NUM}, respectively. These two
4619 environment variables can be used as an alternative to calling
4620 @code{acc_set_device_num()}. As seen in the second use case, the device
4621 type and device number were specified using @code{acc_set_device_num()}.
4622 If however, the aforementioned environment variables were set, then the
4623 call to @code{acc_set_device_num()} would not be required.
4626 The use of the environment variables is only relevant when an OpenACC function
4627 is called prior to a call to @code{cudaCreate()}. If @code{cudaCreate()}
4628 is called prior to a call to an OpenACC function, then you must call
4629 @code{acc_set_device_num()}@footnote{More complete information
4630 about @env{ACC_DEVICE_TYPE} and @env{ACC_DEVICE_NUM} can be found in
4631 sections 4.1 and 4.2 of the @uref{https://www.openacc.org, OpenACC}
4632 Application Programming Interface”, Version 2.6.}
4636 @c ---------------------------------------------------------------------
4637 @c OpenACC Profiling Interface
4638 @c ---------------------------------------------------------------------
4640 @node OpenACC Profiling Interface
4641 @chapter OpenACC Profiling Interface
4643 @section Implementation Status and Implementation-Defined Behavior
4645 We're implementing the OpenACC Profiling Interface as defined by the
4646 OpenACC 2.6 specification. We're clarifying some aspects here as
4647 @emph{implementation-defined behavior}, while they're still under
4648 discussion within the OpenACC Technical Committee.
4650 This implementation is tuned to keep the performance impact as low as
4651 possible for the (very common) case that the Profiling Interface is
4652 not enabled. This is relevant, as the Profiling Interface affects all
4653 the @emph{hot} code paths (in the target code, not in the offloaded
4654 code). Users of the OpenACC Profiling Interface can be expected to
4655 understand that performance will be impacted to some degree once the
4656 Profiling Interface has gotten enabled: for example, because of the
4657 @emph{runtime} (libgomp) calling into a third-party @emph{library} for
4658 every event that has been registered.
4660 We're not yet accounting for the fact that @cite{OpenACC events may
4661 occur during event processing}.
4662 We just handle one case specially, as required by CUDA 9.0
4663 @command{nvprof}, that @code{acc_get_device_type}
4664 (@ref{acc_get_device_type})) may be called from
4665 @code{acc_ev_device_init_start}, @code{acc_ev_device_init_end}
4668 We're not yet implementing initialization via a
4669 @code{acc_register_library} function that is either statically linked
4670 in, or dynamically via @env{LD_PRELOAD}.
4671 Initialization via @code{acc_register_library} functions dynamically
4672 loaded via the @env{ACC_PROFLIB} environment variable does work, as
4673 does directly calling @code{acc_prof_register},
4674 @code{acc_prof_unregister}, @code{acc_prof_lookup}.
4676 As currently there are no inquiry functions defined, calls to
4677 @code{acc_prof_lookup} will always return @code{NULL}.
4679 There aren't separate @emph{start}, @emph{stop} events defined for the
4680 event types @code{acc_ev_create}, @code{acc_ev_delete},
4681 @code{acc_ev_alloc}, @code{acc_ev_free}. It's not clear if these
4682 should be triggered before or after the actual device-specific call is
4683 made. We trigger them after.
4685 Remarks about data provided to callbacks:
4689 @item @code{acc_prof_info.event_type}
4690 It's not clear if for @emph{nested} event callbacks (for example,
4691 @code{acc_ev_enqueue_launch_start} as part of a parent compute
4692 construct), this should be set for the nested event
4693 (@code{acc_ev_enqueue_launch_start}), or if the value of the parent
4694 construct should remain (@code{acc_ev_compute_construct_start}). In
4695 this implementation, the value will generally correspond to the
4696 innermost nested event type.
4698 @item @code{acc_prof_info.device_type}
4702 For @code{acc_ev_compute_construct_start}, and in presence of an
4703 @code{if} clause with @emph{false} argument, this will still refer to
4704 the offloading device type.
4705 It's not clear if that's the expected behavior.
4708 Complementary to the item before, for
4709 @code{acc_ev_compute_construct_end}, this is set to
4710 @code{acc_device_host} in presence of an @code{if} clause with
4711 @emph{false} argument.
4712 It's not clear if that's the expected behavior.
4716 @item @code{acc_prof_info.thread_id}
4717 Always @code{-1}; not yet implemented.
4719 @item @code{acc_prof_info.async}
4723 Not yet implemented correctly for
4724 @code{acc_ev_compute_construct_start}.
4727 In a compute construct, for host-fallback
4728 execution/@code{acc_device_host} it will always be
4729 @code{acc_async_sync}.
4730 It's not clear if that's the expected behavior.
4733 For @code{acc_ev_device_init_start} and @code{acc_ev_device_init_end},
4734 it will always be @code{acc_async_sync}.
4735 It's not clear if that's the expected behavior.
4739 @item @code{acc_prof_info.async_queue}
4740 There is no @cite{limited number of asynchronous queues} in libgomp.
4741 This will always have the same value as @code{acc_prof_info.async}.
4743 @item @code{acc_prof_info.src_file}
4744 Always @code{NULL}; not yet implemented.
4746 @item @code{acc_prof_info.func_name}
4747 Always @code{NULL}; not yet implemented.
4749 @item @code{acc_prof_info.line_no}
4750 Always @code{-1}; not yet implemented.
4752 @item @code{acc_prof_info.end_line_no}
4753 Always @code{-1}; not yet implemented.
4755 @item @code{acc_prof_info.func_line_no}
4756 Always @code{-1}; not yet implemented.
4758 @item @code{acc_prof_info.func_end_line_no}
4759 Always @code{-1}; not yet implemented.
4761 @item @code{acc_event_info.event_type}, @code{acc_event_info.*.event_type}
4762 Relating to @code{acc_prof_info.event_type} discussed above, in this
4763 implementation, this will always be the same value as
4764 @code{acc_prof_info.event_type}.
4766 @item @code{acc_event_info.*.parent_construct}
4770 Will be @code{acc_construct_parallel} for all OpenACC compute
4771 constructs as well as many OpenACC Runtime API calls; should be the
4772 one matching the actual construct, or
4773 @code{acc_construct_runtime_api}, respectively.
4776 Will be @code{acc_construct_enter_data} or
4777 @code{acc_construct_exit_data} when processing variable mappings
4778 specified in OpenACC @emph{declare} directives; should be
4779 @code{acc_construct_declare}.
4782 For implicit @code{acc_ev_device_init_start},
4783 @code{acc_ev_device_init_end}, and explicit as well as implicit
4784 @code{acc_ev_alloc}, @code{acc_ev_free},
4785 @code{acc_ev_enqueue_upload_start}, @code{acc_ev_enqueue_upload_end},
4786 @code{acc_ev_enqueue_download_start}, and
4787 @code{acc_ev_enqueue_download_end}, will be
4788 @code{acc_construct_parallel}; should reflect the real parent
4793 @item @code{acc_event_info.*.implicit}
4794 For @code{acc_ev_alloc}, @code{acc_ev_free},
4795 @code{acc_ev_enqueue_upload_start}, @code{acc_ev_enqueue_upload_end},
4796 @code{acc_ev_enqueue_download_start}, and
4797 @code{acc_ev_enqueue_download_end}, this currently will be @code{1}
4798 also for explicit usage.
4800 @item @code{acc_event_info.data_event.var_name}
4801 Always @code{NULL}; not yet implemented.
4803 @item @code{acc_event_info.data_event.host_ptr}
4804 For @code{acc_ev_alloc}, and @code{acc_ev_free}, this is always
4807 @item @code{typedef union acc_api_info}
4808 @dots{} as printed in @cite{5.2.3. Third Argument: API-Specific
4809 Information}. This should obviously be @code{typedef @emph{struct}
4812 @item @code{acc_api_info.device_api}
4813 Possibly not yet implemented correctly for
4814 @code{acc_ev_compute_construct_start},
4815 @code{acc_ev_device_init_start}, @code{acc_ev_device_init_end}:
4816 will always be @code{acc_device_api_none} for these event types.
4817 For @code{acc_ev_enter_data_start}, it will be
4818 @code{acc_device_api_none} in some cases.
4820 @item @code{acc_api_info.device_type}
4821 Always the same as @code{acc_prof_info.device_type}.
4823 @item @code{acc_api_info.vendor}
4824 Always @code{-1}; not yet implemented.
4826 @item @code{acc_api_info.device_handle}
4827 Always @code{NULL}; not yet implemented.
4829 @item @code{acc_api_info.context_handle}
4830 Always @code{NULL}; not yet implemented.
4832 @item @code{acc_api_info.async_handle}
4833 Always @code{NULL}; not yet implemented.
4837 Remarks about certain event types:
4841 @item @code{acc_ev_device_init_start}, @code{acc_ev_device_init_end}
4845 @c See 'DEVICE_INIT_INSIDE_COMPUTE_CONSTRUCT' in
4846 @c 'libgomp.oacc-c-c++-common/acc_prof-kernels-1.c',
4847 @c 'libgomp.oacc-c-c++-common/acc_prof-parallel-1.c'.
4848 When a compute construct triggers implicit
4849 @code{acc_ev_device_init_start} and @code{acc_ev_device_init_end}
4850 events, they currently aren't @emph{nested within} the corresponding
4851 @code{acc_ev_compute_construct_start} and
4852 @code{acc_ev_compute_construct_end}, but they're currently observed
4853 @emph{before} @code{acc_ev_compute_construct_start}.
4854 It's not clear what to do: the standard asks us provide a lot of
4855 details to the @code{acc_ev_compute_construct_start} callback, without
4856 (implicitly) initializing a device before?
4859 Callbacks for these event types will not be invoked for calls to the
4860 @code{acc_set_device_type} and @code{acc_set_device_num} functions.
4861 It's not clear if they should be.
4865 @item @code{acc_ev_enter_data_start}, @code{acc_ev_enter_data_end}, @code{acc_ev_exit_data_start}, @code{acc_ev_exit_data_end}
4869 Callbacks for these event types will also be invoked for OpenACC
4870 @emph{host_data} constructs.
4871 It's not clear if they should be.
4874 Callbacks for these event types will also be invoked when processing
4875 variable mappings specified in OpenACC @emph{declare} directives.
4876 It's not clear if they should be.
4882 Callbacks for the following event types will be invoked, but dispatch
4883 and information provided therein has not yet been thoroughly reviewed:
4886 @item @code{acc_ev_alloc}
4887 @item @code{acc_ev_free}
4888 @item @code{acc_ev_update_start}, @code{acc_ev_update_end}
4889 @item @code{acc_ev_enqueue_upload_start}, @code{acc_ev_enqueue_upload_end}
4890 @item @code{acc_ev_enqueue_download_start}, @code{acc_ev_enqueue_download_end}
4893 During device initialization, and finalization, respectively,
4894 callbacks for the following event types will not yet be invoked:
4897 @item @code{acc_ev_alloc}
4898 @item @code{acc_ev_free}
4901 Callbacks for the following event types have not yet been implemented,
4902 so currently won't be invoked:
4905 @item @code{acc_ev_device_shutdown_start}, @code{acc_ev_device_shutdown_end}
4906 @item @code{acc_ev_runtime_shutdown}
4907 @item @code{acc_ev_create}, @code{acc_ev_delete}
4908 @item @code{acc_ev_wait_start}, @code{acc_ev_wait_end}
4911 For the following runtime library functions, not all expected
4912 callbacks will be invoked (mostly concerning implicit device
4916 @item @code{acc_get_num_devices}
4917 @item @code{acc_set_device_type}
4918 @item @code{acc_get_device_type}
4919 @item @code{acc_set_device_num}
4920 @item @code{acc_get_device_num}
4921 @item @code{acc_init}
4922 @item @code{acc_shutdown}
4925 Aside from implicit device initialization, for the following runtime
4926 library functions, no callbacks will be invoked for shared-memory
4927 offloading devices (it's not clear if they should be):
4930 @item @code{acc_malloc}
4931 @item @code{acc_free}
4932 @item @code{acc_copyin}, @code{acc_present_or_copyin}, @code{acc_copyin_async}
4933 @item @code{acc_create}, @code{acc_present_or_create}, @code{acc_create_async}
4934 @item @code{acc_copyout}, @code{acc_copyout_async}, @code{acc_copyout_finalize}, @code{acc_copyout_finalize_async}
4935 @item @code{acc_delete}, @code{acc_delete_async}, @code{acc_delete_finalize}, @code{acc_delete_finalize_async}
4936 @item @code{acc_update_device}, @code{acc_update_device_async}
4937 @item @code{acc_update_self}, @code{acc_update_self_async}
4938 @item @code{acc_map_data}, @code{acc_unmap_data}
4939 @item @code{acc_memcpy_to_device}, @code{acc_memcpy_to_device_async}
4940 @item @code{acc_memcpy_from_device}, @code{acc_memcpy_from_device_async}
4943 @c ---------------------------------------------------------------------
4944 @c OpenMP-Implementation Specifics
4945 @c ---------------------------------------------------------------------
4947 @node OpenMP-Implementation Specifics
4948 @chapter OpenMP-Implementation Specifics
4951 * Implementation-defined ICV Initialization::
4952 * OpenMP Context Selectors::
4953 * Memory allocation::
4956 @node Implementation-defined ICV Initialization
4957 @section Implementation-defined ICV Initialization
4958 @cindex Implementation specific setting
4960 @multitable @columnfractions .30 .70
4961 @item @var{affinity-format-var} @tab See @ref{OMP_AFFINITY_FORMAT}.
4962 @item @var{def-allocator-var} @tab See @ref{OMP_ALLOCATOR}.
4963 @item @var{max-active-levels-var} @tab See @ref{OMP_MAX_ACTIVE_LEVELS}.
4964 @item @var{dyn-var} @tab See @ref{OMP_DYNAMIC}.
4965 @item @var{nthreads-var} @tab See @ref{OMP_NUM_THREADS}.
4966 @item @var{num-devices-var} @tab Number of non-host devices found
4967 by GCC's run-time library
4968 @item @var{num-procs-var} @tab The number of CPU cores on the
4969 initial device, except that affinity settings might lead to a
4970 smaller number. On non-host devices, the value of the
4971 @var{nthreads-var} ICV.
4972 @item @var{place-partition-var} @tab See @ref{OMP_PLACES}.
4973 @item @var{run-sched-var} @tab See @ref{OMP_SCHEDULE}.
4974 @item @var{stacksize-var} @tab See @ref{OMP_STACKSIZE}.
4975 @item @var{thread-limit-var} @tab See @ref{OMP_TEAMS_THREAD_LIMIT}
4976 @item @var{wait-policy-var} @tab See @ref{OMP_WAIT_POLICY} and
4977 @ref{GOMP_SPINCOUNT}
4980 @node OpenMP Context Selectors
4981 @section OpenMP Context Selectors
4983 @code{vendor} is always @code{gnu}. References are to the GCC manual.
4985 @multitable @columnfractions .60 .10 .25
4986 @headitem @code{arch} @tab @code{kind} @tab @code{isa}
4987 @item @code{x86}, @code{x86_64}, @code{i386}, @code{i486},
4988 @code{i586}, @code{i686}, @code{ia32}
4990 @tab See @code{-m...} flags in ``x86 Options'' (without @code{-m})
4991 @item @code{amdgcn}, @code{gcn}
4993 @tab See @code{-march=} in ``AMD GCN Options''@footnote{Additionally,
4994 @code{gfx803} is supported as an alias for @code{fiji}.}
4997 @tab See @code{-march=} in ``Nvidia PTX Options''
5000 @node Memory allocation
5001 @section Memory allocation
5003 For the available predefined allocators and, as applicable, their associated
5004 predefined memory spaces and for the available traits and their default values,
5005 see @ref{OMP_ALLOCATOR}. Predefined allocators without an associated memory
5006 space use the @code{omp_default_mem_space} memory space.
5008 For the memory spaces, the following applies:
5010 @item @code{omp_default_mem_space} is supported
5011 @item @code{omp_const_mem_space} maps to @code{omp_default_mem_space}
5012 @item @code{omp_low_lat_mem_space} maps to @code{omp_default_mem_space}
5013 @item @code{omp_large_cap_mem_space} maps to @code{omp_default_mem_space},
5014 unless the memkind library is available
5015 @item @code{omp_high_bw_mem_space} maps to @code{omp_default_mem_space},
5016 unless the memkind library is available
5019 On Linux systems, where the @uref{https://github.com/memkind/memkind, memkind
5020 library} (@code{libmemkind.so.0}) is available at runtime, it is used when
5021 creating memory allocators requesting
5024 @item the memory space @code{omp_high_bw_mem_space}
5025 @item the memory space @code{omp_large_cap_mem_space}
5026 @item the @code{partition} trait @code{interleaved}; note that for
5027 @code{omp_large_cap_mem_space} the allocation will not be interleaved
5030 On Linux systems, where the @uref{https://github.com/numactl/numactl, numa
5031 library} (@code{libnuma.so.1}) is available at runtime, it used when creating
5032 memory allocators requesting
5035 @item the @code{partition} trait @code{nearest}, except when both the
5036 libmemkind library is available and the memory space is either
5037 @code{omp_large_cap_mem_space} or @code{omp_high_bw_mem_space}
5040 Note that the numa library will round up the allocation size to a multiple of
5041 the system page size; therefore, consider using it only with large data or
5042 by sharing allocations via the @code{pool_size} trait. Furthermore, the Linux
5043 kernel does not guarantee that an allocation will always be on the nearest NUMA
5044 node nor that after reallocation the same node will be used. Note additionally
5045 that, on Linux, the default setting of the memory placement policy is to use the
5046 current node; therefore, unless the memory placement policy has been overridden,
5047 the @code{partition} trait @code{environment} (the default) will be effectively
5048 a @code{nearest} allocation.
5050 Additional notes regarding the traits:
5052 @item The @code{pinned} trait is unsupported.
5053 @item The default for the @code{pool_size} trait is no pool and for every
5054 (re)allocation the associated library routine is called, which might
5055 internally use a memory pool.
5056 @item For the @code{partition} trait, the partition part size will be the same
5057 as the requested size (i.e. @code{interleaved} or @code{blocked} has no
5058 effect), except for @code{interleaved} when the memkind library is
5059 available. Furthermore, for @code{nearest} and unless the numa library
5060 is available, the memory might not be on the same NUMA node as thread
5061 that allocated the memory; on Linux, this is in particular the case when
5062 the memory placement policy is set to preferred.
5063 @item The @code{access} trait has no effect such that memory is always
5064 accessible by all threads.
5065 @item The @code{sync_hint} trait has no effect.
5068 @c ---------------------------------------------------------------------
5069 @c Offload-Target Specifics
5070 @c ---------------------------------------------------------------------
5072 @node Offload-Target Specifics
5073 @chapter Offload-Target Specifics
5075 The following sections present notes on the offload-target specifics
5083 @section AMD Radeon (GCN)
5085 On the hardware side, there is the hierarchy (fine to coarse):
5087 @item work item (thread)
5090 @item compute unit (CU)
5093 All OpenMP and OpenACC levels are used, i.e.
5095 @item OpenMP's simd and OpenACC's vector map to work items (thread)
5096 @item OpenMP's threads (``parallel'') and OpenACC's workers map
5098 @item OpenMP's teams and OpenACC's gang use a threadpool with the
5099 size of the number of teams or gangs, respectively.
5104 @item Number of teams is the specified @code{num_teams} (OpenMP) or
5105 @code{num_gangs} (OpenACC) or otherwise the number of CU. It is limited
5106 by two times the number of CU.
5107 @item Number of wavefronts is 4 for gfx900 and 16 otherwise;
5108 @code{num_threads} (OpenMP) and @code{num_workers} (OpenACC)
5109 overrides this if smaller.
5110 @item The wavefront has 102 scalars and 64 vectors
5111 @item Number of workitems is always 64
5112 @item The hardware permits maximally 40 workgroups/CU and
5113 16 wavefronts/workgroup up to a limit of 40 wavefronts in total per CU.
5114 @item 80 scalars registers and 24 vector registers in non-kernel functions
5115 (the chosen procedure-calling API).
5116 @item For the kernel itself: as many as register pressure demands (number of
5117 teams and number of threads, scaled down if registers are exhausted)
5120 The implementation remark:
5122 @item I/O within OpenMP target regions and OpenACC parallel/kernels is supported
5123 using the C library @code{printf} functions and the Fortran
5124 @code{print}/@code{write} statements.
5125 @item Reverse offload regions (i.e. @code{target} regions with
5126 @code{device(ancestor:1)}) are processed serially per @code{target} region
5127 such that the next reverse offload region is only executed after the previous
5129 @item OpenMP code that has a @code{requires} directive with
5130 @code{unified_shared_memory} will remove any GCN device from the list of
5131 available devices (``host fallback'').
5132 @item The available stack size can be changed using the @code{GCN_STACK_SIZE}
5133 environment variable; the default is 32 kiB per thread.
5141 On the hardware side, there is the hierarchy (fine to coarse):
5146 @item streaming multiprocessor
5149 All OpenMP and OpenACC levels are used, i.e.
5151 @item OpenMP's simd and OpenACC's vector map to threads
5152 @item OpenMP's threads (``parallel'') and OpenACC's workers map to warps
5153 @item OpenMP's teams and OpenACC's gang use a threadpool with the
5154 size of the number of teams or gangs, respectively.
5159 @item The @code{warp_size} is always 32
5160 @item CUDA kernel launched: @code{dim=@{#teams,1,1@}, blocks=@{#threads,warp_size,1@}}.
5161 @item The number of teams is limited by the number of blocks the device can
5162 host simultaneously.
5165 Additional information can be obtained by setting the environment variable to
5166 @code{GOMP_DEBUG=1} (very verbose; grep for @code{kernel.*launch} for launch
5169 GCC generates generic PTX ISA code, which is just-in-time compiled by CUDA,
5170 which caches the JIT in the user's directory (see CUDA documentation; can be
5171 tuned by the environment variables @code{CUDA_CACHE_@{DISABLE,MAXSIZE,PATH@}}.
5173 Note: While PTX ISA is generic, the @code{-mptx=} and @code{-march=} commandline
5174 options still affect the used PTX ISA code and, thus, the requirements on
5175 CUDA version and hardware.
5177 The implementation remark:
5179 @item I/O within OpenMP target regions and OpenACC parallel/kernels is supported
5180 using the C library @code{printf} functions. Note that the Fortran
5181 @code{print}/@code{write} statements are not supported, yet.
5182 @item Compilation OpenMP code that contains @code{requires reverse_offload}
5183 requires at least @code{-march=sm_35}, compiling for @code{-march=sm_30}
5185 @item For code containing reverse offload (i.e. @code{target} regions with
5186 @code{device(ancestor:1)}), there is a slight performance penalty
5187 for @emph{all} target regions, consisting mostly of shutdown delay
5188 Per device, reverse offload regions are processed serially such that
5189 the next reverse offload region is only executed after the previous
5191 @item OpenMP code that has a @code{requires} directive with
5192 @code{unified_shared_memory} will remove any nvptx device from the
5193 list of available devices (``host fallback'').
5194 @item The default per-warp stack size is 128 kiB; see also @code{-msoft-stack}
5196 @item The OpenMP routines @code{omp_target_memcpy_rect} and
5197 @code{omp_target_memcpy_rect_async} and the @code{target update}
5198 directive for non-contiguous list items will use the 2D and 3D
5199 memory-copy functions of the CUDA library. Higher dimensions will
5200 call those functions in a loop and are therefore supported.
5204 @c ---------------------------------------------------------------------
5206 @c ---------------------------------------------------------------------
5208 @node The libgomp ABI
5209 @chapter The libgomp ABI
5211 The following sections present notes on the external ABI as
5212 presented by libgomp. Only maintainers should need them.
5215 * Implementing MASTER construct::
5216 * Implementing CRITICAL construct::
5217 * Implementing ATOMIC construct::
5218 * Implementing FLUSH construct::
5219 * Implementing BARRIER construct::
5220 * Implementing THREADPRIVATE construct::
5221 * Implementing PRIVATE clause::
5222 * Implementing FIRSTPRIVATE LASTPRIVATE COPYIN and COPYPRIVATE clauses::
5223 * Implementing REDUCTION clause::
5224 * Implementing PARALLEL construct::
5225 * Implementing FOR construct::
5226 * Implementing ORDERED construct::
5227 * Implementing SECTIONS construct::
5228 * Implementing SINGLE construct::
5229 * Implementing OpenACC's PARALLEL construct::
5233 @node Implementing MASTER construct
5234 @section Implementing MASTER construct
5237 if (omp_get_thread_num () == 0)
5241 Alternately, we generate two copies of the parallel subfunction
5242 and only include this in the version run by the primary thread.
5243 Surely this is not worthwhile though...
5247 @node Implementing CRITICAL construct
5248 @section Implementing CRITICAL construct
5250 Without a specified name,
5253 void GOMP_critical_start (void);
5254 void GOMP_critical_end (void);
5257 so that we don't get COPY relocations from libgomp to the main
5260 With a specified name, use omp_set_lock and omp_unset_lock with
5261 name being transformed into a variable declared like
5264 omp_lock_t gomp_critical_user_<name> __attribute__((common))
5267 Ideally the ABI would specify that all zero is a valid unlocked
5268 state, and so we wouldn't need to initialize this at
5273 @node Implementing ATOMIC construct
5274 @section Implementing ATOMIC construct
5276 The target should implement the @code{__sync} builtins.
5278 Failing that we could add
5281 void GOMP_atomic_enter (void)
5282 void GOMP_atomic_exit (void)
5285 which reuses the regular lock code, but with yet another lock
5286 object private to the library.
5290 @node Implementing FLUSH construct
5291 @section Implementing FLUSH construct
5293 Expands to the @code{__sync_synchronize} builtin.
5297 @node Implementing BARRIER construct
5298 @section Implementing BARRIER construct
5301 void GOMP_barrier (void)
5305 @node Implementing THREADPRIVATE construct
5306 @section Implementing THREADPRIVATE construct
5308 In _most_ cases we can map this directly to @code{__thread}. Except
5309 that OMP allows constructors for C++ objects. We can either
5310 refuse to support this (how often is it used?) or we can
5311 implement something akin to .ctors.
5313 Even more ideally, this ctor feature is handled by extensions
5314 to the main pthreads library. Failing that, we can have a set
5315 of entry points to register ctor functions to be called.
5319 @node Implementing PRIVATE clause
5320 @section Implementing PRIVATE clause
5322 In association with a PARALLEL, or within the lexical extent
5323 of a PARALLEL block, the variable becomes a local variable in
5324 the parallel subfunction.
5326 In association with FOR or SECTIONS blocks, create a new
5327 automatic variable within the current function. This preserves
5328 the semantic of new variable creation.
5332 @node Implementing FIRSTPRIVATE LASTPRIVATE COPYIN and COPYPRIVATE clauses
5333 @section Implementing FIRSTPRIVATE LASTPRIVATE COPYIN and COPYPRIVATE clauses
5335 This seems simple enough for PARALLEL blocks. Create a private
5336 struct for communicating between the parent and subfunction.
5337 In the parent, copy in values for scalar and "small" structs;
5338 copy in addresses for others TREE_ADDRESSABLE types. In the
5339 subfunction, copy the value into the local variable.
5341 It is not clear what to do with bare FOR or SECTION blocks.
5342 The only thing I can figure is that we do something like:
5345 #pragma omp for firstprivate(x) lastprivate(y)
5346 for (int i = 0; i < n; ++i)
5363 where the "x=x" and "y=y" assignments actually have different
5364 uids for the two variables, i.e. not something you could write
5365 directly in C. Presumably this only makes sense if the "outer"
5366 x and y are global variables.
5368 COPYPRIVATE would work the same way, except the structure
5369 broadcast would have to happen via SINGLE machinery instead.
5373 @node Implementing REDUCTION clause
5374 @section Implementing REDUCTION clause
5376 The private struct mentioned in the previous section should have
5377 a pointer to an array of the type of the variable, indexed by the
5378 thread's @var{team_id}. The thread stores its final value into the
5379 array, and after the barrier, the primary thread iterates over the
5380 array to collect the values.
5383 @node Implementing PARALLEL construct
5384 @section Implementing PARALLEL construct
5387 #pragma omp parallel
5396 void subfunction (void *data)
5403 GOMP_parallel_start (subfunction, &data, num_threads);
5404 subfunction (&data);
5405 GOMP_parallel_end ();
5409 void GOMP_parallel_start (void (*fn)(void *), void *data, unsigned num_threads)
5412 The @var{FN} argument is the subfunction to be run in parallel.
5414 The @var{DATA} argument is a pointer to a structure used to
5415 communicate data in and out of the subfunction, as discussed
5416 above with respect to FIRSTPRIVATE et al.
5418 The @var{NUM_THREADS} argument is 1 if an IF clause is present
5419 and false, or the value of the NUM_THREADS clause, if
5422 The function needs to create the appropriate number of
5423 threads and/or launch them from the dock. It needs to
5424 create the team structure and assign team ids.
5427 void GOMP_parallel_end (void)
5430 Tears down the team and returns us to the previous @code{omp_in_parallel()} state.
5434 @node Implementing FOR construct
5435 @section Implementing FOR construct
5438 #pragma omp parallel for
5439 for (i = lb; i <= ub; i++)
5446 void subfunction (void *data)
5449 while (GOMP_loop_static_next (&_s0, &_e0))
5452 for (i = _s0; i < _e1; i++)
5455 GOMP_loop_end_nowait ();
5458 GOMP_parallel_loop_static (subfunction, NULL, 0, lb, ub+1, 1, 0);
5460 GOMP_parallel_end ();
5464 #pragma omp for schedule(runtime)
5465 for (i = 0; i < n; i++)
5474 if (GOMP_loop_runtime_start (0, n, 1, &_s0, &_e0))
5477 for (i = _s0, i < _e0; i++)
5479 @} while (GOMP_loop_runtime_next (&_s0, _&e0));
5484 Note that while it looks like there is trickiness to propagating
5485 a non-constant STEP, there isn't really. We're explicitly allowed
5486 to evaluate it as many times as we want, and any variables involved
5487 should automatically be handled as PRIVATE or SHARED like any other
5488 variables. So the expression should remain evaluable in the
5489 subfunction. We can also pull it into a local variable if we like,
5490 but since its supposed to remain unchanged, we can also not if we like.
5492 If we have SCHEDULE(STATIC), and no ORDERED, then we ought to be
5493 able to get away with no work-sharing context at all, since we can
5494 simply perform the arithmetic directly in each thread to divide up
5495 the iterations. Which would mean that we wouldn't need to call any
5498 There are separate routines for handling loops with an ORDERED
5499 clause. Bookkeeping for that is non-trivial...
5503 @node Implementing ORDERED construct
5504 @section Implementing ORDERED construct
5507 void GOMP_ordered_start (void)
5508 void GOMP_ordered_end (void)
5513 @node Implementing SECTIONS construct
5514 @section Implementing SECTIONS construct
5519 #pragma omp sections
5533 for (i = GOMP_sections_start (3); i != 0; i = GOMP_sections_next ())
5550 @node Implementing SINGLE construct
5551 @section Implementing SINGLE construct
5565 if (GOMP_single_start ())
5573 #pragma omp single copyprivate(x)
5580 datap = GOMP_single_copy_start ();
5585 GOMP_single_copy_end (&data);
5594 @node Implementing OpenACC's PARALLEL construct
5595 @section Implementing OpenACC's PARALLEL construct
5598 void GOACC_parallel ()
5603 @c ---------------------------------------------------------------------
5605 @c ---------------------------------------------------------------------
5607 @node Reporting Bugs
5608 @chapter Reporting Bugs
5610 Bugs in the GNU Offloading and Multi Processing Runtime Library should
5611 be reported via @uref{https://gcc.gnu.org/bugzilla/, Bugzilla}. Please add
5612 "openacc", or "openmp", or both to the keywords field in the bug
5613 report, as appropriate.
5617 @c ---------------------------------------------------------------------
5618 @c GNU General Public License
5619 @c ---------------------------------------------------------------------
5621 @include gpl_v3.texi
5625 @c ---------------------------------------------------------------------
5626 @c GNU Free Documentation License
5627 @c ---------------------------------------------------------------------
5633 @c ---------------------------------------------------------------------
5634 @c Funding Free Software
5635 @c ---------------------------------------------------------------------
5637 @include funding.texi
5639 @c ---------------------------------------------------------------------
5641 @c ---------------------------------------------------------------------
5644 @unnumbered Library Index