doc/ext_ffi_semantics.html

   1 <!DOCTYPE html>
   2 <html>
   3 <head>
   4 <title>FFI Semantics</title>
   5 <meta charset="utf-8">
   6 <meta name="Copyright" content="Copyright (C) 2005-2023">
   7 <meta name="Language" content="en">
   8 <link rel="stylesheet" type="text/css" href="bluequad.css" media="screen">
   9 <link rel="stylesheet" type="text/css" href="bluequad-print.css" media="print">
  10 <style type="text/css">
  11 table.convtable { line-height: 1.2; }
  12 tr.convhead td { font-weight: bold; }
  13 td.convop { font-style: italic; width: 40%; }
  14 </style>
  15 </head>
  16 <body>
  17 <div id="site">
  18 <a href="https://luajit.org"><span>Lua<span id="logo">JIT</span></span></a>
  19 </div>
  20 <div id="head">
  21 <h1>FFI Semantics</h1>
  22 </div>
  23 <div id="nav">
  24 <ul><li>
  25 <a href="luajit.html">LuaJIT</a>
  26 <ul><li>
  27 <a href="https://luajit.org/download.html">Download <span class="ext">&raquo;</span></a>
  28 </li><li>
  29 <a href="install.html">Installation</a>
  30 </li><li>
  31 <a href="running.html">Running</a>
  32 </li></ul>
  33 </li><li>
  34 <a href="extensions.html">Extensions</a>
  35 <ul><li>
  36 <a href="ext_ffi.html">FFI Library</a>
  37 <ul><li>
  38 <a href="ext_ffi_tutorial.html">FFI Tutorial</a>
  39 </li><li>
  40 <a href="ext_ffi_api.html">ffi.* API</a>
  41 </li><li>
  42 <a class="current" href="ext_ffi_semantics.html">FFI Semantics</a>
  43 </li></ul>
  44 </li><li>
  45 <a href="ext_buffer.html">String Buffers</a>
  46 </li><li>
  47 <a href="ext_jit.html">jit.* Library</a>
  48 </li><li>
  49 <a href="ext_c_api.html">Lua/C API</a>
  50 </li><li>
  51 <a href="ext_profiler.html">Profiler</a>
  52 </li></ul>
  53 </li><li>
  54 <a href="https://luajit.org/status.html">Status <span class="ext">&raquo;</span></a>
  55 </li><li>
  56 <a href="https://luajit.org/faq.html">FAQ <span class="ext">&raquo;</span></a>
  57 </li><li>
  58 <a href="https://luajit.org/list.html">Mailing List <span class="ext">&raquo;</span></a>
  59 </li></ul>
  60 </div>
  61 <div id="main">
  62 <p>
  63 This page describes the detailed semantics underlying the FFI library
  64 and its interaction with both Lua and C&nbsp;code.
  65 </p>
  66 <p>
  67 Given that the FFI library is designed to interface with C&nbsp;code
  68 and that declarations can be written in plain C&nbsp;syntax, <b>it
  69 closely follows the C&nbsp;language semantics</b>, wherever possible.
  70 Some minor concessions are needed for smoother interoperation with Lua
  71 language semantics.
  72 </p>
  73 <p>
  74 Please don't be overwhelmed by the contents of this page &mdash; this
  75 is a reference and you may need to consult it, if in doubt. It doesn't
  76 hurt to skim this page, but most of the semantics "just work" as you'd
  77 expect them to work. It should be straightforward to write
  78 applications using the LuaJIT FFI for developers with a C or C++
  79 background.
  80 </p>
  81
  82 <h2 id="clang">C Language Support</h2>
  83 <p>
  84 The FFI library has a built-in C&nbsp;parser with a minimal memory
  85 footprint. It's used by the <a href="ext_ffi_api.html">ffi.* library
  86 functions</a> to declare C&nbsp;types or external symbols.
  87 </p>
  88 <p>
  89 Its only purpose is to parse C&nbsp;declarations, as found e.g. in
  90 C&nbsp;header files. Although it does evaluate constant expressions,
  91 it's <em>not</em> a C&nbsp;compiler. The body of <tt>inline</tt>
  92 C&nbsp;function definitions is simply ignored.
  93 </p>
  94 <p>
  95 Also, this is <em>not</em> a validating C&nbsp;parser. It expects and
  96 accepts correctly formed C&nbsp;declarations, but it may choose to
  97 ignore bad declarations or show rather generic error messages. If in
  98 doubt, please check the input against your favorite C&nbsp;compiler.
  99 </p>
 100 <p>
 101 The C&nbsp;parser complies to the <b>C99 language standard</b> plus
 102 the following extensions:
 103 </p>
 104 <ul>
 105
 106 <li>The <tt>'\e'</tt> escape in character and string literals.</li>
 107
 108 <li>The C99/C++ boolean type, declared with the keywords <tt>bool</tt>
 109 or <tt>_Bool</tt>.</li>
 110
 111 <li>Complex numbers, declared with the keywords <tt>complex</tt> or
 112 <tt>_Complex</tt>.</li>
 113
 114 <li>Two complex number types: <tt>complex</tt> (aka
 115 <tt>complex&nbsp;double</tt>) and <tt>complex&nbsp;float</tt>.</li>
 116
 117 <li>Vector types, declared with the GCC <tt>mode</tt> or
 118 <tt>vector_size</tt> attribute.</li>
 119
 120 <li>Unnamed ('transparent') <tt>struct</tt>/<tt>union</tt> fields
 121 inside a <tt>struct</tt>/<tt>union</tt>.</li>
 122
 123 <li>Incomplete <tt>enum</tt> declarations, handled like incomplete
 124 <tt>struct</tt> declarations.</li>
 125
 126 <li>Unnamed <tt>enum</tt> fields inside a
 127 <tt>struct</tt>/<tt>union</tt>. This is similar to a scoped C++
 128 <tt>enum</tt>, except that declared constants are visible in the
 129 global namespace, too.</li>
 130
 131 <li>Scoped <tt>static&nbsp;const</tt> declarations inside a
 132 <tt>struct</tt>/<tt>union</tt> (from C++).</li>
 133
 134 <li>Zero-length arrays (<tt>[0]</tt>), empty
 135 <tt>struct</tt>/<tt>union</tt>, variable-length arrays (VLA,
 136 <tt>[?]</tt>) and variable-length structs (VLS, with a trailing
 137 VLA).</li>
 138
 139 <li>C++ reference types (<tt>int&nbsp;&amp;x</tt>).</li>
 140
 141 <li>Alternate GCC keywords with '<tt>__</tt>', e.g.
 142 <tt>__const__</tt>.</li>
 143
 144 <li>GCC <tt>__attribute__</tt> with the following attributes:
 145 <tt>aligned</tt>, <tt>packed</tt>, <tt>mode</tt>,
 146 <tt>vector_size</tt>, <tt>cdecl</tt>, <tt>fastcall</tt>,
 147 <tt>stdcall</tt>, <tt>thiscall</tt>.</li>
 148
 149 <li>The GCC <tt>__extension__</tt> keyword and the GCC
 150 <tt>__alignof__</tt> operator.</li>
 151
 152 <li>GCC <tt>__asm__("symname")</tt> symbol name redirection for
 153 function declarations.</li>
 154
 155 <li>MSVC keywords for fixed-length types: <tt>__int8</tt>,
 156 <tt>__int16</tt>, <tt>__int32</tt> and <tt>__int64</tt>.</li>
 157
 158 <li>MSVC <tt>__cdecl</tt>, <tt>__fastcall</tt>, <tt>__stdcall</tt>,
 159 <tt>__thiscall</tt>, <tt>__ptr32</tt>, <tt>__ptr64</tt>,
 160 <tt>__declspec(align(n))</tt> and <tt>#pragma&nbsp;pack</tt>.</li>
 161
 162 <li>All other GCC/MSVC-specific attributes are ignored.</li>
 163
 164 </ul>
 165 <p>
 166 The following C&nbsp;types are predefined by the C&nbsp;parser (like
 167 a <tt>typedef</tt>, except re-declarations will be ignored):
 168 </p>
 169 <ul>
 170
 171 <li>Vararg handling: <tt>va_list</tt>, <tt>__builtin_va_list</tt>,
 172 <tt>__gnuc_va_list</tt>.</li>
 173
 174 <li>From <tt>&lt;stddef.h&gt;</tt>: <tt>ptrdiff_t</tt>,
 175 <tt>size_t</tt>, <tt>wchar_t</tt>.</li>
 176
 177 <li>From <tt>&lt;stdint.h&gt;</tt>: <tt>int8_t</tt>, <tt>int16_t</tt>,
 178 <tt>int32_t</tt>, <tt>int64_t</tt>, <tt>uint8_t</tt>,
 179 <tt>uint16_t</tt>, <tt>uint32_t</tt>, <tt>uint64_t</tt>,
 180 <tt>intptr_t</tt>, <tt>uintptr_t</tt>.</li>
 181
 182 <li>From <tt>&lt;unistd.h&gt;</tt> (POSIX): <tt>ssize_t</tt>.</li>
 183
 184 </ul>
 185 <p>
 186 You're encouraged to use these types in preference to
 187 compiler-specific extensions or target-dependent standard types.
 188 E.g. <tt>char</tt> differs in signedness and <tt>long</tt> differs in
 189 size, depending on the target architecture and platform ABI.
 190 </p>
 191 <p>
 192 The following C&nbsp;features are <b>not</b> supported:
 193 </p>
 194 <ul>
 195
 196 <li>A declaration must always have a type specifier; it doesn't
 197 default to an <tt>int</tt> type.</li>
 198
 199 <li>Old-style empty function declarations (K&amp;R) are not allowed.
 200 All C&nbsp;functions must have a proper prototype declaration. A
 201 function declared without parameters (<tt>int&nbsp;foo();</tt>) is
 202 treated as a function taking zero arguments, like in C++.</li>
 203
 204 <li>The <tt>long double</tt> C&nbsp;type is parsed correctly, but
 205 there's no support for the related conversions, accesses or arithmetic
 206 operations.</li>
 207
 208 <li>Wide character strings and character literals are not
 209 supported.</li>
 210
 211 <li><a href="#status">See below</a> for features that are currently
 212 not implemented.</li>
 213
 214 </ul>
 215
 216 <h2 id="convert">C Type Conversion Rules</h2>
 217
 218 <h3 id="convert_tolua">Conversions from C&nbsp;types to Lua objects</h3>
 219 <p>
 220 These conversion rules apply for <em>read accesses</em> to
 221 C&nbsp;types: indexing pointers, arrays or
 222 <tt>struct</tt>/<tt>union</tt> types; reading external variables or
 223 constant values; retrieving return values from C&nbsp;calls:
 224 </p>
 225 <table class="convtable">
 226 <tr class="convhead">
 227 <td class="convin">Input</td>
 228 <td class="convop">Conversion</td>
 229 <td class="convout">Output</td>
 230 </tr>
 231 <tr class="odd separate">
 232 <td class="convin"><tt>int8_t</tt>, <tt>int16_t</tt></td><td class="convop">&rarr;<sup>sign-ext</sup> <tt>int32_t</tt> &rarr; <tt>double</tt></td><td class="convout">number</td></tr>
 233 <tr class="even">
 234 <td class="convin"><tt>uint8_t</tt>, <tt>uint16_t</tt></td><td class="convop">&rarr;<sup>zero-ext</sup> <tt>int32_t</tt> &rarr; <tt>double</tt></td><td class="convout">number</td></tr>
 235 <tr class="odd">
 236 <td class="convin"><tt>int32_t</tt>, <tt>uint32_t</tt></td><td class="convop">&rarr; <tt>double</tt></td><td class="convout">number</td></tr>
 237 <tr class="even">
 238 <td class="convin"><tt>int64_t</tt>, <tt>uint64_t</tt></td><td class="convop">boxed value</td><td class="convout">64 bit int cdata</td></tr>
 239 <tr class="odd separate">
 240 <td class="convin"><tt>double</tt>, <tt>float</tt></td><td class="convop">&rarr; <tt>double</tt></td><td class="convout">number</td></tr>
 241 <tr class="even separate">
 242 <td class="convin"><tt>bool</tt></td><td class="convop">0 &rarr; <tt>false</tt>, otherwise <tt>true</tt></td><td class="convout">boolean</td></tr>
 243 <tr class="odd separate">
 244 <td class="convin"><tt>enum</tt></td><td class="convop">boxed value</td><td class="convout">enum cdata</td></tr>
 245 <tr class="even">
 246 <td class="convin">Complex number</td><td class="convop">boxed value</td><td class="convout">complex cdata</td></tr>
 247 <tr class="odd">
 248 <td class="convin">Vector</td><td class="convop">boxed value</td><td class="convout">vector cdata</td></tr>
 249 <tr class="even">
 250 <td class="convin">Pointer</td><td class="convop">boxed value</td><td class="convout">pointer cdata</td></tr>
 251 <tr class="odd separate">
 252 <td class="convin">Array</td><td class="convop">boxed reference</td><td class="convout">reference cdata</td></tr>
 253 <tr class="even">
 254 <td class="convin"><tt>struct</tt>/<tt>union</tt></td><td class="convop">boxed reference</td><td class="convout">reference cdata</td></tr>
 255 </table>
 256 <p>
 257 Bitfields are treated like their underlying type.
 258 </p>
 259 <p>
 260 Reference types are dereferenced <em>before</em> a conversion can take
 261 place &mdash; the conversion is applied to the C&nbsp;type pointed to
 262 by the reference.
 263 </p>
 264
 265 <h3 id="convert_fromlua">Conversions from Lua objects to C&nbsp;types</h3>
 266 <p>
 267 These conversion rules apply for <em>write accesses</em> to
 268 C&nbsp;types: indexing pointers, arrays or
 269 <tt>struct</tt>/<tt>union</tt> types; initializing cdata objects;
 270 casts to C&nbsp;types; writing to external variables; passing
 271 arguments to C&nbsp;calls:
 272 </p>
 273 <table class="convtable">
 274 <tr class="convhead">
 275 <td class="convin">Input</td>
 276 <td class="convop">Conversion</td>
 277 <td class="convout">Output</td>
 278 </tr>
 279 <tr class="odd separate">
 280 <td class="convin">number</td><td class="convop">&rarr;</td><td class="convout"><tt>double</tt></td></tr>
 281 <tr class="even">
 282 <td class="convin">boolean</td><td class="convop"><tt>false</tt> &rarr; 0, <tt>true</tt> &rarr; 1</td><td class="convout"><tt>bool</tt></td></tr>
 283 <tr class="odd separate">
 284 <td class="convin">nil</td><td class="convop"><tt>NULL</tt> &rarr;</td><td class="convout"><tt>(void *)</tt></td></tr>
 285 <tr class="even">
 286 <td class="convin">lightuserdata</td><td class="convop">lightuserdata address &rarr;</td><td class="convout"><tt>(void *)</tt></td></tr>
 287 <tr class="odd">
 288 <td class="convin">userdata</td><td class="convop">userdata payload &rarr;</td><td class="convout"><tt>(void *)</tt></td></tr>
 289 <tr class="even">
 290 <td class="convin">io.* file</td><td class="convop">get FILE * handle &rarr;</td><td class="convout"><tt>(void *)</tt></td></tr>
 291 <tr class="odd separate">
 292 <td class="convin">string</td><td class="convop">match against <tt>enum</tt> constant</td><td class="convout"><tt>enum</tt></td></tr>
 293 <tr class="even">
 294 <td class="convin">string</td><td class="convop">copy string data + zero-byte</td><td class="convout"><tt>int8_t[]</tt>, <tt>uint8_t[]</tt></td></tr>
 295 <tr class="odd">
 296 <td class="convin">string</td><td class="convop">string data &rarr;</td><td class="convout"><tt>const char[]</tt></td></tr>
 297 <tr class="even separate">
 298 <td class="convin">function</td><td class="convop"><a href="#callback">create callback</a> &rarr;</td><td class="convout">C function type</td></tr>
 299 <tr class="odd separate">
 300 <td class="convin">table</td><td class="convop"><a href="#init_table">table initializer</a></td><td class="convout">Array</td></tr>
 301 <tr class="even">
 302 <td class="convin">table</td><td class="convop"><a href="#init_table">table initializer</a></td><td class="convout"><tt>struct</tt>/<tt>union</tt></td></tr>
 303 <tr class="odd separate">
 304 <td class="convin">cdata</td><td class="convop">cdata payload &rarr;</td><td class="convout">C type</td></tr>
 305 </table>
 306 <p>
 307 If the result type of this conversion doesn't match the
 308 C&nbsp;type of the destination, the
 309 <a href="#convert_between">conversion rules between C&nbsp;types</a>
 310 are applied.
 311 </p>
 312 <p>
 313 Reference types are immutable after initialization ("no re-seating of
 314 references"). For initialization purposes or when passing values to
 315 reference parameters, they are treated like pointers. Note that unlike
 316 in C++, there's no way to implement automatic reference generation of
 317 variables under the Lua language semantics. If you want to call a
 318 function with a reference parameter, you need to explicitly pass a
 319 one-element array.
 320 </p>
 321
 322 <h3 id="convert_between">Conversions between C&nbsp;types</h3>
 323 <p>
 324 These conversion rules are more or less the same as the standard
 325 C&nbsp;conversion rules. Some rules only apply to casts, or require
 326 pointer or type compatibility:
 327 </p>
 328 <table class="convtable">
 329 <tr class="convhead">
 330 <td class="convin">Input</td>
 331 <td class="convop">Conversion</td>
 332 <td class="convout">Output</td>
 333 </tr>
 334 <tr class="odd separate">
 335 <td class="convin">Signed integer</td><td class="convop">&rarr;<sup>narrow or sign-extend</sup></td><td class="convout">Integer</td></tr>
 336 <tr class="even">
 337 <td class="convin">Unsigned integer</td><td class="convop">&rarr;<sup>narrow or zero-extend</sup></td><td class="convout">Integer</td></tr>
 338 <tr class="odd">
 339 <td class="convin">Integer</td><td class="convop">&rarr;<sup>round</sup></td><td class="convout"><tt>double</tt>, <tt>float</tt></td></tr>
 340 <tr class="even">
 341 <td class="convin"><tt>double</tt>, <tt>float</tt></td><td class="convop">&rarr;<sup>trunc</sup> <tt>int32_t</tt> &rarr;<sup>narrow</sup></td><td class="convout"><tt>(u)int8_t</tt>, <tt>(u)int16_t</tt></td></tr>
 342 <tr class="odd">
 343 <td class="convin"><tt>double</tt>, <tt>float</tt></td><td class="convop">&rarr;<sup>trunc</sup></td><td class="convout"><tt>(u)int32_t</tt>, <tt>(u)int64_t</tt></td></tr>
 344 <tr class="even">
 345 <td class="convin"><tt>double</tt>, <tt>float</tt></td><td class="convop">&rarr;<sup>round</sup></td><td class="convout"><tt>float</tt>, <tt>double</tt></td></tr>
 346 <tr class="odd separate">
 347 <td class="convin">Number</td><td class="convop">n == 0 &rarr; 0, otherwise 1</td><td class="convout"><tt>bool</tt></td></tr>
 348 <tr class="even">
 349 <td class="convin"><tt>bool</tt></td><td class="convop"><tt>false</tt> &rarr; 0, <tt>true</tt> &rarr; 1</td><td class="convout">Number</td></tr>
 350 <tr class="odd separate">
 351 <td class="convin">Complex number</td><td class="convop">convert real part</td><td class="convout">Number</td></tr>
 352 <tr class="even">
 353 <td class="convin">Number</td><td class="convop">convert real part, imag = 0</td><td class="convout">Complex number</td></tr>
 354 <tr class="odd">
 355 <td class="convin">Complex number</td><td class="convop">convert real and imag part</td><td class="convout">Complex number</td></tr>
 356 <tr class="even separate">
 357 <td class="convin">Number</td><td class="convop">convert scalar and replicate</td><td class="convout">Vector</td></tr>
 358 <tr class="odd">
 359 <td class="convin">Vector</td><td class="convop">copy (same size)</td><td class="convout">Vector</td></tr>
 360 <tr class="even separate">
 361 <td class="convin"><tt>struct</tt>/<tt>union</tt></td><td class="convop">take base address (compat)</td><td class="convout">Pointer</td></tr>
 362 <tr class="odd">
 363 <td class="convin">Array</td><td class="convop">take base address (compat)</td><td class="convout">Pointer</td></tr>
 364 <tr class="even">
 365 <td class="convin">Function</td><td class="convop">take function address</td><td class="convout">Function pointer</td></tr>
 366 <tr class="odd separate">
 367 <td class="convin">Number</td><td class="convop">convert via <tt>uintptr_t</tt> (cast)</td><td class="convout">Pointer</td></tr>
 368 <tr class="even">
 369 <td class="convin">Pointer</td><td class="convop">convert address (compat/cast)</td><td class="convout">Pointer</td></tr>
 370 <tr class="odd">
 371 <td class="convin">Pointer</td><td class="convop">convert address (cast)</td><td class="convout">Integer</td></tr>
 372 <tr class="even">
 373 <td class="convin">Array</td><td class="convop">convert base address (cast)</td><td class="convout">Integer</td></tr>
 374 <tr class="odd separate">
 375 <td class="convin">Array</td><td class="convop">copy (compat)</td><td class="convout">Array</td></tr>
 376 <tr class="even">
 377 <td class="convin"><tt>struct</tt>/<tt>union</tt></td><td class="convop">copy (identical type)</td><td class="convout"><tt>struct</tt>/<tt>union</tt></td></tr>
 378 </table>
 379 <p>
 380 Bitfields or <tt>enum</tt> types are treated like their underlying
 381 type.
 382 </p>
 383 <p>
 384 Conversions not listed above will raise an error. E.g. it's not
 385 possible to convert a pointer to a complex number or vice versa.
 386 </p>
 387
 388 <h3 id="convert_vararg">Conversions for vararg C&nbsp;function arguments</h3>
 389 <p>
 390 The following default conversion rules apply when passing Lua objects
 391 to the variable argument part of vararg C&nbsp;functions:
 392 </p>
 393 <table class="convtable">
 394 <tr class="convhead">
 395 <td class="convin">Input</td>
 396 <td class="convop">Conversion</td>
 397 <td class="convout">Output</td>
 398 </tr>
 399 <tr class="odd separate">
 400 <td class="convin">number</td><td class="convop">&rarr;</td><td class="convout"><tt>double</tt></td></tr>
 401 <tr class="even">
 402 <td class="convin">boolean</td><td class="convop"><tt>false</tt> &rarr; 0, <tt>true</tt> &rarr; 1</td><td class="convout"><tt>bool</tt></td></tr>
 403 <tr class="odd separate">
 404 <td class="convin">nil</td><td class="convop"><tt>NULL</tt> &rarr;</td><td class="convout"><tt>(void *)</tt></td></tr>
 405 <tr class="even">
 406 <td class="convin">userdata</td><td class="convop">userdata payload &rarr;</td><td class="convout"><tt>(void *)</tt></td></tr>
 407 <tr class="odd">
 408 <td class="convin">lightuserdata</td><td class="convop">lightuserdata address &rarr;</td><td class="convout"><tt>(void *)</tt></td></tr>
 409 <tr class="even separate">
 410 <td class="convin">string</td><td class="convop">string data &rarr;</td><td class="convout"><tt>const char *</tt></td></tr>
 411 <tr class="odd separate">
 412 <td class="convin"><tt>float</tt> cdata</td><td class="convop">&rarr;</td><td class="convout"><tt>double</tt></td></tr>
 413 <tr class="even">
 414 <td class="convin">Array cdata</td><td class="convop">take base address</td><td class="convout">Element pointer</td></tr>
 415 <tr class="odd">
 416 <td class="convin"><tt>struct</tt>/<tt>union</tt> cdata</td><td class="convop">take base address</td><td class="convout"><tt>struct</tt>/<tt>union</tt> pointer</td></tr>
 417 <tr class="even">
 418 <td class="convin">Function cdata</td><td class="convop">take function address</td><td class="convout">Function pointer</td></tr>
 419 <tr class="odd">
 420 <td class="convin">Any other cdata</td><td class="convop">no conversion</td><td class="convout">C type</td></tr>
 421 </table>
 422 <p>
 423 To pass a Lua object, other than a cdata object, as a specific type,
 424 you need to override the conversion rules: create a temporary cdata
 425 object with a constructor or a cast and initialize it with the value
 426 to pass:
 427 </p>
 428 <p>
 429 Assuming <tt>x</tt> is a Lua number, here's how to pass it as an
 430 integer to a vararg function:
 431 </p>
 432 <pre class="code">
 433 ffi.cdef[[
 434 int printf(const char *fmt, ...);
 435 ]]
 436 ffi.C.printf("integer value: %d\n", ffi.new("int", x))
 437 </pre>
 438 <p>
 439 If you don't do this, the default Lua number &rarr; <tt>double</tt>
 440 conversion rule applies. A vararg C&nbsp;function expecting an integer
 441 will see a garbled or uninitialized value.
 442 </p>
 443
 444 <h2 id="init">Initializers</h2>
 445 <p>
 446 Creating a cdata object with
 447 <a href="ext_ffi_api.html#ffi_new"><tt>ffi.new()</tt></a> or the
 448 equivalent constructor syntax always initializes its contents, too.
 449 Different rules apply, depending on the number of optional
 450 initializers and the C&nbsp;types involved:
 451 </p>
 452 <ul>
 453 <li>If no initializers are given, the object is filled with zero bytes.</li>
 454
 455 <li>Scalar types (numbers and pointers) accept a single initializer.
 456 The Lua object is <a href="#convert_fromlua">converted to the scalar
 457 C&nbsp;type</a>.</li>
 458
 459 <li>Valarrays (complex numbers and vectors) are treated like scalars
 460 when a single initializer is given. Otherwise they are treated like
 461 regular arrays.</li>
 462
 463 <li>Aggregate types (arrays and structs) accept either a single cdata
 464 initializer of the same type (copy constructor), a single
 465 <a href="#init_table">table initializer</a>, or a flat list of
 466 initializers.</li>
 467
 468 <li>The elements of an array are initialized, starting at index zero.
 469 If a single initializer is given for an array, it's repeated for all
 470 remaining elements. This doesn't happen if two or more initializers
 471 are given: all remaining uninitialized elements are filled with zero
 472 bytes.</li>
 473
 474 <li>Byte arrays may also be initialized with a Lua string. This copies
 475 the whole string plus a terminating zero-byte. The copy stops early only
 476 if the array has a known, fixed size.</li>
 477
 478 <li>The fields of a <tt>struct</tt> are initialized in the order of
 479 their declaration. Uninitialized fields are filled with zero
 480 bytes.</li>
 481
 482 <li>Only the first field of a <tt>union</tt> can be initialized with a
 483 flat initializer.</li>
 484
 485 <li>Elements or fields which are aggregates themselves are initialized
 486 with a <em>single</em> initializer, but this may be a table
 487 initializer or a compatible aggregate.</li>
 488
 489 <li>Excess initializers cause an error.</li>
 490
 491 </ul>
 492
 493 <h2 id="init_table">Table Initializers</h2>
 494 <p>
 495 The following rules apply if a Lua table is used to initialize an
 496 Array or a <tt>struct</tt>/<tt>union</tt>:
 497 </p>
 498 <ul>
 499
 500 <li>If the table index <tt>[0]</tt> is non-<tt>nil</tt>, then the
 501 table is assumed to be zero-based. Otherwise it's assumed to be
 502 one-based.</li>
 503
 504 <li>Array elements, starting at index zero, are initialized one-by-one
 505 with the consecutive table elements, starting at either index
 506 <tt>[0]</tt> or <tt>[1]</tt>. This process stops at the first
 507 <tt>nil</tt> table element.</li>
 508
 509 <li>If exactly one array element was initialized, it's repeated for
 510 all the remaining elements. Otherwise all remaining uninitialized
 511 elements are filled with zero bytes.</li>
 512
 513 <li>The above logic only applies to arrays with a known fixed size.
 514 A VLA is only initialized with the element(s) given in the table.
 515 Depending on the use case, you may need to explicitly add a
 516 <tt>NULL</tt> or <tt>0</tt> terminator to a VLA.</li>
 517
 518 <li>A <tt>struct</tt>/<tt>union</tt> can be initialized in the
 519 order of the declaration of its fields. Each field is initialized with
 520 consecutive table elements, starting at either index <tt>[0]</tt>
 521 or <tt>[1]</tt>. This process stops at the first <tt>nil</tt> table
 522 element.</li>
 523
 524 <li>Otherwise, if neither index <tt>[0]</tt> nor <tt>[1]</tt> is present,
 525 a <tt>struct</tt>/<tt>union</tt> is initialized by looking up each field
 526 name (as a string key) in the table. Each non-<tt>nil</tt> value is
 527 used to initialize the corresponding field.</li>
 528
 529 <li>Uninitialized fields of a <tt>struct</tt> are filled with zero
 530 bytes, except for the trailing VLA of a VLS.</li>
 531
 532 <li>Initialization of a <tt>union</tt> stops after one field has been
 533 initialized. If no field has been initialized, the <tt>union</tt> is
 534 filled with zero bytes.</li>
 535
 536 <li>Elements or fields which are aggregates themselves are initialized
 537 with a <em>single</em> initializer, but this may be a nested table
 538 initializer (or a compatible aggregate).</li>
 539
 540 <li>Excess initializers for an array cause an error. Excess
 541 initializers for a <tt>struct</tt>/<tt>union</tt> are ignored.
 542 Unrelated table entries are ignored, too.</li>
 543
 544 </ul>
 545 <p>
 546 Example:
 547 </p>
 548 <pre class="code">
 549 local ffi = require("ffi")
 550
 551 ffi.cdef[[
 552 struct foo { int a, b; };
 553 union bar { int i; double d; };
 554 struct nested { int x; struct foo y; };
 555 ]]
 556
 557 ffi.new("int[3]", {})            --> 0, 0, 0
 558 ffi.new("int[3]", {1})           --> 1, 1, 1
 559 ffi.new("int[3]", {1,2})         --> 1, 2, 0
 560 ffi.new("int[3]", {1,2,3})       --> 1, 2, 3
 561 ffi.new("int[3]", {[0]=1})       --> 1, 1, 1
 562 ffi.new("int[3]", {[0]=1,2})     --> 1, 2, 0
 563 ffi.new("int[3]", {[0]=1,2,3})   --> 1, 2, 3
 564 ffi.new("int[3]", {[0]=1,2,3,4}) --> error: too many initializers
 565
 566 ffi.new("struct foo", {})            --> a = 0, b = 0
 567 ffi.new("struct foo", {1})           --> a = 1, b = 0
 568 ffi.new("struct foo", {1,2})         --> a = 1, b = 2
 569 ffi.new("struct foo", {[0]=1,2})     --> a = 1, b = 2
 570 ffi.new("struct foo", {b=2})         --> a = 0, b = 2
 571 ffi.new("struct foo", {a=1,b=2,c=3}) --> a = 1, b = 2  'c' is ignored
 572
 573 ffi.new("union bar", {})        --> i = 0, d = 0.0
 574 ffi.new("union bar", {1})       --> i = 1, d = ?
 575 ffi.new("union bar", {[0]=1,2}) --> i = 1, d = ?    '2' is ignored
 576 ffi.new("union bar", {d=2})     --> i = ?, d = 2.0
 577
 578 ffi.new("struct nested", {1,{2,3}})     --> x = 1, y.a = 2, y.b = 3
 579 ffi.new("struct nested", {x=1,y={2,3}}) --> x = 1, y.a = 2, y.b = 3
 580 </pre>
 581
 582 <h2 id="cdata_ops">Operations on cdata Objects</h2>
 583 <p>
 584 All standard Lua operators can be applied to cdata objects or a
 585 mix of a cdata object and another Lua object. The following list shows
 586 the predefined operations.
 587 </p>
 588 <p>
 589 Reference types are dereferenced <em>before</em> performing each of
 590 the operations below &mdash; the operation is applied to the
 591 C&nbsp;type pointed to by the reference.
 592 </p>
 593 <p>
 594 The predefined operations are always tried first before deferring to a
 595 metamethod or index table (if any) for the corresponding ctype (except
 596 for <tt>__new</tt>). An error is raised if the metamethod lookup or
 597 index table lookup fails.
 598 </p>
 599
 600 <h3 id="cdata_array">Indexing a cdata object</h3>
 601 <ul>
 602
 603 <li><b>Indexing a pointer/array</b>: a cdata pointer/array can be
 604 indexed by a cdata number or a Lua number. The element address is
 605 computed as the base address plus the number value multiplied by the
 606 element size in bytes. A read access loads the element value and
 607 <a href="#convert_tolua">converts it to a Lua object</a>. A write
 608 access <a href="#convert_fromlua">converts a Lua object to the element
 609 type</a> and stores the converted value to the element. An error is
 610 raised if the element size is undefined or a write access to a
 611 constant element is attempted.</li>
 612
 613 <li><b>Dereferencing a <tt>struct</tt>/<tt>union</tt> field</b>: a
 614 cdata <tt>struct</tt>/<tt>union</tt> or a pointer to a
 615 <tt>struct</tt>/<tt>union</tt> can be dereferenced by a string key,
 616 giving the field name. The field address is computed as the base
 617 address plus the relative offset of the field. A read access loads the
 618 field value and <a href="#convert_tolua">converts it to a Lua
 619 object</a>. A write access <a href="#convert_fromlua">converts a Lua
 620 object to the field type</a> and stores the converted value to the
 621 field. An error is raised if a write access to a constant
 622 <tt>struct</tt>/<tt>union</tt> or a constant field is attempted.
 623 Scoped enum constants or static constants are treated like a constant
 624 field.</li>
 625
 626 <li><b>Indexing a complex number</b>: a complex number can be indexed
 627 either by a cdata number or a Lua number with the values 0 or 1, or by
 628 the strings <tt>"re"</tt> or <tt>"im"</tt>. A read access loads the
 629 real part (<tt>[0]</tt>, <tt>.re</tt>) or the imaginary part
 630 (<tt>[1]</tt>, <tt>.im</tt>) part of a complex number and
 631 <a href="#convert_tolua">converts it to a Lua number</a>. The
 632 sub-parts of a complex number are immutable &mdash; assigning to an
 633 index of a complex number raises an error. Accessing out-of-bound
 634 indexes returns unspecified results, but is guaranteed not to trigger
 635 memory access violations.</li>
 636
 637 <li><b>Indexing a vector</b>: a vector is treated like an array for
 638 indexing purposes, except the vector elements are immutable &mdash;
 639 assigning to an index of a vector raises an error.</li>
 640
 641 </ul>
 642 <p>
 643 A ctype object can be indexed with a string key, too. The only
 644 predefined operation is reading scoped constants of
 645 <tt>struct</tt>/<tt>union</tt> types. All other accesses defer
 646 to the corresponding metamethods or index tables (if any).
 647 </p>
 648 <p>
 649 Note: since there's (deliberately) no address-of operator, a cdata
 650 object holding a value type is effectively immutable after
 651 initialization. The JIT compiler benefits from this fact when applying
 652 certain optimizations.
 653 </p>
 654 <p>
 655 As a consequence, the <em>elements</em> of complex numbers and
 656 vectors are immutable. But the elements of an aggregate holding these
 657 types <em>may</em> be modified, of course. I.e. you cannot assign to
 658 <tt>foo.c.im</tt>, but you can assign a (newly created) complex number
 659 to <tt>foo.c</tt>.
 660 </p>
 661 <p>
 662 The JIT compiler implements strict aliasing rules: accesses to different
 663 types do <b>not</b> alias, except for differences in signedness (this
 664 applies even to <tt>char</tt> pointers, unlike C99). Type punning
 665 through unions is explicitly detected and allowed.
 666 </p>
 667
 668 <h3 id="cdata_call">Calling a cdata object</h3>
 669 <ul>
 670
 671 <li><b>Constructor</b>: a ctype object can be called and used as a
 672 <a href="ext_ffi_api.html#ffi_new">constructor</a>. This is equivalent
 673 to <tt>ffi.new(ct, ...)</tt>, unless a <tt>__new</tt> metamethod is
 674 defined. The <tt>__new</tt> metamethod is called with the ctype object
 675 plus any other arguments passed to the constructor. Note that you have to
 676 use <tt>ffi.new</tt> inside the metamethod, since calling <tt>ct(...)</tt>
 677 would cause infinite recursion.</li>
 678
 679 <li><b>C&nbsp;function call</b>: a cdata function or cdata function
 680 pointer can be called. The passed arguments are
 681 <a href="#convert_fromlua">converted to the C&nbsp;types</a> of the
 682 parameters given by the function declaration. Arguments passed to the
 683 variable argument part of vararg C&nbsp;function use
 684 <a href="#convert_vararg">special conversion rules</a>. This
 685 C&nbsp;function is called and the return value (if any) is
 686 <a href="#convert_tolua">converted to a Lua object</a>.<br>
 687 On Windows/x86 systems, <tt>__stdcall</tt> functions are automatically
 688 detected, and a function declared as <tt>__cdecl</tt> (the default) is
 689 silently fixed up after the first call.</li>
 690
 691 </ul>
 692
 693 <h3 id="cdata_arith">Arithmetic on cdata objects</h3>
 694 <ul>
 695
 696 <li><b>Pointer arithmetic</b>: a cdata pointer/array and a cdata
 697 number or a Lua number can be added or subtracted. The number must be
 698 on the right-hand side for a subtraction. The result is a pointer of
 699 the same type with an address plus or minus the number value
 700 multiplied by the element size in bytes. An error is raised if the
 701 element size is undefined.</li>
 702
 703 <li><b>Pointer difference</b>: two compatible cdata pointers/arrays
 704 can be subtracted. The result is the difference between their
 705 addresses, divided by the element size in bytes. An error is raised if
 706 the element size is undefined or zero.</li>
 707
 708 <li><b>64&nbsp;bit integer arithmetic</b>: the standard arithmetic
 709 operators (<tt>+&nbsp;-&nbsp;*&nbsp;/&nbsp;%&nbsp;^</tt> and unary
 710 minus) can be applied to two cdata numbers, or a cdata number and a
 711 Lua number. If one of them is an <tt>uint64_t</tt>, the other side is
 712 converted to an <tt>uint64_t</tt> and an unsigned arithmetic operation
 713 is performed. Otherwise, both sides are converted to an
 714 <tt>int64_t</tt> and a signed arithmetic operation is performed. The
 715 result is a boxed 64&nbsp;bit cdata object.<br>
 716
 717 If one of the operands is an <tt>enum</tt> and the other operand is a
 718 string, the string is converted to the value of a matching <tt>enum</tt>
 719 constant before the above conversion.<br>
 720
 721 These rules ensure that 64&nbsp;bit integers are "sticky". Any
 722 expression involving at least one 64&nbsp;bit integer operand results
 723 in another one. The undefined cases for the division, modulo and power
 724 operators return <tt>2LL&nbsp;^&nbsp;63</tt> or
 725 <tt>2ULL&nbsp;^&nbsp;63</tt>.<br>
 726
 727 You'll have to explicitly convert a 64&nbsp;bit integer to a Lua
 728 number (e.g. for regular floating-point calculations) with
 729 <tt>tonumber()</tt>. But note this may incur a precision loss.</li>
 730
 731 <li><b>64&nbsp;bit bitwise operations</b>: the rules for 64&nbsp;bit
 732 arithmetic operators apply analogously.<br>
 733
 734 Unlike the other <tt>bit.*</tt> operations, <tt>bit.tobit()</tt>
 735 converts a cdata number via <tt>int64_t</tt> to <tt>int32_t</tt> and
 736 returns a Lua number.<br>
 737
 738 For <tt>bit.band()</tt>, <tt>bit.bor()</tt> and <tt>bit.bxor()</tt>, the
 739 conversion to <tt>int64_t</tt> or <tt>uint64_t</tt> applies to
 740 <em>all</em> arguments, if <em>any</em> argument is a cdata number.<br>
 741
 742 For all other operations, only the first argument is used to determine
 743 the output type. This implies that a cdata number as a shift count for
 744 shifts and rotates is accepted, but that alone does <em>not</em> cause
 745 a cdata number output.
 746
 747 </ul>
 748
 749 <h3 id="cdata_comp">Comparisons of cdata objects</h3>
 750 <ul>
 751
 752 <li><b>Pointer comparison</b>: two compatible cdata pointers/arrays
 753 can be compared. The result is the same as an unsigned comparison of
 754 their addresses. <tt>nil</tt> is treated like a <tt>NULL</tt> pointer,
 755 which is compatible with any other pointer type.</li>
 756
 757 <li><b>64&nbsp;bit integer comparison</b>: two cdata numbers, or a
 758 cdata number and a Lua number can be compared with each other. If one
 759 of them is an <tt>uint64_t</tt>, the other side is converted to an
 760 <tt>uint64_t</tt> and an unsigned comparison is performed. Otherwise,
 761 both sides are converted to an <tt>int64_t</tt> and a signed
 762 comparison is performed.<br>
 763
 764 If one of the operands is an <tt>enum</tt> and the other operand is a
 765 string, the string is converted to the value of a matching <tt>enum</tt>
 766 constant before the above conversion.<br>
 767
 768 <li><b>Comparisons for equality/inequality</b> never raise an error.
 769 Even incompatible pointers can be compared for equality by address. Any
 770 other incompatible comparison (also with non-cdata objects) treats the
 771 two sides as unequal.</li>
 772
 773 </ul>
 774
 775 <h3 id="cdata_key">cdata objects as table keys</h3>
 776 <p>
 777 Lua tables may be indexed by cdata objects, but this doesn't provide
 778 any useful semantics &mdash; <b>cdata objects are unsuitable as table
 779 keys!</b>
 780 </p>
 781 <p>
 782 A cdata object is treated like any other garbage-collected object and
 783 is hashed and compared by its address for table indexing. Since
 784 there's no interning for cdata value types, the same value may be
 785 boxed in different cdata objects with different addresses. Thus,
 786 <tt>t[1LL+1LL]</tt> and <tt>t[2LL]</tt> usually <b>do not</b> point to
 787 the same hash slot, and they certainly <b>do not</b> point to the same
 788 hash slot as <tt>t[2]</tt>.
 789 </p>
 790 <p>
 791 It would seriously drive up implementation complexity and slow down
 792 the common case, if one were to add extra handling for by-value
 793 hashing and comparisons to Lua tables. Given the ubiquity of their use
 794 inside the VM, this is not acceptable.
 795 </p>
 796 <p>
 797 There are three viable alternatives, if you really need to use cdata
 798 objects as keys:
 799 </p>
 800 <ul>
 801
 802 <li>If you can get by with the precision of Lua numbers
 803 (52&nbsp;bits), then use <tt>tonumber()</tt> on a cdata number or
 804 combine multiple fields of a cdata aggregate to a Lua number. Then use
 805 the resulting Lua number as a key when indexing tables.<br>
 806 One obvious benefit: <tt>t[tonumber(2LL)]</tt> <b>does</b> point to
 807 the same slot as <tt>t[2]</tt>.</li>
 808
 809 <li>Otherwise, use either <tt>tostring()</tt> on 64&nbsp;bit integers
 810 or complex numbers or combine multiple fields of a cdata aggregate to
 811 a Lua string (e.g. with
 812 <a href="ext_ffi_api.html#ffi_string"><tt>ffi.string()</tt></a>). Then
 813 use the resulting Lua string as a key when indexing tables.</li>
 814
 815 <li>Create your own specialized hash table implementation using the
 816 C&nbsp;types provided by the FFI library, just like you would in
 817 C&nbsp;code. Ultimately, this may give much better performance than the
 818 other alternatives or what a generic by-value hash table could
 819 possibly provide.</li>
 820
 821 </ul>
 822
 823 <h2 id="param">Parameterized Types</h2>
 824 <p>
 825 To facilitate some abstractions, the two functions
 826 <a href="ext_ffi_api.html#ffi_typeof"><tt>ffi.typeof</tt></a> and
 827 <a href="ext_ffi_api.html#ffi_cdef"><tt>ffi.cdef</tt></a> support
 828 parameterized types in C&nbsp;declarations. Note: none of the other API
 829 functions taking a cdecl allow this.
 830 </p>
 831 <p>
 832 Any place you can write a <b><tt>typedef</tt> name</b>, an
 833 <b>identifier</b> or a <b>number</b> in a declaration, you can write
 834 <tt>$</tt> (the dollar sign) instead. These placeholders are replaced in
 835 order of appearance with the arguments following the cdecl string:
 836 </p>
 837 <pre class="code">
 838 -- Declare a struct with a parameterized field type and name:
 839 ffi.cdef([[
 840 typedef struct { $ $; } foo_t;
 841 ]], type1, name1)
 842
 843 -- Anonymous struct with dynamic names:
 844 local bar_t = ffi.typeof("struct { int $, $; }", name1, name2)
 845 -- Derived pointer type:
 846 local bar_ptr_t = ffi.typeof("$ *", bar_t)
 847
 848 -- Parameterized dimensions work even where a VLA won't work:
 849 local matrix_t = ffi.typeof("uint8_t[$][$]", width, height)
 850 </pre>
 851 <p>
 852 Caveat: this is <em>not</em> simple text substitution! A passed ctype or
 853 cdata object is treated like the underlying type, a passed string is
 854 considered an identifier and a number is considered a number. You must
 855 not mix this up: e.g. passing <tt>"int"</tt> as a string doesn't work in
 856 place of a type, you'd need to use <tt>ffi.typeof("int")</tt> instead.
 857 </p>
 858 <p>
 859 The main use for parameterized types are libraries implementing abstract
 860 data types
 861 (<a href="https://www.freelists.org/post/luajit/ffi-type-of-pointer-to,8"><span class="ext">&raquo;</span>&nbsp;example</a>),
 862 similar to what can be achieved with C++ template metaprogramming.
 863 Another use case are derived types of anonymous structs, which avoids
 864 pollution of the global struct namespace.
 865 </p>
 866 <p>
 867 Please note that parameterized types are a nice tool and indispensable
 868 for certain use cases. But you'll want to use them sparingly in regular
 869 code, e.g. when all types are actually fixed.
 870 </p>
 871
 872 <h2 id="gc">Garbage Collection of cdata Objects</h2>
 873 <p>
 874 All explicitly (<tt>ffi.new()</tt>, <tt>ffi.cast()</tt> etc.) or
 875 implicitly (accessors) created cdata objects are garbage collected.
 876 You need to ensure to retain valid references to cdata objects
 877 somewhere on a Lua stack, an upvalue or in a Lua table while they are
 878 still in use. Once the last reference to a cdata object is gone, the
 879 garbage collector will automatically free the memory used by it (at
 880 the end of the next GC cycle).
 881 </p>
 882 <p>
 883 Please note, that pointers themselves are cdata objects, however they
 884 are <b>not</b> followed by the garbage collector. So e.g. if you
 885 assign a cdata array to a pointer, you must keep the cdata object
 886 holding the array alive as long as the pointer is still in use:
 887 </p>
 888 <pre class="code">
 889 ffi.cdef[[
 890 typedef struct { int *a; } foo_t;
 891 ]]
 892
 893 local s = ffi.new("foo_t", ffi.new("int[10]")) -- <span style="color:#c00000;">WRONG!</span>
 894
 895 local a = ffi.new("int[10]") -- <span style="color:#00a000;">OK</span>
 896 local s = ffi.new("foo_t", a)
 897 -- Now do something with 's', but keep 'a' alive until you're done.
 898 </pre>
 899 <p>
 900 Similar rules apply for Lua strings which are implicitly converted to
 901 <tt>"const&nbsp;char&nbsp;*"</tt>: the string object itself must be
 902 referenced somewhere or it'll be garbage collected eventually. The
 903 pointer will then point to stale data, which may have already been
 904 overwritten. Note that <em>string literals</em> are automatically kept
 905 alive as long as the function containing it (actually its prototype)
 906 is not garbage collected.
 907 </p>
 908 <p>
 909 Objects which are passed as an argument to an external C&nbsp;function
 910 are kept alive until the call returns. So it's generally safe to
 911 create temporary cdata objects in argument lists. This is a common
 912 idiom for <a href="#convert_vararg">passing specific C&nbsp;types to
 913 vararg functions</a>.
 914 </p>
 915 <p>
 916 Memory areas returned by C functions (e.g. from <tt>malloc()</tt>)
 917 must be manually managed, of course (or use
 918 <a href="ext_ffi_api.html#ffi_gc"><tt>ffi.gc()</tt></a>). Pointers to
 919 cdata objects are indistinguishable from pointers returned by C
 920 functions (which is one of the reasons why the GC cannot follow them).
 921 </p>
 922
 923 <h2 id="callback">Callbacks</h2>
 924 <p>
 925 The LuaJIT FFI automatically generates special callback functions
 926 whenever a Lua function is converted to a C&nbsp;function pointer. This
 927 associates the generated callback function pointer with the C&nbsp;type
 928 of the function pointer and the Lua function object (closure).
 929 </p>
 930 <p>
 931 This can happen implicitly due to the usual conversions, e.g. when
 932 passing a Lua function to a function pointer argument. Or, you can use
 933 <tt>ffi.cast()</tt> to explicitly cast a Lua function to a
 934 C&nbsp;function pointer.
 935 </p>
 936 <p>
 937 Currently, only certain C&nbsp;function types can be used as callback
 938 functions. Neither C&nbsp;vararg functions nor functions with
 939 pass-by-value aggregate argument or result types are supported. There
 940 are no restrictions on the kind of Lua functions that can be called
 941 from the callback &mdash; no checks for the proper number of arguments
 942 are made. The return value of the Lua function will be converted to the
 943 result type, and an error will be thrown for invalid conversions.
 944 </p>
 945 <p>
 946 It's allowed to throw errors across a callback invocation, but it's not
 947 advisable in general. Do this only if you know the C&nbsp;function, that
 948 called the callback, copes with the forced stack unwinding and doesn't
 949 leak resources.
 950 </p>
 951 <p>
 952 One thing that's not allowed, is to let an FFI call into a C&nbsp;function
 953 get JIT-compiled, which in turn calls a callback, calling into Lua again.
 954 Usually this attempt is caught by the interpreter first and the
 955 C&nbsp;function is blacklisted for compilation.
 956 </p>
 957 <p>
 958 However, this heuristic may fail under specific circumstances: e.g. a
 959 message polling function might not run Lua callbacks right away and the call
 960 gets JIT-compiled. If it later happens to call back into Lua (e.g. a rarely
 961 invoked error callback), you'll get a VM PANIC with the message
 962 <tt>"bad callback"</tt>. Then you'll need to manually turn off
 963 JIT-compilation with
 964 <a href="ext_jit.html#jit_onoff_func"><tt>jit.off()</tt></a> for the
 965 surrounding Lua function that invokes such a message polling function (or
 966 similar).
 967 </p>
 968
 969 <h3 id="callback_resources">Callback resource handling</h3>
 970 <p>
 971 Callbacks take up resources &mdash; you can only have a limited number
 972 of them at the same time (500&nbsp;-&nbsp;1000, depending on the
 973 architecture). The associated Lua functions are anchored to prevent
 974 garbage collection, too.
 975 </p>
 976 <p>
 977 <b>Callbacks due to implicit conversions are permanent!</b> There is no
 978 way to guess their lifetime, since the C&nbsp;side might store the
 979 function pointer for later use (typical for GUI toolkits). The associated
 980 resources cannot be reclaimed until termination:
 981 </p>
 982 <pre class="code">
 983 ffi.cdef[[
 984 typedef int (__stdcall *WNDENUMPROC)(void *hwnd, intptr_t l);
 985 int EnumWindows(WNDENUMPROC func, intptr_t l);
 986 ]]
 987
 988 -- Implicit conversion to a callback via function pointer argument.
 989 local count = 0
 990 ffi.C.EnumWindows(function(hwnd, l)
 991   count = count + 1
 992   return true
 993 end, 0)
 994 -- The callback is permanent and its resources cannot be reclaimed!
 995 -- Ok, so this may not be a problem, if you do this only once.
 996 </pre>
 997 <p>
 998 Note: this example shows that you <em>must</em> properly declare
 999 <tt>__stdcall</tt> callbacks on Windows/x86 systems. The calling
1000 convention cannot be automatically detected, unlike for
1001 <tt>__stdcall</tt> calls <em>to</em> Windows functions.
1002 </p>
1003 <p>
1004 For some use cases, it's necessary to free up the resources or to
1005 dynamically redirect callbacks. Use an explicit cast to a
1006 C&nbsp;function pointer and keep the resulting cdata object. Then use
1007 the <a href="ext_ffi_api.html#callback_free"><tt>cb:free()</tt></a>
1008 or <a href="ext_ffi_api.html#callback_set"><tt>cb:set()</tt></a> methods
1009 on the cdata object:
1010 </p>
1011 <pre class="code">
1012 -- Explicitly convert to a callback via cast.
1013 local count = 0
1014 local cb = ffi.cast("WNDENUMPROC", function(hwnd, l)
1015   count = count + 1
1016   return true
1017 end)
1018
1019 -- Pass it to a C function.
1020 ffi.C.EnumWindows(cb, 0)
1021 -- EnumWindows doesn't need the callback after it returns, so free it.
1022
1023 cb:free()
1024 -- The callback function pointer is no longer valid and its resources
1025 -- will be reclaimed. The created Lua closure will be garbage collected.
1026 </pre>
1027
1028 <h3 id="callback_performance">Callback performance</h3>
1029 <p>
1030 <b>Callbacks are slow!</b> First, the C&nbsp;to Lua transition itself
1031 has an unavoidable cost, similar to a <tt>lua_call()</tt> or
1032 <tt>lua_pcall()</tt>. Argument and result marshalling add to that cost.
1033 And finally, neither the C&nbsp;compiler nor LuaJIT can inline or
1034 optimize across the language barrier and hoist repeated computations out
1035 of a callback function.
1036 </p>
1037 <p>
1038 Do not use callbacks for performance-sensitive work: e.g. consider a
1039 numerical integration routine which takes a user-defined function to
1040 integrate over. It's a bad idea to call a user-defined Lua function from
1041 C&nbsp;code millions of times. The callback overhead will be absolutely
1042 detrimental for performance.
1043 </p>
1044 <p>
1045 It's considerably faster to write the numerical integration routine
1046 itself in Lua &mdash; the JIT compiler will be able to inline the
1047 user-defined function and optimize it together with its calling context,
1048 with very competitive performance.
1049 </p>
1050 <p>
1051 As a general guideline: <b>use callbacks only when you must</b>, because
1052 of existing C&nbsp;APIs. E.g. callback performance is irrelevant for a
1053 GUI application, which waits for user input most of the time, anyway.
1054 </p>
1055 <p>
1056 For new designs <b>avoid push-style APIs</b>: a C&nbsp;function repeatedly
1057 calling a callback for each result. Instead, <b>use pull-style APIs</b>:
1058 call a C&nbsp;function repeatedly to get a new result. Calls from Lua
1059 to C via the FFI are much faster than the other way round. Most well-designed
1060 libraries already use pull-style APIs (read/write, get/put).
1061 </p>
1062
1063 <h2 id="clib">C Library Namespaces</h2>
1064 <p>
1065 A C&nbsp;library namespace is a special kind of object which allows
1066 access to the symbols contained in shared libraries or the default
1067 symbol namespace. The default
1068 <a href="ext_ffi_api.html#ffi_C"><tt>ffi.C</tt></a> namespace is
1069 automatically created when the FFI library is loaded. C&nbsp;library
1070 namespaces for specific shared libraries may be created with the
1071 <a href="ext_ffi_api.html#ffi_load"><tt>ffi.load()</tt></a> API
1072 function.
1073 </p>
1074 <p>
1075 Indexing a C&nbsp;library namespace object with a symbol name (a Lua
1076 string) automatically binds it to the library. First, the symbol type
1077 is resolved &mdash; it must have been declared with
1078 <a href="ext_ffi_api.html#ffi_cdef"><tt>ffi.cdef</tt></a>. Then the
1079 symbol address is resolved by searching for the symbol name in the
1080 associated shared libraries or the default symbol namespace. Finally,
1081 the resulting binding between the symbol name, the symbol type and its
1082 address is cached. Missing symbol declarations or nonexistent symbol
1083 names cause an error.
1084 </p>
1085 <p>
1086 This is what happens on a <b>read access</b> for the different kinds of
1087 symbols:
1088 </p>
1089 <ul>
1090
1091 <li>External functions: a cdata object with the type of the function
1092 and its address is returned.</li>
1093
1094 <li>External variables: the symbol address is dereferenced and the
1095 loaded value is <a href="#convert_tolua">converted to a Lua object</a>
1096 and returned.</li>
1097
1098 <li>Constant values (<tt>static&nbsp;const</tt> or <tt>enum</tt>
1099 constants): the constant is <a href="#convert_tolua">converted to a
1100 Lua object</a> and returned.</li>
1101
1102 </ul>
1103 <p>
1104 This is what happens on a <b>write access</b>:
1105 </p>
1106 <ul>
1107
1108 <li>External variables: the value to be written is
1109 <a href="#convert_fromlua">converted to the C&nbsp;type</a> of the
1110 variable and then stored at the symbol address.</li>
1111
1112 <li>Writing to constant variables or to any other symbol type causes
1113 an error, like any other attempted write to a constant location.</li>
1114
1115 </ul>
1116 <p>
1117 C&nbsp;library namespaces themselves are garbage collected objects. If
1118 the last reference to the namespace object is gone, the garbage
1119 collector will eventually release the shared library reference and
1120 remove all memory associated with the namespace. Since this may
1121 trigger the removal of the shared library from the memory of the
1122 running process, it's generally <em>not safe</em> to use function
1123 cdata objects obtained from a library if the namespace object may be
1124 unreferenced.
1125 </p>
1126 <p>
1127 Performance notice: the JIT compiler specializes to the identity of
1128 namespace objects and to the strings used to index it. This
1129 effectively turns function cdata objects into constants. It's not
1130 useful and actually counter-productive to explicitly cache these
1131 function objects, e.g. <tt>local strlen = ffi.C.strlen</tt>. OTOH, it
1132 <em>is</em> useful to cache the namespace itself, e.g. <tt>local C =
1133 ffi.C</tt>.
1134 </p>
1135
1136 <h2 id="policy">No Hand-holding!</h2>
1137 <p>
1138 The FFI library has been designed as <b>a low-level library</b>. The
1139 goal is to interface with C&nbsp;code and C&nbsp;data types with a
1140 minimum of overhead. This means <b>you can do anything you can do
1141 from&nbsp;C</b>: access all memory, overwrite anything in memory, call
1142 machine code at any memory address and so on.
1143 </p>
1144 <p>
1145 The FFI library provides <b>no memory safety</b>, unlike regular Lua
1146 code. It will happily allow you to dereference a <tt>NULL</tt>
1147 pointer, to access arrays out of bounds or to misdeclare
1148 C&nbsp;functions. If you make a mistake, your application might crash,
1149 just like equivalent C&nbsp;code would.
1150 </p>
1151 <p>
1152 This behavior is inevitable, since the goal is to provide full
1153 interoperability with C&nbsp;code. Adding extra safety measures, like
1154 bounds checks, would be futile. There's no way to detect
1155 misdeclarations of C&nbsp;functions, since shared libraries only
1156 provide symbol names, but no type information. Likewise, there's no way
1157 to infer the valid range of indexes for a returned pointer.
1158 </p>
1159 <p>
1160 Again: the FFI library is a low-level library. This implies it needs
1161 to be used with care, but it's flexibility and performance often
1162 outweigh this concern. If you're a C or C++ developer, it'll be easy
1163 to apply your existing knowledge. OTOH, writing code for the FFI
1164 library is not for the faint of heart and probably shouldn't be the
1165 first exercise for someone with little experience in Lua, C or C++.
1166 </p>
1167 <p>
1168 As a corollary of the above, the FFI library is <b>not safe for use by
1169 untrusted Lua code</b>. If you're sandboxing untrusted Lua code, you
1170 definitely don't want to give this code access to the FFI library or
1171 to <em>any</em> cdata object (except 64&nbsp;bit integers or complex
1172 numbers). Any properly engineered Lua sandbox needs to provide safety
1173 wrappers for many of the standard Lua library functions &mdash;
1174 similar wrappers need to be written for high-level operations on FFI
1175 data types, too.
1176 </p>
1177
1178 <h2 id="status">Current Status</h2>
1179 <p>
1180 The initial release of the FFI library has some limitations and is
1181 missing some features. Most of these will be fixed in future releases.
1182 </p>
1183 <p>
1184 <a href="#clang">C language support</a> is
1185 currently incomplete:
1186 </p>
1187 <ul>
1188 <li>C&nbsp;declarations are not passed through a C&nbsp;pre-processor,
1189 yet.</li>
1190 <li>The C&nbsp;parser is able to evaluate most constant expressions
1191 commonly found in C&nbsp;header files. However, it doesn't handle the
1192 full range of C&nbsp;expression semantics and may fail for some
1193 obscure constructs.</li>
1194 <li><tt>static const</tt> declarations only work for integer types
1195 up to 32&nbsp;bits. Neither declaring string constants nor
1196 floating-point constants is supported.</li>
1197 <li>Packed <tt>struct</tt> bitfields that cross container boundaries
1198 are not implemented.</li>
1199 <li>Native vector types may be defined with the GCC <tt>mode</tt> or
1200 <tt>vector_size</tt> attribute. But no operations other than loading,
1201 storing and initializing them are supported, yet.</li>
1202 <li>The <tt>volatile</tt> type qualifier is currently ignored by
1203 compiled code.</li>
1204 <li><a href="ext_ffi_api.html#ffi_cdef"><tt>ffi.cdef</tt></a> silently
1205 ignores most re-declarations. Note: avoid re-declarations which do not
1206 conform to C99. The implementation will eventually be changed to
1207 perform strict checks.</li>
1208 </ul>
1209 <p>
1210 The JIT compiler already handles a large subset of all FFI operations.
1211 It automatically falls back to the interpreter for unimplemented
1212 operations (you can check for this with the
1213 <a href="running.html#opt_j"><tt>-jv</tt></a> command line option).
1214 The following operations are currently not compiled and may exhibit
1215 suboptimal performance, especially when used in inner loops:
1216 </p>
1217 <ul>
1218 <li>Vector operations.</li>
1219 <li>Table initializers.</li>
1220 <li>Initialization of nested <tt>struct</tt>/<tt>union</tt> types.</li>
1221 <li>Non-default initialization of VLA/VLS or large C&nbsp;types
1222 (&gt; 128&nbsp;bytes or &gt; 16 array elements).</li>
1223 <li>Bitfield initializations.</li>
1224 <li>Pointer differences for element sizes that are not a power of
1225 two.</li>
1226 <li>Calls to C&nbsp;functions with aggregates passed or returned by
1227 value.</li>
1228 <li>Calls to ctype metamethods which are not plain functions.</li>
1229 <li>ctype <tt>__newindex</tt> tables and non-string lookups in ctype
1230 <tt>__index</tt> tables.</li>
1231 <li><tt>tostring()</tt> for cdata types.</li>
1232 <li>Calls to <tt>ffi.cdef()</tt>, <tt>ffi.load()</tt> and
1233 <tt>ffi.metatype()</tt>.</li>
1234 </ul>
1235 <p>
1236 Other missing features:
1237 </p>
1238 <ul>
1239 <li>Arithmetic for <tt>complex</tt> numbers.</li>
1240 <li>Passing structs by value to vararg C&nbsp;functions.</li>
1241 <li><a href="extensions.html#exceptions">C++ exception interoperability</a>
1242 does not extend to C&nbsp;functions called via the FFI, if the call is
1243 compiled.</li>
1244 </ul>
1245 <br class="flush">
1246 </div>
1247 <div id="foot">
1248 <hr class="hide">
1249 Copyright &copy; 2005-2023
1250 <span class="noprint">
1251 &middot;
1252 <a href="contact.html">Contact</a>
1253 </span>
1254 </div>
1255 </body>
1256 </html>