doc/ext_ffi.html

   1 <!DOCTYPE html>
   2 <html>
   3 <head>
   4 <title>FFI Library</title>
   5 <meta charset="utf-8">
   6 <meta name="Copyright" content="Copyright (C) 2005-2023">
   7 <meta name="Language" content="en">
   8 <link rel="stylesheet" type="text/css" href="bluequad.css" media="screen">
   9 <link rel="stylesheet" type="text/css" href="bluequad-print.css" media="print">
  10 </head>
  11 <body>
  12 <div id="site">
  13 <a href="https://luajit.org"><span>Lua<span id="logo">JIT</span></span></a>
  14 </div>
  15 <div id="head">
  16 <h1>FFI Library</h1>
  17 </div>
  18 <div id="nav">
  19 <ul><li>
  20 <a href="luajit.html">LuaJIT</a>
  21 <ul><li>
  22 <a href="https://luajit.org/download.html">Download <span class="ext">&raquo;</span></a>
  23 </li><li>
  24 <a href="install.html">Installation</a>
  25 </li><li>
  26 <a href="running.html">Running</a>
  27 </li></ul>
  28 </li><li>
  29 <a href="extensions.html">Extensions</a>
  30 <ul><li>
  31 <a class="current" href="ext_ffi.html">FFI Library</a>
  32 <ul><li>
  33 <a href="ext_ffi_tutorial.html">FFI Tutorial</a>
  34 </li><li>
  35 <a href="ext_ffi_api.html">ffi.* API</a>
  36 </li><li>
  37 <a href="ext_ffi_semantics.html">FFI Semantics</a>
  38 </li></ul>
  39 </li><li>
  40 <a href="ext_buffer.html">String Buffers</a>
  41 </li><li>
  42 <a href="ext_jit.html">jit.* Library</a>
  43 </li><li>
  44 <a href="ext_c_api.html">Lua/C API</a>
  45 </li><li>
  46 <a href="ext_profiler.html">Profiler</a>
  47 </li></ul>
  48 </li><li>
  49 <a href="https://luajit.org/status.html">Status <span class="ext">&raquo;</span></a>
  50 </li><li>
  51 <a href="https://luajit.org/faq.html">FAQ <span class="ext">&raquo;</span></a>
  52 </li><li>
  53 <a href="https://luajit.org/list.html">Mailing List <span class="ext">&raquo;</span></a>
  54 </li></ul>
  55 </div>
  56 <div id="main">
  57 <p>
  58
  59 The FFI library allows <b>calling external C&nbsp;functions</b> and
  60 <b>using C&nbsp;data structures</b> from pure Lua code.
  61
  62 </p>
  63 <p>
  64
  65 The FFI library largely obviates the need to write tedious manual
  66 Lua/C bindings in C. No need to learn a separate binding language
  67 &mdash; <b>it parses plain C&nbsp;declarations!</b> These can be
  68 cut-n-pasted from C&nbsp;header files or reference manuals. It's up to
  69 the task of binding large libraries without the need for dealing with
  70 fragile binding generators.
  71
  72 </p>
  73 <p>
  74 The FFI library is tightly integrated into LuaJIT (it's not available
  75 as a separate module). The code generated by the JIT-compiler for
  76 accesses to C&nbsp;data structures from Lua code is on par with the
  77 code a C&nbsp;compiler would generate. Calls to C&nbsp;functions can
  78 be inlined in JIT-compiled code, unlike calls to functions bound via
  79 the classic Lua/C API.
  80 </p>
  81 <p>
  82 This page gives a short introduction to the usage of the FFI library.
  83 <em>Please use the FFI sub-topics in the navigation bar to learn more.</em>
  84 </p>
  85
  86 <h2 id="call">Motivating Example: Calling External C Functions</h2>
  87 <p>
  88 It's really easy to call an external C&nbsp;library function:
  89 </p>
  90 <pre class="code mark">
  91 <span class="codemark">&#9312;
  92 &#9313;
  93
  94
  95 &#9314;</span>local ffi = require("ffi")
  96 ffi.cdef[[
  97 <span style="color:#00a000;">int printf(const char *fmt, ...);</span>
  98 ]]
  99 ffi.C.printf("Hello %s!", "world")
 100 </pre>
 101 <p>
 102 So, let's pick that apart:
 103 </p>
 104 <p>
 105 <span class="mark">&#9312;</span> Load the FFI library.
 106 </p>
 107 <p>
 108 <span class="mark">&#9313;</span> Add a C&nbsp;declaration
 109 for the function. The part inside the double-brackets (in green) is
 110 just standard C&nbsp;syntax.
 111 </p>
 112 <p>
 113 <span class="mark">&#9314;</span> Call the named
 114 C&nbsp;function &mdash; Yes, it's that simple!
 115 </p>
 116 <p style="font-size: 8pt;">
 117 Actually, what goes on behind the scenes is far from simple: <span
 118 style="color:#4040c0;">&#9314;</span> makes use of the standard
 119 C&nbsp;library namespace <tt>ffi.C</tt>. Indexing this namespace with
 120 a symbol name (<tt>"printf"</tt>) automatically binds it to the
 121 standard C&nbsp;library. The result is a special kind of object which,
 122 when called, runs the <tt>printf</tt> function. The arguments passed
 123 to this function are automatically converted from Lua objects to the
 124 corresponding C&nbsp;types.
 125 </p>
 126 <p>
 127 Ok, so maybe the use of <tt>printf()</tt> wasn't such a spectacular
 128 example. You could have done that with <tt>io.write()</tt> and
 129 <tt>string.format()</tt>, too. But you get the idea ...
 130 </p>
 131 <p>
 132 So here's something to pop up a message box on Windows:
 133 </p>
 134 <pre class="code">
 135 local ffi = require("ffi")
 136 ffi.cdef[[
 137 <span style="color:#00a000;">int MessageBoxA(void *w, const char *txt, const char *cap, int type);</span>
 138 ]]
 139 ffi.C.MessageBoxA(nil, "Hello world!", "Test", 0)
 140 </pre>
 141 <p>
 142 Bing! Again, that was far too easy, no?
 143 </p>
 144 <p style="font-size: 8pt;">
 145 Compare this with the effort required to bind that function using the
 146 classic Lua/C API: create an extra C&nbsp;file, add a C&nbsp;function
 147 that retrieves and checks the argument types passed from Lua and calls
 148 the actual C&nbsp;function, add a list of module functions and their
 149 names, add a <tt>luaopen_*</tt> function and register all module
 150 functions, compile and link it into a shared library (DLL), move it to
 151 the proper path, add Lua code that loads the module aaaand ... finally
 152 call the binding function. Phew!
 153 </p>
 154
 155 <h2 id="cdata">Motivating Example: Using C Data Structures</h2>
 156 <p>
 157 The FFI library allows you to create and access C&nbsp;data
 158 structures. Of course, the main use for this is for interfacing with
 159 C&nbsp;functions. But they can be used stand-alone, too.
 160 </p>
 161 <p>
 162 Lua is built upon high-level data types. They are flexible, extensible
 163 and dynamic. That's why we all love Lua so much. Alas, this can be
 164 inefficient for certain tasks, where you'd really want a low-level
 165 data type. E.g. a large array of a fixed structure needs to be
 166 implemented with a big table holding lots of tiny tables. This imposes
 167 both a substantial memory overhead as well as a performance overhead.
 168 </p>
 169 <p>
 170 Here's a sketch of a library that operates on color images, plus a
 171 simple benchmark. First, the plain Lua version:
 172 </p>
 173 <pre class="code">
 174 local floor = math.floor
 175
 176 local function image_ramp_green(n)
 177   local img = {}
 178   local f = 255/(n-1)
 179   for i=1,n do
 180     img[i] = { red = 0, green = floor((i-1)*f), blue = 0, alpha = 255 }
 181   end
 182   return img
 183 end
 184
 185 local function image_to_gray(img, n)
 186   for i=1,n do
 187     local y = floor(0.3*img[i].red + 0.59*img[i].green + 0.11*img[i].blue)
 188     img[i].red = y; img[i].green = y; img[i].blue = y
 189   end
 190 end
 191
 192 local N = 400*400
 193 local img = image_ramp_green(N)
 194 for i=1,1000 do
 195   image_to_gray(img, N)
 196 end
 197 </pre>
 198 <p>
 199 This creates a table with 160.000 pixels, each of which is a table
 200 holding four number values in the range of 0-255. First, an image with
 201 a green ramp is created (1D for simplicity), then the image is
 202 converted to grayscale 1000 times. Yes, that's silly, but I was in
 203 need of a simple example ...
 204 </p>
 205 <p>
 206 And here's the FFI version. The modified parts have been marked in
 207 bold:
 208 </p>
 209 <pre class="code mark">
 210 <span class="codemark">&#9312;
 211
 212
 213
 214
 215
 216 &#9313;
 217
 218 &#9314;
 219 &#9315;
 220
 221
 222
 223
 224
 225
 226 &#9314;
 227 &#9316;</span><b>local ffi = require("ffi")
 228 ffi.cdef[[
 229 </b><span style="color:#00a000;">typedef struct { uint8_t red, green, blue, alpha; } rgba_pixel;</span><b>
 230 ]]</b>
 231
 232 local function image_ramp_green(n)
 233   <b>local img = ffi.new("rgba_pixel[?]", n)</b>
 234   local f = 255/(n-1)
 235   for i=<b>0,n-1</b> do
 236     <b>img[i].green = i*f</b>
 237     <b>img[i].alpha = 255</b>
 238   end
 239   return img
 240 end
 241
 242 local function image_to_grey(img, n)
 243   for i=<b>0,n-1</b> do
 244     local y = <b>0.3*img[i].red + 0.59*img[i].green + 0.11*img[i].blue</b>
 245     img[i].red = y; img[i].green = y; img[i].blue = y
 246   end
 247 end
 248
 249 local N = 400*400
 250 local img = image_ramp_green(N)
 251 for i=1,1000 do
 252   image_to_grey(img, N)
 253 end
 254 </pre>
 255 <p>
 256 Ok, so that wasn't too difficult:
 257 </p>
 258 <p>
 259 <span class="mark">&#9312;</span> First, load the FFI
 260 library and declare the low-level data type. Here we choose a
 261 <tt>struct</tt> which holds four byte fields, one for each component
 262 of a 4x8&nbsp;bit RGBA pixel.
 263 </p>
 264 <p>
 265 <span class="mark">&#9313;</span> Creating the data
 266 structure with <tt>ffi.new()</tt> is straightforward &mdash; the
 267 <tt>'?'</tt> is a placeholder for the number of elements of a
 268 variable-length array.
 269 </p>
 270 <p>
 271 <span class="mark">&#9314;</span> C&nbsp;arrays are
 272 zero-based, so the indexes have to run from <tt>0</tt> to
 273 <tt>n-1</tt>. One might want to allocate one more element instead to
 274 simplify converting legacy code.
 275 </p>
 276 <p>
 277 <span class="mark">&#9315;</span> Since <tt>ffi.new()</tt>
 278 zero-fills the array by default, we only need to set the green and the
 279 alpha fields.
 280 </p>
 281 <p>
 282 <span class="mark">&#9316;</span> The calls to
 283 <tt>math.floor()</tt> can be omitted here, because floating-point
 284 numbers are already truncated towards zero when converting them to an
 285 integer. This happens implicitly when the number is stored in the
 286 fields of each pixel.
 287 </p>
 288 <p>
 289 Now let's have a look at the impact of the changes: first, memory
 290 consumption for the image is down from 22&nbsp;Megabytes to
 291 640&nbsp;Kilobytes (400*400*4 bytes). That's a factor of 35x less! So,
 292 yes, tables do have a noticeable overhead. BTW: The original program
 293 would consume 40&nbsp;Megabytes in plain Lua (on x64).
 294 </p>
 295 <p>
 296 Next, performance: the pure Lua version runs in 9.57 seconds (52.9
 297 seconds with the Lua interpreter) and the FFI version runs in 0.48
 298 seconds on my machine (YMMV). That's a factor of 20x faster (110x
 299 faster than the Lua interpreter).
 300 </p>
 301 <p style="font-size: 8pt;">
 302 The avid reader may notice that converting the pure Lua version over
 303 to use array indexes for the colors (<tt>[1]</tt> instead of
 304 <tt>.red</tt>, <tt>[2]</tt> instead of <tt>.green</tt> etc.) ought to
 305 be more compact and faster. This is certainly true (by a factor of
 306 ~1.7x). Switching to a struct-of-arrays would help, too.
 307 </p>
 308 <p style="font-size: 8pt;">
 309 However, the resulting code would be less idiomatic and rather
 310 error-prone. And it still doesn't get even close to the performance of
 311 the FFI version of the code. Also, high-level data structures cannot
 312 be easily passed to other C&nbsp;functions, especially I/O functions,
 313 without undue conversion penalties.
 314 </p>
 315 <br class="flush">
 316 </div>
 317 <div id="foot">
 318 <hr class="hide">
 319 Copyright &copy; 2005-2023
 320 <span class="noprint">
 321 &middot;
 322 <a href="contact.html">Contact</a>
 323 </span>
 324 </div>
 325 </body>
 326 </html>