doc/ext_ffi.html

   1 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
   2 <html>
   3 <head>
   4 <title>FFI Library</title>
   5 <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
   6 <meta name="Author" content="Mike Pall">
   7 <meta name="Copyright" content="Copyright (C) 2005-2016, Mike Pall">
   8 <meta name="Language" content="en">
   9 <link rel="stylesheet" type="text/css" href="bluequad.css" media="screen">
  10 <link rel="stylesheet" type="text/css" href="bluequad-print.css" media="print">
  11 </head>
  12 <body>
  13 <div id="site">
  14 <a href="http://luajit.org"><span>Lua<span id="logo">JIT</span></span></a>
  15 </div>
  16 <div id="head">
  17 <h1>FFI Library</h1>
  18 </div>
  19 <div id="nav">
  20 <ul><li>
  21 <a href="luajit.html">LuaJIT</a>
  22 <ul><li>
  23 <a href="http://luajit.org/download.html">Download <span class="ext">&raquo;</span></a>
  24 </li><li>
  25 <a href="install.html">Installation</a>
  26 </li><li>
  27 <a href="running.html">Running</a>
  28 </li></ul>
  29 </li><li>
  30 <a href="extensions.html">Extensions</a>
  31 <ul><li>
  32 <a class="current" href="ext_ffi.html">FFI Library</a>
  33 <ul><li>
  34 <a href="ext_ffi_tutorial.html">FFI Tutorial</a>
  35 </li><li>
  36 <a href="ext_ffi_api.html">ffi.* API</a>
  37 </li><li>
  38 <a href="ext_ffi_semantics.html">FFI Semantics</a>
  39 </li></ul>
  40 </li><li>
  41 <a href="ext_jit.html">jit.* Library</a>
  42 </li><li>
  43 <a href="ext_c_api.html">Lua/C API</a>
  44 </li></ul>
  45 </li><li>
  46 <a href="status.html">Status</a>
  47 <ul><li>
  48 <a href="changes.html">Changes</a>
  49 </li></ul>
  50 </li><li>
  51 <a href="faq.html">FAQ</a>
  52 </li><li>
  53 <a href="http://luajit.org/performance.html">Performance <span class="ext">&raquo;</span></a>
  54 </li><li>
  55 <a href="http://wiki.luajit.org/">Wiki <span class="ext">&raquo;</span></a>
  56 </li><li>
  57 <a href="http://luajit.org/list.html">Mailing List <span class="ext">&raquo;</span></a>
  58 </li></ul>
  59 </div>
  60 <div id="main">
  61 <p>
  62
  63 The FFI library allows <b>calling external C&nbsp;functions</b> and
  64 <b>using C&nbsp;data structures</b> from pure Lua code.
  65
  66 </p>
  67 <p>
  68
  69 The FFI library largely obviates the need to write tedious manual
  70 Lua/C bindings in C. No need to learn a separate binding language
  71 &mdash; <b>it parses plain C&nbsp;declarations!</b> These can be
  72 cut-n-pasted from C&nbsp;header files or reference manuals. It's up to
  73 the task of binding large libraries without the need for dealing with
  74 fragile binding generators.
  75
  76 </p>
  77 <p>
  78 The FFI library is tightly integrated into LuaJIT (it's not available
  79 as a separate module). The code generated by the JIT-compiler for
  80 accesses to C&nbsp;data structures from Lua code is on par with the
  81 code a C&nbsp;compiler would generate. Calls to C&nbsp;functions can
  82 be inlined in JIT-compiled code, unlike calls to functions bound via
  83 the classic Lua/C API.
  84 </p>
  85 <p>
  86 This page gives a short introduction to the usage of the FFI library.
  87 <em>Please use the FFI sub-topics in the navigation bar to learn more.</em>
  88 </p>
  89
  90 <h2 id="call">Motivating Example: Calling External C Functions</h2>
  91 <p>
  92 It's really easy to call an external C&nbsp;library function:
  93 </p>
  94 <pre class="code mark">
  95 <span class="codemark">&#9312;
  96 &#9313;
  97
  98
  99 &#9314;</span>local ffi = require("ffi")
 100 ffi.cdef[[
 101 <span style="color:#00a000;">int printf(const char *fmt, ...);</span>
 102 ]]
 103 ffi.C.printf("Hello %s!", "world")
 104 </pre>
 105 <p>
 106 So, let's pick that apart:
 107 </p>
 108 <p>
 109 <span class="mark">&#9312;</span> Load the FFI library.
 110 </p>
 111 <p>
 112 <span class="mark">&#9313;</span> Add a C&nbsp;declaration
 113 for the function. The part inside the double-brackets (in green) is
 114 just standard C&nbsp;syntax.
 115 </p>
 116 <p>
 117 <span class="mark">&#9314;</span> Call the named
 118 C&nbsp;function &mdash; Yes, it's that simple!
 119 </p>
 120 <p style="font-size: 8pt;">
 121 Actually, what goes on behind the scenes is far from simple: <span
 122 style="color:#4040c0;">&#9314;</span> makes use of the standard
 123 C&nbsp;library namespace <tt>ffi.C</tt>. Indexing this namespace with
 124 a symbol name (<tt>"printf"</tt>) automatically binds it to the
 125 standard C&nbsp;library. The result is a special kind of object which,
 126 when called, runs the <tt>printf</tt> function. The arguments passed
 127 to this function are automatically converted from Lua objects to the
 128 corresponding C&nbsp;types.
 129 </p>
 130 <p>
 131 Ok, so maybe the use of <tt>printf()</tt> wasn't such a spectacular
 132 example. You could have done that with <tt>io.write()</tt> and
 133 <tt>string.format()</tt>, too. But you get the idea ...
 134 </p>
 135 <p>
 136 So here's something to pop up a message box on Windows:
 137 </p>
 138 <pre class="code">
 139 local ffi = require("ffi")
 140 ffi.cdef[[
 141 <span style="color:#00a000;">int MessageBoxA(void *w, const char *txt, const char *cap, int type);</span>
 142 ]]
 143 ffi.C.MessageBoxA(nil, "Hello world!", "Test", 0)
 144 </pre>
 145 <p>
 146 Bing! Again, that was far too easy, no?
 147 </p>
 148 <p style="font-size: 8pt;">
 149 Compare this with the effort required to bind that function using the
 150 classic Lua/C API: create an extra C&nbsp;file, add a C&nbsp;function
 151 that retrieves and checks the argument types passed from Lua and calls
 152 the actual C&nbsp;function, add a list of module functions and their
 153 names, add a <tt>luaopen_*</tt> function and register all module
 154 functions, compile and link it into a shared library (DLL), move it to
 155 the proper path, add Lua code that loads the module aaaand ... finally
 156 call the binding function. Phew!
 157 </p>
 158
 159 <h2 id="cdata">Motivating Example: Using C Data Structures</h2>
 160 <p>
 161 The FFI library allows you to create and access C&nbsp;data
 162 structures. Of course the main use for this is for interfacing with
 163 C&nbsp;functions. But they can be used stand-alone, too.
 164 </p>
 165 <p>
 166 Lua is built upon high-level data types. They are flexible, extensible
 167 and dynamic. That's why we all love Lua so much. Alas, this can be
 168 inefficient for certain tasks, where you'd really want a low-level
 169 data type. E.g. a large array of a fixed structure needs to be
 170 implemented with a big table holding lots of tiny tables. This imposes
 171 both a substantial memory overhead as well as a performance overhead.
 172 </p>
 173 <p>
 174 Here's a sketch of a library that operates on color images plus a
 175 simple benchmark. First, the plain Lua version:
 176 </p>
 177 <pre class="code">
 178 local floor = math.floor
 179
 180 local function image_ramp_green(n)
 181   local img = {}
 182   local f = 255/(n-1)
 183   for i=1,n do
 184     img[i] = { red = 0, green = floor((i-1)*f), blue = 0, alpha = 255 }
 185   end
 186   return img
 187 end
 188
 189 local function image_to_grey(img, n)
 190   for i=1,n do
 191     local y = floor(0.3*img[i].red + 0.59*img[i].green + 0.11*img[i].blue)
 192     img[i].red = y; img[i].green = y; img[i].blue = y
 193   end
 194 end
 195
 196 local N = 400*400
 197 local img = image_ramp_green(N)
 198 for i=1,1000 do
 199   image_to_grey(img, N)
 200 end
 201 </pre>
 202 <p>
 203 This creates a table with 160.000 pixels, each of which is a table
 204 holding four number values in the range of 0-255. First an image with
 205 a green ramp is created (1D for simplicity), then the image is
 206 converted to greyscale 1000 times. Yes, that's silly, but I was in
 207 need of a simple example ...
 208 </p>
 209 <p>
 210 And here's the FFI version. The modified parts have been marked in
 211 bold:
 212 </p>
 213 <pre class="code mark">
 214 <span class="codemark">&#9312;
 215
 216
 217
 218
 219
 220 &#9313;
 221
 222 &#9314;
 223 &#9315;
 224
 225
 226
 227
 228
 229
 230 &#9314;
 231 &#9316;</span><b>local ffi = require("ffi")
 232 ffi.cdef[[
 233 </b><span style="color:#00a000;">typedef struct { uint8_t red, green, blue, alpha; } rgba_pixel;</span><b>
 234 ]]</b>
 235
 236 local function image_ramp_green(n)
 237   <b>local img = ffi.new("rgba_pixel[?]", n)</b>
 238   local f = 255/(n-1)
 239   for i=<b>0,n-1</b> do
 240     <b>img[i].green = i*f</b>
 241     <b>img[i].alpha = 255</b>
 242   end
 243   return img
 244 end
 245
 246 local function image_to_grey(img, n)
 247   for i=<b>0,n-1</b> do
 248     local y = <b>0.3*img[i].red + 0.59*img[i].green + 0.11*img[i].blue</b>
 249     img[i].red = y; img[i].green = y; img[i].blue = y
 250   end
 251 end
 252
 253 local N = 400*400
 254 local img = image_ramp_green(N)
 255 for i=1,1000 do
 256   image_to_grey(img, N)
 257 end
 258 </pre>
 259 <p>
 260 Ok, so that wasn't too difficult:
 261 </p>
 262 <p>
 263 <span class="mark">&#9312;</span> First, load the FFI
 264 library and declare the low-level data type. Here we choose a
 265 <tt>struct</tt> which holds four byte fields, one for each component
 266 of a 4x8&nbsp;bit RGBA pixel.
 267 </p>
 268 <p>
 269 <span class="mark">&#9313;</span> Creating the data
 270 structure with <tt>ffi.new()</tt> is straightforward &mdash; the
 271 <tt>'?'</tt> is a placeholder for the number of elements of a
 272 variable-length array.
 273 </p>
 274 <p>
 275 <span class="mark">&#9314;</span> C&nbsp;arrays are
 276 zero-based, so the indexes have to run from <tt>0</tt> to
 277 <tt>n-1</tt>. One might want to allocate one more element instead to
 278 simplify converting legacy code.
 279 </p>
 280 <p>
 281 <span class="mark">&#9315;</span> Since <tt>ffi.new()</tt>
 282 zero-fills the array by default, we only need to set the green and the
 283 alpha fields.
 284 </p>
 285 <p>
 286 <span class="mark">&#9316;</span> The calls to
 287 <tt>math.floor()</tt> can be omitted here, because floating-point
 288 numbers are already truncated towards zero when converting them to an
 289 integer. This happens implicitly when the number is stored in the
 290 fields of each pixel.
 291 </p>
 292 <p>
 293 Now let's have a look at the impact of the changes: first, memory
 294 consumption for the image is down from 22&nbsp;Megabytes to
 295 640&nbsp;Kilobytes (400*400*4 bytes). That's a factor of 35x less! So,
 296 yes, tables do have a noticeable overhead. BTW: The original program
 297 would consume 40&nbsp;Megabytes in plain Lua (on x64).
 298 </p>
 299 <p>
 300 Next, performance: the pure Lua version runs in 9.57 seconds (52.9
 301 seconds with the Lua interpreter) and the FFI version runs in 0.48
 302 seconds on my machine (YMMV). That's a factor of 20x faster (110x
 303 faster than the Lua interpreter).
 304 </p>
 305 <p style="font-size: 8pt;">
 306 The avid reader may notice that converting the pure Lua version over
 307 to use array indexes for the colors (<tt>[1]</tt> instead of
 308 <tt>.red</tt>, <tt>[2]</tt> instead of <tt>.green</tt> etc.) ought to
 309 be more compact and faster. This is certainly true (by a factor of
 310 ~1.7x). Switching to a struct-of-arrays would help, too.
 311 </p>
 312 <p style="font-size: 8pt;">
 313 However the resulting code would be less idiomatic and rather
 314 error-prone. And it still doesn't get even close to the performance of
 315 the FFI version of the code. Also, high-level data structures cannot
 316 be easily passed to other C&nbsp;functions, especially I/O functions,
 317 without undue conversion penalties.
 318 </p>
 319 <br class="flush">
 320 </div>
 321 <div id="foot">
 322 <hr class="hide">
 323 Copyright &copy; 2005-2016 Mike Pall
 324 <span class="noprint">
 325 &middot;
 326 <a href="contact.html">Contact</a>
 327 </span>
 328 </div>
 329 </body>
 330 </html>