doc/ext_ffi.html

   1 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
   2 <html>
   3 <head>
   4 <title>FFI Library</title>
   5 <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
   6 <meta name="Author" content="Mike Pall">
   7 <meta name="Copyright" content="Copyright (C) 2005-2012, Mike Pall">
   8 <meta name="Language" content="en">
   9 <link rel="stylesheet" type="text/css" href="bluequad.css" media="screen">
  10 <link rel="stylesheet" type="text/css" href="bluequad-print.css" media="print">
  11 <style type="text/css">
  12 span.codemark { position:absolute; left: 16em; color: #4040c0; }
  13 span.mark { color: #4040c0; font-family: Courier New, Courier, monospace;
  14   line-height: 1.1; }
  15 pre.mark { padding-left: 2em; }
  16 </style>
  17 </head>
  18 <body>
  19 <div id="site">
  20 <a href="http://luajit.org"><span>Lua<span id="logo">JIT</span></span></a>
  21 </div>
  22 <div id="head">
  23 <h1>FFI Library</h1>
  24 </div>
  25 <div id="nav">
  26 <ul><li>
  27 <a href="luajit.html">LuaJIT</a>
  28 <ul><li>
  29 <a href="install.html">Installation</a>
  30 </li><li>
  31 <a href="running.html">Running</a>
  32 </li></ul>
  33 </li><li>
  34 <a href="extensions.html">Extensions</a>
  35 <ul><li>
  36 <a class="current" href="ext_ffi.html">FFI Library</a>
  37 <ul><li>
  38 <a href="ext_ffi_tutorial.html">FFI Tutorial</a>
  39 </li><li>
  40 <a href="ext_ffi_api.html">ffi.* API</a>
  41 </li><li>
  42 <a href="ext_ffi_semantics.html">FFI Semantics</a>
  43 </li></ul>
  44 </li><li>
  45 <a href="ext_jit.html">jit.* Library</a>
  46 </li><li>
  47 <a href="ext_c_api.html">Lua/C API</a>
  48 </li></ul>
  49 </li><li>
  50 <a href="status.html">Status</a>
  51 <ul><li>
  52 <a href="changes.html">Changes</a>
  53 </li></ul>
  54 </li><li>
  55 <a href="faq.html">FAQ</a>
  56 </li><li>
  57 <a href="http://luajit.org/performance.html">Performance <span class="ext">&raquo;</span></a>
  58 </li><li>
  59 <a href="http://luajit.org/download.html">Download <span class="ext">&raquo;</span></a>
  60 </li></ul>
  61 </div>
  62 <div id="main">
  63 <p>
  64
  65 The FFI library allows <b>calling external C&nbsp;functions</b> and
  66 <b>using C&nbsp;data structures</b> from pure Lua code.
  67
  68 </p>
  69 <p>
  70
  71 The FFI library largely obviates the need to write tedious manual
  72 Lua/C bindings in C. No need to learn a separate binding language
  73 &mdash; <b>it parses plain C&nbsp;declarations!</b> These can be
  74 cut-n-pasted from C&nbsp;header files or reference manuals. It's up to
  75 the task of binding large libraries without the need for dealing with
  76 fragile binding generators.
  77
  78 </p>
  79 <p>
  80 The FFI library is tightly integrated into LuaJIT (it's not available
  81 as a separate module). The code generated by the JIT-compiler for
  82 accesses to C&nbsp;data structures from Lua code is on par with the
  83 code a C&nbsp;compiler would generate. Calls to C&nbsp;functions can
  84 be inlined in JIT-compiled code, unlike calls to functions bound via
  85 the classic Lua/C API.
  86 </p>
  87 <p>
  88 This page gives a short introduction to the usage of the FFI library.
  89 <em>Please use the FFI sub-topics in the navigation bar to learn more.</em>
  90 </p>
  91
  92 <h2 id="call">Motivating Example: Calling External C Functions</h2>
  93 <p>
  94 It's really easy to call an external C&nbsp;library function:
  95 </p>
  96 <pre class="code mark">
  97 <span class="codemark">&#9312;
  98 &#9313;
  99
 100
 101 &#9314;</span>local ffi = require("ffi")
 102 ffi.cdef[[
 103 <span style="color:#00a000;">int printf(const char *fmt, ...);</span>
 104 ]]
 105 ffi.C.printf("Hello %s!", "world")
 106 </pre>
 107 <p>
 108 So, let's pick that apart:
 109 </p>
 110 <p>
 111 <span class="mark">&#9312;</span> Load the FFI library.
 112 </p>
 113 <p>
 114 <span class="mark">&#9313;</span> Add a C&nbsp;declaration
 115 for the function. The part inside the double-brackets (in green) is
 116 just standard C&nbsp;syntax.
 117 </p>
 118 <p>
 119 <span class="mark">&#9314;</span> Call the named
 120 C&nbsp;function &mdash; Yes, it's that simple!
 121 </p>
 122 <p style="font-size: 8pt;">
 123 Actually, what goes on behind the scenes is far from simple: <span
 124 style="color:#4040c0;">&#9314;</span> makes use of the standard
 125 C&nbsp;library namespace <tt>ffi.C</tt>. Indexing this namespace with
 126 a symbol name (<tt>"printf"</tt>) automatically binds it to the the
 127 standard C&nbsp;library. The result is a special kind of object which,
 128 when called, runs the <tt>printf</tt> function. The arguments passed
 129 to this function are automatically converted from Lua objects to the
 130 corresponding C&nbsp;types.
 131 </p>
 132 <p>
 133 Ok, so maybe the use of <tt>printf()</tt> wasn't such a spectacular
 134 example. You could have done that with <tt>io.write()</tt> and
 135 <tt>string.format()</tt>, too. But you get the idea ...
 136 </p>
 137 <p>
 138 So here's something to pop up a message box on Windows:
 139 </p>
 140 <pre class="code">
 141 local ffi = require("ffi")
 142 ffi.cdef[[
 143 <span style="color:#00a000;">int MessageBoxA(void *w, const char *txt, const char *cap, int type);</span>
 144 ]]
 145 ffi.C.MessageBoxA(nil, "Hello world!", "Test", 0)
 146 </pre>
 147 <p>
 148 Bing! Again, that was far too easy, no?
 149 </p>
 150 <p style="font-size: 8pt;">
 151 Compare this with the effort required to bind that function using the
 152 classic Lua/C API: create an extra C&nbsp;file, add a C&nbsp;function
 153 that retrieves and checks the argument types passed from Lua and calls
 154 the actual C&nbsp;function, add a list of module functions and their
 155 names, add a <tt>luaopen_*</tt> function and register all module
 156 functions, compile and link it into a shared library (DLL), move it to
 157 the proper path, add Lua code that loads the module aaaand ... finally
 158 call the binding function. Phew!
 159 </p>
 160
 161 <h2 id="cdata">Motivating Example: Using C Data Structures</h2>
 162 <p>
 163 The FFI library allows you to create and access C&nbsp;data
 164 structures. Of course the main use for this is for interfacing with
 165 C&nbsp;functions. But they can be used stand-alone, too.
 166 </p>
 167 <p>
 168 Lua is built upon high-level data types. They are flexible, extensible
 169 and dynamic. That's why we all love Lua so much. Alas, this can be
 170 inefficient for certain tasks, where you'd really want a low-level
 171 data type. E.g. a large array of a fixed structure needs to be
 172 implemented with a big table holding lots of tiny tables. This imposes
 173 both a substantial memory overhead as well as a performance overhead.
 174 </p>
 175 <p>
 176 Here's a sketch of a library that operates on color images plus a
 177 simple benchmark. First, the plain Lua version:
 178 </p>
 179 <pre class="code">
 180 local floor = math.floor
 181
 182 local function image_ramp_green(n)
 183   local img = {}
 184   local f = 255/(n-1)
 185   for i=1,n do
 186     img[i] = { red = 0, green = floor((i-1)*f), blue = 0, alpha = 255 }
 187   end
 188   return img
 189 end
 190
 191 local function image_to_grey(img, n)
 192   for i=1,n do
 193     local y = floor(0.3*img[i].red + 0.59*img[i].green + 0.11*img[i].blue)
 194     img[i].red = y; img[i].green = y; img[i].blue = y
 195   end
 196 end
 197
 198 local N = 400*400
 199 local img = image_ramp_green(N)
 200 for i=1,1000 do
 201   image_to_grey(img, N)
 202 end
 203 </pre>
 204 <p>
 205 This creates a table with 160.000 pixels, each of which is a table
 206 holding four number values in the range of 0-255. First an image with
 207 a green ramp is created (1D for simplicity), then the image is
 208 converted to greyscale 1000 times. Yes, that's silly, but I was in
 209 need of a simple example ...
 210 </p>
 211 <p>
 212 And here's the FFI version. The modified parts have been marked in
 213 bold:
 214 </p>
 215 <pre class="code mark">
 216 <span class="codemark">&#9312;
 217
 218
 219
 220
 221
 222 &#9313;
 223
 224 &#9314;
 225 &#9315;
 226
 227
 228
 229
 230
 231
 232 &#9314;
 233 &#9316;</span><b>local ffi = require("ffi")
 234 ffi.cdef[[
 235 </b><span style="color:#00a000;">typedef struct { uint8_t red, green, blue, alpha; } rgba_pixel;</span><b>
 236 ]]</b>
 237
 238 local function image_ramp_green(n)
 239   <b>local img = ffi.new("rgba_pixel[?]", n)</b>
 240   local f = 255/(n-1)
 241   for i=<b>0,n-1</b> do
 242     <b>img[i].green = i*f</b>
 243     <b>img[i].alpha = 255</b>
 244   end
 245   return img
 246 end
 247
 248 local function image_to_grey(img, n)
 249   for i=<b>0,n-1</b> do
 250     local y = <b>0.3*img[i].red + 0.59*img[i].green + 0.11*img[i].blue</b>
 251     img[i].red = y; img[i].green = y; img[i].blue = y
 252   end
 253 end
 254
 255 local N = 400*400
 256 local img = image_ramp_green(N)
 257 for i=1,1000 do
 258   image_to_grey(img, N)
 259 end
 260 </pre>
 261 <p>
 262 Ok, so that wasn't too difficult:
 263 </p>
 264 <p>
 265 <span class="mark">&#9312;</span> First, load the FFI
 266 library and declare the low-level data type. Here we choose a
 267 <tt>struct</tt> which holds four byte fields, one for each component
 268 of a 4x8&nbsp;bit RGBA pixel.
 269 </p>
 270 <p>
 271 <span class="mark">&#9313;</span> Creating the data
 272 structure with <tt>ffi.new()</tt> is straightforward &mdash; the
 273 <tt>'?'</tt> is a placeholder for the number of elements of a
 274 variable-length array.
 275 </p>
 276 <p>
 277 <span class="mark">&#9314;</span> C&nbsp;arrays are
 278 zero-based, so the indexes have to run from <tt>0</tt> to
 279 <tt>n-1</tt>. One might want to allocate one more element instead to
 280 simplify converting legacy code.
 281 </p>
 282 <p>
 283 <span class="mark">&#9315;</span> Since <tt>ffi.new()</tt>
 284 zero-fills the array by default, we only need to set the green and the
 285 alpha fields.
 286 </p>
 287 <p>
 288 <span class="mark">&#9316;</span> The calls to
 289 <tt>math.floor()</tt> can be omitted here, because floating-point
 290 numbers are already truncated towards zero when converting them to an
 291 integer. This happens implicitly when the number is stored in the
 292 fields of each pixel.
 293 </p>
 294 <p>
 295 Now let's have a look at the impact of the changes: first, memory
 296 consumption for the image is down from 22&nbsp;Megabytes to
 297 640&nbsp;Kilobytes (400*400*4 bytes). That's a factor of 35x less! So,
 298 yes, tables do have a noticeable overhead. BTW: The original program
 299 would consume 40&nbsp;Megabytes in plain Lua (on x64).
 300 </p>
 301 <p>
 302 Next, performance: the pure Lua version runs in 9.57 seconds (52.9
 303 seconds with the Lua interpreter) and the FFI version runs in 0.48
 304 seconds on my machine (YMMV). That's a factor of 20x faster (110x
 305 faster than the Lua interpreter).
 306 </p>
 307 <p style="font-size: 8pt;">
 308 The avid reader may notice that converting the pure Lua version over
 309 to use array indexes for the colors (<tt>[1]</tt> instead of
 310 <tt>.red</tt>, <tt>[2]</tt> instead of <tt>.green</tt> etc.) ought to
 311 be more compact and faster. This is certainly true (by a factor of
 312 ~1.7x). Switching to a struct-of-arrays would help, too.
 313 </p>
 314 <p style="font-size: 8pt;">
 315 However the resulting code would be less idiomatic and rather
 316 error-prone. And it still doesn't get even close to the performance of
 317 the FFI version of the code. Also, high-level data structures cannot
 318 be easily passed to other C&nbsp;functions, especially I/O functions,
 319 without undue conversion penalties.
 320 </p>
 321 <br class="flush">
 322 </div>
 323 <div id="foot">
 324 <hr class="hide">
 325 Copyright &copy; 2005-2012 Mike Pall
 326 <span class="noprint">
 327 &middot;
 328 <a href="contact.html">Contact</a>
 329 </span>
 330 </div>
 331 </body>
 332 </html>