doc/ext_ffi.html

   1 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
   2 <html>
   3 <head>
   4 <title>FFI Library</title>
   5 <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
   6 <meta name="Author" content="Mike Pall">
   7 <meta name="Copyright" content="Copyright (C) 2005-2012, Mike Pall">
   8 <meta name="Language" content="en">
   9 <link rel="stylesheet" type="text/css" href="bluequad.css" media="screen">
  10 <link rel="stylesheet" type="text/css" href="bluequad-print.css" media="print">
  11 <style type="text/css">
  12 span.codemark { position:absolute; left: 16em; color: #4040c0; }
  13 span.mark { color: #4040c0; font-family: Courier New, Courier, monospace;
  14   line-height: 1.1; }
  15 pre.mark { padding-left: 2em; }
  16 </style>
  17 </head>
  18 <body>
  19 <div id="site">
  20 <a href="http://luajit.org"><span>Lua<span id="logo">JIT</span></span></a>
  21 </div>
  22 <div id="head">
  23 <h1>FFI Library</h1>
  24 </div>
  25 <div id="nav">
  26 <ul><li>
  27 <a href="luajit.html">LuaJIT</a>
  28 <ul><li>
  29 <a href="install.html">Installation</a>
  30 </li><li>
  31 <a href="running.html">Running</a>
  32 </li></ul>
  33 </li><li>
  34 <a href="extensions.html">Extensions</a>
  35 <ul><li>
  36 <a class="current" href="ext_ffi.html">FFI Library</a>
  37 <ul><li>
  38 <a href="ext_ffi_tutorial.html">FFI Tutorial</a>
  39 </li><li>
  40 <a href="ext_ffi_api.html">ffi.* API</a>
  41 </li><li>
  42 <a href="ext_ffi_semantics.html">FFI Semantics</a>
  43 </li></ul>
  44 </li><li>
  45 <a href="ext_jit.html">jit.* Library</a>
  46 </li><li>
  47 <a href="ext_c_api.html">Lua/C API</a>
  48 </li></ul>
  49 </li><li>
  50 <a href="status.html">Status</a>
  51 <ul><li>
  52 <a href="changes.html">Changes</a>
  53 </li></ul>
  54 </li><li>
  55 <a href="faq.html">FAQ</a>
  56 </li><li>
  57 <a href="http://luajit.org/performance.html">Performance <span class="ext">&raquo;</span></a>
  58 </li><li>
  59 <a href="http://luajit.org/download.html">Download <span class="ext">&raquo;</span></a>
  60 </li><li>
  61 <a href="http://wiki.luajit.org/">Wiki <span class="ext">&raquo;</span></a>
  62 </li><li>
  63 <a href="http://luajit.org/list.html">Mailing List <span class="ext">&raquo;</span></a>
  64 </li></ul>
  65 </div>
  66 <div id="main">
  67 <p>
  68
  69 The FFI library allows <b>calling external C&nbsp;functions</b> and
  70 <b>using C&nbsp;data structures</b> from pure Lua code.
  71
  72 </p>
  73 <p>
  74
  75 The FFI library largely obviates the need to write tedious manual
  76 Lua/C bindings in C. No need to learn a separate binding language
  77 &mdash; <b>it parses plain C&nbsp;declarations!</b> These can be
  78 cut-n-pasted from C&nbsp;header files or reference manuals. It's up to
  79 the task of binding large libraries without the need for dealing with
  80 fragile binding generators.
  81
  82 </p>
  83 <p>
  84 The FFI library is tightly integrated into LuaJIT (it's not available
  85 as a separate module). The code generated by the JIT-compiler for
  86 accesses to C&nbsp;data structures from Lua code is on par with the
  87 code a C&nbsp;compiler would generate. Calls to C&nbsp;functions can
  88 be inlined in JIT-compiled code, unlike calls to functions bound via
  89 the classic Lua/C API.
  90 </p>
  91 <p>
  92 This page gives a short introduction to the usage of the FFI library.
  93 <em>Please use the FFI sub-topics in the navigation bar to learn more.</em>
  94 </p>
  95
  96 <h2 id="call">Motivating Example: Calling External C Functions</h2>
  97 <p>
  98 It's really easy to call an external C&nbsp;library function:
  99 </p>
 100 <pre class="code mark">
 101 <span class="codemark">&#9312;
 102 &#9313;
 103
 104
 105 &#9314;</span>local ffi = require("ffi")
 106 ffi.cdef[[
 107 <span style="color:#00a000;">int printf(const char *fmt, ...);</span>
 108 ]]
 109 ffi.C.printf("Hello %s!", "world")
 110 </pre>
 111 <p>
 112 So, let's pick that apart:
 113 </p>
 114 <p>
 115 <span class="mark">&#9312;</span> Load the FFI library.
 116 </p>
 117 <p>
 118 <span class="mark">&#9313;</span> Add a C&nbsp;declaration
 119 for the function. The part inside the double-brackets (in green) is
 120 just standard C&nbsp;syntax.
 121 </p>
 122 <p>
 123 <span class="mark">&#9314;</span> Call the named
 124 C&nbsp;function &mdash; Yes, it's that simple!
 125 </p>
 126 <p style="font-size: 8pt;">
 127 Actually, what goes on behind the scenes is far from simple: <span
 128 style="color:#4040c0;">&#9314;</span> makes use of the standard
 129 C&nbsp;library namespace <tt>ffi.C</tt>. Indexing this namespace with
 130 a symbol name (<tt>"printf"</tt>) automatically binds it to the
 131 standard C&nbsp;library. The result is a special kind of object which,
 132 when called, runs the <tt>printf</tt> function. The arguments passed
 133 to this function are automatically converted from Lua objects to the
 134 corresponding C&nbsp;types.
 135 </p>
 136 <p>
 137 Ok, so maybe the use of <tt>printf()</tt> wasn't such a spectacular
 138 example. You could have done that with <tt>io.write()</tt> and
 139 <tt>string.format()</tt>, too. But you get the idea ...
 140 </p>
 141 <p>
 142 So here's something to pop up a message box on Windows:
 143 </p>
 144 <pre class="code">
 145 local ffi = require("ffi")
 146 ffi.cdef[[
 147 <span style="color:#00a000;">int MessageBoxA(void *w, const char *txt, const char *cap, int type);</span>
 148 ]]
 149 ffi.C.MessageBoxA(nil, "Hello world!", "Test", 0)
 150 </pre>
 151 <p>
 152 Bing! Again, that was far too easy, no?
 153 </p>
 154 <p style="font-size: 8pt;">
 155 Compare this with the effort required to bind that function using the
 156 classic Lua/C API: create an extra C&nbsp;file, add a C&nbsp;function
 157 that retrieves and checks the argument types passed from Lua and calls
 158 the actual C&nbsp;function, add a list of module functions and their
 159 names, add a <tt>luaopen_*</tt> function and register all module
 160 functions, compile and link it into a shared library (DLL), move it to
 161 the proper path, add Lua code that loads the module aaaand ... finally
 162 call the binding function. Phew!
 163 </p>
 164
 165 <h2 id="cdata">Motivating Example: Using C Data Structures</h2>
 166 <p>
 167 The FFI library allows you to create and access C&nbsp;data
 168 structures. Of course the main use for this is for interfacing with
 169 C&nbsp;functions. But they can be used stand-alone, too.
 170 </p>
 171 <p>
 172 Lua is built upon high-level data types. They are flexible, extensible
 173 and dynamic. That's why we all love Lua so much. Alas, this can be
 174 inefficient for certain tasks, where you'd really want a low-level
 175 data type. E.g. a large array of a fixed structure needs to be
 176 implemented with a big table holding lots of tiny tables. This imposes
 177 both a substantial memory overhead as well as a performance overhead.
 178 </p>
 179 <p>
 180 Here's a sketch of a library that operates on color images plus a
 181 simple benchmark. First, the plain Lua version:
 182 </p>
 183 <pre class="code">
 184 local floor = math.floor
 185
 186 local function image_ramp_green(n)
 187   local img = {}
 188   local f = 255/(n-1)
 189   for i=1,n do
 190     img[i] = { red = 0, green = floor((i-1)*f), blue = 0, alpha = 255 }
 191   end
 192   return img
 193 end
 194
 195 local function image_to_grey(img, n)
 196   for i=1,n do
 197     local y = floor(0.3*img[i].red + 0.59*img[i].green + 0.11*img[i].blue)
 198     img[i].red = y; img[i].green = y; img[i].blue = y
 199   end
 200 end
 201
 202 local N = 400*400
 203 local img = image_ramp_green(N)
 204 for i=1,1000 do
 205   image_to_grey(img, N)
 206 end
 207 </pre>
 208 <p>
 209 This creates a table with 160.000 pixels, each of which is a table
 210 holding four number values in the range of 0-255. First an image with
 211 a green ramp is created (1D for simplicity), then the image is
 212 converted to greyscale 1000 times. Yes, that's silly, but I was in
 213 need of a simple example ...
 214 </p>
 215 <p>
 216 And here's the FFI version. The modified parts have been marked in
 217 bold:
 218 </p>
 219 <pre class="code mark">
 220 <span class="codemark">&#9312;
 221
 222
 223
 224
 225
 226 &#9313;
 227
 228 &#9314;
 229 &#9315;
 230
 231
 232
 233
 234
 235
 236 &#9314;
 237 &#9316;</span><b>local ffi = require("ffi")
 238 ffi.cdef[[
 239 </b><span style="color:#00a000;">typedef struct { uint8_t red, green, blue, alpha; } rgba_pixel;</span><b>
 240 ]]</b>
 241
 242 local function image_ramp_green(n)
 243   <b>local img = ffi.new("rgba_pixel[?]", n)</b>
 244   local f = 255/(n-1)
 245   for i=<b>0,n-1</b> do
 246     <b>img[i].green = i*f</b>
 247     <b>img[i].alpha = 255</b>
 248   end
 249   return img
 250 end
 251
 252 local function image_to_grey(img, n)
 253   for i=<b>0,n-1</b> do
 254     local y = <b>0.3*img[i].red + 0.59*img[i].green + 0.11*img[i].blue</b>
 255     img[i].red = y; img[i].green = y; img[i].blue = y
 256   end
 257 end
 258
 259 local N = 400*400
 260 local img = image_ramp_green(N)
 261 for i=1,1000 do
 262   image_to_grey(img, N)
 263 end
 264 </pre>
 265 <p>
 266 Ok, so that wasn't too difficult:
 267 </p>
 268 <p>
 269 <span class="mark">&#9312;</span> First, load the FFI
 270 library and declare the low-level data type. Here we choose a
 271 <tt>struct</tt> which holds four byte fields, one for each component
 272 of a 4x8&nbsp;bit RGBA pixel.
 273 </p>
 274 <p>
 275 <span class="mark">&#9313;</span> Creating the data
 276 structure with <tt>ffi.new()</tt> is straightforward &mdash; the
 277 <tt>'?'</tt> is a placeholder for the number of elements of a
 278 variable-length array.
 279 </p>
 280 <p>
 281 <span class="mark">&#9314;</span> C&nbsp;arrays are
 282 zero-based, so the indexes have to run from <tt>0</tt> to
 283 <tt>n-1</tt>. One might want to allocate one more element instead to
 284 simplify converting legacy code.
 285 </p>
 286 <p>
 287 <span class="mark">&#9315;</span> Since <tt>ffi.new()</tt>
 288 zero-fills the array by default, we only need to set the green and the
 289 alpha fields.
 290 </p>
 291 <p>
 292 <span class="mark">&#9316;</span> The calls to
 293 <tt>math.floor()</tt> can be omitted here, because floating-point
 294 numbers are already truncated towards zero when converting them to an
 295 integer. This happens implicitly when the number is stored in the
 296 fields of each pixel.
 297 </p>
 298 <p>
 299 Now let's have a look at the impact of the changes: first, memory
 300 consumption for the image is down from 22&nbsp;Megabytes to
 301 640&nbsp;Kilobytes (400*400*4 bytes). That's a factor of 35x less! So,
 302 yes, tables do have a noticeable overhead. BTW: The original program
 303 would consume 40&nbsp;Megabytes in plain Lua (on x64).
 304 </p>
 305 <p>
 306 Next, performance: the pure Lua version runs in 9.57 seconds (52.9
 307 seconds with the Lua interpreter) and the FFI version runs in 0.48
 308 seconds on my machine (YMMV). That's a factor of 20x faster (110x
 309 faster than the Lua interpreter).
 310 </p>
 311 <p style="font-size: 8pt;">
 312 The avid reader may notice that converting the pure Lua version over
 313 to use array indexes for the colors (<tt>[1]</tt> instead of
 314 <tt>.red</tt>, <tt>[2]</tt> instead of <tt>.green</tt> etc.) ought to
 315 be more compact and faster. This is certainly true (by a factor of
 316 ~1.7x). Switching to a struct-of-arrays would help, too.
 317 </p>
 318 <p style="font-size: 8pt;">
 319 However the resulting code would be less idiomatic and rather
 320 error-prone. And it still doesn't get even close to the performance of
 321 the FFI version of the code. Also, high-level data structures cannot
 322 be easily passed to other C&nbsp;functions, especially I/O functions,
 323 without undue conversion penalties.
 324 </p>
 325 <br class="flush">
 326 </div>
 327 <div id="foot">
 328 <hr class="hide">
 329 Copyright &copy; 2005-2012 Mike Pall
 330 <span class="noprint">
 331 &middot;
 332 <a href="contact.html">Contact</a>
 333 </span>
 334 </div>
 335 </body>
 336 </html>