1 A guide to benchmarking nbdkit
2 ==============================
8 * The plugin matters! Different plugins have completely different
9 uses, implementations and threading models. There is little point
10 in talking generically about “the performance of nbdkit” without
11 mentioning what plugin you are testing.
13 * The client matters! Does the client support multi-conn? Does the
14 client use the oldstyle or newstyle protocol? Has the client been
15 written with performance in mind? The currently best clients are
16 (a) the Linux kernel (nbd.ko), (b) qemu, and (c) fio. Make sure you
17 are using recent versions and have multi-conn enabled.
19 * Filters affect performance. When benchmarking you should never use
20 filters unless filters are what you are trying to benchmark.
26 FIO is a Flexible I/O tester written by Jens Axboe, and it is the
27 primary tool used for generating the load to test filesystems and
30 (1) Clone and compile fio:
32 https://github.com/axboe/fio
34 (2) Add the NBD engine patch. The current version is linked below,
35 although we are hoping it will eventually get upstream:
37 https://www.spinics.net/lists/fio/msg07832.html
39 (3) Edit the test file in examples/nbd.fio, if required.
41 (4) Run nbdkit and fio together. From the fio source directory:
44 nbdkit -f -U /tmp/socket null 1G --run './fio examples/nbd.fio'
46 If you want to use nbdkit from the source directory too, change
47 ‘nbdkit’ to the path of the wrapper, eg:
50 ../nbdkit/nbdkit -f -U /tmp/socket null 1G --run './fio examples/nbd.fio'
56 * Try adjusting the number of fio jobs (threads).
58 * Try adjusting the number of nbdkit threads (nbdkit -t option).
60 * Use other plugins. Both nbdkit-memory-plugin and nbdkit-file-plugin
61 are important ones to test.
63 * Run nbdkit under perf:
65 perf record -a -g --call-graph=dwarf -- \
66 server/nbdkit -f -U /tmp/socket \
67 ./plugins/null/.libs/nbdkit-null-plugin.so 1G
70 Testing using the Linux kernel client
71 -------------------------------------
73 Step (1) is the same as above - obtain or compile fio.
75 (2) Create the fio configuation file.
77 Create /var/tmp/test.fio containing:
79 ----------------------------------------------------------------------
83 directory=/var/tmp/nbd
91 ----------------------------------------------------------------------
95 From the nbdkit source directory:
98 ./nbdkit -f -U /tmp/socket memory 1G
100 (4) Loop mount the NBD server:
103 nbd-client -C 8 -b 512 -unix /tmp/socket /dev/nbd0
104 mkfs.xfs -f /dev/nbd0
106 mount /dev/nbd0 /var/tmp/nbd
108 (5) Run the fio test:
110 fio /var/tmp/test.fio