1 Block I/O error injection using blkdebug
2 ----------------------------------------
3 Copyright (C) 2014-2015 Red Hat Inc
5 This work is licensed under the terms of the GNU GPL, version 2 or later. See
6 the COPYING file in the top-level directory.
8 The blkdebug block driver is a rule-based error injection engine. It can be
9 used to exercise error code paths in block drivers including ENOSPC (out of
12 This document gives an overview of the features available in blkdebug.
16 Block drivers have many error code paths that handle I/O errors. Image formats
17 are especially complex since metadata I/O errors during cluster allocation or
18 while updating tables happen halfway through request processing and require
19 discipline to keep image files consistent.
21 Error injection allows test cases to trigger I/O errors at specific points.
22 This way, all error paths can be tested to make sure they are correct.
26 The blkdebug block driver takes a list of "rules" that tell the error injection
27 engine when to fail an I/O request.
29 Each I/O request is evaluated against the rules. If a rule matches the request
30 then its "action" is executed.
32 Rules can be placed in a configuration file; the configuration file
33 follows the same .ini-like format used by QEMU's -readconfig option, and
34 each section of the file represents a rule.
36 The following configuration file defines a single rule:
43 This rule fails all aio read requests with ENOSPC (28). Note that the errno
44 value depends on the host. On Linux, see
45 /usr/include/asm-generic/errno-base.h for errno values.
47 Invoke QEMU as follows:
50 -drive if=none,cache=none,file=blkdebug:blkdebug.conf:test.img,id=drive0 \
51 -device virtio-blk-pci,drive=drive0,id=virtio-blk-pci0
53 Rules support the following attributes:
55 event - which type of operation to match (e.g. read_aio, write_aio,
56 flush_to_os, flush_to_disk). See the "Events" section for
57 information on events.
59 state - (optional) the engine must be in this state number in order for this
60 rule to match. See the "State transitions" section for information
63 errno - the numeric errno value to return when a request matches this rule.
64 The errno values depend on the host since the numeric values are not
65 standarized in the POSIX specification.
67 sector - (optional) a sector number that the request must overlap in order to
70 once - (optional, default "off") only execute this action on the first
73 immediately - (optional, default "off") return a NULL BlockAIOCB
74 pointer and fail without an errno instead. This
75 exercises the code path where BlockAIOCB fails and the
76 caller's BlockCompletionFunc is not invoked.
80 Block drivers provide information about the type of I/O request they are about
81 to make so rules can match specific types of requests. For example, the qcow2
82 block driver tells blkdebug when it accesses the L1 table so rules can match
83 only L1 table accesses and not other metadata or guest data requests.
87 read_aio - guest data read
89 write_aio - guest data write
91 flush_to_os - write out unwritten block driver state (e.g. cached metadata)
93 flush_to_disk - flush the host block device's disk cache
95 See qapi/block-core.json:BlkdebugEvent for the full list of events.
96 You may need to grep block driver source code to understand the
97 meaning of specific events.
101 There are cases where more power is needed to match a particular I/O request in
102 a longer sequence of requests. For example:
108 How do we match the 2nd write_aio but not the first? This is where state
111 The error injection engine has an integer called the "state" that always starts
112 initialized to 1. The state integer is internal to blkdebug and cannot be
113 observed from outside but rules can interact with it for powerful matching
116 Rules can be conditional on the current state and they can transition to a new
119 When a rule's "state" attribute is non-zero then the current state must equal
120 the attribute in order for the rule to match.
122 For example, to match the 2nd write_aio:
134 The first write_aio request matches the set-state rule and transitions from
135 state 1 to state 2. Once state 2 has been entered, the set-state rule no
136 longer matches since it requires state 1. But the inject-error rule now
137 matches the next write_aio request and injects EIO (5).
139 State transition rules support the following attributes:
141 event - which type of operation to match (e.g. read_aio, write_aio,
142 flush_to_os, flush_to_disk). See the "Events" section for
143 information on events.
145 state - (optional) the engine must be in this state number in order for this
148 new_state - transition to this state number
152 Exercising code paths in block drivers may require specific ordering amongst
153 concurrent requests. The "breakpoint" feature allows requests to be halted on
154 a blkdebug event and resumed later. This makes it possible to achieve
155 deterministic ordering when multiple requests are in flight.
157 Breakpoints on blkdebug events are associated with a user-defined "tag" string.
158 This tag serves as an identifier by which the request can be resumed at a later
161 See the qemu-io(1) break, resume, remove_break, and wait_break commands for