| blob | 17d2413b6a1976202f73c6436a9a245eb4f31286 |
1 To-do list for nbdkit
2 ======================================================================
4 General ideas for improvements
5 ------------------------------
7 * Listen on specific interfaces or protocols.
9 * Performance - measure and improve it. Chart it over various buffer
10 sizes and threads, as that should make it easier to identify
11 systematic issues.
13 * Exit on last connection (the default behaviour of qemu-nbd unless
14 you use -t).
16 * Limit number of incoming connections (like qemu-nbd -e).
18 * For parallel plugins, only create threads on demand from parallel
19 client requests, rather than pre-creating all threads at connection
20 time, up to the thread pool size limit. Of course, once created, a
21 thread is reused as possible until the connection closes.
23 * Async callbacks. The current parallel support requires one thread
24 per pending message; a solution with fewer threads would split
25 low-level code between request and response, where the callback has
26 to inform nbdkit when the response is ready:
27 https://www.redhat.com/archives/libguestfs/2018-January/msg00149.html
29 * More NBD protocol features. The currently missing features are
30 structured replies for sparse reads, block size constraints, and
31 online resize.
33 * Add a callback to let plugins request minimum alignment for the
34 buffer to pread/pwrite; useful for a plugin utilizing O_DIRECT or
35 other situation where pre-aligned buffers are more efficient.
36 Ideally, a blocksize filter would honor strict alignment below and
37 advertise loose alignment above; all other filters (particularly
38 ones like offset) can fail to initialize if they can't guarantee
39 strict alignment and don't want to deal with bounce buffers.
41 * Test that zero-length read/write/extents requests behave sanely
42 (NBD protocol says they are unspecified).
44 * If a client negotiates structured replies, and issues a read/extents
45 call that exceeds EOF (qemu 3.1 is one such client, when nbdkit
46 serves non-sector-aligned images), return the valid answer for the
47 subset of the request in range and then NBD_REPLY_TYPE_ERROR_OFFSET
48 for the tail, rather than erroring the entire request.
50 * Test and document how to run nbdkit from inetd and xinetd in
51 nbdkit-service(1).
53 * Audit the code base to get rid of strerror() usage (the function is
54 not thread-safe); however, using geterror_r() can be tricky as it
55 has a different signature in glibc than in POSIX.
57 * Teach nbdkit_error() to have smart newline appending (for existing
58 inconsistent clients), while fixing internal uses to omit the
59 newline. Commit ef4f72ef has some ideas on smart newlines, but that
60 should probably be factored into a utility function.
62 * Add a mode of operation where nbdkit is handed a pre-opened fd to be
63 used immediately in transmission phase (skipping handshake). There
64 are already third-party clients of the kernel's /dev/nbdX which rely
65 on their own protocol instead of NBD handshake, before calling
66 ioctl(NBD_SET_SOCK); this mode would let the third-party client
67 continue to keep their non-standard handshake while utilizing nbdkit
68 to prototype new behaviors in serving the kernel.
70 * Clients should be able to list export names supported by plugins.
71 Current behaviour is not really correct: We only list the -e
72 parameter from the command line, which is different from the export
73 name(s) that a plugin might want to support. Probably we should
74 deprecate the -e option entirely since it does nothing useful.
76 * Background thread for filters. Some filters (readahead, cache and
77 proposed scan filter - see below) could be more effective if they
78 were able to defer work to a background thread. However it's not as
79 simple as just creating a background thread, because you only have
80 access to the connection / next_ops from the context of a filter
81 callback. (Also you can't "save" next_ops because it becomes
82 invalid outside the callback). The background thread would need to
83 have its own connection to the plugin, which would be independent of
84 any client connection, and this requires some care because it breaks
85 an existing assumption.
87 * Add scan filter. This would be placed on top of cache filters and
88 would scan (read) the whole disk in the background, ensuring it is
89 copied into the cache. Useful if you have a slow plugin, limited
90 size device, and lots of local disk space, especially if you know
91 that the NBD clients will eventually read all of the device. RWMJ
92 wrote an implementation of this but it doesn't work well without a
93 background thread.
95 * "nbdkit.so": nbdkit as a loadable shared library. The aim of nbdkit
96 is to make it reusable from other programs (see nbdkit-captive(1)).
97 If it was a loadable shared library it would be even more reusable.
98 API would allow you to create an nbdkit instance, configure it (same
99 as the current command line), start it serving on a socket, etc.
100 However perhaps the current ability to work well as a subprocess is
101 good enough? Also allowing multiple instances of nbdkit to be
102 loaded in the same process is probably impossible.
104 * Examine other fuzzers: https://gitlab.com/akihe/radamsa
106 * Analyze code with Coverity.
108 Suggestions for plugins
109 -----------------------
111 Note: qemu supports other formats such as libnfs, iscsi, gluster and
112 ceph/rbd, and while similar plugins could be written for nbdkit there
113 is no compelling reason unless the result is better than qemu-nbd.
114 For the majority of users it would be better if they were directed to
115 qemu-nbd for these use cases.
117 * XVA files
119 https://lists.gnu.org/archive/html/qemu-devel/2017-11/msg02971.html
120 is a partial solution but it needs cleaning up.
122 nbdkit-floppy-plugin:
124 * Add boot sector support. In theory this is easy (eg. using
125 SYSLINUX), but the practical reality of making a fully bootable
126 floppy is rather more complex.
128 * Add multiple dir merging.
130 nbdkit-linuxdisk-plugin:
132 * Add multiple dir merging (in e2fsprogs mke2fs).
134 Suggestions for language plugins
135 --------------------------------
137 Python:
139 * Get the __docstring__ from the module and print it in --help output.
140 This requires changes to the core API so that config_help is a
141 function rather than a variable (see V3 suggestions below).
143 Rust:
145 * Consider supporting a more idiomatic style for writing Rust plugins.
147 * Better documentation.
149 * Add tests.
151 * There is no attempt to ‘make install’ or otherwise package the
152 crate. Since it looks as if Rust code is normally distributed as
153 source it's not clear what that would even mean.
155 Suggestions for filters
156 -----------------------
158 * tar plugin should really be a filter
160 * gzip plugin should really be a filter
162 * libarchive could be used to implement a general tar/zip filter
164 * LUKS encrypt/decrypt filter, bonus points if compatible with qemu
165 LUKS-encrypted disk images
167 * masking plugin features for testing clients (see 'nozero' and 'fua'
168 filters for examples)
170 * nbdkit-cache-filter should handle ENOSPC errors automatically by
171 reclaiming blocks from the cache
173 * zstd filter was requested as a way to do what we currently do with
174 xz but saving many hours on compression (at the cost of hundreds of
175 MBs of extra data)
176 https://github.com/facebook/zstd/issues/395#issuecomment-535875379
178 nbdkit-rate-filter:
180 * allow other kinds of traffic shaping such as VBR
182 * limit traffic per client (ie. per IP address)
184 * split large requests to avoid long, lumpy sleeps when request size
185 is much larger than rate limit
187 nbdkit-retry-filter:
189 * allow user to specify which errors cause a retry and which ones are
190 passed through; for example there's probably no point retrying on
191 ENOMEM
193 * implement a softer mode (retry-reopen=no) where we don't reopen the
194 plugin, we just retry the data command that failed
196 * there are all kinds of extra complications possible here,
197 eg. specifying a pattern of retrying and reopening:
198 retry-method=RRRORRRRRORRRRR meaning to retry the data command 3
199 times, reopen, retry 5 times, etc.
201 * subsecond times
203 nbdkit-ip-filter:
205 * permit hostnames and hostname wildcards to be used in the
206 allow and deny lists
208 * the allow and deny lists should be updatable while nbdkit is
209 running, for example by storing them in a database file
211 Filters for security
212 --------------------
214 Things like blacklisting or whitelisting IP addresses can be done
215 using external wrappers (TCP wrappers, systemd), or nbdkit-ip-filter.
217 However it might be nice to have a configurable filter for preventing
218 valid but not sensible requests. The server already filters invalid
219 requests. This would be like seccomp, and could be implemented using
220 an eBPF-based parser. Unfortunately actual eBPF is difficult to use
221 for userspace processes. The "standard" isn't solidly defined - the
222 Linux kernel implementation is the standard - and Linux has by far the
223 best implementation, particularly around bytecode verification and
224 JITting. There is a userspace VM (ubpf) but it has very limited
225 capabilities compared to Linux.
227 Composing nbdkit
228 ----------------
230 Filters allow certain types of composition, but others would not be
231 possible, for example RAIDing over multiple nbd sources. Because the
232 plugin API limits us to loading a single plugin to the server, the
233 best way to do this (and the most robust) is to compose multiple
234 nbdkit processes. Perhaps libnbd will prove useful for this purpose.
236 Build-related
237 -------------
239 * Figure out how to get 'make distcheck' working. VPATH builds are
240 working, but various pkg-config results that try to stick
241 bash-completion and ocaml add-ons into their system-wide home do
242 not play nicely with --prefix builds for a non-root user.
244 * Port to Windows.
246 V3 plugin protocol
247 ------------------
249 From time to time we may update the plugin protocol. This section
250 collects ideas for things which might be fixed in the next version of
251 the protocol.
253 Note that we keep the old protocol(s) around so that source
254 compatibility is retained. Plugins must opt in to the new protocol
255 using ‘#define NBDKIT_API_VERSION <version>’.
257 * All methods taking a ‘count’ field should be uint64_t (instead of
258 uint32_t). Although the NBD protocol does not support 64 bit
259 lengths, it might do in future.
261 * pread could be changed to allow it to support Structured Replies
262 (SRs). This could mean allowing it to return partial data, holes,
263 zeroes, etc. For a client that negotiates SR coupled with a plugin
264 that supports .extents, the v2 protocol would allow us to at least
265 synthesize NBD_REPLY_TYPE_OFFSET_HOLE for less network traffic, even
266 though the plugin will still have to fully populate the .pread
267 buffer; the v3 protocol should make sparse reads more direct.
269 * Parameters should be systematized so that they aren't just (key,
270 value) strings. nbdkit should know the possible keys for the plugin
271 and filters, and the type of the values, and both check and parse
272 them for the plugin.
274 * Modify open() API so it takes an export name parameter.
276 * Change config_help from a variable to a function.