2 ======================================================================
4 General ideas for improvements
5 ------------------------------
7 * Listen on specific interfaces or protocols (eg. only IPv6).
9 * Performance - measure and improve it.
11 * Exit on last connection (the default behaviour of qemu-nbd unless
14 * Limit number of incoming connections (like qemu-nbd -e).
16 * For parallel plugins, only create threads on demand from parallel
17 client requests, rather than pre-creating all threads at connection
18 time, up to the thread pool size limit. Of course, once created, a
19 thread is reused as possible until the connection closes.
21 * Async callbacks. The current parallel support requires one thread
22 per pending message; a solution with fewer threads would split
23 low-level code between request and response, where the callback has
24 to inform nbdkit when the response is ready:
25 https://www.redhat.com/archives/libguestfs/2018-January/msg00149.html
27 * More NBD protocol features. The currently missing features are
28 structured replies for sparse reads, block size constraints, and
31 * Add a callback to let plugins request minimum alignment for the
32 buffer to pread/pwrite; useful for a plugin utilizing O_DIRECT or
33 other situation where pre-aligned buffers are more efficient.
34 Ideally, a blocksize filter would honor strict alignment below and
35 advertise loose alignment above; all other filters (particularly
36 ones like offset) can fail to initialize if they can't guarantee
37 strict alignment and don't want to deal with bounce buffers.
39 * Test that zero-length read/write/extents requests behave sanely
40 (NBD protocol says they are unspecified).
42 * If a client negotiates structured replies, and issues a read/extents
43 call that exceeds EOF (qemu 3.1 is one such client, when nbdkit
44 serves non-sector-aligned images), return the valid answer for the
45 subset of the request in range and then NBD_REPLY_TYPE_ERROR_OFFSET
46 for the tail, rather than erroring the entire request.
48 * Test and document how to run nbdkit from inetd and xinetd in
51 * Audit the code base to get rid of strerror() usage (the function is
52 not thread-safe); however, using geterror_r() can be tricky as it
53 has a different signature in glibc than in POSIX.
55 * Teach nbdkit_error() to have smart newline appending (for existing
56 inconsistent clients), while fixing internal uses to omit the
57 newline. Commit ef4f72ef has some ideas on smart newlines, but that
58 should probably be factored into a utility function.
60 * We may need a way to lock nbdkit into memory and adjust the OOM
61 killer score. See this LKML discussion:
62 https://www.mail-archive.com/linux-kernel@vger.kernel.org/msg1933394.html
63 and also look at the implementation of the -swap option in
66 * Add a mode of operation where nbdkit is handed a pre-opened fd to be
67 used immediately in transmission phase (skipping handshake). There
68 are already third-party clients of the kernel's /dev/nbdX which rely
69 on their own protocol instead of NBD handshake, before calling
70 ioctl(NBD_SET_SOCK); this mode would let the third-party client
71 continue to keep their non-standard handshake while utilizing nbdkit
72 to prototype new behaviors in serving the kernel.
74 * Clients should be able to list export names supported by plugins.
75 Current behaviour is not really correct: We only list the -e
76 parameter from the command line, which is different from the export
77 name(s) that a plugin might want to support. Probably we should
78 deprecate the -e option entirely since it does nothing useful.
80 Suggestions for plugins
81 -----------------------
83 Note: qemu supports other formats such as libnfs, iscsi, gluster and
84 ceph/rbd, and while similar plugins could be written for nbdkit there
85 is no compelling reason unless the result is better than qemu-nbd.
86 For the majority of users it would be better if they were directed to
87 qemu-nbd for these use cases.
91 https://lists.gnu.org/archive/html/qemu-devel/2017-11/msg02971.html
92 is a partial solution but it needs cleaning up.
96 * Add boot sector support. In theory this is easy (eg. using
97 SYSLINUX), but the practical reality of making a fully bootable
98 floppy is rather more complex.
100 * Add multiple dir merging.
102 nbdkit-linuxdisk-plugin:
104 * Add multiple dir merging (in e2fsprogs mke2fs).
106 Suggestions for filters
107 -----------------------
109 * tar plugin should really be a filter
111 * gzip plugin should really be a filter
113 * libarchive could be used to implement a general tar/zip filter
115 * LUKS encrypt/decrypt filter, bonus points if compatible with qemu
116 LUKS-encrypted disk images
118 * masking plugin features for testing clients (see 'nozero' and 'fua'
119 filters for examples)
121 * nbdkit-cache-filter should handle ENOSPC errors automatically by
122 reclaiming blocks from the cache
126 * allow other kinds of traffic shaping such as VBR
128 * limit traffic per client (ie. per IP address)
130 * split large requests to avoid long, lumpy sleeps when request size
131 is much larger than rate limit
136 Things like blacklisting or whitelisting IP addresses can be done
137 using external wrappers (TCP wrappers, systemd).
139 However it might be nice to have a configurable filter for preventing
140 valid but not sensible requests. The server already filters invalid
141 requests. This would be like seccomp, and could be implemented using
142 an eBPF-based parser. Unfortunately actual eBPF is difficult to use
143 for userspace processes. The "standard" isn't solidly defined - the
144 Linux kernel implementation is the standard - and Linux has by far the
145 best implementation, particularly around bytecode verification and
146 JITting. There is a userspace VM (ubpf) but it has very limited
147 capabilities compared to Linux.
152 Filters allow certain types of composition, but others would not be
153 possible, for example RAIDing over multiple nbd sources. Because the
154 plugin API limits us to loading a single plugin to the server, the
155 best way to do this (and the most robust) is to compose multiple
156 nbdkit processes. Perhaps libnbd will prove useful for this purpose.
161 * Figure out how to get 'make distcheck' working. VPATH builds are
162 working, but various pkg-config results that try to stick
163 bash-completion and ocaml add-ons into their system-wide home do
164 not play nicely with --prefix builds for a non-root user.
171 * Consider supporting a more idiomatic style for writing Rust plugins.
173 * Better documentation.
177 * There is no attempt to ‘make install’ or otherwise package the
178 crate. Since it looks as if Rust code is normally distributed as
179 source it's not clear what that would even mean.
184 From time to time we may update the plugin protocol. This section
185 collects ideas for things which might be fixed in the next version of
188 Note that we keep the old protocol(s) around so that source
189 compatibility is retained. Plugins must opt in to the new protocol
190 using ‘#define NBDKIT_API_VERSION <version>’.
192 * All methods taking a ‘count’ field should be uint64_t (instead of
193 uint32_t). Although the NBD protocol does not support 64 bit
194 lengths, it might do in future.
196 * pread could be changed to allow it to support Structured Replies
197 (SRs). This could mean allowing it to return partial data, holes,
198 zeroes, etc. For a client that negotiates SR coupled with a plugin
199 that supports .extents, the v2 protocol would allow us to at least
200 synthesize NBD_REPLY_TYPE_OFFSET_HOLE for less network traffic, even
201 though the plugin will still have to fully populate the .pread
202 buffer; the v3 protocol should make sparse reads more direct.
204 * Parameters should be systematized so that they aren't just (key,
205 value) strings. nbdkit should know the possible keys for the plugin
206 and filters, and the type of the values, and both check and parse
209 * Modify open() API so it takes an export name parameter.