4 :Copyright: 2014 Virtual Open Systems Sarl.
5 :Copyright: 2019 Intel Corporation
6 :Licence: This work is licensed under the terms of the GNU GPL,
7 version 2 or later. See the COPYING file in the top-level
10 .. contents:: Table of Contents
15 This protocol is aiming to complement the ``ioctl`` interface used to
16 control the vhost implementation in the Linux kernel. It implements
17 the control plane needed to establish virtqueue sharing with a user
18 space process on the same host. It uses communication over a Unix
19 domain socket to share file descriptors in the ancillary data of the
22 The protocol defines 2 sides of the communication, *master* and
23 *slave*. *Master* is the application that shares its virtqueues, in
24 our case QEMU. *Slave* is the consumer of the virtqueues.
26 In the current implementation QEMU is the *master*, and the *slave* is
27 the external process consuming the virtio queues, for example a
28 software Ethernet switch running in user space, such as Snabbswitch,
29 or a block device backend processing read & write to a virtual
30 disk. In order to facilitate interoperability between various backend
31 implementations, it is recommended to follow the :ref:`Backend program
32 conventions <backend_conventions>`.
34 *Master* and *slave* can be either a client (i.e. connecting) or
35 server (listening) in the socket communication.
40 .. Note:: All numbers are in the machine native byte order.
42 A vhost-user message consists of 3 header fields and a payload.
44 +---------+-------+------+---------+
45 | request | flags | size | payload |
46 +---------+-------+------+---------+
51 :request: 32-bit type of the request
53 :flags: 32-bit bit field
55 - Lower 2 bits are the version (currently 0x01)
56 - Bit 2 is the reply flag - needs to be sent on each reply from the slave
57 - Bit 3 is the need_reply flag - see :ref:`REPLY_ACK <reply_ack>` for
60 :size: 32-bit size of the payload
65 Depending on the request type, **payload** can be:
67 A single 64-bit integer
68 ^^^^^^^^^^^^^^^^^^^^^^^
74 :u64: a 64-bit unsigned integer
76 A vring state description
77 ^^^^^^^^^^^^^^^^^^^^^^^^^
83 :index: a 32-bit index
87 A vring address description
88 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
90 +-------+-------+------+------------+------+-----------+-----+
91 | index | flags | size | descriptor | used | available | log |
92 +-------+-------+------+------------+------+-----------+-----+
94 :index: a 32-bit vring index
96 :flags: a 32-bit vring flags
98 :descriptor: a 64-bit ring address of the vring descriptor table
100 :used: a 64-bit ring address of the vring used ring
102 :available: a 64-bit ring address of the vring available ring
104 :log: a 64-bit guest address for logging
106 Note that a ring address is an IOVA if ``VIRTIO_F_IOMMU_PLATFORM`` has
107 been negotiated. Otherwise it is a user address.
109 Memory regions description
110 ^^^^^^^^^^^^^^^^^^^^^^^^^^
112 +-------------+---------+---------+-----+---------+
113 | num regions | padding | region0 | ... | region7 |
114 +-------------+---------+---------+-----+---------+
116 :num regions: a 32-bit number of regions
122 +---------------+------+--------------+-------------+
123 | guest address | size | user address | mmap offset |
124 +---------------+------+--------------+-------------+
126 :guest address: a 64-bit guest address of the region
130 :user address: a 64-bit user address
132 :mmap offset: 64-bit offset where region starts in the mapped memory
134 Single memory region description
135 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
137 +---------+---------------+------+--------------+-------------+
138 | padding | guest address | size | user address | mmap offset |
139 +---------+---------------+------+--------------+-------------+
143 :guest address: a 64-bit guest address of the region
147 :user address: a 64-bit user address
149 :mmap offset: 64-bit offset where region starts in the mapped memory
154 +----------+------------+
155 | log size | log offset |
156 +----------+------------+
158 :log size: size of area used for logging
160 :log offset: offset from start of supplied file descriptor where
161 logging starts (i.e. where guest address 0 would be
167 +------+------+--------------+-------------------+------+
168 | iova | size | user address | permissions flags | type |
169 +------+------+--------------+-------------------+------+
171 :iova: a 64-bit I/O virtual address programmed by the guest
175 :user address: a 64-bit user address
177 :permissions flags: an 8-bit value:
181 - 3: Read/Write access
183 :type: an 8-bit IOTLB message type:
186 - 3: IOTLB invalidate
187 - 4: IOTLB access fail
189 Virtio device config space
190 ^^^^^^^^^^^^^^^^^^^^^^^^^^
192 +--------+------+-------+---------+
193 | offset | size | flags | payload |
194 +--------+------+-------+---------+
196 :offset: a 32-bit offset of virtio device's configuration space
198 :size: a 32-bit configuration space access size in bytes
200 :flags: a 32-bit value:
201 - 0: Vhost master messages used for writeable fields
202 - 1: Vhost master messages used for live migration
204 :payload: Size bytes array holding the contents of the virtio
205 device's configuration space
207 Vring area description
208 ^^^^^^^^^^^^^^^^^^^^^^
210 +-----+------+--------+
211 | u64 | size | offset |
212 +-----+------+--------+
214 :u64: a 64-bit integer contains vring index and flags
216 :size: a 64-bit size of this area
218 :offset: a 64-bit offset of this area from the start of the
219 supplied file descriptor
224 +-----------+-------------+------------+------------+
225 | mmap size | mmap offset | num queues | queue size |
226 +-----------+-------------+------------+------------+
228 :mmap size: a 64-bit size of area to track inflight I/O
230 :mmap offset: a 64-bit offset of this area from the start
231 of the supplied file descriptor
233 :num queues: a 16-bit number of virtqueues
235 :queue size: a 16-bit size of virtqueues
240 In QEMU the vhost-user message is implemented with the following struct:
244 typedef struct VhostUserMsg {
245 VhostUserRequest request;
250 struct vhost_vring_state state;
251 struct vhost_vring_addr addr;
252 VhostUserMemory memory;
254 struct vhost_iotlb_msg iotlb;
255 VhostUserConfig config;
256 VhostUserVringArea area;
257 VhostUserInflight inflight;
259 } QEMU_PACKED VhostUserMsg;
264 The protocol for vhost-user is based on the existing implementation of
265 vhost for the Linux Kernel. Most messages that can be sent via the
266 Unix domain socket implementing vhost-user have an equivalent ioctl to
267 the kernel implementation.
269 The communication consists of *master* sending message requests and
270 *slave* sending message replies. Most of the requests don't require
271 replies. Here is a list of the ones that do:
273 * ``VHOST_USER_GET_FEATURES``
274 * ``VHOST_USER_GET_PROTOCOL_FEATURES``
275 * ``VHOST_USER_GET_VRING_BASE``
276 * ``VHOST_USER_SET_LOG_BASE`` (if ``VHOST_USER_PROTOCOL_F_LOG_SHMFD``)
277 * ``VHOST_USER_GET_INFLIGHT_FD`` (if ``VHOST_USER_PROTOCOL_F_INFLIGHT_SHMFD``)
281 :ref:`REPLY_ACK <reply_ack>`
282 The section on ``REPLY_ACK`` protocol extension.
284 There are several messages that the master sends with file descriptors passed
285 in the ancillary data:
287 * ``VHOST_USER_SET_MEM_TABLE``
288 * ``VHOST_USER_SET_LOG_BASE`` (if ``VHOST_USER_PROTOCOL_F_LOG_SHMFD``)
289 * ``VHOST_USER_SET_LOG_FD``
290 * ``VHOST_USER_SET_VRING_KICK``
291 * ``VHOST_USER_SET_VRING_CALL``
292 * ``VHOST_USER_SET_VRING_ERR``
293 * ``VHOST_USER_SET_SLAVE_REQ_FD``
294 * ``VHOST_USER_SET_INFLIGHT_FD`` (if ``VHOST_USER_PROTOCOL_F_INFLIGHT_SHMFD``)
296 If *master* is unable to send the full message or receives a wrong
297 reply it will close the connection. An optional reconnection mechanism
300 If *slave* detects some error such as incompatible features, it may also
301 close the connection. This should only happen in exceptional circumstances.
303 Any protocol extensions are gated by protocol feature bits, which
304 allows full backwards compatibility on both master and slave. As
305 older slaves don't support negotiating protocol features, a feature
306 bit was dedicated for this purpose::
308 #define VHOST_USER_F_PROTOCOL_FEATURES 30
310 Starting and stopping rings
311 ---------------------------
313 Client must only process each ring when it is started.
315 Client must only pass data between the ring and the backend, when the
318 If ring is started but disabled, client must process the ring without
319 talking to the backend.
321 For example, for a networking device, in the disabled state client
322 must not supply any new RX packets, but must process and discard any
325 If ``VHOST_USER_F_PROTOCOL_FEATURES`` has not been negotiated, the
326 ring is initialized in an enabled state.
328 If ``VHOST_USER_F_PROTOCOL_FEATURES`` has been negotiated, the ring is
329 initialized in a disabled state. Client must not pass data to/from the
330 backend until ring is enabled by ``VHOST_USER_SET_VRING_ENABLE`` with
331 parameter 1, or after it has been disabled by
332 ``VHOST_USER_SET_VRING_ENABLE`` with parameter 0.
334 Each ring is initialized in a stopped state, client must not process
335 it until ring is started, or after it has been stopped.
337 Client must start ring upon receiving a kick (that is, detecting that
338 file descriptor is readable) on the descriptor specified by
339 ``VHOST_USER_SET_VRING_KICK`` or receiving the in-band message
340 ``VHOST_USER_VRING_KICK`` if negotiated, and stop ring upon receiving
341 ``VHOST_USER_GET_VRING_BASE``.
343 While processing the rings (whether they are enabled or not), client
344 must support changing some configuration aspects on the fly.
346 Multiple queue support
347 ----------------------
349 Many devices have a fixed number of virtqueues. In this case the master
350 already knows the number of available virtqueues without communicating with the
353 Some devices do not have a fixed number of virtqueues. Instead the maximum
354 number of virtqueues is chosen by the slave. The number can depend on host
355 resource availability or slave implementation details. Such devices are called
356 multiple queue devices.
358 Multiple queue support allows the slave to advertise the maximum number of
359 queues. This is treated as a protocol extension, hence the slave has to
360 implement protocol features first. The multiple queues feature is supported
361 only when the protocol feature ``VHOST_USER_PROTOCOL_F_MQ`` (bit 0) is set.
363 The max number of queues the slave supports can be queried with message
364 ``VHOST_USER_GET_QUEUE_NUM``. Master should stop when the number of requested
365 queues is bigger than that.
367 As all queues share one connection, the master uses a unique index for each
368 queue in the sent message to identify a specified queue.
370 The master enables queues by sending message ``VHOST_USER_SET_VRING_ENABLE``.
371 vhost-user-net has historically automatically enabled the first queue pair.
373 Slaves should always implement the ``VHOST_USER_PROTOCOL_F_MQ`` protocol
374 feature, even for devices with a fixed number of virtqueues, since it is simple
375 to implement and offers a degree of introspection.
377 Masters must not rely on the ``VHOST_USER_PROTOCOL_F_MQ`` protocol feature for
378 devices with a fixed number of virtqueues. Only true multiqueue devices
379 require this protocol feature.
384 During live migration, the master may need to track the modifications
385 the slave makes to the memory mapped regions. The client should mark
386 the dirty pages in a log. Once it complies to this logging, it may
387 declare the ``VHOST_F_LOG_ALL`` vhost feature.
389 To start/stop logging of data/used ring writes, server may send
390 messages ``VHOST_USER_SET_FEATURES`` with ``VHOST_F_LOG_ALL`` and
391 ``VHOST_USER_SET_VRING_ADDR`` with ``VHOST_VRING_F_LOG`` in ring's
392 flags set to 1/0, respectively.
394 All the modifications to memory pointed by vring "descriptor" should
395 be marked. Modifications to "used" vring should be marked if
396 ``VHOST_VRING_F_LOG`` is part of ring's flags.
398 Dirty pages are of size::
400 #define VHOST_LOG_PAGE 0x1000
402 The log memory fd is provided in the ancillary data of
403 ``VHOST_USER_SET_LOG_BASE`` message when the slave has
404 ``VHOST_USER_PROTOCOL_F_LOG_SHMFD`` protocol feature.
406 The size of the log is supplied as part of ``VhostUserMsg`` which
407 should be large enough to cover all known guest addresses. Log starts
408 at the supplied offset in the supplied file descriptor. The log
409 covers from address 0 to the maximum of guest regions. In pseudo-code,
410 to mark page at ``addr`` as dirty::
412 page = addr / VHOST_LOG_PAGE
413 log[page / 8] |= 1 << page % 8
415 Where ``addr`` is the guest physical address.
417 Use atomic operations, as the log may be concurrently manipulated.
419 Note that when logging modifications to the used ring (when
420 ``VHOST_VRING_F_LOG`` is set for this ring), ``log_guest_addr`` should
421 be used to calculate the log offset: the write to first byte of the
422 used ring is logged at this offset from log start. Also note that this
423 value might be outside the legal guest physical address range
424 (i.e. does not have to be covered by the ``VhostUserMemory`` table), but
425 the bit offset of the last byte of the ring must fall within the size
426 supplied by ``VhostUserLog``.
428 ``VHOST_USER_SET_LOG_FD`` is an optional message with an eventfd in
429 ancillary data, it may be used to inform the master that the log has
432 Once the source has finished migration, rings will be stopped by the
433 source. No further update must be done before rings are restarted.
435 In postcopy migration the slave is started before all the memory has
436 been received from the source host, and care must be taken to avoid
437 accessing pages that have yet to be received. The slave opens a
438 'userfault'-fd and registers the memory with it; this fd is then
439 passed back over to the master. The master services requests on the
440 userfaultfd for pages that are accessed and when the page is available
441 it performs WAKE ioctl's on the userfaultfd to wake the stalled
442 slave. The client indicates support for this via the
443 ``VHOST_USER_PROTOCOL_F_PAGEFAULT`` feature.
448 The master sends a list of vhost memory regions to the slave using the
449 ``VHOST_USER_SET_MEM_TABLE`` message. Each region has two base
450 addresses: a guest address and a user address.
452 Messages contain guest addresses and/or user addresses to reference locations
453 within the shared memory. The mapping of these addresses works as follows.
455 User addresses map to the vhost memory region containing that user address.
457 When the ``VIRTIO_F_IOMMU_PLATFORM`` feature has not been negotiated:
459 * Guest addresses map to the vhost memory region containing that guest
462 When the ``VIRTIO_F_IOMMU_PLATFORM`` feature has been negotiated:
464 * Guest addresses are also called I/O virtual addresses (IOVAs). They are
465 translated to user addresses via the IOTLB.
467 * The vhost memory region guest address is not used.
472 When the ``VIRTIO_F_IOMMU_PLATFORM`` feature has been negotiated, the
473 master sends IOTLB entries update & invalidation by sending
474 ``VHOST_USER_IOTLB_MSG`` requests to the slave with a ``struct
475 vhost_iotlb_msg`` as payload. For update events, the ``iotlb`` payload
476 has to be filled with the update message type (2), the I/O virtual
477 address, the size, the user virtual address, and the permissions
478 flags. Addresses and size must be within vhost memory regions set via
479 the ``VHOST_USER_SET_MEM_TABLE`` request. For invalidation events, the
480 ``iotlb`` payload has to be filled with the invalidation message type
481 (3), the I/O virtual address and the size. On success, the slave is
482 expected to reply with a zero payload, non-zero otherwise.
484 The slave relies on the slave communication channel (see :ref:`Slave
485 communication <slave_communication>` section below) to send IOTLB miss
486 and access failure events, by sending ``VHOST_USER_SLAVE_IOTLB_MSG``
487 requests to the master with a ``struct vhost_iotlb_msg`` as
488 payload. For miss events, the iotlb payload has to be filled with the
489 miss message type (1), the I/O virtual address and the permissions
490 flags. For access failure event, the iotlb payload has to be filled
491 with the access failure message type (4), the I/O virtual address and
492 the permissions flags. For synchronization purpose, the slave may
493 rely on the reply-ack feature, so the master may send a reply when
494 operation is completed if the reply-ack feature is negotiated and
495 slaves requests a reply. For miss events, completed operation means
496 either master sent an update message containing the IOTLB entry
497 containing requested address and permission, or master sent nothing if
498 the IOTLB miss message is invalid (invalid IOVA or permission).
500 The master isn't expected to take the initiative to send IOTLB update
501 messages, as the slave sends IOTLB miss messages for the guest virtual
502 memory areas it needs to access.
504 .. _slave_communication:
509 An optional communication channel is provided if the slave declares
510 ``VHOST_USER_PROTOCOL_F_SLAVE_REQ`` protocol feature, to allow the
511 slave to make requests to the master.
513 The fd is provided via ``VHOST_USER_SET_SLAVE_REQ_FD`` ancillary data.
515 A slave may then send ``VHOST_USER_SLAVE_*`` messages to the master
516 using this fd communication channel.
518 If ``VHOST_USER_PROTOCOL_F_SLAVE_SEND_FD`` protocol feature is
519 negotiated, slave can send file descriptors (at most 8 descriptors in
520 each message) to master via ancillary data using this fd communication
523 Inflight I/O tracking
524 ---------------------
526 To support reconnecting after restart or crash, slave may need to
527 resubmit inflight I/Os. If virtqueue is processed in order, we can
528 easily achieve that by getting the inflight descriptors from
529 descriptor table (split virtqueue) or descriptor ring (packed
530 virtqueue). However, it can't work when we process descriptors
531 out-of-order because some entries which store the information of
532 inflight descriptors in available ring (split virtqueue) or descriptor
533 ring (packed virtqueue) might be overrided by new entries. To solve
534 this problem, slave need to allocate an extra buffer to store this
535 information of inflight descriptors and share it with master for
536 persistent. ``VHOST_USER_GET_INFLIGHT_FD`` and
537 ``VHOST_USER_SET_INFLIGHT_FD`` are used to transfer this buffer
538 between master and slave. And the format of this buffer is described
541 +---------------+---------------+-----+---------------+
542 | queue0 region | queue1 region | ... | queueN region |
543 +---------------+---------------+-----+---------------+
545 N is the number of available virtqueues. Slave could get it from num
546 queues field of ``VhostUserInflight``.
548 For split virtqueue, queue region can be implemented as:
552 typedef struct DescStateSplit {
553 /* Indicate whether this descriptor is inflight or not.
554 * Only available for head-descriptor. */
560 /* Maintain a list for the last batch of used descriptors.
561 * Only available when batching is used for submitting */
564 /* Used to preserve the order of fetching available descriptors.
565 * Only available for head-descriptor. */
569 typedef struct QueueRegionSplit {
570 /* The feature flags of this region. Now it's initialized to 0. */
573 /* The version of this region. It's 1 currently.
574 * Zero value indicates an uninitialized buffer */
577 /* The size of DescStateSplit array. It's equal to the virtqueue
578 * size. Slave could get it from queue size field of VhostUserInflight. */
581 /* The head of list that track the last batch of used descriptors. */
582 uint16_t last_batch_head;
584 /* Store the idx value of used ring */
587 /* Used to track the state of each descriptor in descriptor table */
588 DescStateSplit desc[];
591 To track inflight I/O, the queue region should be processed as follows:
593 When receiving available buffers from the driver:
595 #. Get the next available head-descriptor index from available ring, ``i``
597 #. Set ``desc[i].counter`` to the value of global counter
599 #. Increase global counter by 1
601 #. Set ``desc[i].inflight`` to 1
603 When supplying used buffers to the driver:
605 1. Get corresponding used head-descriptor index, i
607 2. Set ``desc[i].next`` to ``last_batch_head``
609 3. Set ``last_batch_head`` to ``i``
611 #. Steps 1,2,3 may be performed repeatedly if batching is possible
613 #. Increase the ``idx`` value of used ring by the size of the batch
615 #. Set the ``inflight`` field of each ``DescStateSplit`` entry in the batch to 0
617 #. Set ``used_idx`` to the ``idx`` value of used ring
621 #. If the value of ``used_idx`` does not match the ``idx`` value of
622 used ring (means the inflight field of ``DescStateSplit`` entries in
623 last batch may be incorrect),
625 a. Subtract the value of ``used_idx`` from the ``idx`` value of
626 used ring to get last batch size of ``DescStateSplit`` entries
628 #. Set the ``inflight`` field of each ``DescStateSplit`` entry to 0 in last batch
629 list which starts from ``last_batch_head``
631 #. Set ``used_idx`` to the ``idx`` value of used ring
633 #. Resubmit inflight ``DescStateSplit`` entries in order of their
636 For packed virtqueue, queue region can be implemented as:
640 typedef struct DescStatePacked {
641 /* Indicate whether this descriptor is inflight or not.
642 * Only available for head-descriptor. */
648 /* Link to the next free entry */
651 /* Link to the last entry of descriptor list.
652 * Only available for head-descriptor. */
655 /* The length of descriptor list.
656 * Only available for head-descriptor. */
659 /* Used to preserve the order of fetching available descriptors.
660 * Only available for head-descriptor. */
666 /* The descriptor flags */
669 /* The buffer length */
672 /* The buffer address */
676 typedef struct QueueRegionPacked {
677 /* The feature flags of this region. Now it's initialized to 0. */
680 /* The version of this region. It's 1 currently.
681 * Zero value indicates an uninitialized buffer */
684 /* The size of DescStatePacked array. It's equal to the virtqueue
685 * size. Slave could get it from queue size field of VhostUserInflight. */
688 /* The head of free DescStatePacked entry list */
691 /* The old head of free DescStatePacked entry list */
692 uint16_t old_free_head;
694 /* The used index of descriptor ring */
697 /* The old used index of descriptor ring */
698 uint16_t old_used_idx;
700 /* Device ring wrap counter */
701 uint8_t used_wrap_counter;
703 /* The old device ring wrap counter */
704 uint8_t old_used_wrap_counter;
709 /* Used to track the state of each descriptor fetched from descriptor ring */
710 DescStatePacked desc[];
713 To track inflight I/O, the queue region should be processed as follows:
715 When receiving available buffers from the driver:
717 #. Get the next available descriptor entry from descriptor ring, ``d``
719 #. If ``d`` is head descriptor,
721 a. Set ``desc[old_free_head].num`` to 0
723 #. Set ``desc[old_free_head].counter`` to the value of global counter
725 #. Increase global counter by 1
727 #. Set ``desc[old_free_head].inflight`` to 1
729 #. If ``d`` is last descriptor, set ``desc[old_free_head].last`` to
732 #. Increase ``desc[old_free_head].num`` by 1
734 #. Set ``desc[free_head].addr``, ``desc[free_head].len``,
735 ``desc[free_head].flags``, ``desc[free_head].id`` to ``d.addr``,
736 ``d.len``, ``d.flags``, ``d.id``
738 #. Set ``free_head`` to ``desc[free_head].next``
740 #. If ``d`` is last descriptor, set ``old_free_head`` to ``free_head``
742 When supplying used buffers to the driver:
744 1. Get corresponding used head-descriptor entry from descriptor ring,
747 2. Get corresponding ``DescStatePacked`` entry, ``e``
749 3. Set ``desc[e.last].next`` to ``free_head``
751 4. Set ``free_head`` to the index of ``e``
753 #. Steps 1,2,3,4 may be performed repeatedly if batching is possible
755 #. Increase ``used_idx`` by the size of the batch and update
756 ``used_wrap_counter`` if needed
758 #. Update ``d.flags``
760 #. Set the ``inflight`` field of each head ``DescStatePacked`` entry
763 #. Set ``old_free_head``, ``old_used_idx``, ``old_used_wrap_counter``
764 to ``free_head``, ``used_idx``, ``used_wrap_counter``
768 #. If ``used_idx`` does not match ``old_used_idx`` (means the
769 ``inflight`` field of ``DescStatePacked`` entries in last batch may
772 a. Get the next descriptor ring entry through ``old_used_idx``, ``d``
774 #. Use ``old_used_wrap_counter`` to calculate the available flags
776 #. If ``d.flags`` is not equal to the calculated flags value (means
777 slave has submitted the buffer to guest driver before crash, so
778 it has to commit the in-progres update), set ``old_free_head``,
779 ``old_used_idx``, ``old_used_wrap_counter`` to ``free_head``,
780 ``used_idx``, ``used_wrap_counter``
782 #. Set ``free_head``, ``used_idx``, ``used_wrap_counter`` to
783 ``old_free_head``, ``old_used_idx``, ``old_used_wrap_counter``
784 (roll back any in-progress update)
786 #. Set the ``inflight`` field of each ``DescStatePacked`` entry in
789 #. Resubmit inflight ``DescStatePacked`` entries in order of their
792 In-band notifications
793 ---------------------
795 In some limited situations (e.g. for simulation) it is desirable to
796 have the kick, call and error (if used) signals done via in-band
797 messages instead of asynchronous eventfd notifications. This can be
798 done by negotiating the ``VHOST_USER_PROTOCOL_F_INBAND_NOTIFICATIONS``
801 Note that due to the fact that too many messages on the sockets can
802 cause the sending application(s) to block, it is not advised to use
803 this feature unless absolutely necessary. It is also considered an
804 error to negotiate this feature without also negotiating
805 ``VHOST_USER_PROTOCOL_F_SLAVE_REQ`` and ``VHOST_USER_PROTOCOL_F_REPLY_ACK``,
806 the former is necessary for getting a message channel from the slave
807 to the master, while the latter needs to be used with the in-band
808 notification messages to block until they are processed, both to avoid
809 blocking later and for proper processing (at least in the simulation
810 use case.) As it has no other way of signalling this error, the slave
811 should close the connection as a response to a
812 ``VHOST_USER_SET_PROTOCOL_FEATURES`` message that sets the in-band
813 notifications feature flag without the other two.
820 #define VHOST_USER_PROTOCOL_F_MQ 0
821 #define VHOST_USER_PROTOCOL_F_LOG_SHMFD 1
822 #define VHOST_USER_PROTOCOL_F_RARP 2
823 #define VHOST_USER_PROTOCOL_F_REPLY_ACK 3
824 #define VHOST_USER_PROTOCOL_F_MTU 4
825 #define VHOST_USER_PROTOCOL_F_SLAVE_REQ 5
826 #define VHOST_USER_PROTOCOL_F_CROSS_ENDIAN 6
827 #define VHOST_USER_PROTOCOL_F_CRYPTO_SESSION 7
828 #define VHOST_USER_PROTOCOL_F_PAGEFAULT 8
829 #define VHOST_USER_PROTOCOL_F_CONFIG 9
830 #define VHOST_USER_PROTOCOL_F_SLAVE_SEND_FD 10
831 #define VHOST_USER_PROTOCOL_F_HOST_NOTIFIER 11
832 #define VHOST_USER_PROTOCOL_F_INFLIGHT_SHMFD 12
833 #define VHOST_USER_PROTOCOL_F_RESET_DEVICE 13
834 #define VHOST_USER_PROTOCOL_F_INBAND_NOTIFICATIONS 14
835 #define VHOST_USER_PROTOCOL_F_CONFIGURE_MEM_SLOTS 15
836 #define VHOST_USER_PROTOCOL_F_STATUS 16
841 ``VHOST_USER_GET_FEATURES``
843 :equivalent ioctl: ``VHOST_GET_FEATURES``
845 :slave payload: ``u64``
847 Get from the underlying vhost implementation the features bitmask.
848 Feature bit ``VHOST_USER_F_PROTOCOL_FEATURES`` signals slave support
849 for ``VHOST_USER_GET_PROTOCOL_FEATURES`` and
850 ``VHOST_USER_SET_PROTOCOL_FEATURES``.
852 ``VHOST_USER_SET_FEATURES``
854 :equivalent ioctl: ``VHOST_SET_FEATURES``
855 :master payload: ``u64``
857 Enable features in the underlying vhost implementation using a
858 bitmask. Feature bit ``VHOST_USER_F_PROTOCOL_FEATURES`` signals
859 slave support for ``VHOST_USER_GET_PROTOCOL_FEATURES`` and
860 ``VHOST_USER_SET_PROTOCOL_FEATURES``.
862 ``VHOST_USER_GET_PROTOCOL_FEATURES``
864 :equivalent ioctl: ``VHOST_GET_FEATURES``
866 :slave payload: ``u64``
868 Get the protocol feature bitmask from the underlying vhost
869 implementation. Only legal if feature bit
870 ``VHOST_USER_F_PROTOCOL_FEATURES`` is present in
871 ``VHOST_USER_GET_FEATURES``.
874 Slave that reported ``VHOST_USER_F_PROTOCOL_FEATURES`` must
875 support this message even before ``VHOST_USER_SET_FEATURES`` was
878 ``VHOST_USER_SET_PROTOCOL_FEATURES``
880 :equivalent ioctl: ``VHOST_SET_FEATURES``
881 :master payload: ``u64``
883 Enable protocol features in the underlying vhost implementation.
885 Only legal if feature bit ``VHOST_USER_F_PROTOCOL_FEATURES`` is present in
886 ``VHOST_USER_GET_FEATURES``.
889 Slave that reported ``VHOST_USER_F_PROTOCOL_FEATURES`` must support
890 this message even before ``VHOST_USER_SET_FEATURES`` was called.
892 ``VHOST_USER_SET_OWNER``
894 :equivalent ioctl: ``VHOST_SET_OWNER``
897 Issued when a new connection is established. It sets the current
898 *master* as an owner of the session. This can be used on the *slave*
899 as a "session start" flag.
901 ``VHOST_USER_RESET_OWNER``
905 .. admonition:: Deprecated
907 This is no longer used. Used to be sent to request disabling all
908 rings, but some clients interpreted it to also discard connection
909 state (this interpretation would lead to bugs). It is recommended
910 that clients either ignore this message, or use it to disable all
913 ``VHOST_USER_SET_MEM_TABLE``
915 :equivalent ioctl: ``VHOST_SET_MEM_TABLE``
916 :master payload: memory regions description
917 :slave payload: (postcopy only) memory regions description
919 Sets the memory map regions on the slave so it can translate the
920 vring addresses. In the ancillary data there is an array of file
921 descriptors for each memory mapped region. The size and ordering of
922 the fds matches the number and ordering of memory regions.
924 When ``VHOST_USER_POSTCOPY_LISTEN`` has been received,
925 ``SET_MEM_TABLE`` replies with the bases of the memory mapped
926 regions to the master. The slave must have mmap'd the regions but
927 not yet accessed them and should not yet generate a userfault
931 ``NEED_REPLY_MASK`` is not set in this case. QEMU will then
932 reply back to the list of mappings with an empty
933 ``VHOST_USER_SET_MEM_TABLE`` as an acknowledgement; only upon
934 reception of this message may the guest start accessing the memory
935 and generating faults.
937 ``VHOST_USER_SET_LOG_BASE``
939 :equivalent ioctl: ``VHOST_SET_LOG_BASE``
943 Sets logging shared memory space.
945 When slave has ``VHOST_USER_PROTOCOL_F_LOG_SHMFD`` protocol feature,
946 the log memory fd is provided in the ancillary data of
947 ``VHOST_USER_SET_LOG_BASE`` message, the size and offset of shared
948 memory area provided in the message.
950 ``VHOST_USER_SET_LOG_FD``
952 :equivalent ioctl: ``VHOST_SET_LOG_FD``
955 Sets the logging file descriptor, which is passed as ancillary data.
957 ``VHOST_USER_SET_VRING_NUM``
959 :equivalent ioctl: ``VHOST_SET_VRING_NUM``
960 :master payload: vring state description
962 Set the size of the queue.
964 ``VHOST_USER_SET_VRING_ADDR``
966 :equivalent ioctl: ``VHOST_SET_VRING_ADDR``
967 :master payload: vring address description
970 Sets the addresses of the different aspects of the vring.
972 ``VHOST_USER_SET_VRING_BASE``
974 :equivalent ioctl: ``VHOST_SET_VRING_BASE``
975 :master payload: vring state description
977 Sets the base offset in the available vring.
979 ``VHOST_USER_GET_VRING_BASE``
981 :equivalent ioctl: ``VHOST_USER_GET_VRING_BASE``
982 :master payload: vring state description
983 :slave payload: vring state description
985 Get the available vring base offset.
987 ``VHOST_USER_SET_VRING_KICK``
989 :equivalent ioctl: ``VHOST_SET_VRING_KICK``
990 :master payload: ``u64``
992 Set the event file descriptor for adding buffers to the vring. It is
993 passed in the ancillary data.
995 Bits (0-7) of the payload contain the vring index. Bit 8 is the
996 invalid FD flag. This flag is set when there is no file descriptor
997 in the ancillary data. This signals that polling should be used
998 instead of waiting for the kick. Note that if the protocol feature
999 ``VHOST_USER_PROTOCOL_F_INBAND_NOTIFICATIONS`` has been negotiated
1000 this message isn't necessary as the ring is also started on the
1001 ``VHOST_USER_VRING_KICK`` message, it may however still be used to
1002 set an event file descriptor (which will be preferred over the
1003 message) or to enable polling.
1005 ``VHOST_USER_SET_VRING_CALL``
1007 :equivalent ioctl: ``VHOST_SET_VRING_CALL``
1008 :master payload: ``u64``
1010 Set the event file descriptor to signal when buffers are used. It is
1011 passed in the ancillary data.
1013 Bits (0-7) of the payload contain the vring index. Bit 8 is the
1014 invalid FD flag. This flag is set when there is no file descriptor
1015 in the ancillary data. This signals that polling will be used
1016 instead of waiting for the call. Note that if the protocol features
1017 ``VHOST_USER_PROTOCOL_F_INBAND_NOTIFICATIONS`` and
1018 ``VHOST_USER_PROTOCOL_F_SLAVE_REQ`` have been negotiated this message
1019 isn't necessary as the ``VHOST_USER_SLAVE_VRING_CALL`` message can be
1020 used, it may however still be used to set an event file descriptor
1021 or to enable polling.
1023 ``VHOST_USER_SET_VRING_ERR``
1025 :equivalent ioctl: ``VHOST_SET_VRING_ERR``
1026 :master payload: ``u64``
1028 Set the event file descriptor to signal when error occurs. It is
1029 passed in the ancillary data.
1031 Bits (0-7) of the payload contain the vring index. Bit 8 is the
1032 invalid FD flag. This flag is set when there is no file descriptor
1033 in the ancillary data. Note that if the protocol features
1034 ``VHOST_USER_PROTOCOL_F_INBAND_NOTIFICATIONS`` and
1035 ``VHOST_USER_PROTOCOL_F_SLAVE_REQ`` have been negotiated this message
1036 isn't necessary as the ``VHOST_USER_SLAVE_VRING_ERR`` message can be
1037 used, it may however still be used to set an event file descriptor
1038 (which will be preferred over the message).
1040 ``VHOST_USER_GET_QUEUE_NUM``
1042 :equivalent ioctl: N/A
1043 :master payload: N/A
1046 Query how many queues the backend supports.
1048 This request should be sent only when ``VHOST_USER_PROTOCOL_F_MQ``
1049 is set in queried protocol features by
1050 ``VHOST_USER_GET_PROTOCOL_FEATURES``.
1052 ``VHOST_USER_SET_VRING_ENABLE``
1054 :equivalent ioctl: N/A
1055 :master payload: vring state description
1057 Signal slave to enable or disable corresponding vring.
1059 This request should be sent only when
1060 ``VHOST_USER_F_PROTOCOL_FEATURES`` has been negotiated.
1062 ``VHOST_USER_SEND_RARP``
1064 :equivalent ioctl: N/A
1065 :master payload: ``u64``
1067 Ask vhost user backend to broadcast a fake RARP to notify the migration
1068 is terminated for guest that does not support GUEST_ANNOUNCE.
1070 Only legal if feature bit ``VHOST_USER_F_PROTOCOL_FEATURES`` is
1071 present in ``VHOST_USER_GET_FEATURES`` and protocol feature bit
1072 ``VHOST_USER_PROTOCOL_F_RARP`` is present in
1073 ``VHOST_USER_GET_PROTOCOL_FEATURES``. The first 6 bytes of the
1074 payload contain the mac address of the guest to allow the vhost user
1075 backend to construct and broadcast the fake RARP.
1077 ``VHOST_USER_NET_SET_MTU``
1079 :equivalent ioctl: N/A
1080 :master payload: ``u64``
1082 Set host MTU value exposed to the guest.
1084 This request should be sent only when ``VIRTIO_NET_F_MTU`` feature
1085 has been successfully negotiated, ``VHOST_USER_F_PROTOCOL_FEATURES``
1086 is present in ``VHOST_USER_GET_FEATURES`` and protocol feature bit
1087 ``VHOST_USER_PROTOCOL_F_NET_MTU`` is present in
1088 ``VHOST_USER_GET_PROTOCOL_FEATURES``.
1090 If ``VHOST_USER_PROTOCOL_F_REPLY_ACK`` is negotiated, slave must
1091 respond with zero in case the specified MTU is valid, or non-zero
1094 ``VHOST_USER_SET_SLAVE_REQ_FD``
1096 :equivalent ioctl: N/A
1097 :master payload: N/A
1099 Set the socket file descriptor for slave initiated requests. It is passed
1100 in the ancillary data.
1102 This request should be sent only when
1103 ``VHOST_USER_F_PROTOCOL_FEATURES`` has been negotiated, and protocol
1104 feature bit ``VHOST_USER_PROTOCOL_F_SLAVE_REQ`` bit is present in
1105 ``VHOST_USER_GET_PROTOCOL_FEATURES``. If
1106 ``VHOST_USER_PROTOCOL_F_REPLY_ACK`` is negotiated, slave must
1107 respond with zero for success, non-zero otherwise.
1109 ``VHOST_USER_IOTLB_MSG``
1111 :equivalent ioctl: N/A (equivalent to ``VHOST_IOTLB_MSG`` message type)
1112 :master payload: ``struct vhost_iotlb_msg``
1113 :slave payload: ``u64``
1115 Send IOTLB messages with ``struct vhost_iotlb_msg`` as payload.
1117 Master sends such requests to update and invalidate entries in the
1118 device IOTLB. The slave has to acknowledge the request with sending
1119 zero as ``u64`` payload for success, non-zero otherwise.
1121 This request should be send only when ``VIRTIO_F_IOMMU_PLATFORM``
1122 feature has been successfully negotiated.
1124 ``VHOST_USER_SET_VRING_ENDIAN``
1126 :equivalent ioctl: ``VHOST_SET_VRING_ENDIAN``
1127 :master payload: vring state description
1129 Set the endianness of a VQ for legacy devices. Little-endian is
1130 indicated with state.num set to 0 and big-endian is indicated with
1131 state.num set to 1. Other values are invalid.
1133 This request should be sent only when
1134 ``VHOST_USER_PROTOCOL_F_CROSS_ENDIAN`` has been negotiated.
1135 Backends that negotiated this feature should handle both
1136 endiannesses and expect this message once (per VQ) during device
1137 configuration (ie. before the master starts the VQ).
1139 ``VHOST_USER_GET_CONFIG``
1141 :equivalent ioctl: N/A
1142 :master payload: virtio device config space
1143 :slave payload: virtio device config space
1145 When ``VHOST_USER_PROTOCOL_F_CONFIG`` is negotiated, this message is
1146 submitted by the vhost-user master to fetch the contents of the
1147 virtio device configuration space, vhost-user slave's payload size
1148 MUST match master's request, vhost-user slave uses zero length of
1149 payload to indicate an error to vhost-user master. The vhost-user
1150 master may cache the contents to avoid repeated
1151 ``VHOST_USER_GET_CONFIG`` calls.
1153 ``VHOST_USER_SET_CONFIG``
1155 :equivalent ioctl: N/A
1156 :master payload: virtio device config space
1159 When ``VHOST_USER_PROTOCOL_F_CONFIG`` is negotiated, this message is
1160 submitted by the vhost-user master when the Guest changes the virtio
1161 device configuration space and also can be used for live migration
1162 on the destination host. The vhost-user slave must check the flags
1163 field, and slaves MUST NOT accept SET_CONFIG for read-only
1164 configuration space fields unless the live migration bit is set.
1166 ``VHOST_USER_CREATE_CRYPTO_SESSION``
1168 :equivalent ioctl: N/A
1169 :master payload: crypto session description
1170 :slave payload: crypto session description
1172 Create a session for crypto operation. The server side must return
1173 the session id, 0 or positive for success, negative for failure.
1174 This request should be sent only when
1175 ``VHOST_USER_PROTOCOL_F_CRYPTO_SESSION`` feature has been
1176 successfully negotiated. It's a required feature for crypto
1179 ``VHOST_USER_CLOSE_CRYPTO_SESSION``
1181 :equivalent ioctl: N/A
1182 :master payload: ``u64``
1184 Close a session for crypto operation which was previously
1185 created by ``VHOST_USER_CREATE_CRYPTO_SESSION``.
1187 This request should be sent only when
1188 ``VHOST_USER_PROTOCOL_F_CRYPTO_SESSION`` feature has been
1189 successfully negotiated. It's a required feature for crypto
1192 ``VHOST_USER_POSTCOPY_ADVISE``
1194 :master payload: N/A
1195 :slave payload: userfault fd
1197 When ``VHOST_USER_PROTOCOL_F_PAGEFAULT`` is supported, the master
1198 advises slave that a migration with postcopy enabled is underway,
1199 the slave must open a userfaultfd for later use. Note that at this
1200 stage the migration is still in precopy mode.
1202 ``VHOST_USER_POSTCOPY_LISTEN``
1204 :master payload: N/A
1206 Master advises slave that a transition to postcopy mode has
1207 happened. The slave must ensure that shared memory is registered
1208 with userfaultfd to cause faulting of non-present pages.
1210 This is always sent sometime after a ``VHOST_USER_POSTCOPY_ADVISE``,
1211 and thus only when ``VHOST_USER_PROTOCOL_F_PAGEFAULT`` is supported.
1213 ``VHOST_USER_POSTCOPY_END``
1215 :slave payload: ``u64``
1217 Master advises that postcopy migration has now completed. The slave
1218 must disable the userfaultfd. The response is an acknowledgement
1221 When ``VHOST_USER_PROTOCOL_F_PAGEFAULT`` is supported, this message
1222 is sent at the end of the migration, after
1223 ``VHOST_USER_POSTCOPY_LISTEN`` was previously sent.
1225 The value returned is an error indication; 0 is success.
1227 ``VHOST_USER_GET_INFLIGHT_FD``
1229 :equivalent ioctl: N/A
1230 :master payload: inflight description
1232 When ``VHOST_USER_PROTOCOL_F_INFLIGHT_SHMFD`` protocol feature has
1233 been successfully negotiated, this message is submitted by master to
1234 get a shared buffer from slave. The shared buffer will be used to
1235 track inflight I/O by slave. QEMU should retrieve a new one when vm
1238 ``VHOST_USER_SET_INFLIGHT_FD``
1240 :equivalent ioctl: N/A
1241 :master payload: inflight description
1243 When ``VHOST_USER_PROTOCOL_F_INFLIGHT_SHMFD`` protocol feature has
1244 been successfully negotiated, this message is submitted by master to
1245 send the shared inflight buffer back to slave so that slave could
1246 get inflight I/O after a crash or restart.
1248 ``VHOST_USER_GPU_SET_SOCKET``
1250 :equivalent ioctl: N/A
1251 :master payload: N/A
1253 Sets the GPU protocol socket file descriptor, which is passed as
1254 ancillary data. The GPU protocol is used to inform the master of
1255 rendering state and updates. See vhost-user-gpu.rst for details.
1257 ``VHOST_USER_RESET_DEVICE``
1259 :equivalent ioctl: N/A
1260 :master payload: N/A
1263 Ask the vhost user backend to disable all rings and reset all
1264 internal device state to the initial state, ready to be
1265 reinitialized. The backend retains ownership of the device
1266 throughout the reset operation.
1268 Only valid if the ``VHOST_USER_PROTOCOL_F_RESET_DEVICE`` protocol
1269 feature is set by the backend.
1271 ``VHOST_USER_VRING_KICK``
1273 :equivalent ioctl: N/A
1274 :slave payload: vring state description
1275 :master payload: N/A
1277 When the ``VHOST_USER_PROTOCOL_F_INBAND_NOTIFICATIONS`` protocol
1278 feature has been successfully negotiated, this message may be
1279 submitted by the master to indicate that a buffer was added to
1280 the vring instead of signalling it using the vring's kick file
1281 descriptor or having the slave rely on polling.
1283 The state.num field is currently reserved and must be set to 0.
1285 ``VHOST_USER_GET_MAX_MEM_SLOTS``
1287 :equivalent ioctl: N/A
1290 When the ``VHOST_USER_PROTOCOL_F_CONFIGURE_MEM_SLOTS`` protocol
1291 feature has been successfully negotiated, this message is submitted
1292 by master to the slave. The slave should return the message with a
1293 u64 payload containing the maximum number of memory slots for
1294 QEMU to expose to the guest. The value returned by the backend
1295 will be capped at the maximum number of ram slots which can be
1296 supported by the target platform.
1298 ``VHOST_USER_ADD_MEM_REG``
1300 :equivalent ioctl: N/A
1301 :slave payload: single memory region description
1303 When the ``VHOST_USER_PROTOCOL_F_CONFIGURE_MEM_SLOTS`` protocol
1304 feature has been successfully negotiated, this message is submitted
1305 by the master to the slave. The message payload contains a memory
1306 region descriptor struct, describing a region of guest memory which
1307 the slave device must map in. When the
1308 ``VHOST_USER_PROTOCOL_F_CONFIGURE_MEM_SLOTS`` protocol feature has
1309 been successfully negotiated, along with the
1310 ``VHOST_USER_REM_MEM_REG`` message, this message is used to set and
1311 update the memory tables of the slave device.
1313 ``VHOST_USER_REM_MEM_REG``
1315 :equivalent ioctl: N/A
1316 :slave payload: single memory region description
1318 When the ``VHOST_USER_PROTOCOL_F_CONFIGURE_MEM_SLOTS`` protocol
1319 feature has been successfully negotiated, this message is submitted
1320 by the master to the slave. The message payload contains a memory
1321 region descriptor struct, describing a region of guest memory which
1322 the slave device must unmap. When the
1323 ``VHOST_USER_PROTOCOL_F_CONFIGURE_MEM_SLOTS`` protocol feature has
1324 been successfully negotiated, along with the
1325 ``VHOST_USER_ADD_MEM_REG`` message, this message is used to set and
1326 update the memory tables of the slave device.
1328 ``VHOST_USER_SET_STATUS``
1330 :equivalent ioctl: VHOST_VDPA_SET_STATUS
1332 :master payload: ``u64``
1334 When the ``VHOST_USER_PROTOCOL_F_STATUS`` protocol feature has been
1335 successfully negotiated, this message is submitted by the master to
1336 notify the backend with updated device status as defined in the Virtio
1339 ``VHOST_USER_GET_STATUS``
1341 :equivalent ioctl: VHOST_VDPA_GET_STATUS
1342 :slave payload: ``u64``
1343 :master payload: N/A
1345 When the ``VHOST_USER_PROTOCOL_F_STATUS`` protocol feature has been
1346 successfully negotiated, this message is submitted by the master to
1347 query the backend for its device status as defined in the Virtio
1354 ``VHOST_USER_SLAVE_IOTLB_MSG``
1356 :equivalent ioctl: N/A (equivalent to ``VHOST_IOTLB_MSG`` message type)
1357 :slave payload: ``struct vhost_iotlb_msg``
1358 :master payload: N/A
1360 Send IOTLB messages with ``struct vhost_iotlb_msg`` as payload.
1361 Slave sends such requests to notify of an IOTLB miss, or an IOTLB
1362 access failure. If ``VHOST_USER_PROTOCOL_F_REPLY_ACK`` is
1363 negotiated, and slave set the ``VHOST_USER_NEED_REPLY`` flag, master
1364 must respond with zero when operation is successfully completed, or
1365 non-zero otherwise. This request should be send only when
1366 ``VIRTIO_F_IOMMU_PLATFORM`` feature has been successfully
1369 ``VHOST_USER_SLAVE_CONFIG_CHANGE_MSG``
1371 :equivalent ioctl: N/A
1373 :master payload: N/A
1375 When ``VHOST_USER_PROTOCOL_F_CONFIG`` is negotiated, vhost-user
1376 slave sends such messages to notify that the virtio device's
1377 configuration space has changed, for those host devices which can
1378 support such feature, host driver can send ``VHOST_USER_GET_CONFIG``
1379 message to slave to get the latest content. If
1380 ``VHOST_USER_PROTOCOL_F_REPLY_ACK`` is negotiated, and slave set the
1381 ``VHOST_USER_NEED_REPLY`` flag, master must respond with zero when
1382 operation is successfully completed, or non-zero otherwise.
1384 ``VHOST_USER_SLAVE_VRING_HOST_NOTIFIER_MSG``
1386 :equivalent ioctl: N/A
1387 :slave payload: vring area description
1388 :master payload: N/A
1390 Sets host notifier for a specified queue. The queue index is
1391 contained in the ``u64`` field of the vring area description. The
1392 host notifier is described by the file descriptor (typically it's a
1393 VFIO device fd) which is passed as ancillary data and the size
1394 (which is mmap size and should be the same as host page size) and
1395 offset (which is mmap offset) carried in the vring area
1396 description. QEMU can mmap the file descriptor based on the size and
1397 offset to get a memory range. Registering a host notifier means
1398 mapping this memory range to the VM as the specified queue's notify
1399 MMIO region. Slave sends this request to tell QEMU to de-register
1400 the existing notifier if any and register the new notifier if the
1401 request is sent with a file descriptor.
1403 This request should be sent only when
1404 ``VHOST_USER_PROTOCOL_F_HOST_NOTIFIER`` protocol feature has been
1405 successfully negotiated.
1407 ``VHOST_USER_SLAVE_VRING_CALL``
1409 :equivalent ioctl: N/A
1410 :slave payload: vring state description
1411 :master payload: N/A
1413 When the ``VHOST_USER_PROTOCOL_F_INBAND_NOTIFICATIONS`` protocol
1414 feature has been successfully negotiated, this message may be
1415 submitted by the slave to indicate that a buffer was used from
1416 the vring instead of signalling this using the vring's call file
1417 descriptor or having the master relying on polling.
1419 The state.num field is currently reserved and must be set to 0.
1421 ``VHOST_USER_SLAVE_VRING_ERR``
1423 :equivalent ioctl: N/A
1424 :slave payload: vring state description
1425 :master payload: N/A
1427 When the ``VHOST_USER_PROTOCOL_F_INBAND_NOTIFICATIONS`` protocol
1428 feature has been successfully negotiated, this message may be
1429 submitted by the slave to indicate that an error occurred on the
1430 specific vring, instead of signalling the error file descriptor
1431 set by the master via ``VHOST_USER_SET_VRING_ERR``.
1433 The state.num field is currently reserved and must be set to 0.
1437 VHOST_USER_PROTOCOL_F_REPLY_ACK
1438 -------------------------------
1440 The original vhost-user specification only demands replies for certain
1441 commands. This differs from the vhost protocol implementation where
1442 commands are sent over an ``ioctl()`` call and block until the client
1445 With this protocol extension negotiated, the sender (QEMU) can set the
1446 ``need_reply`` [Bit 3] flag to any command. This indicates that the
1447 client MUST respond with a Payload ``VhostUserMsg`` indicating success
1448 or failure. The payload should be set to zero on success or non-zero
1449 on failure, unless the message already has an explicit reply body.
1451 The response payload gives QEMU a deterministic indication of the result
1452 of the command. Today, QEMU is expected to terminate the main vhost-user
1453 loop upon receiving such errors. In future, qemu could be taught to be more
1454 resilient for selective requests.
1456 For the message types that already solicit a reply from the client,
1457 the presence of ``VHOST_USER_PROTOCOL_F_REPLY_ACK`` or need_reply bit
1458 being set brings no behavioural change. (See the Communication_
1459 section for details.)
1461 .. _backend_conventions:
1463 Backend program conventions
1464 ===========================
1466 vhost-user backends can provide various devices & services and may
1467 need to be configured manually depending on the use case. However, it
1468 is a good idea to follow the conventions listed here when
1469 possible. Users, QEMU or libvirt, can then rely on some common
1470 behaviour to avoid heterogeneous configuration and management of the
1471 backend programs and facilitate interoperability.
1473 Each backend installed on a host system should come with at least one
1474 JSON file that conforms to the vhost-user.json schema. Each file
1475 informs the management applications about the backend type, and binary
1476 location. In addition, it defines rules for management apps for
1477 picking the highest priority backend when multiple match the search
1478 criteria (see ``@VhostUserBackend`` documentation in the schema file).
1480 If the backend is not capable of enabling a requested feature on the
1481 host (such as 3D acceleration with virgl), or the initialization
1482 failed, the backend should fail to start early and exit with a status
1483 != 0. It may also print a message to stderr for further details.
1485 The backend program must not daemonize itself, but it may be
1486 daemonized by the management layer. It may also have a restricted
1487 access to the system.
1489 File descriptors 0, 1 and 2 will exist, and have regular
1490 stdin/stdout/stderr usage (they may have been redirected to /dev/null
1491 by the management layer, or to a log handler).
1493 The backend program must end (as quickly and cleanly as possible) when
1494 the SIGTERM signal is received. Eventually, it may receive SIGKILL by
1495 the management layer after a few seconds.
1497 The following command line options have an expected behaviour. They
1498 are mandatory, unless explicitly said differently:
1502 This option specify the location of the vhost-user Unix domain socket.
1503 It is incompatible with --fd.
1507 When this argument is given, the backend program is started with the
1508 vhost-user socket as file descriptor FDNUM. It is incompatible with
1511 --print-capabilities
1513 Output to stdout the backend capabilities in JSON format, and then
1514 exit successfully. Other options and arguments should be ignored, and
1515 the backend program should not perform its normal function. The
1516 capabilities can be reported dynamically depending on the host
1519 The JSON output is described in the ``vhost-user.json`` schema, by
1520 ```@VHostUserBackendCapabilities``. Example:
1535 Command line options:
1539 Specify the linux input device.
1545 Do no request exclusive access to the input device.
1552 Command line options:
1556 Specify the GPU DRM render node.
1562 Enable virgl rendering support.
1569 Command line options:
1573 Specify block device or file path.