docs/pci-hotplug.html.in

   1 <?xml version="1.0" encoding="UTF-8"?>
   2 <!DOCTYPE html>
   3 <html xmlns="http://www.w3.org/1999/xhtml">
   4   <body>
   5     <h1>PCI topology and hotplug</h1>
   6
   7     <ul id="toc"></ul>
   8
   9     <p>
  10       Perhaps surprisingly, most libvirt guests support only limited PCI
  11       device hotplug out of the box, or even none at all.
  12     </p>
  13     <p>
  14       The reason for this apparent limitation is the fact that each
  15       hotplugged PCI device might require additional PCI controllers to
  16       be added to the guest. Since most PCI controllers can't be
  17       hotplugged, they need to be added before the guest is started;
  18       however, libvirt has no way of knowing in advance how many devices
  19       will be hotplugged during the guest's lifetime, thus making it
  20       impossible to automatically provide the right amount of PCI
  21       controllers: any arbitrary number would end up being too big
  22       for some users, and too small for others.
  23     </p>
  24     <p>
  25       Ultimately, the user is the only one who knows how much the guest
  26       will need to grow dynamically, so the responsibility of planning
  27       a suitable PCI topology in advance falls on them.
  28     </p>
  29     <p>
  30       This document aims at providing all the information needed to
  31       successfully plan the PCI topology of a guest. Note that the
  32       details can vary a lot between architectures and even machine
  33       types, hence the way it's organized.
  34     </p>
  35
  36     <h2><a id="x86_64">x86_64 architecture</a></h2>
  37
  38     <h3><a id="x86_64-q35">q35 machine type</a></h3>
  39
  40     <p>
  41       This is a PCI Express native machine type. The default PCI topology
  42       looks like
  43     </p>
  44
  45 <pre>
  46 &lt;controller type='pci' index='0' model='pcie-root'/&gt;
  47 &lt;controller type='pci' index='1' model='pcie-root-port'&gt;
  48   &lt;model name='pcie-root-port'/&gt;
  49   &lt;target chassis='1' port='0x10'/&gt;
  50   &lt;address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/&gt;
  51 &lt;/controller&gt;</pre>
  52
  53     <p>
  54       and supports hotplugging a single PCI Express device, either
  55       emulated or assigned from the host.
  56     </p>
  57     <p>
  58       If you have a very specific use case, such as the appliances
  59       used by <a href="http://libguestfs.org/">libguestfs</a> behind
  60       the scenes to access disk images, and this automatically-added
  61       <code>pcie-root-port</code> controller ends up being a nuisance,
  62       you can prevent libvirt from adding it by manually managing PCI
  63       controllers and addresses according to your needs.
  64     </p>
  65     <p>
  66       Slots on the <code>pcie-root</code> controller do not support
  67       hotplug, so the device will be hotplugged into the
  68       <code>pcie-root-port</code> controller. If you plan to hotplug
  69       more than a single PCI Express device, you should add a suitable
  70       number of <code>pcie-root-port</code> controllers when defining
  71       the guest: for example, add
  72     </p>
  73
  74 <pre>
  75 &lt;controller type='pci' model='pcie-root'/&gt;
  76 &lt;controller type='pci' model='pcie-root-port'/&gt;
  77 &lt;controller type='pci' model='pcie-root-port'/&gt;
  78 &lt;controller type='pci' model='pcie-root-port'/&gt;</pre>
  79
  80     <p>
  81       if you expect to hotplug up to three PCI Express devices,
  82       either emulated or assigned from the host. That's all the
  83       information you need to provide: libvirt will fill in the
  84       remaining details automatically. Note that you need to add
  85       the <code>pcie-root</code> controller along with the
  86       <code>pcie-root-port</code> controllers or you will get an
  87       error.
  88     </p>
  89     <p>
  90       Note that if you're adding PCI controllers to a guest and at
  91       the same time you're also adding PCI devices, some of the
  92       controllers will be used for the newly-added devices and won't
  93       be available for hotplug once the guest has been started.
  94     </p>
  95     <p>
  96       If you expect to hotplug legacy PCI devices, then you will need
  97       specialized controllers, since all those mentioned above are
  98       intended for PCI Express devices only: add
  99     </p>
 100
 101 <pre>
 102 &lt;controller type='pci' model='pcie-to-pci-bridge'/&gt;</pre>
 103
 104     <p>
 105       and you'll be able to hotplug up to 31 legacy PCI devices,
 106       either emulated or assigned from the host, in the slots
 107       from 0x01 to 0x1f of the <code>pcie-to-pci-bridge</code> controller.
 108     </p>
 109
 110     <h3><a id="x86_64-i440fx">i440fx (pc) machine type</a></h3>
 111
 112     <p>
 113       This is a legacy PCI native machine type. The default PCI
 114       topology looks like
 115     </p>
 116
 117 <pre>
 118 &lt;controller type='pci' index='0' model='pci-root'/&gt;</pre>
 119
 120     <p>
 121       where each of the 31 slots (from 0x01 to 0x1f) on the
 122       <code>pci-root</code> controller is hotplug capable and
 123       can accept a legacy PCI device, either emulated or
 124       assigned from the guest.
 125     </p>
 126
 127     <h2><a id="ppc64">ppc64 architecture</a></h2>
 128
 129     <h3><a id="ppc64-pseries">pseries machine type</a></h3>
 130
 131     <p>
 132       The default PCI topology for the <code>pseries</code> machine
 133       type looks like
 134     </p>
 135
 136 <pre>
 137 &lt;controller type='pci' index='0' model='pci-root'&gt;
 138   &lt;model name='spapr-pci-host-bridge'/&gt;
 139   &lt;target index='0'/&gt;
 140 &lt;/controller&gt;</pre>
 141
 142     <p>
 143       The 31 slots, from 0x01 to 0x1f, on a <code>pci-root</code>
 144       controller are all hotplug capable and, despite the name
 145       suggesting otherwise, starting with QEMU 2.9 all of them
 146       can accept PCI Express devices in addition to legacy PCI
 147       devices; however, libvirt will only place emulated devices
 148       on the default <code>pci-root</code> controller.
 149     </p>
 150     <p>
 151       In order to take advantage of improved error reporting and
 152       recovering capabilities, PCI devices assigned from the
 153       host need to be isolated by placing each on a separate
 154       <code>pci-root</code> controller, which has to be prepared
 155       in advance for hotplug to work: for example, add
 156     </p>
 157
 158 <pre>
 159 &lt;controller type='pci' model='pci-root'/&gt;
 160 &lt;controller type='pci' model='pci-root'/&gt;
 161 &lt;controller type='pci' model='pci-root'/&gt;</pre>
 162
 163     <p>
 164       if you expect to hotplug up to three PCI devices assigned
 165       from the host.
 166     </p>
 167
 168     <h2><a id="aarch64">aarch64 architecture</a></h2>
 169
 170     <h3><a id="aarch64-virt">mach-virt (virt) machine type</a></h3>
 171
 172     <p>
 173       This machine type mostly behaves the same as the
 174       <a href="#x86_64-q35">q35 machine type</a>, so you can just
 175       refer to that section for information.
 176     </p>
 177     <p>
 178       The only difference worth mentioning is that using legacy
 179       PCI for <code>mach-virt</code> guests is extremely uncommon,
 180       so you'll probably never need to add controllers other than
 181       <code>pcie-root-port</code>.
 182     </p>
 183
 184   </body>
 185 </html>