3 nbdkit-vddk-plugin - nbdkit VMware VDDK plugin
7 nbdkit vddk file=FILENAME
8 [config=FILENAME] [cookie=COOKIE] [libdir=LIBRARY]
9 [nfchostport=PORT] [single-link=true]
10 [password=PASSWORD | password=- | password=+FILENAME
12 [port=PORT] [server=HOSTNAME] [snapshot=MOREF]
13 [thumbprint=THUMBPRINT] [transports=MODE:MODE:...]
14 [unbuffered=true] [user=USERNAME] [vm=moref=ID]
15 nbdkit vddk --dump-plugin
19 C<nbdkit-vddk-plugin> is an L<nbdkit(1)> plugin that serves files from
20 local VMware VMDK files, VMware ESXi servers, VMware VCenter servers,
21 and other sources. It requires VMware's proprietary VDDK library that
22 you must download yourself separately.
24 The plugin can serve read-only (if the I<-r> option is used) or
29 =head2 Open a local VMDK file
31 nbdkit vddk file=/absolute/path/to/file.vmdk
33 Note that when opening local files the C<file=> parameter B<must> be
36 Because VDDK needs to take a lock on this file, the file must be on a
37 writable filesystem (unless you use the I<-r> option).
39 =head2 Open a file on a remote VMware ESXi hypervisor
41 Connect directly to a VMware ESXi hypervisor and export a particular
44 nbdkit vddk user=root password=+/tmp/rootpw \
45 server=esxi.example.com thumbprint=xx:xx:xx:... \
47 file="[datastore1] Fedora/Fedora.vmdk"
49 C<user> and C<password> must be specified. Use C<password=+FILENAME>
50 to provide the password securely in a file.
52 C<server> is the hostname of the ESXi server.
54 C<thumbprint> is the thumb print for validating the SSL certificate.
55 How to find the thumb print of a server is described in
56 L</THUMBPRINTS> below.
58 C<vm> is the Managed Object Reference ("moref") of the virtual
59 machine. See L</MANAGED OBJECT REFERENCE> below.
61 C<file> is the file you want to open, usually in the form
62 S<C<"[datastore] vmname/vmname.vmdk">>. See L</FILE PARAMETER> below.
64 =head2 Open a file on a remote VMware vCenter server
66 Connect via VMware vCenter and export a particular file:
68 nbdkit vddk user=root password=vmware \
69 server=vcenter.example.com thumbprint=xx:xx:xx:... \
71 file="[datastore1] Fedora/Fedora.vmdk"
73 C<user> and C<password> must be specified. Use C<password=+FILENAME>
74 to provide the password securely in a file.
76 C<server> is the hostname of the vCenter server.
78 C<thumbprint> is the thumb print for validating the SSL certificate.
79 How to find the thumb print of a server is described in
80 L</THUMBPRINTS> below.
82 C<vm> is the Managed Object Reference ("moref") of the virtual
83 machine. See L</MANAGED OBJECT REFERENCE> below.
85 C<file> is the file you want to open, usually in the form
86 S<C<"[datastore] vmname/vmname.vmdk">>. See L</FILE PARAMETER> below.
90 All parameters are optional except:
96 is required for opening a local VMDK file.
110 When making a remote connection you must supply all of these
117 =item B<config=>FILENAME
119 The name of the VDDK configuration file.
121 =item B<cookie=>COOKIE
123 Cookie from existing authenticated session on the host.
125 =item B<file=>FILENAME
127 =item B<file=[>datastoreB<] >vmname/vmnameB<.vmdk>
129 Set the name of the VMDK file to serve.
131 For local files you B<must> supply an absolute path.
132 For remote files see L</FILE PARAMETER> section below.
134 If a VM has multiple disks, nbdkit can only serve one at a time. To
135 serve more than one you must run multiple copies of nbdkit. (See
138 =item B<libdir=>PATHNAME
140 This sets the path of the VMware VDDK distribution.
142 VDDK uses this to load its own plugins, if this path is unspecified or
143 wrong then VDDK will work with reduced functionality.
145 If the parameter is not given, then a hard-coded path determined at
146 compile time is used, see L</DUMP-PLUGIN OUTPUT> below.
148 =item B<nfchostport=>PORT
150 Port used to establish an NFC connection to ESXi. Defaults to 902.
152 (Only supported in VDDK ≥ 5.5.5 and ≥ 6.0.1)
154 =item B<password=>PASSWORD
156 Set the password to use when connecting to the remote server.
158 Note that passing this on the command line is not secure on shared
163 Ask for the password (interactively) when nbdkit starts up.
165 =item B<password=+>FILENAME
167 Read the password from the named file. This is a secure method
168 to supply a password, as long as you set the permissions on the file
171 =item B<password=->FD
173 Read the password from file descriptor number C<FD>, inherited from
174 the parent process when nbdkit starts up. This is also a secure
175 method to supply a password.
179 The port on the VCenter/ESXi host. Defaults to 443.
181 =item B<server=>HOSTNAME
183 The hostname or IP address of VCenter or ESXi host.
185 =item B<single-link=true>
187 Open the current link, not the entire chain. This corresponds to the
188 C<VIXDISKLIB_FLAG_OPEN_SINGLE_LINK> flag.
190 =item B<snapshot=>MOREF
192 The Managed Object Reference of the snapshot.
193 See L</MANAGED OBJECT REFERENCE> below.
195 =item B<thumbprint=>THUMBPRINT
197 The SSL (SHA1) thumbprint for validating the SSL certificate.
200 C<xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx>
201 (20 hex digit pairs).
203 See L</THUMBPRINTS> below for how to get this.
205 =item B<transports=>MODEB<:>MODEB<:>...
207 List of one or more transport modes to use. Possible values include
208 ‘nbd’, ‘nbdssl’, ‘san’, ‘hotadd’, ‘file’ (there may be others). If
209 not given, VDDK will try to choose the best transport mode.
211 =item B<unbuffered=true>
213 Disable host caching. This corresponds to the
214 C<VIXDISKLIB_FLAG_OPEN_UNBUFFERED> flag.
216 =item B<user=>USERNAME
218 The username to connect to the remote server as.
222 The Managed Object Reference ("moref") of the virtual machine.
223 See L</MANAGED OBJECT REFERENCE> below.
225 =item B<vimapiver=>APIVER
227 This parameter is ignored for backwards compatibility.
231 =head1 LIBRARY AND CONFIG FILE LOCATIONS
233 If the VDDK library (F<libvixDiskLib.so.I<N>>) is located on a
234 non-standard path, you may need to set C<LD_LIBRARY_PATH> or modify
235 F</etc/ld.so.conf> before this plugin will work. In addition you may
236 want to set the C<libdir> parameter so that the VDDK library can load
237 plugins like Advanced Transport.
239 For 64 bit platforms pass the F<lib64> subdirectory:
241 export LD_LIBRARY_PATH=/path/to/vmware-vix-disklib-distrib/lib64
243 For 32 bit platforms pass the F<lib32> subdirectory:
245 export LD_LIBRARY_PATH=/path/to/vmware-vix-disklib-distrib/lib32
247 Then pass the VDDK distribution directory as C<libdir> along with
248 other parameters as required:
251 libdir=/path/to/vmware-vix-disklib-distrib \
254 VDDK itself looks in a few default locations for the optional
255 configuration file, usually including F</etc/vmware/config> and
256 F<$HOME/.vmware/config>, but you can override this using the C<config>
259 =head1 FILE PARAMETER
261 The C<file> parameter can either be a local file, in which case it
262 must be the absolute path. Or it can refer to a remote file on the
263 VMware server in the format S<C<"[datastore] vmname/vmname.vmdk">>.
265 For remote files you can find the path using L<virsh(1)>. For ESXi:
267 virsh -c 'esx://esxi.example.com?no_verify=1' dumpxml guestname
271 virsh -c 'vpx://vcenter.example.com/Datacenter/esxi.example.com?no_verify=1' \
276 The thumbprint is a 20 byte string containing the SSL (SHA1)
277 fingerprint of the remote VMware server and it is required when making
278 a remote connection. There are two ways to obtain this.
280 =head2 Extracting thumbprint from ESXi or vCenter server
282 To extract the thumbprint, log in to the ESXi hypervisor shell and run
285 # openssl x509 -in /etc/vmware/ssl/rui.crt -fingerprint -sha1 -noout
287 For VMware vCenter servers the thumbprint is printed on the text
288 console of the server or is available by logging in to the server and
291 # openssl x509 -in /etc/vmware-vpx/ssl/rui.crt -fingerprint -sha1 -noout
293 =head2 Trick: Get VDDK to tell you the thumbprint
295 Another (easier) way to get the thumbprint of a server is to connect
296 to the server using a bogus thumbprint with debugging enabled:
298 nbdkit -f -v vddk server=esxi.example.com [...] thumbprint=12
299 qemu-img info nbd:localhost:10809
301 The nbdkit process will try to connect (and fail because the
302 thumbprint is wrong). However in the debug output will be a message
305 nbdkit: debug: VixDiskLibVim: Failed to verify SSL certificate: actual thumbprint=B2:31:BD:DE:9F:DB:9D:E0:78:EF:30:42:8A:41:B0:28:92:93:C8:DD expected=12
307 This gives you the server’s real thumbprint. Of course this method is
308 not secure since it allows a Man-in-the-Middle (MITM) attack.
310 =head1 MANAGED OBJECT REFERENCE
312 Some parameters require you to pass in the Managed Object Reference
313 ("moref") of an object on the VMware server.
315 For VMware ESXi hypervisors, the C<vm> moref is a number
316 (eg. C<vm=moref=2>). For VMware VCenter it is a string beginning with
317 C<"vm-">) (eg. C<vm=moref=vm-16>). Across ESXi and vCenter the
318 numbers are different even for the same virtual machine.
320 If you have libvirt E<ge> 3.7, the moref is available in the
321 L<virsh(1)> C<dumpxml> output:
323 $ virsh -c 'esx://esxi.example.com?no_verify=1' dumpxml guestname
325 <vmware:moref>2</vmware:moref>
330 $ virsh -c 'vpx://vcenter.example.com/Datacenter/esxi.example.com?no_verify=1' \
333 <vmware:moref>vm-16</vmware:moref>
336 An alternative way to find the moref of a VM is using the
337 C<moRefFinder.pl> script written by William Lam
338 (L<http://www.virtuallyghetto.com/2011/11/vsphere-moref-managed-object-reference.html>
339 L<https://blogs.vmware.com/vsphere/2012/02/uniquely-identifying-virtual-machines-in-vsphere-and-vcloud-part-2-technical.html>).
341 =head1 DUMP-PLUGIN OUTPUT
343 To query more information about the plugin (and whether it is
346 nbdkit vddk --dump-plugin
348 If the plugin is not present, not working or the library path is wrong
349 you will get an error.
351 If it works the output will include:
355 =item C<vddk_default_libdir=...>
357 The compiled-in library path. Use C<libdir=PATHNAME> to override this
360 =item C<vddk_has_nfchostport=1>
362 If this is printed then the C<nfchostport=PORT> parameter is supported
365 =item C<vddk_dll=...>
367 Prints the full path to the VDDK shared library. Since this requires
368 a glibc extension it may not be available in all builds of the plugin.
372 =head1 DEBUGGING VDDK
374 Debugging messages can be very helpful if you have problems connecting
375 to VMware servers, or to find the list of available transport modes,
376 or to diagnose SAN problems.
378 Run nbdkit like this to see all debugging messages:
380 nbdkit -f -v vddk file=FILENAME [...]
384 =head2 Sector size limitation
386 The VDDK plugin can only answer read/write requests on whole 512 byte
387 sector boundaries. This is because the VDDK Read and Write APIs only
390 The plugin could be extended in future to support byte granularity,
391 but common NBD clients don't need it so it's not a priority.
395 Handling threads in the VDDK API is complex and does not map well to
396 any of the thread models offered by nbdkit (see
397 L<nbdkit-plugin(3)/THREADS>). The plugin uses the nbdkit
398 C<SERIALIZE_ALL_REQUESTS> model, but technically even this is not
399 completely safe. This is a subject of future work.
403 For VMs with multiple disks, it would be nice to map the disk names to
404 NBD export names. However nbdkit core will need to be extended to
407 =head2 Out of memory errors
409 In the verbose log you may see errors like:
411 nbdkit: vddk[3]: error: [NFC ERROR] NfcFssrvrProcessErrorMsg:
412 received NFC error 5 from server: Failed to allocate the
413 requested 2097176 bytes
415 This seems especially common when there are multiple parallel
416 connections open to the VMware server.
418 These can be caused by resource limits set on the VMware server. You
419 can increase the limit for the NFC service by editing
420 F</etc/vmware/hostd/config.xml> and adjusting the
421 C<E<lt>maxMemoryE<gt>> setting:
424 <path>libnfcsvc.so</path>
425 <enabled>true</enabled>
426 <maxMemory>50331648</maxMemory>
427 <maxStreamMemory>10485760</maxStreamMemory>
430 and restarting the C<hostd> service:
432 # /etc/init.d/hostd restart
434 For more information see L<https://bugzilla.redhat.com/1614276>.
436 =head1 SUPPORTED VERSIONS OF VDDK
438 This plugin requires VDDK E<ge> 5.1.1.
440 It has been tested with all versions up to 6.7 (but should work with
443 VDDK E<ge> 6.0 should be used if possible. This is the first version
444 which added Flush support which is crucial for data integrity when
451 =item F<$plugindir/nbdkit-vddk-plugin.so>
455 Use C<nbdkit --dump-config> to find the location of C<$plugindir>.
461 C<nbdkit-vddk-plugin> first appeared in nbdkit 1.2.
467 L<nbdkit-readahead-filter(1)>,
468 L<nbdkit-retry-filter(1)>,
470 L<https://www.vmware.com/support/developer/vddk/>
478 Copyright (C) 2013-2019 Red Hat Inc.