3 nbdkit-xz-filter - nbdkit xz filter
7 nbdkit --filter=xz file FILENAME.xz
11 nbdkit --filter=xz curl https://example.com/FILENAME.xz
15 C<nbdkit-xz-filter> is a filter for L<nbdkit(1)> which uncompresses
16 the underlying plugin on the fly. The filter only supports read-only
19 =head2 Getting best random access performance from xz
21 L<xz(1)> files are split into streams and blocks. Most xz files
22 contain only one stream which contains one or more blocks. You can
23 find out how many streams and blocks are in an xz file by doing:
25 $ xz --list winxp.img.xz
26 Strms Blocks Compressed Uncompressed Ratio Check Filename
27 1 1 2,100.0 MiB 6,144.0 MiB 0.342 CRC64 winxp.img.xz
31 xz files are seekable on block boundaries only. Seeking is done by
32 seeking directly to the lower block boundary, then uncompressing data
33 until the precise byte is reached. As uncompressing xz data is slow,
34 B<to get best random access performance, you must prepare your xz
35 files with many small blocks.>
37 Use the I<--block-size> option with a small-ish block size. For
38 example this is the same image compressed with a 16 MB block size:
40 $ xz --best --block-size=16777216 winxp.img
42 $ xz --list winxp.img.xz
43 Strms Blocks Compressed Uncompressed Ratio Check Filename
44 1 384 2,120.1 MiB 6,144.0 MiB 0.345 CRC64 winxp.img.xz
48 This file can be accessed randomly. At most 16 MB will have to be
49 uncompressed to seek to any byte.
51 As you would expect, xz cannot compress as efficiently when using a
52 smaller block size. The space penalty in the above example is
53 S<E<lt> 1%> of the compressed file size.
59 =item B<xz-max-block=>SIZE
61 The maximum block size that the filter will read. The filter will
62 refuse to read xz files that contain any block larger than this size.
64 See the discussion above about creating xz files with small block
65 sizes in order to reduce memory usage and increase performance.
67 This parameter is optional. If not specified it defaults to 512M.
69 =item B<xz-max-depth=>N
71 Maximum number of blocks stored in the LRU block cache.
73 This parameter is optional. If not specified it defaults to 8.
75 The filter may allocate up to
76 S<maximum block size in file × maxdepth>
77 bytes of memory I<per connection>.
85 =item F<$filterdir/nbdkit-xz-filter.so>
89 Use C<nbdkit --dump-config> to find the location of C<$filterdir>.
95 C<nbdkit-xz-filter> first appeared in nbdkit 1.10.
96 It is derived from an earlier plugin that first appeared in nbdkit 1.0.
102 L<nbdkit-curl-plugin(1)>,
103 L<nbdkit-file-plugin(1)>,