curl: Enable multi-conn for read-only connections
Comparing before and after this commit shows approximately double the
performance. In other tests this allowed us to download files from
web servers at line speed.
Benchmark 1: nbdkit -r curl file:/var/tmp/jammy-server-cloudimg-amd64.raw --run "nbdcopy -p \$uri null:"
Time (mean ± σ): 943.8 ms ± 18.8 ms [User: 316.2 ms, System: 1029.7 ms]
Range (min … max): 923.7 ms … 989.2 ms 10 runs
Benchmark 2: ~/d/nbdkit/nbdkit -r curl file:/var/tmp/jammy-server-cloudimg-amd64.raw --run "nbdcopy -p \$uri null:"
Time (mean ± σ): 455.0 ms ± 6.2 ms [User: 542.2 ms, System: 1824.7 ms]
Range (min … max): 449.1 ms … 471.6 ms 10 runs
Summary
' ~/d/nbdkit/nbdkit -r curl file:/var/tmp/jammy-server-cloudimg-amd64.raw --run "nbdcopy -p \$uri null:" ' ran
2.07 ± 0.05 times faster than ' nbdkit -r curl file:/var/tmp/jammy-server-cloudimg-amd64.raw --run "nbdcopy -p \$uri null:" '
Multi-conn is enabled only when we know the connection is read-only:
$ ./nbdkit -r curl file:/var/tmp/jammy-server-cloudimg-amd64.raw --run ' nbdinfo $uri ' | grep can_multi_conn
can_multi_conn: true
$ ./nbdkit curl file:/var/tmp/jammy-server-cloudimg-amd64.raw --run ' nbdinfo $uri ' | grep can_multi_conn
can_multi_conn: false
See also:
https://listman.redhat.com/archives/libguestfs/2023-February/030581.html
Reviewed-by: Eric Blake <eblake@redhat.com>