tests: vsock: Use VMADDR_CID_LOCAL for loopback testing.

[nbdkit/ericb.git] / docs / nbdkit-plugin.pod

=head1 NAME

nbdkit-plugin - how to write nbdkit plugins

=head1 SYNOPSIS

 #define NBDKIT_API_VERSION 2
 
 #include <nbdkit-plugin.h>
 
 #define THREAD_MODEL NBDKIT_THREAD_MODEL_SERIALIZE_ALL_REQUESTS
 
 static void *
 myplugin_open (void)
 {
   /* create a handle ... */
   return handle;
 }
 
 static struct nbdkit_plugin plugin = {
   .name              = "myplugin",
   .open              = myplugin_open,
   .get_size          = myplugin_get_size,
   .pread             = myplugin_pread,
   .pwrite            = myplugin_pwrite,
   /* etc */
 };
 
 NBDKIT_REGISTER_PLUGIN(plugin)

When this has been compiled to a shared library, do:

 nbdkit [--args ...] ./myplugin.so [key=value ...]

When debugging, use the I<-fv> options:

 nbdkit -fv ./myplugin.so [key=value ...]

=head1 DESCRIPTION

An nbdkit plugin is a new source device which can be served using the
Network Block Device (NBD) protocol.  This manual page describes how
to create an nbdkit plugin in C.

To see example plugins:
L<https://github.com/libguestfs/nbdkit/tree/master/plugins>

Plugins written in C have an ABI guarantee: a plugin compiled against
an older version of nbdkit will still work correctly when loaded with
a newer nbdkit.  We also try (but cannot guarantee) to support plugins
compiled against a newer version of nbdkit when loaded with an older
nbdkit, although the plugin may have reduced functionality if it
depends on features only provided by newer nbdkit.

For plugins written in C, we also provide an API guarantee: a plugin
written against an older header will still compile unmodified with a
newer nbdkit.

The API guarantee does not always apply to plugins written in other
(non-C) languages which may have to adapt to changes when recompiled
against a newer nbdkit.

To write plugins in other languages, see:
__LANG_PLUGIN_LINKS__.

=head1 C<#define NBDKIT_API_VERSION 2>

Plugins must choose which API version they want to use, by defining
NBDKIT_API_VERSION to a positive integer prior to including
C<nbdkit-plugin.h> (or any other nbdkit header).  The default version
is 1 for backwards-compatibility with nbdkit v1.1.26 and earlier;
however, it is recommended that new plugins be written to the maximum
version (currently 2) as it enables more features and better
interaction with nbdkit filters.  Therefore, the rest of this document
only covers the version 2 interface.  A newer nbdkit will always
support plugins written in C which were compiled against any prior API
version.

=head1 C<#include E<lt>nbdkit-plugin.hE<gt>>

All plugins should start by including this header file, after
optionally choosing an API version.

=head1 C<#define THREAD_MODEL>

All plugins must define a thread model.  See L</THREADS> below for
details.  It is generally safe to use:

 #define THREAD_MODEL NBDKIT_THREAD_MODEL_SERIALIZE_ALL_REQUESTS

=head1 C<struct nbdkit_plugin>

All plugins must define and register one C<struct nbdkit_plugin>,
which contains the name of the plugin and pointers to callback
functions.

 static struct nbdkit_plugin plugin = {
   .name              = "myplugin",
   .longname          = "My Plugin",
   .description       = "This is my great plugin for nbdkit",
   .open              = myplugin_open,
   .get_size          = myplugin_get_size,
   .pread             = myplugin_pread,
   .pwrite            = myplugin_pwrite,
   /* etc */
 };
 
 NBDKIT_REGISTER_PLUGIN(plugin)

The C<.name> field is the name of the plugin.

The callbacks are described below (see L</CALLBACKS>).  Only C<.name>,
C<.open>, C<.get_size> and C<.pread> are required.  All other
callbacks can be omitted.  However almost all plugins should have a
C<.close> callback.  Most real-world plugins will also want to declare
some of the other callbacks.

The nbdkit server calls the callbacks in the following order over the
lifetime of the plugin:

=over 4

=item C<.load>

is called once just after the plugin is loaded into memory.

=item C<.config> and C<.config_complete>

C<.config> is called zero or more times during command line parsing.
C<.config_complete> is called once after all configuration information
has been passed to the plugin (but not during C<nbdkit --dump-plugin>).

Both are called after loading the plugin but before any connections
are accepted.

=item C<.thread_model>

In normal operation, C<.thread_model> is called once after
C<.config_complete> has validated all configuration information, and
before any connections are accepted.  However, during C<nbdkit
--dump-plugin>, it is called after any C<.config> calls but without
C<.config_complete> (so a plugin which determines the results from a
script must be prepared for a missing script).

=item C<.preconnect>

Called when a TCP connection has been made to the server.  This
happens early, before NBD or TLS negotiation.

=item C<.open>

A new client has connected and finished the NBD handshake.  TLS
negotiation (if required) has been completed successfully.

=item C<.can_write>, C<.get_size> and other option negotiation callbacks

These are called during option negotiation with the client, but before
any data is served.  These callbacks may return different values
across different C<.open> calls, but within a single connection, they
are called at most once and cached by nbdkit for that connection.

=item C<.pread>, C<.pwrite> and other data serving callbacks

After option negotiation has finished, these may be called to serve
data.  Depending on the thread model chosen, they might be called in
parallel from multiple threads.  The data serving callbacks include a
flags argument; the results of the negotiation callbacks influence
whether particular flags will ever be passed to a data callback.

=item C<.close>

The client has disconnected.

=item C<.preconnect>, C<.open> ... C<.close>

The sequence C<.preconnect>, C<.open> ... C<.close> can be called
repeatedly over the lifetime of the plugin, and can be called in
parallel (depending on the thread model).

=item C<.unload>

is called once just before the plugin is unloaded from memory.

=back

=head1 FLAGS

The following flags are defined by nbdkit, and used in various data
serving callbacks as follows:

=over 4

=item C<NBDKIT_FLAG_MAY_TRIM>

This flag is used by the C<.zero> callback; there is no way to disable
this flag, although a plugin that does not support trims as a way to
write zeroes may ignore the flag without violating expected semantics.

=item C<NBDKIT_FLAG_FUA>

This flag represents Forced Unit Access semantics.  It is used by the
C<.pwrite>, C<.zero>, and C<.trim> callbacks to indicate that the
plugin must not return a result until the action has landed in
persistent storage.  This flag will not be sent to the plugin unless
C<.can_fua> is provided and returns C<NBDKIT_FUA_NATIVE>.

=back

The following defines are valid as successful return values for
C<.can_fua>:

=over 4

=item C<NBDKIT_FUA_NONE>

Forced Unit Access is not supported; the client must manually request
a flush after writes have completed.  The C<NBDKIT_FLAG_FUA> flag will
not be passed to the plugin's write callbacks.

=item C<NBDKIT_FUA_EMULATE>

The client may request Forced Unit Access, but it is implemented by
emulation, where nbdkit calls C<.flush> after a write operation; this
is semantically correct, but may hurt performance as it tends to flush
more data than just what the client requested.  The C<NBDKIT_FLAG_FUA>
flag will not be passed to the plugin's write callbacks.

=item C<NBDKIT_FUA_NATIVE>

The client may request Forced Unit Access, which results in the
C<NBDKIT_FLAG_FUA> flag being passed to the plugin's write callbacks
(C<.pwrite>, C<.trim>, and C<.zero>).  When the flag is set, these
callbacks must not return success until the client's request has
landed in persistent storage.

=back

The following defines are valid as successful return values for
C<.can_cache>:

=over 4

=item C<NBDKIT_CACHE_NONE>

The server does not advertise caching support, and rejects any
client-requested caching. Any C<.cache> callback is ignored.

=item C<NBDKIT_CACHE_EMULATE>

The nbdkit server advertises cache support to the client, where the
client may request that the server cache a region of the export to
potentially speed up future read and/or write operations on that
region. The nbdkit server implements the caching by calling C<.pread>
and ignoring the results. This option exists to ease the
implementation of a common form of caching; any C<.cache> callback is
ignored.

=item C<NBDKIT_CACHE_NATIVE>

The nbdkit server advertises cache support to the client, where the
client may request that the server cache a region of the export to
potentially speed up future read and/or write operations on that
region. The nbdkit server calls the C<.cache> callback to perform the
caching; if that callback is missing, the client's cache request
succeeds without doing anything.

=back

=head1 ERROR HANDLING

If there is an error in the plugin, the plugin should call
C<nbdkit_error> to report an error message; additionally, if the
callback is involved in serving data, the plugin should call
C<nbdkit_set_error> to influence the error code that will be sent to
the client.  These two functions can be called in either order.  Then,
the callback should return the appropriate error indication,
eg. C<NULL> or C<-1>.

If the call to C<nbdkit_set_error> is omitted while serving data, then
the global variable C<errno> may be used.  For plugins which have
C<.errno_is_preserved != 0> the core code will use C<errno>.  In
plugins written in non-C languages, we usually cannot trust that
C<errno> will not be overwritten when returning from that language to
C.  In that case, either the plugin must call C<nbdkit_set_error> or
hard-coded C<EIO> is used.

C<nbdkit_error> has the following prototype and works like
L<printf(3)>:

 void nbdkit_error (const char *fs, ...);
 void nbdkit_verror (const char *fs, va_list args);

For convenience, C<nbdkit_error> preserves the value of C<errno>, and
also supports the glibc extension of a single C<%m> in a format string
expanding to C<strerror(errno)>, even on platforms that don't support
that natively.

C<nbdkit_set_error> can be called at any time, but only has an impact
during callbacks for serving data, and only when the callback returns
an indication of failure.  It has the following prototype:

 void nbdkit_set_error (int err);

=head1 CALLBACKS

=head2 C<.name>

 const char *name;

This field (a string) is required, and B<must> contain only ASCII
alphanumeric characters and be unique amongst all plugins.

=head2 C<.version>

 const char *version;

Plugins may optionally set a version string which is displayed in help
and debugging output.

=head2 C<.longname>

 const char *longname;

An optional free text name of the plugin.  This field is used in error
messages.

=head2 C<.description>

 const char *description;

An optional multi-line description of the plugin.

=head2 C<.load>

 void load (void);

This is called once just after the plugin is loaded into memory.  You
can use this to perform any global initialization needed by the
plugin.

=head2 C<.unload>

 void unload (void);

This may be called once just before the plugin is unloaded from
memory.  Note that it's not guaranteed that C<.unload> will always be
called (eg. the server might be killed or segfault), so you should try
to make the plugin as robust as possible by not requiring cleanup.
See also L</SHUTDOWN> below.

=head2 C<.dump_plugin>

 void dump_plugin (void);

This optional callback is called when the
S<C<nbdkit plugin --dump-plugin>> command is used.  It should print
any additional informative C<key=value> fields to stdout as needed.
Prefixing the keys with the name of the plugin will avoid conflicts.

=head2 C<.config>

 int config (const char *key, const char *value);

On the nbdkit command line, after the plugin filename, come an
optional list of C<key=value> arguments.  These are passed to the
plugin through this callback when the plugin is first loaded and
before any connections are accepted.

This callback may be called zero or more times.

Both C<key> and C<value> parameters will be non-NULL.  The strings are
owned by nbdkit but will remain valid for the lifetime of the plugin,
so the plugin does not need to copy them.

The key will be a non-empty string beginning with an ASCII alphabetic
character (C<A-Z> C<a-z>).  The rest of the key must contain only
ASCII alphanumeric plus period, underscore or dash characters (C<A-Z>
C<a-z> C<0-9> C<.> C<_> C<->).  The value may be an arbitrary string,
including an empty string.

The names of C<key>s accepted by plugins is up to the plugin, but you
should probably look at other plugins and follow the same conventions.

If the value is a relative path, then note that the server changes
directory when it starts up.  See L</FILENAMES AND PATHS> above.

If the C<.config> callback is not provided by the plugin, and the user
tries to specify any C<key=value> arguments, then nbdkit will exit
with an error.

If there is an error, C<.config> should call C<nbdkit_error> with an
error message and return C<-1>.

=head2 C<.magic_config_key>

 const char *magic_config_key;

This optional string can be used to set a "magic" key used when
parsing plugin parameters.  It affects how "bare parameters" (those
which do not contain an C<=> character) are parsed on the command
line.

If C<magic_config_key != NULL> then any bare parameters are passed to
the C<.config> method as: S<C<config (magic_config_key, argv[i]);>>.

If C<magic_config_key> is not set then we behave as in nbdkit E<lt>
1.7: If the first parameter on the command line is bare then it is
passed to the C<.config> method as: S<C<config ("script", value);>>.
Any other bare parameters give errors.

=head2 C<.config_complete>

 int config_complete (void);

This optional callback is called after all the configuration has been
passed to the plugin.  It is a good place to do checks, for example
that the user has passed the required parameters to the plugin.

If there is an error, C<.config_complete> should call C<nbdkit_error>
with an error message and return C<-1>.

=head2 C<.config_help>

 const char *config_help;

This optional multi-line help message should summarize any
C<key=value> parameters that it takes.  It does I<not> need to repeat
what already appears in C<.description>.

If the plugin doesn't take any config parameters you should probably
omit this.

=head2 C<.thread_model>

 int thread_model (void)

This optional callback is called after all the configuration has been
passed to the plugin.  It can be used to force a stricter thread model
based on configuration, compared to C<THREAD_MODEL>.  See L</THREADS>
below for details.  Attempts to request a looser (more parallel) model
are silently ignored.

If there is an error, C<.thread_model> should call C<nbdkit_error>
with an error message and return C<-1>.

=head2 C<.preconnect>

 int preconnect (int readonly);

This optional callback is called when a TCP connection has been made
to the server.  This happens early, before NBD or TLS negotiation.  If
TLS authentication is required to access the server, then it has
B<not> been negotiated at this point.

For security reasons (to avoid denial of service attacks) this
callback should be written to be as fast and take as few resources as
possible.  If you use this callback, only use it to do basic access
control, such as checking C<nbdkit_peer_name> against a whitelist (see
L</PEER NAME> and L<nbdkit-ip-filter(1)>).  It may be better to do
access control outside the server, for example using TCP wrappers or a
firewall.

The C<readonly> flag informs the plugin that the server was started
with the I<-r> flag on the command line.

Returning C<0> will allow the connection to continue.  If there is an
error or you want to deny the connection, call C<nbdkit_error> with an
error message and return C<-1>.

=head2 C<.open>

 void *open (int readonly);

This is called when a new client connects to the nbdkit server.  The
callback should allocate a handle and return it.  This handle
is passed back to other callbacks and could be freed in the C<.close>
callback.

Note that the handle is completely opaque to nbdkit, but it must not
be NULL.  If you don't need to use a handle, return
C<NBDKIT_HANDLE_NOT_NEEDED> which is a static non-NULL pointer.

The C<readonly> flag informs the plugin that the server was started
with the I<-r> flag on the command line which forces connections to be
read-only.  Note that the plugin may I<additionally> force the
connection to be readonly (even if this flag is false) by returning
false from the C<.can_write> callback.  So if your plugin can only
serve read-only, you can ignore this parameter.

This callback is called after the NBD handshake has completed, which
includes TLS authentication (if required).  If the plugin defines a
C<.preconnect> callback, then it must be called and return with
success before C<.open> is called.

If there is an error, C<.open> should call C<nbdkit_error> with an
error message and return C<NULL>.

=head2 C<.close>

 void close (void *handle);

This is called when the client closes the connection.  It should clean
up any per-connection resources.

Note there is no way in the NBD protocol to communicate close errors
back to the client, for example if your plugin calls L<close(2)> and
you are checking for errors (as you should do).  Therefore the best
you can do is to log the error on the server.  Well-behaved NBD
clients I<should> try to flush the connection before it is closed and
check for errors, but obviously this is outside the scope of nbdkit.

=head2 C<.get_size>

 int64_t get_size (void *handle);

This is called during the option negotiation phase of the protocol
to get the size (in bytes) of the block device being exported.

The returned size must be E<ge> 0.  If there is an error, C<.get_size>
should call C<nbdkit_error> with an error message and return C<-1>.

=head2 C<.can_write>

 int can_write (void *handle);

This is called during the option negotiation phase to find out if the
handle supports writes.

If there is an error, C<.can_write> should call C<nbdkit_error> with
an error message and return C<-1>.

This callback is not required.  If omitted, then we return true iff a
C<.pwrite> callback has been defined.

=head2 C<.can_flush>

 int can_flush (void *handle);

This is called during the option negotiation phase to find out if the
handle supports the flush-to-disk operation.

If there is an error, C<.can_flush> should call C<nbdkit_error> with
an error message and return C<-1>.

This callback is not required.  If omitted, then we return true iff a
C<.flush> callback has been defined.

=head2 C<.is_rotational>

 int is_rotational (void *handle);

This is called during the option negotiation phase to find out if the
backing disk is a rotational medium (like a traditional hard disk) or
not (like an SSD).  If true, this may cause the client to reorder
requests to make them more efficient for a slow rotating disk.

If there is an error, C<.is_rotational> should call C<nbdkit_error>
with an error message and return C<-1>.

This callback is not required.  If omitted, then we return false.

=head2 C<.can_trim>

 int can_trim (void *handle);

This is called during the option negotiation phase to find out if the
plugin supports the trim/discard operation for punching holes in the
backing storage.

If there is an error, C<.can_trim> should call C<nbdkit_error> with an
error message and return C<-1>.

This callback is not required.  If omitted, then we return true iff a
C<.trim> callback has been defined.

=head2 C<.can_zero>

 int can_zero (void *handle);

This is called during the option negotiation phase to find out if the
plugin wants the C<.zero> callback to be utilized.  Support for
writing zeroes is still advertised to the client (unless the
L<nbdkit-nozero-filter(1)> is also used), so returning false merely
serves as a way to avoid complicating the C<.zero> callback to have to
fail with C<ENOTSUP> or C<EOPNOTSUPP> on the connections where it will
never be more efficient than using C<.pwrite> up front.

If there is an error, C<.can_zero> should call C<nbdkit_error> with an
error message and return C<-1>.

This callback is not required.  If omitted, then for a normal zero
request, nbdkit always tries C<.zero> first if it is present, and
gracefully falls back to C<.pwrite> if C<.zero> was absent or failed
with C<ENOTSUP> or C<EOPNOTSUPP>.

=head2 C<.can_fast_zero>

 int can_fast_zero (void *handle);

This is called during the option negotiation phase to find out if the
plugin wants to advertise support for fast zero requests.  If this
support is not advertised, a client cannot attempt fast zero requests,
and has no way to tell if writing zeroes offers any speedups compared
to using C<.pwrite> (other than compressed network traffic).  If
support is advertised, then C<.zero> will have
C<NBDKIT_FLAG_FAST_ZERO> set when the client has requested a fast
zero, in which case the plugin must fail with C<ENOTSUP> or
C<EOPNOTSUPP> up front if the request would not offer any benefits
over C<.pwrite>.  Advertising support for fast zero requests does not
require that writing zeroes be fast, only that the result (whether
success or failure) is fast, so this should be advertised when
feasible.

If there is an error, C<.can_fast_zero> should call C<nbdkit_error>
with an error message and return C<-1>.

This callback is not required.  If omitted, then nbdkit returns true
if C<.zero> is absent or C<.can_zero> returns false (in those cases,
nbdkit fails all fast zero requests, as its fallback to C<.pwrite> is
not inherently faster), otherwise false (since it cannot be determined
in advance if the plugin's C<.zero> will properly honor the semantics
of C<NBDKIT_FLAG_FAST_ZERO>).

=head2 C<.can_extents>

 int can_extents (void *handle);

This is called during the option negotiation phase to find out if the
plugin supports detecting allocated (non-sparse) regions of the disk
with the C<.extents> callback.

If there is an error, C<.can_extents> should call C<nbdkit_error> with
an error message and return C<-1>.

This callback is not required.  If omitted, then we return true iff a
C<.extents> callback has been defined.

=head2 C<.can_fua>

 int can_fua (void *handle);

This is called during the option negotiation phase to find out if the
plugin supports the Forced Unit Access (FUA) flag on write, zero, and
trim requests.  If this returns C<NBDKIT_FUA_NONE>, FUA support is not
advertised to the client; if this returns C<NBDKIT_FUA_EMULATE>, the
C<.flush> callback must work (even if C<.can_flush> returns false),
and FUA support is emulated by calling C<.flush> after any write
operation; if this returns C<NBDKIT_FUA_NATIVE>, then the C<.pwrite>,
C<.zero>, and C<.trim> callbacks (if implemented) must handle the flag
C<NBDKIT_FLAG_FUA>, by not returning until that action has landed in
persistent storage.

If there is an error, C<.can_fua> should call C<nbdkit_error> with an
error message and return C<-1>.

This callback is not required unless a plugin wants to specifically
handle FUA requests.  If omitted, nbdkit checks whether C<.flush>
exists, and behaves as if this function returns C<NBDKIT_FUA_NONE> or
C<NBDKIT_FUA_EMULATE> as appropriate.

=head2 C<.can_multi_conn>

 int can_multi_conn (void *handle);

This is called during the option negotiation phase to find out if the
plugin is prepared to handle multiple connections from a single
client.  If the plugin sets this to true then a client may try to open
multiple connections to the nbdkit server and spread requests across
all connections to maximize parallelism.  If the plugin sets it to
false (which is the default) then well-behaved clients should only
open a single connection, although we cannot control what clients do
in practice.

Specifically it means that either the plugin does not cache requests
at all.  Or if it does cache them then the effects of a C<.flush>
request or setting C<NBDKIT_FLAG_FUA> on a request must be visible
across all connections to the plugin before the plugin replies to that
request.

Properly working clients should send the same export name for each of
these connections.

If you use Linux L<nbd-client(8)> option S<I<-C num>> with
S<num E<gt> 1> then Linux checks this flag and will refuse to connect
if C<.can_multi_conn> is false.

If there is an error, C<.can_multi_conn> should call C<nbdkit_error>
with an error message and return C<-1>.

This callback is not required.  If omitted, then we return false.

=head2 C<.can_cache>

 int can_cache (void *handle);

This is called during the option negotiation phase to find out if the
plugin supports a cache operation. The nature of the caching is
unspecified (including whether there are limits on how much can be
cached at once, and whether writes to a cached region have
write-through or write-back semantics), but the command exists to let
clients issue a hint to the server that they will be accessing that
region of the export.

If this returns C<NBDKIT_CACHE_NONE>, cache support is not advertised
to the client; if this returns C<NBDKIT_CACHE_EMULATE>, caching is
emulated by the server calling C<.pread> and ignoring the results; if
this returns C<NBDKIT_CACHE_NATIVE>, then the C<.cache> callback will
be used.  If there is an error, C<.can_cache> should call
C<nbdkit_error> with an error message and return C<-1>.

This callback is not required.  If omitted, then we return
C<NBDKIT_CACHE_NONE> if the C<.cache> callback is missing, or
C<NBDKIT_CACHE_NATIVE> if it is defined.

=head2 C<.pread>

 int pread (void *handle, void *buf, uint32_t count, uint64_t offset,
            uint32_t flags);

During the data serving phase, nbdkit calls this callback to read data
from the backing store.  C<count> bytes starting at C<offset> in the
backing store should be read and copied into C<buf>.  nbdkit takes
care of all bounds- and sanity-checking, so the plugin does not need
to worry about that.

The parameter C<flags> exists in case of future NBD protocol
extensions; at this time, it will be 0 on input.

The callback must read the whole C<count> bytes if it can.  The NBD
protocol doesn't allow partial reads (instead, these would be errors).
If the whole C<count> bytes was read, the callback should return C<0>
to indicate there was I<no> error.

If there is an error (including a short read which couldn't be
recovered from), C<.pread> should call C<nbdkit_error> with an error
message, and C<nbdkit_set_error> to record an appropriate error
(unless C<errno> is sufficient), then return C<-1>.

=head2 C<.pwrite>

 int pwrite (void *handle, const void *buf, uint32_t count, uint64_t offset,
             uint32_t flags);

During the data serving phase, nbdkit calls this callback to write
data to the backing store.  C<count> bytes starting at C<offset> in
the backing store should be written using the data in C<buf>.  nbdkit
takes care of all bounds- and sanity-checking, so the plugin does not
need to worry about that.

This function will not be called if C<.can_write> returned false.  The
parameter C<flags> may include C<NBDKIT_FLAG_FUA> on input based on
the result of C<.can_fua>.

The callback must write the whole C<count> bytes if it can.  The NBD
protocol doesn't allow partial writes (instead, these would be
errors).  If the whole C<count> bytes was written successfully, the
callback should return C<0> to indicate there was I<no> error.

If there is an error (including a short write which couldn't be
recovered from), C<.pwrite> should call C<nbdkit_error> with an error
message, and C<nbdkit_set_error> to record an appropriate error
(unless C<errno> is sufficient), then return C<-1>.

=head2 C<.flush>

 int flush (void *handle, uint32_t flags);

During the data serving phase, this callback is used to
L<fdatasync(2)> the backing store, ie. to ensure it has been
completely written to a permanent medium.  If that is not possible
then you can omit this callback.

This function will not be called directly by the client if
C<.can_flush> returned false; however, it may still be called by
nbdkit if C<.can_fua> returned C<NBDKIT_FUA_EMULATE>.  The parameter
C<flags> exists in case of future NBD protocol extensions; at this
time, it will be 0 on input.

If there is an error, C<.flush> should call C<nbdkit_error> with an
error message, and C<nbdkit_set_error> to record an appropriate error
(unless C<errno> is sufficient), then return C<-1>.

=head2 C<.trim>

 int trim (void *handle, uint32_t count, uint64_t offset, uint32_t flags);

During the data serving phase, this callback is used to "punch holes"
in the backing store.  If that is not possible then you can omit this
callback.

This function will not be called if C<.can_trim> returned false.  The
parameter C<flags> may include C<NBDKIT_FLAG_FUA> on input based on
the result of C<.can_fua>.

If there is an error, C<.trim> should call C<nbdkit_error> with an
error message, and C<nbdkit_set_error> to record an appropriate error
(unless C<errno> is sufficient), then return C<-1>.

=head2 C<.zero>

 int zero (void *handle, uint32_t count, uint64_t offset, uint32_t flags);

During the data serving phase, this callback is used to write C<count>
bytes of zeroes at C<offset> in the backing store.

This function will not be called if C<.can_zero> returned false.  On
input, the parameter C<flags> may include C<NBDKIT_FLAG_MAY_TRIM>
unconditionally, C<NBDKIT_FLAG_FUA> based on the result of
C<.can_fua>, and C<NBDKIT_FLAG_FAST_ZERO> based on the result of
C<.can_fast_zero>.

If C<NBDKIT_FLAG_MAY_TRIM> is requested, the operation can punch a
hole instead of writing actual zero bytes, but only if subsequent
reads from the hole read as zeroes.

If C<NBDKIT_FLAG_FAST_ZERO> is requested, the plugin must decide up
front if the implementation is likely to be faster than a
corresponding C<.pwrite>; if not, then it must immediately fail with
C<ENOTSUP> or C<EOPNOTSUPP> (whether by C<nbdkit_set_error> or
C<errno>) and preferably without modifying the exported image.  It is
acceptable to always fail a fast zero request (as a fast failure is
better than attempting the write only to find out after the fact that
it was not fast after all).  Note that on Linux, support for
C<ioctl(BLKZEROOUT)> is insufficient for determining whether a zero
request to a block device will be fast (because the kernel will
perform a slow fallback when needed).

The callback must write the whole C<count> bytes if it can.  The NBD
protocol doesn't allow partial writes (instead, these would be
errors).  If the whole C<count> bytes was written successfully, the
callback should return C<0> to indicate there was I<no> error.

If there is an error, C<.zero> should call C<nbdkit_error> with an
error message, and C<nbdkit_set_error> to record an appropriate error
(unless C<errno> is sufficient), then return C<-1>.

If this callback is omitted, or if it fails with C<ENOTSUP> or
C<EOPNOTSUPP> (whether by C<nbdkit_set_error> or C<errno>), then
C<.pwrite> will be used as an automatic fallback except when the
client requested a fast zero.

=head2 C<.extents>

 int extents (void *handle, uint32_t count, uint64_t offset,
              uint32_t flags, struct nbdkit_extents *extents);

During the data serving phase, this callback is used to detect
allocated, sparse and zeroed regions of the disk.

This function will not be called if C<.can_extents> returned false.
nbdkit's default behaviour in this case is to treat the whole virtual
disk as if it was allocated.  Also, this function will not be called
by a client that does not request structured replies (the I<--no-sr>
option of nbdkit can be used to test behavior when C<.extents> is
unavailable to the client).

The callback should detect and return the list of extents overlapping
the range C<[offset...offset+count-1]>.  The C<extents> parameter
points to an opaque object which the callback should fill in by
calling C<nbdkit_add_extent>.  See L</Extents list> below.

If there is an error, C<.extents> should call C<nbdkit_error> with an
error message, and C<nbdkit_set_error> to record an appropriate error
(unless C<errno> is sufficient), then return C<-1>.

=head3 Extents list

The plugin C<extents> callback is passed an opaque pointer C<struct
nbdkit_extents *extents>.  This structure represents a list of
L<filesystem extents|https://en.wikipedia.org/wiki/Extent_(file_systems)>
describing which areas of the disk are allocated, which are sparse
(“holes”), and, if supported, which are zeroes.

The C<extents> callback should scan the disk starting at C<offset> and
call C<nbdkit_add_extent> for each extent found.

Extents overlapping the range C<[offset...offset+count-1]> should be
returned if possible.  However nbdkit ignores extents E<lt> offset so
the plugin may, if it is easier to implement, return all extent
information for the whole disk.  The plugin may return extents beyond
the end of the range.  It may also return extent information for less
than the whole range, but it must return at least one extent
overlapping C<offset>.

The extents B<must> be added in ascending order, and B<must> be
contiguous.

The C<flags> parameter of the C<.extents> callback may contain the
flag C<NBDKIT_FLAG_REQ_ONE>.  This means that the client is only
requesting information about the extent overlapping C<offset>.  The
plugin may ignore this flag, or as an optimization it may return just
a single extent for C<offset>.

 int nbdkit_add_extent (struct nbdkit_extents *extents,
                        uint64_t offset, uint64_t length, uint32_t type);

Add an extent covering C<[offset...offset+length-1]> of one of
the following four types:

=over 4

=item C<type = 0>

A normal, allocated data extent.

=item C<type = NBDKIT_EXTENT_HOLE|NBDKIT_EXTENT_ZERO>

An unallocated extent, a.k.a. a “hole”, which reads back as zeroes.
This is the normal type of hole applicable to most disks.

=item C<type = NBDKIT_EXTENT_ZERO>

An allocated extent which is known to contain only zeroes.

=item C<type = NBDKIT_EXTENT_HOLE>

An unallocated extent (hole) which does not read back as zeroes.  Note
this should only be used in specialized circumstances such as when
writing a plugin for (or to emulate) certain SCSI drives which do not
guarantee that trimmed blocks read back as zeroes.

=back

C<nbdkit_add_extent> returns C<0> on success or C<-1> on failure.  On
failure C<nbdkit_error> and/or C<nbdkit_set_error> has already been
called.  C<errno> will be set to a suitable value.

=head2 C<.cache>

 int cache (void *handle, uint32_t count, uint64_t offset, uint32_t flags);

During the data serving phase, this callback is used to give the
plugin a hint that the client intends to make further accesses to the
given region of the export.  The nature of caching is not specified
further by the NBD specification (for example, a server may place
limits on how much may be cached at once, and there is no way to
control if writes to a cached area have write-through or write-back
semantics).  In fact, the cache command can always fail and still be
compliant, and success might not guarantee a performance gain.  If
this callback is omitted, then the results of C<.can_cache> determine
whether nbdkit will reject cache requests, treat them as instant
success, or emulate caching by calling C<.pread> over the same region
and ignoring the results.

This function will not be called if C<.can_cache> did not return
C<NBDKIT_CACHE_NATIVE>.  The parameter C<flags> exists in case of
future NBD protocol extensions; at this time, it will be 0 on input. A
plugin must fail this function if C<flags> includes an unrecognized
flag, as that may indicate a requirement that the plugin comply must
with a specific caching semantic.

If there is an error, C<.cache> should call C<nbdkit_error> with an
error message, and C<nbdkit_set_error> to record an appropriate error
(unless C<errno> is sufficient), then return C<-1>.

=head1 OTHER FIELDS

The plugin struct also contains an integer field used as a
boolean in C code, but unlikely to be exposed in other language
bindings:

=over 4

=item C<.errno_is_preserved>

This defaults to 0; if non-zero, nbdkit can reliably use the value of
C<errno> when a callback reports failure, rather than the plugin
having to call C<nbdkit_set_error>.

=back

=head1 THREADS

Each nbdkit plugin must declare its maximum thread safety model by
defining the C<THREAD_MODEL> macro.  (This macro is used by
C<NBDKIT_REGISTER_PLUGIN>).  Additionally, a plugin may implement the
C<.thread_model> callback, called right after C<.config_complete> to
make a runtime decision on which thread model to use.  The nbdkit
server chooses the most restrictive model between the plugin's
C<THREAD_MODEL>, the C<.thread_model> if present, any restrictions
requested by filters, and any limitations imposed by the system (for
example, a system without atomic C<FD_CLOEXEC> will serialize all
requests, so as to avoid nbdkit leaking a new file descriptor from one
thread into a child process created by another thread).

In C<nbdkit --dump-plugin PLUGIN> output, the C<max_thread_model> line
matches the C<THREAD_MODEL> macro, and the C<thread_model> line
matches what the system finally settled on after applying all
restrictions.

The possible settings for C<THREAD_MODEL> are defined below.

=over 4

=item C<#define THREAD_MODEL NBDKIT_THREAD_MODEL_SERIALIZE_CONNECTIONS>

Only a single handle can be open at any time, and all requests happen
from one thread.

Note this means only one client can connect to the server at any time.
If a second client tries to connect it will block waiting for the
first client to close the connection.

=item C<#define THREAD_MODEL NBDKIT_THREAD_MODEL_SERIALIZE_ALL_REQUESTS>

I<This is a safe default for most plugins>.

Multiple handles can be open at the same time, but requests are
serialized so that for the plugin as a whole only one
open/read/write/close (etc) request will be in progress at any time.

This is a useful setting if the library you are using is not
thread-safe.  However performance may not be good.

=item C<#define THREAD_MODEL NBDKIT_THREAD_MODEL_SERIALIZE_REQUESTS>

Multiple handles can be open and multiple data requests can happen in
parallel.  However only one request will happen per handle at a time
(but requests on different handles might happen concurrently).

=item C<#define THREAD_MODEL NBDKIT_THREAD_MODEL_PARALLEL>

Multiple handles can be open and multiple data requests can happen in
parallel (even on the same handle).  The server may reorder replies,
answering a later request before an earlier one.

All the libraries you use must be thread-safe and reentrant, and any
code that creates a file descriptor should atomically set
C<FD_CLOEXEC> if you do not want it accidentally leaked to another
thread's child process.  You may also need to provide mutexes for
fields in your connection handle.

=back

If none of the above thread models are suitable, then use
C<NBDKIT_THREAD_MODEL_PARALLEL> and implement your own locking using
C<pthread_mutex_t> etc.

=head1 SHUTDOWN

When nbdkit receives certain signals it will shut down (see
L<nbdkit(1)/SIGNALS>).  The server will wait for any currently running
plugin callbacks to finish and also call the C<.unload> callback
before unloading the plugin.

Note that it's not guaranteed this can always happen (eg. the server
might be killed by C<SIGKILL> or segfault).

=head1 PARSING COMMAND LINE PARAMETERS

=head2 Parsing numbers

There are several functions for parsing numbers.  These all deal
correctly with overflow, out of range and parse errors, and you should
use them instead of unsafe functions like L<sscanf(3)>, L<atoi(3)> and
similar.

 int nbdkit_parse_int (const char *what, const char *str, int *r);
 int nbdkit_parse_unsigned (const char *what,
                            const char *str, unsigned *r);
 int nbdkit_parse_int8_t (const char *what,
                          const char *str, int8_t *r);
 int nbdkit_parse_uint8_t (const char *what,
                           const char *str, uint8_t *r);
 int nbdkit_parse_int16_t (const char *what,
                           const char *str, int16_t *r);
 int nbdkit_parse_uint16_t (const char *what,
                            const char *str, uint16_t *r);
 int nbdkit_parse_int32_t (const char *what,
                           const char *str, int32_t *r);
 int nbdkit_parse_uint32_t (const char *what,
                            const char *str, uint32_t *r);
 int nbdkit_parse_int64_t (const char *what,
                           const char *str, int64_t *r);
 int nbdkit_parse_uint64_t (const char *what,
                            const char *str, uint64_t *r);

Parse string C<str> into an integer of various types.  These functions
parse a decimal, hexadecimal (C<"0x...">) or octal (C<"0...">) number.

On success the functions return C<0> and set C<*r> to the parsed value
(unless C<*r == NULL> in which case the result is discarded).  On
error, C<nbdkit_error> is called and the functions return C<-1>.  On
error C<*r> is always unchanged.

The C<what> parameter is printed in error messages to provide context.
It should usually be a short descriptive string of what you are trying
to parse, eg:

 if (nbdkit_parse_int ("random seed", argv[1], &seed) == -1)
   return -1;

might print an error:

 random seed: could not parse number: "lalala"

=head2 Parsing sizes

Use the C<nbdkit_parse_size> utility function to parse human-readable
size strings such as "100M" into the size in bytes.

 int64_t nbdkit_parse_size (const char *str);

C<str> can be a string in a number of common formats.  The function
returns the size in bytes.  If there was an error, it returns C<-1>.

=head2 Parsing booleans

Use the C<nbdkit_parse_bool> utility function to parse human-readable
strings such as "on" into a boolean value.

 int nbdkit_parse_bool (const char *str);

C<str> can be a string containing a case-insensitive form of various
common toggle values.  The function returns 0 or 1 if the parse was
successful.  If there was an error, it returns C<-1>.

=head2 Reading passwords

The C<nbdkit_read_password> utility function can be used to read
passwords from config parameters:

 int nbdkit_read_password (const char *value, char **password);

For example:

 char *password = NULL;
 
 static int
 myplugin_config (const char *key, const char *value)
 {
   ..
   if (strcmp (key, "password") == 0) {
     free (password);
     if (nbdkit_read_password (value, &password) == -1)
       return -1;
   }
   ..
 }

The C<password> result string is allocated by malloc, and so you may
need to free it.

This function recognizes several password formats.  A password may be
used directly on the command line, eg:

 nbdkit myplugin password=mostsecret

But more securely this function can also read a password
interactively:

 nbdkit myplugin password=-

or from a file:

 nbdkit myplugin password=+/tmp/secret

or from a file descriptor inherited by nbdkit:

 nbdkit myplugin password=-99

(If the password begins with a C<-> or C<+> character then it must be
passed in a file).

=head1 FILENAMES AND PATHS

The server usually (not always) changes directory to C</> before it
starts serving connections.  This means that any relative paths passed
during configuration will not work when the server is running
(example: S<C<nbdkit plugin.so disk.img>>).

To avoid problems, prepend relative paths with the current directory
before storing them in the handle.  Or open files and store the file
descriptor.

=head2 C<nbdkit_absolute_path>

 char *nbdkit_absolute_path (const char *filename);

The utility function C<nbdkit_absolute_path> converts any path to an
absolute path: if it is relative, then all this function does is
prepend the current working directory to the path, with no extra
checks.

Note that this function works I<only> when used in the C<.config>, and
C<.config_complete> callbacks.

If conversion was not possible, this calls C<nbdkit_error> and returns
C<NULL>.  Note that this function does not check that the file exists.

The returned string must be freed by the caller.

=head2 C<nbdkit_realpath>

 char *nbdkit_realpath (const char *filename);

The utility function C<nbdkit_realpath> converts any path to an
absolute path, resolving symlinks.  Under the hood it uses the
C<realpath> function, and thus it fails if the path does not exist,
or it is not possible to access to any of the components of the path.

Note that this function works I<only> when used in the C<.config>, and
C<.config_complete> callbacks.

If the path resolution was not possible, this calls C<nbdkit_error>
and returns C<NULL>.

The returned string must be freed by the caller.

=head2 umask

All plugins will see a L<umask(2)> of C<0022>.

=head1 SLEEPING

A plugin that needs to sleep may call L<sleep(2)>, L<nanosleep(2)> and
similar.  However that can cause nbdkit to delay excessively when
shutting down (since it must wait for any plugin or filter which is
sleeping).  To avoid this there is a special wrapper around nanosleep
which plugins and filters should use instead.

=head2 C<nbdkit_nanosleep>

 int nbdkit_nanosleep (unsigned sec, unsigned nsec);

The utility function C<nbdkit_nanosleep> suspends the current thread,
and returns 0 if it slept at least as many seconds and nanoseconds as
requested, or -1 after calling C<nbdkit_error> if there is no point in
continuing the current command.  Attempts to sleep more than
C<INT_MAX> seconds are treated as an error.

=head1 EXPORT NAME

If the client negotiated an NBD export name with nbdkit then plugins
may read this from any connected callbacks.  Nbdkit's normal behaviour
is to accept any export name passed by the client, log it in debug
output, but otherwise ignore it.  By using C<nbdkit_export_name>
plugins may choose to filter by export name or serve different
content.

=head2 C<nbdkit_export_name>

 const char *nbdkit_export_name (void);

Return the optional NBD export name if one was negotiated with the
current client (this uses thread-local magic so no parameter is
required).  The returned string is only valid while the client is
connected, so if you need to store it in the plugin you must copy it.

The export name is a free-form text string, it is not necessarily a
path or filename and it does not need to begin with a C<'/'>
character.  The NBD protocol describes the empty string (C<"">) as a
representing a "default export" or to be used in cases where the
export name does not make sense.  B<The export name is untrusted
client data, be cautious when parsing it.>

On error, C<nbdkit_error> is called and the call returns C<NULL>.

=head1 PEER NAME

It is possible to get the address of the client when you are running
in any connected callback.

=head2 C<nbdkit_peer_name>

 int nbdkit_peer_name (struct sockaddr *addr, socklen_t *addrlen);

Return the peer (client) address, if available.  The C<addr> and
C<addrlen> parameters behave like L<getpeername(2)>.  In particular
you must initialize C<addrlen> with the size of the buffer pointed to
by C<addr>, and if C<addr> is not large enough then the address will
be truncated.

In some cases this is not available or the address returned will be
meaningless (eg. if there is a proxy between the client and nbdkit).
This call uses thread-local magic so no parameter is required to
specify the current connection.

On success this returns C<0>.  On error, C<nbdkit_error> is called and
this call returns C<-1>.

=head1 DEBUGGING

Run the server with I<-f> and I<-v> options so it doesn't fork and you
can see debugging information:

 nbdkit -fv ./myplugin.so [key=value [key=value [...]]]

To print debugging information from within the plugin, call
C<nbdkit_debug>, which has the following prototype and works like
L<printf(3)>:

 void nbdkit_debug (const char *fs, ...);
 void nbdkit_vdebug (const char *fs, va_list args);

For convenience, C<nbdkit_debug> preserves the value of C<errno>, and
also supports the glibc extension of a single C<%m> in a format string
expanding to C<strerror(errno)>, even on platforms that don't support
that natively. Note that C<nbdkit_debug> only prints things when the
server is in verbose mode (I<-v> option).

=head2 Debug Flags

The I<-v> option switches general debugging on or off, and this
debugging should be used for messages which are useful for all users
of your plugin.

In cases where you want to enable specific extra debugging to track
down bugs in plugins or filters — mainly for use by the plugin/filter
developers themselves — you can define Debug Flags.  These are global
ints called C<myplugin_debug_*>:

 int myplugin_debug_foo;
 int myplugin_debug_bar;
 ...
 if (myplugin_debug_foo) {
   nbdkit_debug ("lots of extra debugging about foo: ...");
 }

Debug Flags can be controlled on the command line using the I<-D> (or
I<--debug>) option:

 nbdkit -f -v -D myplugin.foo=1 -D myplugin.bar=2 myplugin [...]

Note C<myplugin> is the name passed to C<.name> in the C<struct
nbdkit_plugin>.

You should only use this feature for debug settings.  For general
settings use ordinary plugin parameters.  Debug Flags can only be C
ints.  They are not supported by non-C language plugins.

For convenience C<'.'> characters are replaced with C<'_'> characters
in the variable name, so both of these parameters:

 -D myplugin.foo_bar=1
 -D myplugin.foo.bar=1

correspond to the plugin variable C<myplugin_debug_foo_bar>.

=head1 INSTALLING THE PLUGIN

The plugin is a C<*.so> file and possibly a manual page.  You can of
course install the plugin C<*.so> file wherever you want, and users
will be able to use it by running:

 nbdkit /path/to/plugin.so [args]

However B<if> the shared library has a name of the form
C<nbdkit-I<name>-plugin.so> B<and if> the library is installed in the
C<$plugindir> directory, then users can be run it by only typing:

 nbdkit name [args]

The location of the C<$plugindir> directory is set when nbdkit is
compiled and can be found by doing:

 nbdkit --dump-config

If using the pkg-config/pkgconf system then you can also find the
plugin directory at compile time by doing:

 pkgconf nbdkit --variable=plugindir

=head1 PKG-CONFIG/PKGCONF

nbdkit provides a pkg-config/pkgconf file called C<nbdkit.pc> which
should be installed on the correct path when the nbdkit plugin
development environment is installed.  You can use this in autoconf
F<configure.ac> scripts to test for the development environment:

 PKG_CHECK_MODULES([NBDKIT], [nbdkit >= 1.2.3])

The above will fail unless nbdkit E<ge> 1.2.3 and the header file is
installed, and will set C<NBDKIT_CFLAGS> and C<NBDKIT_LIBS>
appropriately for compiling plugins.

You can also run pkg-config/pkgconf directly, for example:

 if ! pkgconf nbdkit --exists; then
   echo "you must install the nbdkit plugin development environment"
   exit 1
 fi

You can also substitute the plugindir variable by doing:

 PKG_CHECK_VAR([NBDKIT_PLUGINDIR], [nbdkit], [plugindir])

which defines C<$(NBDKIT_PLUGINDIR)> in automake-generated Makefiles.

=head1 WRITING PLUGINS IN OTHER PROGRAMMING LANGUAGES

You can also write nbdkit plugins in Lua, OCaml, Perl, Python, Ruby,
Rust, shell script or Tcl.  Other programming languages may be offered
in future.

For more information see:
__LANG_PLUGIN_LINKS__.

Plugins written in scripting languages may also be installed in
C<$plugindir>.  These must be called C<nbdkit-I<name>-plugin> without
any extension.  They must be executable, and they must use the shebang
header (see L<nbdkit(1)/Shebang scripts>).  For example a plugin
written in Perl called C<foo.pl> might be installed like this:

 $ head -1 foo.pl
 #!/usr/sbin/nbdkit perl

 $ sudo install -m 0755 foo.pl $plugindir/nbdkit-foo-plugin

and then users will be able to run it like this:

 $ nbdkit foo [args ...]

=head1 SEE ALSO

L<nbdkit(1)>,
L<nbdkit-nozero-filter(3)>,
L<nbdkit-filter(3)>.

Standard plugins provided by nbdkit:

__PLUGIN_LINKS__.

=head1 AUTHORS

Eric Blake

Richard W.M. Jones

Pino Toscano

=head1 COPYRIGHT

Copyright (C) 2013-2018 Red Hat Inc.