1 <?xml version="1.0" encoding="iso-8859-1"?>
2 <!DOCTYPE chapter PUBLIC "-//Samba-Team//DTD DocBook V4.2-Based Variant V1.0//EN" "http://www.samba.org/samba/DTD/samba-doc">
6 <firstname>Alexander</firstname><surname>Bokovoy</surname>
8 <address><email>ab@samba.org</email></address>
12 <firstname>Stefan</firstname><surname>Metzmacher</surname>
14 <address><email>metze@samba.org</email></address>
17 <pubdate> 27 May 2003 </pubdate>
20 <title>VFS Modules</title>
23 <title>The Samba (Posix) VFS layer</title>
26 <title>The general interface</title>
29 Each VFS operation has a vfs_op_type, a function pointer and a handle pointer in the
30 struct vfs_ops and tree macros to make it easier to call the operations.
31 (Take a look at <filename>include/vfs.h</filename> and <filename>include/vfs_macros.h</filename>.)
34 <para><programlisting>
35 typedef enum _vfs_op_type {
53 </programlisting></para>
55 <para>This struct contains the function and handle pointers for all operations.<programlisting>
57 struct vfs_fn_pointers {
62 int (*open)(struct vfs_handle_struct *handle,
63 struct connection_struct *conn,
64 const char *fname, int flags, mode_t mode);
65 int (*close)(struct vfs_handle_struct *handle,
66 struct files_struct *fsp, int fd);
67 ssize_t (*read)(struct vfs_handle_struct *handle,
68 struct files_struct *fsp, int fd, void *data, size_t n);
69 ssize_t (*write)(struct vfs_handle_struct *handle,
70 struct files_struct *fsp, int fd,
71 const void *data, size_t n);
72 SMB_OFF_T (*lseek)(struct vfs_handle_struct *handle,
73 struct files_struct *fsp, int fd,
74 SMB_OFF_T offset, int whence);
75 ssize_t (*sendfile)(struct vfs_handle_struct *handle,
76 int tofd, files_struct *fsp, int fromfd,
77 const DATA_BLOB *header, SMB_OFF_T offset, size_t count);
82 struct vfs_handles_pointers {
87 struct vfs_handle_struct *open;
88 struct vfs_handle_struct *close;
89 struct vfs_handle_struct *read;
90 struct vfs_handle_struct *write;
91 struct vfs_handle_struct *lseek;
92 struct vfs_handle_struct *sendfile;
97 </programlisting></para>
100 This macros SHOULD be used to call any vfs operation.
101 DO NOT ACCESS conn->vfs.ops.* directly !!!
105 /* File operations */
106 #define SMB_VFS_OPEN(conn, fname, flags, mode) \
107 ((conn)->vfs.ops.open((conn)->vfs.handles.open,\
108 (conn), (fname), (flags), (mode)))
109 #define SMB_VFS_CLOSE(fsp, fd) \
110 ((fsp)->conn->vfs.ops.close(\
111 (fsp)->conn->vfs.handles.close, (fsp), (fd)))
112 #define SMB_VFS_READ(fsp, fd, data, n) \
113 ((fsp)->conn->vfs.ops.read(\
114 (fsp)->conn->vfs.handles.read,\
115 (fsp), (fd), (data), (n)))
116 #define SMB_VFS_WRITE(fsp, fd, data, n) \
117 ((fsp)->conn->vfs.ops.write(\
118 (fsp)->conn->vfs.handles.write,\
119 (fsp), (fd), (data), (n)))
120 #define SMB_VFS_LSEEK(fsp, fd, offset, whence) \
121 ((fsp)->conn->vfs.ops.lseek(\
122 (fsp)->conn->vfs.handles.lseek,\
123 (fsp), (fd), (offset), (whence)))
124 #define SMB_VFS_SENDFILE(tofd, fsp, fromfd, header, offset, count) \
125 ((fsp)->conn->vfs.ops.sendfile(\
126 (fsp)->conn->vfs.handles.sendfile,\
127 (tofd), (fsp), (fromfd), (header), (offset), (count)))
130 </programlisting></para>
135 <title>Possible VFS operation layers</title>
138 These values are used by the VFS subsystem when building the conn->vfs
139 and conn->vfs_opaque structs for a connection with multiple VFS modules.
140 Internally, Samba differentiates only opaque and transparent layers at this process.
141 Other types are used for providing better diagnosing facilities.
145 Most modules will provide transparent layers. Opaque layer is for modules
146 which implement actual file system calls (like DB-based VFS). For example,
147 default POSIX VFS which is built in into Samba is an opaque VFS module.
151 Other layer types (logger, splitter, scanner) were designed to provide different
152 degree of transparency and for diagnosing VFS module behaviour.
156 Each module can implement several layers at the same time provided that only
157 one layer is used per each operation.
160 <para><programlisting>
161 typedef enum _vfs_op_layer {
162 SMB_VFS_LAYER_NOOP = -1, /* - For using in VFS module to indicate end of array */
163 /* of operations description */
164 SMB_VFS_LAYER_OPAQUE = 0, /* - Final level, does not call anything beyond itself */
165 SMB_VFS_LAYER_TRANSPARENT, /* - Normal operation, calls underlying layer after */
166 /* possibly changing passed data */
167 SMB_VFS_LAYER_LOGGER, /* - Logs data, calls underlying layer, logging may not */
169 SMB_VFS_LAYER_SPLITTER, /* - Splits operation, calls underlying layer _and_ own facility, */
170 /* then combines result */
171 SMB_VFS_LAYER_SCANNER /* - Checks data and possibly initiates additional */
172 /* file activity like logging to files _inside_ samba VFS */
174 </programlisting></para>
181 <title>The Interaction between the Samba VFS subsystem and the modules</title>
184 <title>Initialization and registration</title>
187 As each Samba module a VFS module should have a
188 <programlisting>NTSTATUS vfs_example_init(void);</programlisting> function if it's staticly linked to samba or
189 <programlisting>NTSTATUS init_module(void);</programlisting> function if it's a shared module.
193 This should be the only non static function inside the module.
194 Global variables should also be static!
198 The module should register its functions via the
200 NTSTATUS smb_register_vfs(int version, const char *name, vfs_op_tuple *vfs_op_tuples);
201 </programlisting> function.
206 <varlistentry><term>version</term>
207 <listitem><para>should be filled with SMB_VFS_INTERFACE_VERSION</para></listitem>
210 <varlistentry><term>name</term>
211 <listitem><para>this is the name witch can be listed in the
212 <command>vfs objects</command> parameter to use this module.</para></listitem>
215 <varlistentry><term>vfs_op_tuples</term>
217 this is an array of vfs_op_tuple's.
218 (vfs_op_tuples is descripted in details below.)
225 For each operation the module wants to provide it has a entry in the
230 typedef struct _vfs_op_tuple {
239 <varlistentry><term>op</term>
240 <listitem><para>the function pointer to the specified function.</para></listitem>
243 <varlistentry><term>type</term>
244 <listitem><para>the vfs_op_type of the function to specified witch operation the function provides.</para></listitem>
247 <varlistentry><term>layer</term>
248 <listitem><para>the vfs_op_layer in whitch the function operates.</para></listitem>
253 <para>A simple example:</para>
256 static vfs_op_tuple example_op_tuples[] = {
257 {SMB_VFS_OP(example_connect), SMB_VFS_OP_CONNECT, SMB_VFS_LAYER_TRANSPARENT},
258 {SMB_VFS_OP(example_disconnect), SMB_VFS_OP_DISCONNECT, SMB_VFS_LAYER_TRANSPARENT},
260 {SMB_VFS_OP(example_rename), SMB_VFS_OP_RENAME, SMB_VFS_LAYER_OPAQUE},
262 /* This indicates the end of the array */
263 {SMB_VFS_OP(NULL), SMB_VFS_OP_NOOP, SMB_VFS_LAYER_NOOP}
266 NTSTATUS init_module(void)
268 return smb_register_vfs(SMB_VFS_INTERFACE_VERSION, "example", example_op_tuples);
275 <title>How the Modules handle per connection data</title>
277 <para>Each VFS function has as first parameter a pointer to the modules vfs_handle_struct.
281 typedef struct vfs_handle_struct {
282 struct vfs_handle_struct *next, *prev;
284 struct vfs_ops vfs_next;
285 struct connection_struct *conn;
287 void (*free_data)(void **data);
293 <varlistentry><term>param</term>
294 <listitem><para>this is the module parameter specified in the <command>vfs objects</command> parameter.</para>
295 <para>e.g. for 'vfs objects = example:test' param would be "test".</para></listitem>
298 <varlistentry><term>vfs_next</term>
299 <listitem><para>This vfs_ops struct contains the information for calling the next module operations.
300 Use the SMB_VFS_NEXT_* macros to call a next module operations and
301 don't access handle->vfs_next.ops.* directly!</para></listitem>
304 <varlistentry><term>conn</term>
305 <listitem><para>This is a pointer back to the connection_struct to witch the handle belongs.</para></listitem>
308 <varlistentry><term>data</term>
309 <listitem><para>This is a pointer for holding module private data.
310 You can alloc data with connection life time on the handle->conn->mem_ctx TALLOC_CTX.
311 But you can also manage the memory allocation yourself.</para></listitem>
314 <varlistentry><term>free_data</term>
315 <listitem><para>This is a function pointer to a function that free's the module private data.
316 If you talloc your private data on the TALLOC_CTX handle->conn->mem_ctx,
317 you can set this function pointer to NULL.</para></listitem>
322 <para>Some useful MACROS for handle private data.
326 #define SMB_VFS_HANDLE_GET_DATA(handle, datap, type, ret) { \
327 if (!(handle)||((datap=(type *)(handle)->data)==NULL)) { \
328 DEBUG(0,("%s() failed to get vfs_handle->data!\n",FUNCTION_MACRO)); \
333 #define SMB_VFS_HANDLE_SET_DATA(handle, datap, free_fn, type, ret) { \
335 DEBUG(0,("%s() failed to set handle->data!\n",FUNCTION_MACRO)); \
338 if ((handle)->free_data) { \
339 (handle)->free_data(&(handle)->data); \
341 (handle)->data = (void *)datap; \
342 (handle)->free_data = free_fn; \
346 #define SMB_VFS_HANDLE_FREE_DATA(handle) { \
347 if ((handle) && (handle)->free_data) { \
348 (handle)->free_data(&(handle)->data); \
353 <para>How SMB_VFS_LAYER_TRANSPARENT functions can call the SMB_VFS_LAYER_OPAQUE functions.</para>
355 <para>The easiest way to do this is to use the SMB_VFS_OPAQUE_* macros.
360 /* File operations */
361 #define SMB_VFS_OPAQUE_OPEN(conn, fname, flags, mode) \
362 ((conn)->vfs_opaque.ops.open(\
363 (conn)->vfs_opaque.handles.open,\
364 (conn), (fname), (flags), (mode)))
365 #define SMB_VFS_OPAQUE_CLOSE(fsp, fd) \
366 ((fsp)->conn->vfs_opaque.ops.close(\
367 (fsp)->conn->vfs_opaque.handles.close,\
369 #define SMB_VFS_OPAQUE_READ(fsp, fd, data, n) \
370 ((fsp)->conn->vfs_opaque.ops.read(\
371 (fsp)->conn->vfs_opaque.handles.read,\
372 (fsp), (fd), (data), (n)))
373 #define SMB_VFS_OPAQUE_WRITE(fsp, fd, data, n) \
374 ((fsp)->conn->vfs_opaque.ops.write(\
375 (fsp)->conn->vfs_opaque.handles.write,\
376 (fsp), (fd), (data), (n)))
377 #define SMB_VFS_OPAQUE_LSEEK(fsp, fd, offset, whence) \
378 ((fsp)->conn->vfs_opaque.ops.lseek(\
379 (fsp)->conn->vfs_opaque.handles.lseek,\
380 (fsp), (fd), (offset), (whence)))
381 #define SMB_VFS_OPAQUE_SENDFILE(tofd, fsp, fromfd, header, offset, count) \
382 ((fsp)->conn->vfs_opaque.ops.sendfile(\
383 (fsp)->conn->vfs_opaque.handles.sendfile,\
384 (tofd), (fsp), (fromfd), (header), (offset), (count)))
388 <para>How SMB_VFS_LAYER_TRANSPARENT functions can call the next modules functions.</para>
390 <para>The easiest way to do this is to use the SMB_VFS_NEXT_* macros.
395 /* File operations */
396 #define SMB_VFS_NEXT_OPEN(handle, conn, fname, flags, mode) \
397 ((handle)->vfs_next.ops.open(\
398 (handle)->vfs_next.handles.open,\
399 (conn), (fname), (flags), (mode)))
400 #define SMB_VFS_NEXT_CLOSE(handle, fsp, fd) \
401 ((handle)->vfs_next.ops.close(\
402 (handle)->vfs_next.handles.close,\
404 #define SMB_VFS_NEXT_READ(handle, fsp, fd, data, n) \
405 ((handle)->vfs_next.ops.read(\
406 (handle)->vfs_next.handles.read,\
407 (fsp), (fd), (data), (n)))
408 #define SMB_VFS_NEXT_WRITE(handle, fsp, fd, data, n) \
409 ((handle)->vfs_next.ops.write(\
410 (handle)->vfs_next.handles.write,\
411 (fsp), (fd), (data), (n)))
412 #define SMB_VFS_NEXT_LSEEK(handle, fsp, fd, offset, whence) \
413 ((handle)->vfs_next.ops.lseek(\
414 (handle)->vfs_next.handles.lseek,\
415 (fsp), (fd), (offset), (whence)))
416 #define SMB_VFS_NEXT_SENDFILE(handle, tofd, fsp, fromfd, header, offset, count) \
417 ((handle)->vfs_next.ops.sendfile(\
418 (handle)->vfs_next.handles.sendfile,\
419 (tofd), (fsp), (fromfd), (header), (offset), (count)))
428 <title>Upgrading to the New VFS Interface</title>
431 <title>Upgrading from 2.2.* and 3.0aplha modules</title>
435 Add "vfs_handle_struct *handle, " as first parameter to all vfs operation functions.
436 e.g. example_connect(connection_struct *conn, const char *service, const char *user);
437 -> example_connect(vfs_handle_struct *handle, connection_struct *conn, const char *service, const char *user);
441 Replace "default_vfs_ops." with "smb_vfs_next_".
442 e.g. default_vfs_ops.connect(conn, service, user);
443 -> smb_vfs_next_connect(conn, service, user);
447 Uppercase all "smb_vfs_next_*" functions.
448 e.g. smb_vfs_next_connect(conn, service, user);
449 -> SMB_VFS_NEXT_CONNECT(conn, service, user);
453 Add "handle, " as first parameter to all SMB_VFS_NEXT_*() calls.
454 e.g. SMB_VFS_NEXT_CONNECT(conn, service, user);
455 -> SMB_VFS_NEXT_CONNECT(handle, conn, service, user);
459 (Only for 2.2.* modules)
460 Convert the old struct vfs_ops example_ops to
461 a vfs_op_tuple example_op_tuples[] array.
464 struct vfs_ops example_ops = {
465 /* Disk operations */
466 example_connect, /* connect */
467 example_disconnect, /* disconnect */
469 /* Directory operations */
475 /* File operations */
484 example_stat, /* stat */
485 example_fstat, /* fstat */
486 example_lstat, /* lstat */
495 NULL, /* ftruncate */
502 NULL, /* fget_nt_acl */
503 NULL, /* get_nt_acl */
504 NULL, /* fset_nt_acl */
505 NULL, /* set_nt_acl */
507 NULL, /* chmod_acl */
508 NULL, /* fchmod_acl */
510 NULL, /* sys_acl_get_entry */
511 NULL, /* sys_acl_get_tag_type */
512 NULL, /* sys_acl_get_permset */
513 NULL, /* sys_acl_get_qualifier */
514 NULL, /* sys_acl_get_file */
515 NULL, /* sys_acl_get_fd */
516 NULL, /* sys_acl_clear_perms */
517 NULL, /* sys_acl_add_perm */
518 NULL, /* sys_acl_to_text */
519 NULL, /* sys_acl_init */
520 NULL, /* sys_acl_create_entry */
521 NULL, /* sys_acl_set_tag_type */
522 NULL, /* sys_acl_set_qualifier */
523 NULL, /* sys_acl_set_permset */
524 NULL, /* sys_acl_valid */
525 NULL, /* sys_acl_set_file */
526 NULL, /* sys_acl_set_fd */
527 NULL, /* sys_acl_delete_def_file */
528 NULL, /* sys_acl_get_perm */
529 NULL, /* sys_acl_free_text */
530 NULL, /* sys_acl_free_acl */
531 NULL /* sys_acl_free_qualifier */
536 static vfs_op_tuple example_op_tuples[] = {
537 {SMB_VFS_OP(example_connect), SMB_VFS_OP_CONNECT, SMB_VFS_LAYER_TRANSPARENT},
538 {SMB_VFS_OP(example_disconnect), SMB_VFS_OP_DISCONNECT, SMB_VFS_LAYER_TRANSPARENT},
540 {SMB_VFS_OP(example_fstat), SMB_VFS_OP_FSTAT, SMB_VFS_LAYER_TRANSPARENT},
541 {SMB_VFS_OP(example_stat), SMB_VFS_OP_STAT, SMB_VFS_LAYER_TRANSPARENT},
542 {SMB_VFS_OP(example_lstat), SMB_VFS_OP_LSTAT, SMB_VFS_LAYER_TRANSPARENT},
544 {SMB_VFS_OP(NULL), SMB_VFS_OP_NOOP, SMB_VFS_LAYER_NOOP}
550 Move the example_op_tuples[] array to the end of the file.
554 Add the init_module() function at the end of the file.
557 NTSTATUS init_module(void)
559 return smb_register_vfs(SMB_VFS_INTERFACE_VERSION,"example",example_op_tuples);
565 Check if your vfs_init() function does more then just prepare the vfs_ops structs or
566 remember the struct smb_vfs_handle_struct.
568 <member>If NOT you can remove the vfs_init() function.</member>
569 <member>If YES decide if you want to move the code to the example_connect() operation or to the init_module(). And then remove vfs_init().
570 e.g. a debug class registration should go into init_module() and the allocation of private data should go to example_connect().</member>
575 (Only for 3.0alpha* modules)
576 Check if your vfs_done() function contains needed code.
578 <member>If NOT you can remove the vfs_done() function.</member>
579 <member>If YES decide if you can move the code to the example_disconnect() operation. Otherwise register a SMB_EXIT_EVENT with smb_register_exit_event(); (Described in the <link linkend="modules">modules section</link>) And then remove vfs_done(). e.g. the freeing of private data should go to example_disconnect().
585 Check if you have any global variables left.
586 Decide if it wouldn't be better to have this data on a connection basis.
588 <member>If NOT leave them as they are. (e.g. this could be the variable for the private debug class.)</member>
589 <member>If YES pack all this data into a struct. You can use handle->data to point to such a struct on a per connection basis.</member>
592 e.g. if you have such a struct:
594 struct example_privates {
599 first way of doing it:
601 static int example_connect(vfs_handle_struct *handle,
602 connection_struct *conn, const char *service,
605 struct example_privates *data = NULL;
607 /* alloc our private data */
608 data = (struct example_privates *)talloc_zero(conn->mem_ctx, sizeof(struct example_privates));
610 DEBUG(0,("talloc_zero() failed\n"));
614 /* init out private data */
615 data->some_string = talloc_strdup(conn->mem_ctx,"test");
616 if (!data->some_string) {
617 DEBUG(0,("talloc_strdup() failed\n"));
621 data->db_connection = open_db_conn();
623 /* and now store the private data pointer in handle->data
624 * we don't need to specify a free_function here because
625 * we use the connection TALLOC context.
626 * (return -1 if something failed.)
628 VFS_HANDLE_SET_DATA(handle, data, NULL, struct example_privates, return -1);
630 return SMB_VFS_NEXT_CONNECT(handle,conn,service,user);
633 static int example_close(vfs_handle_struct *handle, files_struct *fsp, int fd)
635 struct example_privates *data = NULL;
637 /* get the pointer to our private data
638 * return -1 if something failed
640 SMB_VFS_HANDLE_GET_DATA(handle, data, struct example_privates, return -1);
642 /* do something here...*/
643 DEBUG(0,("some_string: %s\n",data->some_string));
645 return SMB_VFS_NEXT_CLOSE(handle, fsp, fd);
648 second way of doing it:
650 static void free_example_privates(void **datap)
652 struct example_privates *data = (struct example_privates *)*datap;
654 SAFE_FREE(data->some_string);
662 static int example_connect(vfs_handle_struct *handle,
663 connection_struct *conn, const char *service,
666 struct example_privates *data = NULL;
668 /* alloc our private data */
669 data = (struct example_privates *)malloc(sizeof(struct example_privates));
671 DEBUG(0,("malloc() failed\n"));
675 /* init out private data */
676 data->some_string = strdup("test");
677 if (!data->some_string) {
678 DEBUG(0,("strdup() failed\n"));
682 data->db_connection = open_db_conn();
684 /* and now store the private data pointer in handle->data
685 * we need to specify a free_function because we used malloc() and strdup().
686 * (return -1 if something failed.)
688 SMB_VFS_HANDLE_SET_DATA(handle, data, free_example_privates, struct example_privates, return -1);
690 return SMB_VFS_NEXT_CONNECT(handle,conn,service,user);
693 static int example_close(vfs_handle_struct *handle, files_struct *fsp, int fd)
695 struct example_privates *data = NULL;
697 /* get the pointer to our private data
698 * return -1 if something failed
700 SMB_VFS_HANDLE_GET_DATA(handle, data, struct example_privates, return -1);
702 /* do something here...*/
703 DEBUG(0,("some_string: %s\n",data->some_string));
705 return SMB_VFS_NEXT_CLOSE(handle, fsp, fd);
711 To make it easy to build 3rd party modules it would be usefull to provide
712 configure.in, (configure), install.sh and Makefile.in with the module.
713 (Take a look at the example in <filename>examples/VFS</filename>.)
717 The configure script accepts <option>--with-samba-source</option> to specify
718 the path to the samba source tree.
719 It also accept <option>--enable-developer</option> which lets the compiler
720 give you more warnings.
724 The idea is that you can extend this
725 <filename>configure.in</filename> and <filename>Makefile.in</filename> scripts
730 Compiling & Testing...
732 <member><userinput>./configure <option>--enable-developer</option></userinput> ...</member>
733 <member><userinput>make</userinput></member>
734 <member>Try to fix all compiler warnings</member>
735 <member><userinput>make</userinput></member>
736 <member>Testing, Testing, Testing ...</member>
745 <title>Some Notes</title>
748 <title>Implement TRANSPARENT functions</title>
751 Avoid writing functions like this:
754 static int example_close(vfs_handle_struct *handle, files_struct *fsp, int fd)
756 return SMB_VFS_NEXT_CLOSE(handle, fsp, fd);
760 Overload only the functions you really need to!
766 <title>Implement OPAQUE functions</title>
769 If you want to just implement a better version of a
770 default samba opaque function
771 (e.g. like a disk_free() function for a special filesystem)
772 it's ok to just overload that specific function.
776 If you want to implement a database filesystem or
777 something different from a posix filesystem.
778 Make sure that you overload every vfs operation!!!
781 Functions your FS does not support should be overloaded by something like this:
782 e.g. for a readonly filesystem.
786 static int example_rename(vfs_handle_struct *handle, connection_struct *conn,
787 char *oldname, char *newname)
789 DEBUG(10,("function rename() not allowed on vfs 'example'\n"));