1 <?xml version="1.0" encoding="iso-8859-1"?>
3 PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
4 "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd">
8 <refentrytitle>ctdb</refentrytitle>
9 <manvolnum>7</manvolnum>
10 <refmiscinfo class="source">ctdb</refmiscinfo>
11 <refmiscinfo class="manual">CTDB - clustered TDB database</refmiscinfo>
16 <refname>ctdb</refname>
17 <refpurpose>Clustered TDB</refpurpose>
21 <title>DESCRIPTION</title>
24 CTDB is a clustered database component in clustered Samba that
25 provides a high-availability load-sharing CIFS server cluster.
29 The main functions of CTDB are:
35 Provide a clustered version of the TDB database with automatic
36 rebuild/recovery of the databases upon node failures.
42 Monitor nodes in the cluster and services running on each node.
48 Manage a pool of public IP addresses that are used to provide
49 services to clients. Alternatively, CTDB can be used with
56 Combined with a cluster filesystem CTDB provides a full
57 high-availablity (HA) environment for services such as clustered
58 Samba, NFS and other services.
62 In addition to the CTDB manual pages there is much more
63 information available at
64 <ulink url="https://wiki.samba.org/index.php/CTDB_and_Clustered_Samba"/>.
69 <title>ANATOMY OF A CTDB CLUSTER</title>
72 A CTDB cluster is a collection of nodes with 2 or more network
73 interfaces. All nodes provide network (usually file/NAS) services
74 to clients. Data served by file services is stored on shared
75 storage (usually a cluster filesystem) that is accessible by all
79 CTDB provides an "all active" cluster, where services are load
80 balanced across all nodes.
85 <title>Cluster leader</title>
88 CTDB uses a <emphasis>cluster leader and follower</emphasis>
89 model of cluster management. All nodes in a cluster elect one
90 node to be the leader. The leader node coordinates privileged
91 operations such as database recovery and IP address failover.
95 CTDB previously referred to the leader as the <emphasis>recovery
96 master</emphasis> or <emphasis>recmaster</emphasis>. References
97 to these terms may still be found in documentation and code.
102 <title>Cluster Lock</title>
105 CTDB uses a cluster lock to assert its privileged role in the
106 cluster. This node takes the cluster lock when it becomes
107 leader and holds the lock until it is no longer leader. The
108 <emphasis>cluster lock</emphasis> helps CTDB to avoid a
109 <emphasis>split brain</emphasis>, where a cluster becomes
110 partitioned and each partition attempts to operate
111 independently. Issues that can result from a split brain
112 include file data corruption, because file locking metadata may
113 not be tracked correctly.
117 CTDB previously referred to the cluster lock as the
118 <emphasis>recovery lock</emphasis>. The abbreviation
119 <emphasis>reclock</emphasis> is still used - just "clock" would
124 <emphasis>CTDB is unable configure a default cluster
125 lock</emphasis>, because this would depend on factors such as
126 cluster filesystem mountpoints. However, <emphasis>running CTDB
127 without a cluster lock is not recommended</emphasis> as there
128 will be no split brain protection.
132 When a cluster lock is configured it is used as the election
133 mechanism. Nodes race to take the cluster lock and the winner
134 is the cluster leader. This avoids problems when a node wins an
135 election but is unable to take the lock - this can occur if a
136 cluster becomes partitioned (for example, due to a communication
137 failure) and a different leader is elected by the nodes in each
138 partition, or if the cluster filesystem has a high failover
143 By default, the cluster lock is implemented using a file
144 (specified by <parameter>cluster lock</parameter> in the
145 <literal>[cluster]</literal> section of
146 <citerefentry><refentrytitle>ctdb.conf</refentrytitle>
147 <manvolnum>5</manvolnum></citerefentry>) residing in shared
148 storage (usually) on a cluster filesystem. To support a
149 cluster lock the cluster filesystem must support lock
151 <citerefentry><refentrytitle>ping_pong</refentrytitle>
152 <manvolnum>1</manvolnum></citerefentry> for more details.
156 The cluster lock can also be implemented using an arbitrary
157 cluster mutex helper (or call-out). This is indicated by using
158 an exclamation point ('!') as the first character of the
159 <parameter>cluster lock</parameter> parameter. For example, a
160 value of <command>!/usr/local/bin/myhelper cluster</command>
161 would run the given helper with the specified arguments. The
162 helper will continue to run as long as it holds its mutex. See
163 <filename>ctdb/doc/cluster_mutex_helper.txt</filename> in the
164 source tree, and related code, for clues about writing helpers.
168 When a file is specified for the <parameter>cluster
169 lock</parameter> parameter (i.e. no leading '!') the file lock
170 is implemented by a default helper
171 (<command>/usr/local/libexec/ctdb/ctdb_mutex_fcntl_helper</command>).
172 This helper has arguments as follows:
174 <!-- cmdsynopsis would not require long line but does not work :-( -->
176 <command>ctdb_mutex_fcntl_helper</command> <parameter>FILE</parameter> <optional><parameter>RECHECK-INTERVAL</parameter></optional>
179 <command>ctdb_mutex_fcntl_helper</command> will take a lock on
180 FILE and then check every RECHECK-INTERVAL seconds to ensure
181 that FILE still exists and that its inode number is unchanged
182 from when the lock was taken. The default value for
183 RECHECK-INTERVAL is 5.
187 CTDB does sanity checks to ensure that the cluster lock is held
193 <title>Private vs Public addresses</title>
196 Each node in a CTDB cluster has multiple IP addresses assigned
202 A single private IP address that is used for communication
208 One or more public IP addresses that are used to provide
209 NAS or other services.
216 <title>Private address</title>
219 Each node is configured with a unique, permanently assigned
220 private address. This address is configured by the operating
221 system. This address uniquely identifies a physical node in
222 the cluster and is the address that CTDB daemons will use to
223 communicate with the CTDB daemons on other nodes.
227 Private addresses are listed in the file
228 <filename>/usr/local/etc/ctdb/nodes</filename>). This file
229 contains the list of private addresses for all nodes in the
230 cluster, one per line. This file must be the same on all nodes
235 Some users like to put this configuration file in their
236 cluster filesystem. A symbolic link should be used in this
241 Private addresses should not be used by clients to connect to
242 services provided by the cluster.
245 It is strongly recommended that the private addresses are
246 configured on a private network that is separate from client
247 networks. This is because the CTDB protocol is both
248 unauthenticated and unencrypted. If clients share the private
249 network then steps need to be taken to stop injection of
250 packets to relevant ports on the private addresses. It is
251 also likely that CTDB protocol traffic between nodes could
252 leak sensitive information if it can be intercepted.
256 Example <filename>/usr/local/etc/ctdb/nodes</filename> for a four node
259 <screen format="linespecific">
268 <title>Public addresses</title>
271 Public addresses are used to provide services to clients.
272 Public addresses are not configured at the operating system
273 level and are not permanently associated with a particular
274 node. Instead, they are managed by CTDB and are assigned to
275 interfaces on physical nodes at runtime.
278 The CTDB cluster will assign/reassign these public addresses
279 across the available healthy nodes in the cluster. When one
280 node fails, its public addresses will be taken over by one or
281 more other nodes in the cluster. This ensures that services
282 provided by all public addresses are always available to
283 clients, as long as there are nodes available capable of
284 hosting this address.
288 The public address configuration is stored in
289 <filename>/usr/local/etc/ctdb/public_addresses</filename> on
290 each node. This file contains a list of the public addresses
291 that the node is capable of hosting, one per line. Each entry
292 also contains the netmask and the interface to which the
293 address should be assigned. If this file is missing then no
294 public addresses are configured.
298 Some users who have the same public addresses on all nodes
299 like to put this configuration file in their cluster
300 filesystem. A symbolic link should be used in this case.
304 Example <filename>/usr/local/etc/ctdb/public_addresses</filename> for a
305 node that can host 4 public addresses, on 2 different
308 <screen format="linespecific">
316 In many cases the public addresses file will be the same on
317 all nodes. However, it is possible to use different public
318 address configurations on different nodes.
322 Example: 4 nodes partitioned into two subgroups:
324 <screen format="linespecific">
325 Node 0:/usr/local/etc/ctdb/public_addresses
329 Node 1:/usr/local/etc/ctdb/public_addresses
333 Node 2:/usr/local/etc/ctdb/public_addresses
337 Node 3:/usr/local/etc/ctdb/public_addresses
342 In this example nodes 0 and 1 host two public addresses on the
343 10.1.1.x network while nodes 2 and 3 host two public addresses
344 for the 10.1.2.x network.
347 Public address 10.1.1.1 can be hosted by either of nodes 0 or
348 1 and will be available to clients as long as at least one of
349 these two nodes are available.
352 If both nodes 0 and 1 become unavailable then public address
353 10.1.1.1 also becomes unavailable. 10.1.1.1 can not be failed
354 over to nodes 2 or 3 since these nodes do not have this public
358 The <command>ctdb ip</command> command can be used to view the
359 current assignment of public addresses to physical nodes.
366 <title>Node status</title>
369 The current status of each node in the cluster can be viewed by the
370 <command>ctdb status</command> command.
374 A node can be in one of the following states:
382 This node is healthy and fully functional. It hosts public
383 addresses to provide services.
389 <term>DISCONNECTED</term>
392 This node is not reachable by other nodes via the private
393 network. It is not currently participating in the cluster.
394 It <emphasis>does not</emphasis> host public addresses to
395 provide services. It might be shut down.
401 <term>DISABLED</term>
404 This node has been administratively disabled. This node is
405 partially functional and participates in the cluster.
406 However, it <emphasis>does not</emphasis> host public
407 addresses to provide services.
413 <term>UNHEALTHY</term>
416 A service provided by this node has failed a health check
417 and should be investigated. This node is partially
418 functional and participates in the cluster. However, it
419 <emphasis>does not</emphasis> host public addresses to
420 provide services. Unhealthy nodes should be investigated
421 and may require an administrative action to rectify.
430 CTDB is not behaving as designed on this node. For example,
431 it may have failed too many recovery attempts. Such nodes
432 are banned from participating in the cluster for a
433 configurable time period before they attempt to rejoin the
434 cluster. A banned node <emphasis>does not</emphasis> host
435 public addresses to provide services. All banned nodes
436 should be investigated and may require an administrative
446 This node has been administratively exclude from the
447 cluster. A stopped node does no participate in the cluster
448 and <emphasis>does not</emphasis> host public addresses to
449 provide services. This state can be used while performing
450 maintenance on a node.
456 <term>PARTIALLYONLINE</term>
459 A node that is partially online participates in a cluster
460 like a healthy (OK) node. Some interfaces to serve public
461 addresses are down, but at least one interface is up. See
462 also <command>ctdb ifaces</command>.
471 <title>CAPABILITIES</title>
474 Cluster nodes can have several different capabilities enabled.
475 These are listed below.
484 Indicates that a node can become the CTDB cluster leader.
485 The current leader is decided via an
486 election held by all active nodes with this capability.
498 Indicates that a node can be the location master (LMASTER)
499 for database records. The LMASTER always knows which node
500 has the latest copy of a record in a volatile database.
511 The LEADER and LMASTER capabilities can be disabled when CTDB
512 is used to create a cluster spanning across WAN links. In this
513 case CTDB acts as a WAN accelerator.
522 LVS is a mode where CTDB presents one single IP address for the
523 entire cluster. This is an alternative to using public IP
524 addresses and round-robin DNS to loadbalance clients across the
529 This is similar to using a layer-4 loadbalancing switch but with
534 One extra LVS public address is assigned on the public network
535 to each LVS group. Each LVS group is a set of nodes in the
536 cluster that presents the same LVS address public address to the
537 outside world. Normally there would only be one LVS group
538 spanning an entire cluster, but in situations where one CTDB
539 cluster spans multiple physical sites it might be useful to have
540 one LVS group for each site. There can be multiple LVS groups
541 in a cluster but each node can only be member of one LVS group.
545 Client access to the cluster is load-balanced across the HEALTHY
546 nodes in an LVS group. If no HEALTHY nodes exists then all
547 nodes in the group are used, regardless of health status. CTDB
548 will, however never load-balance LVS traffic to nodes that are
549 BANNED, STOPPED, DISABLED or DISCONNECTED. The <command>ctdb
550 lvs</command> command is used to show which nodes are currently
551 load-balanced across.
555 In each LVS group, one of the nodes is selected by CTDB to be
556 the LVS leader. This node receives all traffic from clients
557 coming in to the LVS public address and multiplexes it across
558 the internal network to one of the nodes that LVS is using.
559 When responding to the client, that node will send the data back
560 directly to the client, bypassing the LVS leader node. The
561 command <command>ctdb lvs leader</command> will show which node
562 is the current LVS leader.
566 The path used for a client I/O is:
570 Client sends request packet to LVS leader.
575 LVS leader passes the request on to one node across the
581 Selected node processes the request.
586 Node responds back to client.
593 This means that all incoming traffic to the cluster will pass
594 through one physical node, which limits scalability. You can
595 send more data to the LVS address that one physical node can
596 multiplex. This means that you should not use LVS if your I/O
597 pattern is write-intensive since you will be limited in the
598 available network bandwidth that node can handle. LVS does work
599 very well for read-intensive workloads where only smallish READ
600 requests are going through the LVS leader bottleneck and the
601 majority of the traffic volume (the data in the read replies)
602 goes straight from the processing node back to the clients. For
603 read-intensive i/o patterns you can achieve very high throughput
608 Note: you can use LVS and public addresses at the same time.
612 If you use LVS, you must have a permanent address configured for
613 the public interface on each node. This address must be routable
614 and the cluster nodes must be configured so that all traffic
615 back to client hosts are routed through this interface. This is
616 also required in order to allow samba/winbind on the node to
617 talk to the domain controller. This LVS IP address can not be
618 used to initiate outgoing traffic.
621 Make sure that the domain controller and the clients are
622 reachable from a node <emphasis>before</emphasis> you enable
623 LVS. Also ensure that outgoing traffic to these hosts is routed
624 out through the configured public interface.
628 <title>Configuration</title>
631 To activate LVS on a CTDB node you must specify the
632 <varname>CTDB_LVS_PUBLIC_IFACE</varname>,
633 <varname>CTDB_LVS_PUBLIC_IP</varname> and
634 <varname>CTDB_LVS_NODES</varname> configuration variables.
635 <varname>CTDB_LVS_NODES</varname> specifies a file containing
636 the private address of all nodes in the current node's LVS
642 <screen format="linespecific">
643 CTDB_LVS_PUBLIC_IFACE=eth1
644 CTDB_LVS_PUBLIC_IP=10.1.1.237
645 CTDB_LVS_NODES=/usr/local/etc/ctdb/lvs_nodes
650 Example <filename>/usr/local/etc/ctdb/lvs_nodes</filename>:
652 <screen format="linespecific">
659 Normally any node in an LVS group can act as the LVS leader.
660 Nodes that are highly loaded due to other demands maybe
661 flagged with the "follower-only" option in the
662 <varname>CTDB_LVS_NODES</varname> file to limit the LVS
663 functionality of those nodes.
667 LVS nodes file that excludes 192.168.1.4 from being
670 <screen format="linespecific">
673 192.168.1.4 follower-only
680 <title>TRACKING AND RESETTING TCP CONNECTIONS</title>
683 CTDB tracks TCP connections from clients to public IP addresses,
684 on known ports. When an IP address moves from one node to
685 another, all existing TCP connections to that IP address are
686 reset. The node taking over this IP address will also send
687 gratuitous ARPs (for IPv4, or neighbour advertisement, for
688 IPv6). This allows clients to reconnect quickly, rather than
689 waiting for TCP timeouts, which can be very long.
693 It is important that established TCP connections do not survive
694 a release and take of a public IP address on the same node.
695 Such connections can get out of sync with sequence and ACK
696 numbers, potentially causing a disruptive ACK storm.
702 <title>NAT GATEWAY</title>
705 NAT gateway (NATGW) is an optional feature that is used to
706 configure fallback routing for nodes. This allows cluster nodes
707 to connect to external services (e.g. DNS, AD, NIS and LDAP)
708 when they do not host any public addresses (e.g. when they are
712 This also applies to node startup because CTDB marks nodes as
713 UNHEALTHY until they have passed a "monitor" event. In this
714 context, NAT gateway helps to avoid a "chicken and egg"
715 situation where a node needs to access an external service to
719 Another way of solving this type of problem is to assign an
720 extra static IP address to a public interface on every node.
721 This is simpler but it uses an extra IP address per node, while
722 NAT gateway generally uses only one extra IP address.
726 <title>Operation</title>
729 One extra NATGW public address is assigned on the public
730 network to each NATGW group. Each NATGW group is a set of
731 nodes in the cluster that shares the same NATGW address to
732 talk to the outside world. Normally there would only be one
733 NATGW group spanning an entire cluster, but in situations
734 where one CTDB cluster spans multiple physical sites it might
735 be useful to have one NATGW group for each site.
738 There can be multiple NATGW groups in a cluster but each node
739 can only be member of one NATGW group.
742 In each NATGW group, one of the nodes is selected by CTDB to
743 be the NATGW leader and the other nodes are consider to be
744 NATGW followers. NATGW followers establish a fallback default route
745 to the NATGW leader via the private network. When a NATGW
746 follower hosts no public IP addresses then it will use this route
747 for outbound connections. The NATGW leader hosts the NATGW
748 public IP address and routes outgoing connections from
749 follower nodes via this IP address. It also establishes a
750 fallback default route.
755 <title>Configuration</title>
758 NATGW is usually configured similar to the following example configuration:
760 <screen format="linespecific">
761 CTDB_NATGW_NODES=/usr/local/etc/ctdb/natgw_nodes
762 CTDB_NATGW_PRIVATE_NETWORK=192.168.1.0/24
763 CTDB_NATGW_PUBLIC_IP=10.0.0.227/24
764 CTDB_NATGW_PUBLIC_IFACE=eth0
765 CTDB_NATGW_DEFAULT_GATEWAY=10.0.0.1
769 Normally any node in a NATGW group can act as the NATGW
770 leader. Some configurations may have special nodes that lack
771 connectivity to a public network. In such cases, those nodes
772 can be flagged with the "follower-only" option in the
773 <varname>CTDB_NATGW_NODES</varname> file to limit the NATGW
774 functionality of those nodes.
778 See the <citetitle>NAT GATEWAY</citetitle> section in
779 <citerefentry><refentrytitle>ctdb-script.options</refentrytitle>
780 <manvolnum>5</manvolnum></citerefentry> for more details of
787 <title>Implementation details</title>
790 When the NATGW functionality is used, one of the nodes is
791 selected to act as a NAT gateway for all the other nodes in
792 the group when they need to communicate with the external
793 services. The NATGW leader is selected to be a node that is
794 most likely to have usable networks.
798 The NATGW leader hosts the NATGW public IP address
799 <varname>CTDB_NATGW_PUBLIC_IP</varname> on the configured public
800 interfaces <varname>CTDB_NATGW_PUBLIC_IFACE</varname> and acts as
801 a router, masquerading outgoing connections from follower nodes
802 via this IP address. If
803 <varname>CTDB_NATGW_DEFAULT_GATEWAY</varname> is set then it
804 also establishes a fallback default route to the configured
805 this gateway with a metric of 10. A metric 10 route is used
806 so it can co-exist with other default routes that may be
811 A NATGW follower establishes its fallback default route to the
812 NATGW leader via the private network
813 <varname>CTDB_NATGW_PRIVATE_NETWORK</varname>with a metric of 10.
814 This route is used for outbound connections when no other
815 default route is available because the node hosts no public
816 addresses. A metric 10 routes is used so that it can co-exist
817 with other default routes that may be available when the node
818 is hosting public addresses.
822 <varname>CTDB_NATGW_STATIC_ROUTES</varname> can be used to
823 have NATGW create more specific routes instead of just default
828 This is implemented in the <filename>11.natgw</filename>
829 eventscript. Please see the eventscript file and the
830 <citetitle>NAT GATEWAY</citetitle> section in
831 <citerefentry><refentrytitle>ctdb-script.options</refentrytitle>
832 <manvolnum>5</manvolnum></citerefentry> for more details.
839 <title>POLICY ROUTING</title>
842 Policy routing is an optional CTDB feature to support complex
843 network topologies. Public addresses may be spread across
844 several different networks (or VLANs) and it may not be possible
845 to route packets from these public addresses via the system's
846 default route. Therefore, CTDB has support for policy routing
847 via the <filename>13.per_ip_routing</filename> eventscript.
848 This allows routing to be specified for packets sourced from
849 each public address. The routes are added and removed as CTDB
850 moves public addresses between nodes.
854 <title>Configuration variables</title>
857 There are 4 configuration variables related to policy routing:
858 <varname>CTDB_PER_IP_ROUTING_CONF</varname>,
859 <varname>CTDB_PER_IP_ROUTING_RULE_PREF</varname>,
860 <varname>CTDB_PER_IP_ROUTING_TABLE_ID_LOW</varname>,
861 <varname>CTDB_PER_IP_ROUTING_TABLE_ID_HIGH</varname>. See the
862 <citetitle>POLICY ROUTING</citetitle> section in
863 <citerefentry><refentrytitle>ctdb-script.options</refentrytitle>
864 <manvolnum>5</manvolnum></citerefentry> for more details.
869 <title>Configuration</title>
872 The format of each line of
873 <varname>CTDB_PER_IP_ROUTING_CONF</varname> is:
877 <public_address> <network> [ <gateway> ]
881 Leading whitespace is ignored and arbitrary whitespace may be
882 used as a separator. Lines that have a "public address" item
883 that doesn't match an actual public address are ignored. This
884 means that comment lines can be added using a leading
885 character such as '#', since this will never match an IP
890 A line without a gateway indicates a link local route.
894 For example, consider the configuration line:
898 192.168.1.99 192.168.1.0/24
902 If the corresponding public_addresses line is:
906 192.168.1.99/24 eth2,eth3
910 <varname>CTDB_PER_IP_ROUTING_RULE_PREF</varname> is 100, and
911 CTDB adds the address to eth2 then the following routing
912 information is added:
916 ip rule add from 192.168.1.99 pref 100 table ctdb.192.168.1.99
917 ip route add 192.168.1.0/24 dev eth2 table ctdb.192.168.1.99
921 This causes traffic from 192.168.1.99 to 192.168.1.0/24 go via
926 The <command>ip rule</command> command will show (something
927 like - depending on other public addresses and other routes on
932 0: from all lookup local
933 100: from 192.168.1.99 lookup ctdb.192.168.1.99
934 32766: from all lookup main
935 32767: from all lookup default
939 <command>ip route show table ctdb.192.168.1.99</command> will show:
943 192.168.1.0/24 dev eth2 scope link
947 The usual use for a line containing a gateway is to add a
948 default route corresponding to a particular source address.
949 Consider this line of configuration:
953 192.168.1.99 0.0.0.0/0 192.168.1.1
957 In the situation described above this will cause an extra
958 routing command to be executed:
962 ip route add 0.0.0.0/0 via 192.168.1.1 dev eth2 table ctdb.192.168.1.99
966 With both configuration lines, <command>ip route show table
967 ctdb.192.168.1.99</command> will show:
971 192.168.1.0/24 dev eth2 scope link
972 default via 192.168.1.1 dev eth2
977 <title>Sample configuration</title>
980 Here is a more complete example configuration.
984 /usr/local/etc/ctdb/public_addresses:
986 192.168.1.98 eth2,eth3
987 192.168.1.99 eth2,eth3
989 /usr/local/etc/ctdb/policy_routing:
991 192.168.1.98 192.168.1.0/24
992 192.168.1.98 192.168.200.0/24 192.168.1.254
993 192.168.1.98 0.0.0.0/0 192.168.1.1
994 192.168.1.99 192.168.1.0/24
995 192.168.1.99 192.168.200.0/24 192.168.1.254
996 192.168.1.99 0.0.0.0/0 192.168.1.1
1000 The routes local packets as expected, the default route is as
1001 previously discussed, but packets to 192.168.200.0/24 are
1002 routed via the alternate gateway 192.168.1.254.
1009 <title>NOTIFICATIONS</title>
1012 When certain state changes occur in CTDB, it can be configured
1013 to perform arbitrary actions via notifications. For example,
1014 sending SNMP traps or emails when a node becomes unhealthy or
1019 The notification mechanism runs all executable files ending in
1021 <filename>/usr/local/etc/ctdb/events/notification/</filename>,
1022 ignoring any failures and continuing to run all files.
1026 CTDB currently generates notifications after CTDB changes to
1031 <member>init</member>
1032 <member>setup</member>
1033 <member>startup</member>
1034 <member>healthy</member>
1035 <member>unhealthy</member>
1041 <title>LOG LEVELS</title>
1044 Valid log levels, in increasing order of verbosity, are:
1048 <member>ERROR</member>
1049 <member>WARNING</member>
1050 <member>NOTICE</member>
1051 <member>INFO</member>
1052 <member>DEBUG</member>
1058 <title>REMOTE CLUSTER NODES</title>
1060 It is possible to have a CTDB cluster that spans across a WAN link.
1061 For example where you have a CTDB cluster in your datacentre but you also
1062 want to have one additional CTDB node located at a remote branch site.
1063 This is similar to how a WAN accelerator works but with the difference
1064 that while a WAN-accelerator often acts as a Proxy or a MitM, in
1065 the ctdb remote cluster node configuration the Samba instance at the remote site
1066 IS the genuine server, not a proxy and not a MitM, and thus provides 100%
1067 correct CIFS semantics to clients.
1071 See the cluster as one single multihomed samba server where one of
1072 the NICs (the remote node) is very far away.
1076 NOTE: This does require that the cluster filesystem you use can cope
1077 with WAN-link latencies. Not all cluster filesystems can handle
1078 WAN-link latencies! Whether this will provide very good WAN-accelerator
1079 performance or it will perform very poorly depends entirely
1080 on how optimized your cluster filesystem is in handling high latency
1081 for data and metadata operations.
1085 To activate a node as being a remote cluster node you need to
1086 set the following two parameters in
1087 /usr/local/etc/ctdb/ctdb.conf for the remote node:
1088 <screen format="linespecific">
1090 lmaster capability = false
1091 leader capability = false
1096 Verify with the command "ctdb getcapabilities" that that node no longer
1097 has the leader or the lmaster capabilities.
1104 <title>SEE ALSO</title>
1107 <citerefentry><refentrytitle>ctdb</refentrytitle>
1108 <manvolnum>1</manvolnum></citerefentry>,
1110 <citerefentry><refentrytitle>ctdbd</refentrytitle>
1111 <manvolnum>1</manvolnum></citerefentry>,
1113 <citerefentry><refentrytitle>ctdb_diagnostics</refentrytitle>
1114 <manvolnum>1</manvolnum></citerefentry>,
1116 <citerefentry><refentrytitle>ltdbtool</refentrytitle>
1117 <manvolnum>1</manvolnum></citerefentry>,
1119 <citerefentry><refentrytitle>onnode</refentrytitle>
1120 <manvolnum>1</manvolnum></citerefentry>,
1122 <citerefentry><refentrytitle>ping_pong</refentrytitle>
1123 <manvolnum>1</manvolnum></citerefentry>,
1125 <citerefentry><refentrytitle>ctdb.conf</refentrytitle>
1126 <manvolnum>5</manvolnum></citerefentry>,
1128 <citerefentry><refentrytitle>ctdb-script.options</refentrytitle>
1129 <manvolnum>5</manvolnum></citerefentry>,
1131 <citerefentry><refentrytitle>ctdb.sysconfig</refentrytitle>
1132 <manvolnum>5</manvolnum></citerefentry>,
1134 <citerefentry><refentrytitle>ctdb-statistics</refentrytitle>
1135 <manvolnum>7</manvolnum></citerefentry>,
1137 <citerefentry><refentrytitle>ctdb-tunables</refentrytitle>
1138 <manvolnum>7</manvolnum></citerefentry>,
1140 <ulink url="https://wiki.samba.org/index.php/CTDB_and_Clustered_Samba"/>,
1142 <ulink url="http://ctdb.samba.org/"/>
1149 This documentation was written by
1158 <holder>Andrew Tridgell</holder>
1159 <holder>Ronnie Sahlberg</holder>
1163 This program is free software; you can redistribute it and/or
1164 modify it under the terms of the GNU General Public License as
1165 published by the Free Software Foundation; either version 3 of
1166 the License, or (at your option) any later version.
1169 This program is distributed in the hope that it will be
1170 useful, but WITHOUT ANY WARRANTY; without even the implied
1171 warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR
1172 PURPOSE. See the GNU General Public License for more details.
1175 You should have received a copy of the GNU General Public
1176 License along with this program; if not, see
1177 <ulink url="http://www.gnu.org/licenses"/>.