docs/whitepaper/architecture.tex

   1 \chapter{Framework Architecture}
   2
   3 \section{IPC}
   4 Part of the OpenSync framework is an Interprocess Communication system to
   5 communicate with the Plugins. To avoid confusion between the different process
   6 types of Plugins, and to make the IPC independent of Plugins, the neutral terms
   7 ">Client"< and >"Client Proxy<" got introduced. The IPC is allows full duplex
   8 (bidirectional) communication between Client and Proxy. Independent of the
   9 process type of the Client. The frameworks allows to communicate between three
  10 different process types of Clients:
  11 \begin{itemize}
  12 \item Threaded via pipes (two pipes for full duplex/bidirectional ipc. See
  13 pipe(7))
  14 \item Forked via pipes (two pipes for full duplex/bidirectional ipc. See pipe(7))
  15 \item External process via named pipes (FIFO)
  16 \end{itemize}
  17 This IPC also to send specific type of messages between the Client and the
  18 Proxy. The Proxy takes care about timeout handling, if a client doesn't
  19 response in time. This is only supported for messages which have a callback
  20 function assigned. Such callbacks (command handler) get executed as soon if the
  21 message reply by the Client is retrieved. On a timeout or error the callback
  22 function got called with the custom error or timeout message.
  23
  24 \subsection{Client}
  25 Clients handle all requests from the Proxy. Each request from the proxy gets
  26 replied. Threaded and Forked clients get started by the Proxy. Most common use of
  27 the Client is to call Plugin specific functions, requested with message send by
  28 the Proxy. The client replies the result of the function calls with replying to
  29 the messaged which requested the event.
  30
  31 \subsection{Proxy}
  32 The Proxy handles all request to and from the Client. For each Client there is
  33 one Proxy. The proxy got mainly used by the OpenSync framework engines to
  34 communicate with the Plugin processes.
  35
  36 \subsection{Messaging Queues}
  37 The communication between Client and Proxy is based on message queues. There are
  38 two types of such Queues: Sender- and Receiver-Queue. The Queue object supports
  39 (anonymous) pipes as well as named pipes (FIFO).\\
  40 \\
  41 Note: When using Queues along forked processes the read and write ends have to
  42 be disconnected/closed for proper communication. See pipe(7).\\
  43 \\
  44 There is a specified set of Message Commands for simple communication between
  45 Client and Proxy. For synchronized communication between Client and Proxy the
  46 Client simply replies the Message Command from the Proxy, when Client executed
  47 the requested command. If Client failed while executing the command, it has to
  48 reply with an error reply. Synchronize communication requires that the caller
  49 assign a message handler to the Message object, which got called when the
  50 message got replied. This message handler should check carefully the received
  51 replied message type, and handle provide proper error handling.\\
  52 \\
  53 For synchronized communication it's recommended to send message with a timeout
  54 via the message queue. To avoid deadlocks when the counterpart fails to reply.
  55 No timeout handling is needed for asynchronized communication without command
  56 handler, since no reply is expected.\\
  57 \\
  58 Additional content of the message have to be marshaled/demarshaled, to make the
  59 content usable even it's running in a different address space. (rough rule of
  60 thumb: everything which includes a pointers/memory addresses)
  61
  62 \section{Engines}
  63 So called engines are the core of the entire framework and control the entire
  64 sequence of every single step which is done during a synchronization. When the
  65 engine got initialized it checks if a previous synchronization was unclean
  66 (failed) and a slow sync is required. If the previous synchronization was
  67 unclean or not could be only determined by engine itself if an group environment
  68 is used. Beside the group environment it also initializes the plugin, format.
  69 The engine isn't restricted to get called with a group environment, the group
  70 environment is only of optional use.\\
  71 \\
  72 If the group environment is used, the engine locks the group when the engine
  73 gets initialized. If the group detects an unassigned lock the engine will
  74 request while synchronization a slow sync.\\
  75 \\
  76 To determine a previous unclean synchronization without a group environment ,
  77 it's up to OpenSync framework using program to provide facilities to store such
  78 information. If so, the engine MUST be set to trigger a slow sync, after it got
  79 initialized. When the engine got finalized or synchronized successful the slow
  80 sync engine status got reseted.\\
  81 \\
  82 With the engine initialization also the members of the group got initialized.
  83 The member initialization creates for corresponding member plugin a Client Proxy
  84 which got initialized and spawns the plugin. For more details check the IPC
  85 section about the Proxy.\\
  86 \\
  87 After all members got initialized, the >"Object Engines<" for each enabled
  88 Object Type got initialized. If no Object Type is enabled or discovered the
  89 engine initialization aborts with an error. The engine needs at least one Object
  90 Type to synchronize. Often this error appears if not all or none members got
  91 discovered. See discover section of "Merging capabilities" for more details.
  92
  93 \subsection{Object Engine}
  94 For each Object Type, which have to be synchronized, one Object Engine gets
  95 started by the (main) engine. If the engine detected that previous
  96 synchronization was unclean, each Object Engine get the slow sync flag set to
  97 perform a slow sync on the next sync. If the Object Engine got finalized or
  98 synchronized (successful or unsuccessful) the slow sync flag get reseted. If
  99 the synchronization failed, it's most likely that the (main) engine will set
 100 the slow sync flag again on the next synchronization.\\
 101 \\
 102 The Object Engines allows to perform actions only for a certain Object Type. For
 103 example to request a slow sync only for specific Object Engines (this means also
 104 for specific Object Types).\\
 105 \\
 106 The initialization of an Object Engine creates for each Client Proxy (read
 107 Members' Plugin) a Sink Engine, which maps each Client Proxy with an Object
 108 Type. (More about the Sink Engine in the next section.)\\
 109 \\
 110 If a slow sync got set for the Object Engine, then all >"Mappings<" with the
 111 Object Type of the Object Engine got deleted from the >"Mapping Table<". This is
 112 done to avoid loss of the data inconsistence while performing a slow sync.\\
 113 \\
 114 If no slow sync got set for this Object Engine the >"Mapping Table<" got loaded
 115 and creates for each Mapping a >"Mapping Engine<".
 116
 117 %TODO: Injecting ignored entries in mapping engine
 118
 119 %TODO
 120
 121
 122 \subsection{Mapping Engine}
 123 \subsection{Mapping Entry Engine}
 124
 125
 126 \subsection{Sink Engine}
 127 The Sink Engine currently doesn't consists of any logic. The only relation to
 128 Engines of the Sink Engine object is, it maps all the already mentioned engines
 129 together.\\
 130 \\
 131 The Sink Engine maps the Client Proxy and Object Engine together. With this
 132 combination the Sink Engine is able to handle Object Type specific (aka.
 133 ObjTypeSink) tasks. Additionally it contains of a list of Mapping Engines,
 134 which handle Object Type and Client related Mappings only.
 135
 136 \section{Merging Different Capabilities}
 137 \subsection{Capabilities}
 138 \subsection{Archive}
 139 \subsection{Merger \& Demerger}
 140 \section{Helpers}
 141 \subsection{Anchor Table}
 142 \subsection{Hash Table}
 143 \subsection{Time Helper}
 144 \section{Filter}
 145 \subsection{Custom Fiter}