doc/user-manual.html

   1 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN" "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
   2 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">
   3 <head>
   4   <title>bwmon: Application-level bandwidth monitoring (thp.io)</title>
   5   <style type="text/css">
   6             body { margin: 0px; background-color: rgb( 30%, 30%, 30%); font-family: sans-serif; font-size: 10pt; }
   7             h1 { font-size: 16px; }
   8             h2 { font-size: 15px; }
   9             h3 { font-size: 14px; }
  10             a { text-decoration: none; color: rgb( 24%, 24%, 24%); }
  11             a:hover { text-decoration: underline; color: #333; background-color: rgb( 95%, 95%, 95%); }
  12             img { border-width: 0px; }
  13             .nav { padding: 10px; margin: 0px; background-color: rgb( 10%, 10%, 10%); color: rgb( 80%, 80%, 80%); letter-spacing: 3px; font-size: 11px; border-bottom: 1px rgb( 10%, 10%, 10%) solid; font-family: sans-serif; }
  14             .nav a { color: rgb( 40%, 40%, 40%); background-color: inherit; font-weight: bold; }
  15             .taglinks a { color: gray; }
  16             address { text-align: right; color: #666; }
  17             code, pre { background-color: rgb( 95%, 95%, 95%); }
  18             pre { overflow: auto; max-height: 500px; }
  19             hr { border: 0px; height: 1px; background-image: url(/2007/archive/dotted.gif); margin-top: 6px; margin-bottom: 6px; clear: both; }
  20             table.main { z-index: 200; margin-top: 20px; margin-left: auto; margin-right: auto; border-width: 0px; }
  21             table.main td { z-index: 200; padding: 10px; vertical-align: top;}
  22             table.main div.cont, table.main div.sidebar { background-color: rgb(98%, 98%, 98%); border: 1px rgb( 20%, 20%, 20%) solid; padding: 5px; }
  23             table.main div.cont { width: 660px; }
  24             div.important { border: 3px green solid; background-color: lightgreen; margin: 10px; padding: 10px; }
  25             p.quote { font-style: italic; text-align: center; padding-left: 20%; padding-right: 20%; color: rgb( 20%, 0%, 20%); }
  26             p.quote:after { font-size: 8pt; font-style: normal; content: attr(title); display: block; text-align: right; padding-right: 10%; padding-top: 5px; }
  27
  28             p { text-align: justify; }
  29         </style>
  30 </head>
  31 <body>
  32 <p style="text-align: right;" class="nav"> <a href="/" rel="home">home</a>
  33 &#8226; <a href="/about">contact</a> &#8226; thp.io</p>
  34 <table class="main">
  35   <tbody>
  36     <tr>
  37       <td>
  38       <div class="cont">
  39       <div style="padding: 0px 30px 15px;">
  40       <p style="font-size: 10px; color: rgb(204, 204, 204);"> <a
  41  href="/2010/">2010</a> / <a href="/2010/bwmon/"><strong>bwmon: Application-level bandwidth monitoring</strong></a> </p>
  42 <!-- BEGIN CONTENT -->
  43       <div style="text-align: center;">
  44           <img src="http://thp.io/2010/bwmon/bwmon.png" alt="" width="320" height="100"/>
  45       </div>
  46       <h1>bwmon User Manual</h1>
  47
  48 <p>bwmon is a per-application bandwith monitoring framework written in Python.
  49 It can monitor generated traffic and bandwidth (traffic over time) per process
  50 or application (group of processes).</p>
  51
  52 <p>bwmon has two different means of monitoring: the Monitor which relies on
  53 data gathered from the /proc filesystem and and Pipe which accounts data passed
  54 through it.</p>
  55
  56 <p>Both monitoring types can be used standalone or be combined through the
  57 Aggregator. The Aggregator can group processes to applications and issue
  58 notifications if one process/application passes its configured threshold.</p>
  59
  60
  61 <h2>The Monitor</h2>
  62
  63 <p>The Monitor is used for general-purpose traffic and bandwith monitoring. It
  64 uses data from the /proc filesystem, especially from /proc/net/ip_conntrack
  65 which is provided by the Kernel module ip_conntrack. If this file does not
  66 exist, try to run</p>
  67
  68 <pre>
  69   sudo modprobe ip_conntrack
  70 </pre>
  71
  72
  73 <h3>Using the Monitor</h3>
  74
  75 <p>The Monitor can be used standalone by running</p>
  76
  77 <pre>
  78   sudo python runmonitor.py [--no-lookback] [--bandwidth] [--ignore-local]
  79       [--include=<regex> [--include=<regex> ...] [--exclude=<regex> [...]]
  80 </pre>
  81
  82 <p>When started without arguments, Monitor displays the accumulated traffic
  83 since monitoring for the process was started by ip_conntrack. However this
  84 timespan can vary between processes and can (currently) not be determined. To
  85 display only traffic generated after the start of the Monitor, use the
  86 <strong>--no-lookback</strong> switch.</p>
  87
  88
  89 <p>To display bandwith usage instead of accumulated traffic, you can use
  90 <strong>--bandwidth</strong> switch. Monitor will then calculate the bandwidth
  91 usage per second.</p>
  92
  93
  94 <p>By default Monitor also includes traffic via the loopback interface. By
  95 using the <strong>--ignore-local</strong> switch loopback traffic can be
  96 ignored.</p>
  97
  98
  99 <p>To select the processes that are monitored by Monitor you can use the
 100 <strong>--include</strong> and <strong>--exclude</strong> options to pass
 101 regular expressions. Both flags can be combined and each can be used multiple
 102 times. When --include is used, only processes that match at least one of the
 103 "include" regular expressions are monitored. When --exclude is used, processes
 104 that match the "exclude" regular expressions are excluded. Exclude overrides
 105 include.</p>
 106
 107
 108
 109 <h2>The Pipe</h2>
 110
 111 <p>The Pipe opens a local port and connects to a remote host. It them forwards
 112 and accounts all traffic passed through it.</p>
 113
 114
 115 <h3>Using the Pipe</h3>
 116
 117 <p>The Pipe is started with</p>
 118
 119 <pre>
 120     python runpipe.py <em>port</em> <em>remote-host</em>
 121     python runpipe.py <em>local-port</em> <em>remote-host</em> <em>remote-port</em>
 122 </pre>
 123
 124 <p>When started with two arguments, Pipe opens <em>port</em> locally and
 125 establishes a connection to <em>remote-host</em> on the same port. When started
 126 with three arguments, different local and remote ports can be used.</p>
 127
 128
 129 <h2>The Aggregator</h2>
 130
 131 <p>The aggregator combines multiple instances of Monitor and Pipe to provide a
 132 consistant configuration and notification interface.</p>
 133
 134 <p>Multiple processes (even with traffic data coming from different monitors)
 135 can be grouped to applications. Notification thresholds can then be assigned to
 136 applications instead of simple processes.</p>
 137
 138
 139 <h3>Using the Aggregator</h3>
 140
 141 <p>The aggregator is started with</p>
 142
 143 <pre>
 144     python runaggregator.py [--app-config=<file>] [--monitor-config=<file>]
 145         [--notification-config=<file>] [--auto-group]
 146 </pre>
 147
 148 <p>It then prints one line per application that has used bandwidth during the
 149 last interval (1 second). The output can be divided into three columns. The
 150 first containins the incoming and the second the bandwidth. The third columns
 151 is used to distinguish the processes and contains the commandline (for
 152 processes) or the application name (for grouped processes).</p>
 153
 154
 155 <h3>Specifying Monitors</h3>
 156
 157 <p>By default the Aggregator starts one Monitor and no Pipes. To override this
 158 behaviour, the switch <strong>--monitor-config</strong> can be used to specify
 159 a monitor configuration file.</p>
 160
 161 <p>The configuration file is expected to have the following format.</p>
 162
 163 <pre>
 164 [Monitor Name]
 165 type=<em>monitor-type</em>
 166 <em>parameter</em>=<em>value</em>
 167 </pre>
 168
 169 <p><em>Monitor Name</em> is a description of the monitoring instance and is not
 170 processed by the Aggregator. The type can either be "monitor" or pipe. The
 171 available parameters depend on the type.
 172
 173 <p>Paramters for Monitors:
 174  <ul>
 175   <li>include</li>
 176   <li>exclude</li>
 177   <li>ignorelocal</li>
 178  </ul>
 179 </p>
 180
 181 <p>The parameters correspond to the commandline switches of the standalone
 182 monitor. Only one include and exclude regular expression can be given per
 183 Monitor instance.</p>
 184
 185
 186 <p>Parameters for Pipes
 187  <ul>
 188   <li>port</li>
 189   <li>newhost</li>
 190   <li>newport</li>
 191  </ul>
 192 </p>
 193
 194 <p>The parameters correspond to the commandline switches of the standalone
 195 pipe.</p>
 196
 197 <p>A sample configuration can be found in <a
 198 href="http://repo.or.cz/w/bwmon.git/blob_plain/HEAD:/monitors.sample.cfg">monitors.sample.cfg</a></p>
 199
 200
 201 <h3>Grouping Processes</h3>
 202
 203 <p>The Aggregator can aggregate the bandwidth usage for a group of processes.
 204 The grouping can either happen automatically or through a supplied
 205 configuration.</p>
 206
 207 <p>The switch <strong>--auto-group</strong> activates automatic grouping based
 208 on the basename of the processes. Grouping is tried for all processes without
 209 exceptions. If only some processes shall be grouped or different rules shall be
 210 applied, a configuration file can be specified with the
 211 <strong>--app-config</strong> switch.</p>
 212
 213 <p>The supplied configuration file is expected to have the following format:</p>
 214
 215 <pre>
 216 [<em>Application Name</em>]
 217 <em>shortname1</em>=<em>regex</em>
 218 <em>shortname2</em>=<em>regex</em>
 219 </pre>
 220
 221 <p>Where <em>Application Name</em> is the name under which the aggregated
 222 bandwidth of the matched processes will be shown. Each application section can
 223 have multiple rules labeled with a "shortname" each with a regular expression
 224 that is matched against the processes' commandline. The shortname is used to
 225 describe the rule and is not used by the Aggregator.</p>
 226
 227 <p>A sample configuration file can be found in <a
 228 href="http://repo.or.cz/w/bwmon.git/blob_plain/HEAD:/group.sample.cfg">group.sample.cfg</a>.</p>
 229
 230
 231 <h3>Notifications</h3>
 232
 233 <p>The Aggregator can issue notifications if a process or application exceeds
 234 its configured bandwidth threshold. Aggregator does not use default thresholds
 235 and does therefor not issue notifications unless explicitly configured.</p>
 236
 237 <p>The configuration file is provided via the parameter
 238 <strong>--notification-config</strong> and is expected to have the following
 239 format.</p>
 240
 241 <pre>
 242 [<em>Application Name</em>]
 243 process_filter=<em>regex</em>
 244 notification_threshold=<em>bandwidth in kB/s</em>
 245 interval=<em>average timestamp in s</em>
 246 notification_command=<em>cmd</em>
 247 </pre>
 248
 249 <p><em>Application Name</em> is a label that is used to describe the following
 250 entry and is not processed by the Aggregator.</p>
 251
 252 <p><em>process_filter</em> specifies a regular expression that is used to limit
 253 the scope of the rule. The regular expression is matched against
 254  <ul>
 255   <li>the process commandline for processes that are not grouped</li>
 256   <li>the application name for grouped processes</li>
 257  </ul>
 258 </p>
 259
 260 <p>The rule is only valid for matching processes/groups.</p>
 261
 262 <p><em>Incoming</em> and <em>outgoing</em> bandwidth thresholds are specified
 263 in kB/s.</p>
 264
 265 <p>Notifications are triggered if the average bandwidth over the given
 266 <em>interval</em> (in seconds) exceeds the configured threshold.</p>
 267
 268 <p>A triggered notification writes its information to stderr. If a
 269 <em>notification_command</em> is configured, the given program is started.</p>
 270
 271
 272       <br>
 273       <address>Thomas Perl (thp at this domain), jabber:
 274 thp@jabber.org</address>
 275       </div>
 276       </div>
 277       </td>
 278       <td> <br>
 279       </td>
 280     </tr>
 281   </tbody>
 282 </table>
 283 </body>
 284 </html>