Version 1.1 with conntrack-tools support
[bwmon.git] / doc / user-manual.html
blob4763f1f544e20ee8ef9c75b6aaa4d3e7f7061193
1 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN" "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
2 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">
3 <head>
4 <title>bwmon: Application-level bandwidth monitoring (thp.io)</title>
5 <style type="text/css">
6 body { margin: 0px; background-color: rgb( 30%, 30%, 30%); font-family: sans-serif; font-size: 10pt; }
7 h1 { font-size: 16px; }
8 h2 { font-size: 15px; }
9 h3 { font-size: 14px; }
10 a { text-decoration: none; color: rgb( 24%, 24%, 24%); }
11 a:hover { text-decoration: underline; color: #333; background-color: rgb( 95%, 95%, 95%); }
12 img { border-width: 0px; }
13 .nav { padding: 10px; margin: 0px; background-color: rgb( 10%, 10%, 10%); color: rgb( 80%, 80%, 80%); letter-spacing: 3px; font-size: 11px; border-bottom: 1px rgb( 10%, 10%, 10%) solid; font-family: sans-serif; }
14 .nav a { color: rgb( 40%, 40%, 40%); background-color: inherit; font-weight: bold; }
15 .taglinks a { color: gray; }
16 address { text-align: right; color: #666; }
17 code, pre { background-color: rgb( 95%, 95%, 95%); }
18 pre { overflow: auto; max-height: 500px; }
19 hr { border: 0px; height: 1px; background-image: url(/2007/archive/dotted.gif); margin-top: 6px; margin-bottom: 6px; clear: both; }
20 table.main { z-index: 200; margin-top: 20px; margin-left: auto; margin-right: auto; border-width: 0px; }
21 table.main td { z-index: 200; padding: 10px; vertical-align: top;}
22 table.main div.cont, table.main div.sidebar { background-color: rgb(98%, 98%, 98%); border: 1px rgb( 20%, 20%, 20%) solid; padding: 5px; }
23 table.main div.cont { width: 660px; }
24 div.important { border: 3px green solid; background-color: lightgreen; margin: 10px; padding: 10px; }
25 p.quote { font-style: italic; text-align: center; padding-left: 20%; padding-right: 20%; color: rgb( 20%, 0%, 20%); }
26 p.quote:after { font-size: 8pt; font-style: normal; content: attr(title); display: block; text-align: right; padding-right: 10%; padding-top: 5px; }
28 p { text-align: justify; }
29 </style>
30 </head>
31 <body>
32 <p style="text-align: right;" class="nav"> <a href="/" rel="home">home</a>
33 &#8226; <a href="/about">contact</a> &#8226; thp.io</p>
34 <table class="main">
35 <tbody>
36 <tr>
37 <td>
38 <div class="cont">
39 <div style="padding: 0px 30px 15px;">
40 <p style="font-size: 10px; color: rgb(204, 204, 204);"> <a
41 href="/2010/">2010</a> / <a href="/2010/bwmon/"><strong>bwmon: Application-level bandwidth monitoring</strong></a> </p>
42 <!-- BEGIN CONTENT -->
43 <div style="text-align: center;">
44 <img src="http://thp.io/2010/bwmon/bwmon.png" alt="" width="320" height="100"/>
45 </div>
46 <h1>bwmon User Manual</h1>
48 <p>bwmon is a per-application bandwith monitoring framework written in Python.
49 It can monitor generated traffic and bandwidth (traffic over time) per process
50 or application (group of processes).</p>
52 <p>bwmon has two different means of monitoring: the Monitor which relies on
53 data gathered from the /proc filesystem and and Pipe which accounts data passed
54 through it.</p>
56 <p>Both monitoring types can be used standalone or be combined through the
57 Aggregator. The Aggregator can group processes to applications and issue
58 notifications if one process/application passes its configured threshold.</p>
61 <h2>The Monitor</h2>
63 <p>The Monitor is used for general-purpose traffic and bandwith monitoring. It
64 uses data from the /proc filesystem, especially from /proc/net/ip_conntrack
65 which is provided by the Kernel module ip_conntrack. If this file does not
66 exist, try to run</p>
68 <pre>
69 sudo modprobe ip_conntrack
70 </pre>
73 <h3>Using the Monitor</h3>
75 <p>The Monitor can be used standalone by running</p>
77 <pre>
78 sudo python runmonitor.py [--no-lookback] [--bandwidth] [--ignore-local]
79 [--include=<regex> [--include=<regex> ...] [--exclude=<regex> [...]]
80 </pre>
82 <p>When started without arguments, Monitor displays the accumulated traffic
83 since monitoring for the process was started by ip_conntrack. However this
84 timespan can vary between processes and can (currently) not be determined. To
85 display only traffic generated after the start of the Monitor, use the
86 <strong>--no-lookback</strong> switch.</p>
89 <p>To display bandwith usage instead of accumulated traffic, you can use
90 <strong>--bandwidth</strong> switch. Monitor will then calculate the bandwidth
91 usage per second.</p>
94 <p>By default Monitor also includes traffic via the loopback interface. By
95 using the <strong>--ignore-local</strong> switch loopback traffic can be
96 ignored.</p>
99 <p>To select the processes that are monitored by Monitor you can use the
100 <strong>--include</strong> and <strong>--exclude</strong> options to pass
101 regular expressions. Both flags can be combined and each can be used multiple
102 times. When --include is used, only processes that match at least one of the
103 "include" regular expressions are monitored. When --exclude is used, processes
104 that match the "exclude" regular expressions are excluded. Exclude overrides
105 include.</p>
109 <h2>The Pipe</h2>
111 <p>The Pipe opens a local port and connects to a remote host. It them forwards
112 and accounts all traffic passed through it.</p>
115 <h3>Using the Pipe</h3>
117 <p>The Pipe is started with</p>
119 <pre>
120 python runpipe.py <em>port</em> <em>remote-host</em>
121 python runpipe.py <em>local-port</em> <em>remote-host</em> <em>remote-port</em>
122 </pre>
124 <p>When started with two arguments, Pipe opens <em>port</em> locally and
125 establishes a connection to <em>remote-host</em> on the same port. When started
126 with three arguments, different local and remote ports can be used.</p>
129 <h2>The Aggregator</h2>
131 <p>The aggregator combines multiple instances of Monitor and Pipe to provide a
132 consistant configuration and notification interface.</p>
134 <p>Multiple processes (even with traffic data coming from different monitors)
135 can be grouped to applications. Notification thresholds can then be assigned to
136 applications instead of simple processes.</p>
139 <h3>Using the Aggregator</h3>
141 <p>The aggregator is started with</p>
143 <pre>
144 python runaggregator.py [--app-config=<file>] [--monitor-config=<file>]
145 [--notification-config=<file>] [--auto-group]
146 </pre>
148 <p>It then prints one line per application that has used bandwidth during the
149 last interval (1 second). The output can be divided into three columns. The
150 first containins the incoming and the second the bandwidth. The third columns
151 is used to distinguish the processes and contains the commandline (for
152 processes) or the application name (for grouped processes).</p>
155 <h3>Specifying Monitors</h3>
157 <p>By default the Aggregator starts one Monitor and no Pipes. To override this
158 behaviour, the switch <strong>--monitor-config</strong> can be used to specify
159 a monitor configuration file.</p>
161 <p>The configuration file is expected to have the following format.</p>
163 <pre>
164 [Monitor Name]
165 type=<em>monitor-type</em>
166 <em>parameter</em>=<em>value</em>
167 </pre>
169 <p><em>Monitor Name</em> is a description of the monitoring instance and is not
170 processed by the Aggregator. The type can either be "monitor" or pipe. The
171 available parameters depend on the type.
173 <p>Paramters for Monitors:
174 <ul>
175 <li>include</li>
176 <li>exclude</li>
177 <li>ignorelocal</li>
178 </ul>
179 </p>
181 <p>The parameters correspond to the commandline switches of the standalone
182 monitor. Only one include and exclude regular expression can be given per
183 Monitor instance.</p>
186 <p>Parameters for Pipes
187 <ul>
188 <li>port</li>
189 <li>newhost</li>
190 <li>newport</li>
191 </ul>
192 </p>
194 <p>The parameters correspond to the commandline switches of the standalone
195 pipe.</p>
197 <p>A sample configuration can be found in <a
198 href="http://repo.or.cz/w/bwmon.git/blob_plain/HEAD:/monitors.sample.cfg">monitors.sample.cfg</a></p>
201 <h3>Grouping Processes</h3>
203 <p>The Aggregator can aggregate the bandwidth usage for a group of processes.
204 The grouping can either happen automatically or through a supplied
205 configuration.</p>
207 <p>The switch <strong>--auto-group</strong> activates automatic grouping based
208 on the basename of the processes. Grouping is tried for all processes without
209 exceptions. If only some processes shall be grouped or different rules shall be
210 applied, a configuration file can be specified with the
211 <strong>--app-config</strong> switch.</p>
213 <p>The supplied configuration file is expected to have the following format:</p>
215 <pre>
216 [<em>Application Name</em>]
217 <em>shortname1</em>=<em>regex</em>
218 <em>shortname2</em>=<em>regex</em>
219 </pre>
221 <p>Where <em>Application Name</em> is the name under which the aggregated
222 bandwidth of the matched processes will be shown. Each application section can
223 have multiple rules labeled with a "shortname" each with a regular expression
224 that is matched against the processes' commandline. The shortname is used to
225 describe the rule and is not used by the Aggregator.</p>
227 <p>A sample configuration file can be found in <a
228 href="http://repo.or.cz/w/bwmon.git/blob_plain/HEAD:/group.sample.cfg">group.sample.cfg</a>.</p>
231 <h3>Notifications</h3>
233 <p>The Aggregator can issue notifications if a process or application exceeds
234 its configured bandwidth threshold. Aggregator does not use default thresholds
235 and does therefor not issue notifications unless explicitly configured.</p>
237 <p>The configuration file is provided via the parameter
238 <strong>--notification-config</strong> and is expected to have the following
239 format.</p>
241 <pre>
242 [<em>Application Name</em>]
243 process_filter=<em>regex</em>
244 notification_threshold=<em>bandwidth in kB/s</em>
245 interval=<em>average timestamp in s</em>
246 notification_command=<em>cmd</em>
247 </pre>
249 <p><em>Application Name</em> is a label that is used to describe the following
250 entry and is not processed by the Aggregator.</p>
252 <p><em>process_filter</em> specifies a regular expression that is used to limit
253 the scope of the rule. The regular expression is matched against
254 <ul>
255 <li>the process commandline for processes that are not grouped</li>
256 <li>the application name for grouped processes</li>
257 </ul>
258 </p>
260 <p>The rule is only valid for matching processes/groups.</p>
262 <p><em>Incoming</em> and <em>outgoing</em> bandwidth thresholds are specified
263 in kB/s.</p>
265 <p>Notifications are triggered if the average bandwidth over the given
266 <em>interval</em> (in seconds) exceeds the configured threshold.</p>
268 <p>A triggered notification writes its information to stderr. If a
269 <em>notification_command</em> is configured, the given program is started.</p>
272 <br>
273 <address>Thomas Perl (thp at this domain), jabber:
274 thp@jabber.org</address>
275 </div>
276 </div>
277 </td>
278 <td> <br>
279 </td>
280 </tr>
281 </tbody>
282 </table>
283 </body>
284 </html>