3 These aren't strictly necessary for hacking on Tor, but they can help track
11 * https://travis-ci.org/torproject/tor
12 * https://ci.appveyor.com/project/torproject/tor
14 Travis builds and runs tests on Linux, and eventually macOS (#24629).
15 Appveyor builds and runs tests on Windows (using Windows Services for Linux).
17 Runs automatically on Pull Requests sent to torproject/tor. You can set it up
18 for your fork to build commits outside of PRs too:
20 1. sign up for GitHub: https://github.com/join
21 2. fork https://github.com/torproject/tor:
22 https://help.github.com/articles/fork-a-repo/
23 3. follow https://docs.travis-ci.com/user/getting-started/#To-get-started-with-Travis-CI.
24 skip steps involving `.travis.yml` (we already have one).
25 4. go to https://ci.appveyor.com/login , log in with github, and select
28 Builds should show up on the web at travis-ci.com and on IRC at #tor-ci on
29 OFTC. If they don't, ask #tor-dev (also on OFTC).
33 It's CI/builders. Looks like this: https://jenkins.torproject.org
35 Runs automatically on commits merged to git.torproject.org. We CI the
36 master branch and all supported tor versions. We also build nightly debian
39 Builds Linux and Windows cross-compilation. Runs Linux tests.
41 Builds should show up on the web at jenkins.torproject.org and on IRC at
42 #tor-bots on OFTC. If they don't, ask #tor-dev (also on OFTC).
47 $ valgrind --leak-check=yes --error-limit=no --show-reachable=yes src/app/tor
50 (Note that if you get a zillion openssl warnings, you will also need to
51 pass `--undef-value-errors=no` to valgrind, or rebuild your openssl
56 Nick regularly runs the coverity static analyzer on the Tor codebase.
58 The preprocessor define `__COVERITY__` is used to work around instances
59 where coverity picks up behavior that we wish to permit.
61 ## clang Static Analyzer
63 The clang static analyzer can be run on the Tor codebase using Xcode (WIP)
64 or a command-line build.
66 The preprocessor define `__clang_analyzer__` is used to work around instances
67 where clang picks up behavior that we wish to permit.
69 ## clang Runtime Sanitizers
71 To build the Tor codebase with the clang Address and Undefined Behavior
72 sanitizers, see the file `contrib/clang/sanitize_blacklist.txt`.
74 Preprocessor workarounds for instances where clang picks up behavior that
75 we wish to permit are also documented in the blacklist file.
77 ## Running lcov for unit test coverage
79 Lcov is a utility that generates pretty HTML reports of test code coverage.
80 To generate such a report:
83 $ ./configure --enable-coverage
86 $ $BROWSER ./coverage_html/index.html
89 This will run the tor unit test suite `./src/test/test` and generate the HTML
90 coverage code report under the directory `./coverage_html/`. To change the
91 output directory, use `make coverage-html HTML_COVER_DIR=./funky_new_cov_dir`.
93 Coverage diffs using lcov are not currently implemented, but are being
94 investigated (as of July 2014).
96 ## Running the unit tests
98 To quickly run all the tests distributed with Tor:
104 To run the fast unit tests only:
110 To selectively run just some tests (the following can be combined
114 $ ./src/test/test <name_of_test> [<name of test 2>] ...
115 $ ./src/test/test <prefix_of_name_of_test>.. [<prefix_of_name_of_test2>..] ...
116 $ ./src/test/test :<name_of_excluded_test> [:<name_of_excluded_test2]...
119 To run all tests, including those based on Stem or Chutney:
125 To run all tests, including those based on Stem or Chutney that require a
126 working connection to the internet:
129 $ make test-full-online
132 ## Running gcov for unit test coverage
135 $ ./configure --enable-coverage
138 $ # or--- make test-full ? make test-full-online?
139 $ mkdir coverage-output
140 $ ./scripts/test/coverage coverage-output
143 (On OSX, you'll need to start with `--enable-coverage CC=clang`.)
145 If that doesn't work:
147 * Try configuring Tor with `--disable-gcc-hardening`
148 * You might need to run `make clean` after you run `./configure`.
150 Then, look at the .gcov files in `coverage-output`. '-' before a line means
151 that the compiler generated no code for that line. '######' means that the
152 line was never reached. Lines with numbers were called that number of times.
154 For more details about how to read gcov output, see the [Invoking
155 gcov](https://gcc.gnu.org/onlinedocs/gcc/Invoking-Gcov.html) chapter
158 If you make changes to Tor and want to get another set of coverage results,
159 you can run `make reset-gcov` to clear the intermediary gcov output.
161 If you have two different `coverage-output` directories, and you want to see
162 a meaningful diff between them, you can run:
165 $ ./scripts/test/cov-diff coverage-output1 coverage-output2 | less
168 In this diff, any lines that were visited at least once will have coverage "1",
169 and line numbers are deleted. This lets you inspect what you (probably) really
170 want to know: which untested lines were changed? Are there any new untested
173 If you run ./scripts/test/cov-exclude, it marks excluded unreached
174 lines with 'x', and excluded reached lines with '!!!'.
176 ## Running integration tests
178 We have the beginnings of a set of scripts to run integration tests using
179 Chutney. To try them, set CHUTNEY_PATH to your chutney source directory, and
180 run `make test-network`.
182 We also have scripts to run integration tests using Stem. To try them, set
183 `STEM_SOURCE_DIR` to your Stem source directory, and run `test-stem`.
187 Ongoing notes about Tor profiling can be found at
188 https://pad.riseup.net/p/profiling-tor
190 ## Profiling Tor with oprofile
192 The oprofile tool runs (on Linux only!) to tell you what functions Tor is
193 spending its CPU time in, so we can identify performance bottlenecks.
195 Here are some basic instructions
197 - Build tor with debugging symbols (you probably already have, unless
198 you messed with CFLAGS during the build process).
199 - Build all the libraries you care about with debugging symbols
200 (probably you only care about libssl, maybe zlib and Libevent).
201 - Copy this tor to a new directory
202 - Copy all the libraries it uses to that dir too (`ldd ./tor` will
204 - Set LD_LIBRARY_PATH to include that dir. `ldd ./tor` should now
205 show you it's using the libs in that dir
207 - Reset oprofiles counters/start it
208 * `opcontrol --reset; opcontrol --start`, if Nick remembers right.
209 - After a while, have it dump the stats on tor and all the libs
210 in that dir you created.
211 * `opcontrol --dump;`
212 * `opreport -l that_dir/*`
215 ## Profiling Tor with perf
217 This works with a running Tor, and requires root.
219 1. Decide how long you want to profile for. Start with (say) 30 seconds. If that
220 works, try again with longer times.
222 2. Find the PID of your running tor process.
224 3. Run `perf record --call-graph dwarf -p <PID> sleep <SECONDS>`
226 (You may need to do this as root.)
228 You might need to add `-e cpu-clock` as an option to the perf record line
229 above, if you are on an older CPU without access to hardware profiling
230 events, or in a VM, or something.
232 4. Now you have a perf.data file. Have a look at it with `perf report
233 --no-children --sort symbol,dso` or `perf report --no-children --sort
234 symbol,dso --stdio --header`. How does it look?
236 5a. Once you have a nice big perf.data file, you can compress it, encrypt it,
237 and send it to your favorite Tor developers.
239 5b. Or maybe you'd rather not send a nice big perf.data file. Who knows what's
240 in that!? It's kinda scary. To generate a less scary file, you can use `perf
241 report -g > <FILENAME>.out`. Then you can compress that and put it somewhere
244 ## Profiling Tor with gperftools aka Google-performance-tools
246 This should work on nearly any unixy system. It doesn't seem to be compatible
247 with RunAsDaemon though.
249 Beforehand, install google-perftools.
251 1. You need to rebuild Tor, hack the linking steps to add `-lprofiler` to the
252 libs. You can do this by adding `LIBS=-lprofiler` when you call `./configure`.
254 Now you can run Tor with profiling enabled, and use the pprof utility to look at
255 performance! See the gperftools manual for more info, but basically:
257 2. Run `env CPUPROFILE=/tmp/profile src/app/tor -f <path/torrc>`. The profile file
258 is not written to until Tor finishes execution.
260 3. Run `pprof src/app/tor /tmp/profile` to start the REPL.
262 ## Generating and analyzing a callgraph
264 0. Build Tor on linux or mac, ideally with -O0 or -fno-inline.
266 1. Clone 'https://git.torproject.org/user/nickm/calltool.git/' .
267 Follow the README in that repository.
269 Note that currently the callgraph generator can't detect calls that pass
270 through function pointers.
272 ## Getting emacs to edit Tor source properly
274 Nick likes to put the following snippet in his .emacs file:
277 (add-hook 'c-mode-hook
280 (set-variable 'show-trailing-whitespace t)
282 (let ((fname (expand-file-name (buffer-file-name))))
284 ((string-match "^/home/nickm/src/libevent" fname)
285 (set-variable 'indent-tabs-mode t)
286 (set-variable 'c-basic-offset 4)
287 (set-variable 'tab-width 4))
288 ((string-match "^/home/nickm/src/tor" fname)
289 (set-variable 'indent-tabs-mode nil)
290 (set-variable 'c-basic-offset 2))
291 ((string-match "^/home/nickm/src/openssl" fname)
292 (set-variable 'indent-tabs-mode t)
293 (set-variable 'c-basic-offset 8)
294 (set-variable 'tab-width 8))
298 You'll note that it defaults to showing all trailing whitespace. The `cond`
299 test detects whether the file is one of a few C free software projects that I
300 often edit, and sets up the indentation level and tab preferences to match
303 If you want to try this out, you'll need to change the filename regex
304 patterns to match where you keep your Tor files.
306 If you use emacs for editing Tor and nothing else, you could always just say:
309 (add-hook 'c-mode-hook
312 (set-variable 'show-trailing-whitespace t)
313 (set-variable 'indent-tabs-mode nil)
314 (set-variable 'c-basic-offset 2)))
317 There is probably a better way to do this. No, we are probably not going
318 to clutter the files with emacs stuff.
320 ## Building a tag file (code index)
322 Many functions in tor use `MOCK_IMPL` wrappers for unit tests. Your
323 tag-building program must be told how to handle this syntax.
325 If you're using emacs, you can generate an emacs-compatible tag file using
326 `make tags`. This will run your system's `etags`. Tor's build system assumes
327 that you're using the emacs-specific version of `etags` (bundled under the
328 `xemacs21-bin` package on Debian). This is incompatible with other versions of
329 `etags` such as the version provided by Exuberant Ctags.
331 If you're using vim or emacs, you can also use Universal Ctags to build a tag
332 file using the syntax:
335 $ ctags -R -D 'MOCK_IMPL(r,h,a)=r h a' .
338 If you're using an older version of Universal Ctags, you can use the following
342 ctags -R --mline-regex-c='/MOCK_IMPL\([^,]+,\W*([a-zA-Z0-9_]+)\W*,/\1/f/{mgroup=1}' .
345 A vim-compatible tag file will be generated by default. If you use emacs, add
346 the `-e` flag to generate an emacs-compatible tag file.
350 We use the 'doxygen' utility to generate documentation from our
351 source code. Here's how to use it:
353 1. Begin every file that should be documented with
358 * \brief Short description of the file.
362 (Doxygen will recognize any comment beginning with /** as special.)
364 2. Before any function, structure, #define, or variable you want to
365 document, add a comment of the form:
368 /** Describe the function's actions in imperative sentences.
370 * Use blank lines for paragraph breaks
376 * Write <b>argument_names</b> in boldface.
379 * place_example_code();
380 * between_code_and_endcode_commands();
385 3. Make sure to escape the characters `<`, `>`, `\`, `%` and `#` as `\<`,
386 `\>`, `\\`, `\%` and `\#`.
388 4. To document structure members, you can use two forms:
392 /** You can put the comment before an element; */
394 int b; /**< Or use the less-than symbol to put the comment
395 * after the element. */
399 5. To generate documentation from the Tor source code, type:
405 to generate a file called `Doxyfile`. Edit that file and run
406 `doxygen` to generate the API documentation.
408 6. See the Doxygen manual for more information; this summary just
409 scratches the surface.
411 ## Style and best-practices checking
413 We use scripts to check for various problems in the formatting and style
414 of our source code. The "check-spaces" test detects a bunch of violations
415 of our coding style on the local level. The "check-best-practices" test
416 looks for violations of some of our complexity guidelines.
418 You can tell the tool about exceptions to the complexity guidelines via its
419 exceptions file (scripts/maint/practracker/exceptions.txt). But before you
420 do this, consider whether you shouldn't fix the underlying problem. Maybe
421 that file really _is_ too big. Maybe that function really _is_ doing too
422 much. (On the other hand, for stable release series, it is sometimes better
423 to leave things unrefactored.)