docs: added notes for tag of version 0.5.6
[netsniff-ng.git] / Documentation / Notes-0.5.6
blob0f039e440ac7a1c510540629b848addc4e7091f5
1 netsniff-ng, release 0.5.6:
2 ///////////////////////////
4 Date: 29.03.2012
6 We are pleased to announce the immediate and free availability of netsniff-ng
7 in version 0.5.6! This is a major release with lots of new features. If you
8 are using netsniff-ng 0.5.5, we highly recommend upgrading!
10 So 18 months with late-night spare time hacking have passed. Promised, the next
11 timespan will be shorter. There are still a lot of things to be done in future
12 as our projects file suggests, so keep in mind that the version number 0.5.6
13 indicates that this is not a mature product yet.
15 No Linux kernel patch is required to make usage of the zero-copy facilities in
16 the kernel. And, when we speak of zero-copy, we mean that network packets are
17 not copied between user space and kernel space. Internally, we are using the
18 built-in RX_RING and TX_RING functionality, especially in netsniff-ng and
19 trafgen. And yes, you don't need to have PF_RING for that [1]! Netsniff-ng
20 users have reported performance numbers to us that indicate that the packet
21 per second performance has no significant differences. Own measurements agree
22 to that. So out of the box, RX_RING and TX_RING is the fastest you can get.
24 Please find documentation about the individual tools in the Documentation/
25 folder. The netsniff-ng toolkit is purely non-profit and provided in the hope,
26 that it is found useful.
28  [1] e.g. http://www.spinics.net/lists/netfilter-devel/msg20212.html
30 Obtaining the sources:
32 - Via Git:
33   - git clone git://github.com/gnumaniacs/netsniff-ng.git
34   - git checkout 0.5.6
35 - Via HTTP:
36   - wget http://pub.netsniff-ng.org/netsniff-ng/netsniff-ng-0.5.6.tar.gz
38 Highlights:
40 - We have thrown away the old netsniff-ng 0.5.5 code and have rewritten
41   netsniff-ng from scratch. It has even grown into a toolkit. Thus, next to
42   netsniff-ng, the tools trafgen, bpfc, ifpps, flowtop, curvetun and ashunt
43   are available:
45   - netsniff-ng: a zero-copy protocol analyzer and traffic capturing utility.
46     It can record and also replay pcap files with different file I/O techniques
47     such as memory mapped I/O or scatter gather I/O. netsniff-ng supports packet
48     filtering with Berkeley Packet Filters. The dissector has also been improved
49     with further IPv6 functionality.
51   - trafgen: is a zero-copy network packet generator. It uses the Linux' TX_RING
52     for high-speed transmissions, but also has a slower transmission mode where
53     inter-departure gaps are possible. Packets can be easily defined in a
54     text-based configuration file that is passed to trafgen. Note that
55     netsniff-ng also has a possibility of transforming pcap files into txf files
56     for usage with trafgen.
58   - bpfc: a Berkeley Packet Filter compiler that speaks Steven McCanne and
59     Van Jacobson's filter language that is defined in "The BSD packet filter:
60     a new architecture for user-level packet capture", from Proceedings of the
61     USENIX Winter 1993 Conference Proceedings on USENIX Winter 1993 Conference
62     Proceedings. It also supports undocumented Linux kernel extensions. We think
63     it is useful to also have the possibility to experiment with filters on a
64     lower level that gives you _full control_ over filtering and haven't found
65     an implementation of that language yet. The output of bpfc can be used in
66     netsniff-ng.
68   - ifpps: For measurement purposes, we have implemented a tool called ifpps,
69     which periodically provides top-like networking and system statistics from
70     the kernel. ifpps gathers its data directly from procfs files and does not
71     apply any user space monitoring libraries such as libpcap which is used in
72     tools like iptraf, for instance. Hence, no statistical distortion will come
73     up on high packet loads. ifpps presents what i.e. the network driver
74     calculates in kernel space.
76   - flowtop: flowtop is a top-like connection tracking tool that can run on an
77     end host or router. It is able to present TCP or UDP flows that have been
78     collected by the kernel space netfilter framework. Next to reverse DNS data,
79     connection states and ports, geographical information about the connection
80     end points are supplied. If flowtop runs on an end host, it is able to
81     detect the corresponding user space application of a particular flow. For
82     instance, it will output 'chromium-browser' with its process id, if you
83     surf the web from this machine with (guess what?!) chromium.
85   - curvetun: curvetun is a lightweight, high-speed ECDH multiuser IP tunnel
86     for Linux that is based on epoll(2). curvetun uses the Linux TUN/TAP
87     interface and supports {IPv4,IPv6} over {IPv4,IPv6} with UDP or TCP as
88     carrier protocols. As key management, public-key cryptography based on
89     elliptic curves are being used and packets are encrypted by a symmetric
90     stream cipher (Salsa20) and authenticated by a MAC (Poly1305), where
91     session keys have previously been computed with the ECDH key agreement
92     protocol (Curve25519). Cryptography is based on Daniel J. Bernsteins
93     Networking and Cryptography library (NaCl). We also provide a small script
94     for generating a user-pubkey text configuration file for curvetun servers
95     with information supplied from LDAP.
97   - ashunt: is an autonomous system trace route utility. It uses TCP- and also
98     ICMP-based probes to detect intermediate nodes. Next to reverse DNS
99     information that is also gathered by traceroute, information about the
100     autonomous system of that hop is presented. Furthermore, geographical data
101     such as country and city that is connected to a hop is supplied as well as
102     some other data. For experimenting, ashunt gives full control to the TCP/IP
103     header fields to the user. Also, sending a probe with a specified ASCII
104     cleartext payload is possible.
106 Summary:
108 - 18 months timespan
109 - Git commits excluding merges since 0.5.5:
111  1107  Daniel Borkmann
112   126  Emmanuel Roullit
114   Plus further contributions by:
116        Markus Amend
117        Ronald W. Henderson
118        James S. Binder
119        Markus Kötter
121   Plus distribution maintenance by:
123        Kartik Mistry
124        Jiří Skála
125        Can Celasun
126        Michael Weber
127        Corrado Franco
128        Pascal Bleser
129        Guillaume Rousse
130        Michael Prokop
131        Ronald W. Henderson
132        Fabian Affolter
134 Reporting bugs:
136 - E-mail to <bugs@netsniff-ng.org>
137 - Online bug tracker: http://bugs.netsniff-ng.org/
139 Git short log:
141  (see Git history, too long to put here this time)
143 Detailed file changes:
145  .gitattributes                                     |    3 +
146  .gitignore                                         |    2 -
147  .mailmap                                           |   14 +
148  CHANGELOG                                          | 2137 ----
149  CODING                                             |  831 --
150  COPYING                                            |   10 +
151  CREDITS                                            |  113 -
152  Documentation/Ashunt                               |   86 +
153  Documentation/Bpfc                                 |  236 +
154  Documentation/ChangeLog                            |   79 +
155  Documentation/CodingStyle                          |  831 ++
156  Documentation/Curvetun                             |  236 +
157  Documentation/Flowtop                              |   74 +
158  Documentation/Ifpps                                |   90 +
159  Documentation/Manpages                             |    2 +
160  Documentation/Netsniff-ng                          |   99 +
161  Documentation/Performance                          |  286 +
162  Documentation/SubmittingPatches                    |  121 +
163  Documentation/Trafgen                              |  129 +
164  Documentation/logo.png                             |  Bin 0 -> 12215 bytes
165  Documentation/logo.txt                             |    3 +
166  HACKING                                            |   67 -
167  INSTALL                                            |  100 +-
168  MAINTAINER                                         |  100 +
169  MIRRORS                                            |    9 +
170  PROJECTS                                           |  204 +
171  README                                             |  151 +-
172  REPORTING-BUGS                                     |    8 +
173  THANKS                                             |   55 +
174  TODO                                               |   12 -
175  VERSION                                            |    2 +-
176  contrib/art/logo.png                               |  Bin 0 -> 12215 bytes
177  contrib/art/logo_only.png                          |  Bin 0 -> 3928 bytes
178  contrib/art/netsniff_Tshirt.jpg                    |  Bin 0 -> 1351501 bytes
179  contrib/art/netsniff_logo.ai                       | 4168 ++++++
180  contrib/art/netsniff_logo2.svg                     |  156 +
181  contrib/art/netsniff_logo2_paths.pdf               |  Bin 0 -> 17198 bytes
182  contrib/art/netsniff_logo2_paths.svg               |  278 +
183  contrib/art/netsniff_logo2_paths_white.pdf         |  Bin 0 -> 17203 bytes
184  contrib/art/netsniff_logo3.svg                     |  143 +
185  contrib/art/netsniff_logo3_paths.pdf               |  Bin 0 -> 8764 bytes
186  contrib/art/netsniff_logo3_paths.svg               |  181 +
187  contrib/art/netsniff_logo3_paths_white.pdf         |  Bin 0 -> 8769 bytes
188  contrib/art/qr_netsniff_ng2_black_corner.pdf       | 1462 +++
189  contrib/art/qr_netsniff_ng2_black_corner.svg       |  121 +
190  contrib/art/qr_netsniff_ng_black_corner.eps        | 1527 +++
191  contrib/html/bpf.pdf                               |  Bin 0 -> 135803 bytes
192  contrib/html/faq.html                              |  516 +
193  contrib/html/img/debian.png                        |  Bin 0 -> 2761 bytes
194  contrib/html/img/logo.png                          |  Bin 0 -> 3928 bytes
195  contrib/html/img/logo2.png                         |  Bin 0 -> 7349 bytes
196  contrib/html/img/no_epatent.png                    |  Bin 0 -> 2267 bytes
197  contrib/html/img/osmc.jpg                          |  Bin 0 -> 120430 bytes
198  contrib/html/img/qdn.png                           |  Bin 0 -> 1908 bytes
199  contrib/html/img/tiny-logo.png                     |  Bin 0 -> 449 bytes
200  contrib/html/img/vim.png                           |  Bin 0 -> 3109 bytes
201  contrib/html/img/vt100.gif                         |  Bin 0 -> 409 bytes
202  contrib/html/index.html                            |  230 +
203  .../Sending_and_receiving_zero-copy_networking.png |  Bin 0 -> 6787 bytes
204  .../Sending_and_receiving_zero-copy_networking.txt | 3527 +++++
205  contrib/html/pub/netsniff-ng/MD5SUMS               |    5 +
206  contrib/html/pub/netsniff-ng/SHA256SUMS            |    5 +
207  .../pub/netsniff-ng/netsniff-ng-0.5.3.0.tar.gz     |  Bin 0 -> 21469 bytes
208  .../pub/netsniff-ng/netsniff-ng-0.5.4.0.tar.gz     |  Bin 0 -> 137012 bytes
209  .../pub/netsniff-ng/netsniff-ng-0.5.4.1.tar.gz     |  Bin 0 -> 137477 bytes
210  .../pub/netsniff-ng/netsniff-ng-0.5.4.2.tar.gz     |  Bin 0 -> 141979 bytes
211  .../pub/netsniff-ng/netsniff-ng-0.5.5.0.tar.gz     |  Bin 0 -> 265313 bytes
212  contrib/html/style.css                             |  149 +
213  contrib/nacl/nacl-20110221.tar.bz2                 |  Bin 0 -> 163415 bytes
214  netsniff-ng.8                                      |  692 -
215  scripts/bpf.vim                                    |   45 +
216  scripts/curvetun-ldap                              |   98 +
217  scripts/geoip-database-update                      |   42 +
218  src/.gitattributes                                 |    3 +
219  src/CMakeLists.txt                                 |   77 +
220  src/Makefile                                       |   69 -
221  src/ashunt.c                                       | 1116 ++
222  src/ashunt/.gitignore                              |    5 +
223  src/ashunt/CMakeLists.txt                          |   29 +
224  src/aslookup.c                                     |  184 +
225  src/aslookup.h                                     |   24 +
226  src/bootstrap.c                                    |  549 -
227  src/bpf.c                                          |  327 +-
228  src/bpf.h                                          |  144 +
229  src/bpf_lexer.l                                    |  110 +
230  src/bpf_parser.y                                   |  542 +
231  src/bpfc.c                                         |  186 +
232  src/bpfc/.gitignore                                |    5 +
233  src/bpfc/CMakeLists.txt                            |   25 +
234  src/built_in.h                                     |   82 +
235  src/cmake/modules/CheckBPFAttach.cmake             |   39 +
236  src/cmake/modules/CheckPFPacket.cmake              |   67 +
237  src/cmake/modules/CheckStrictAlign.cmake           |   95 +
238  src/cmake/modules/CheckTxRing.cmake                |   67 +
239  src/cmake/modules/FindLibGeoIP.cmake               |   52 +
240  src/cmake/modules/FindLibNaCl.cmake                |   25 +
241  src/cmake/modules/FindLibNetFilterConnTrack.cmake  |   44 +
242  src/cmake/modules/FindLibURCU.cmake                |   44 +
243  src/cmake/modules/Pod2Man.cmake                    |   59 +
244  src/conf/ether.conf                                |  290 +
245  src/conf/oui.conf                                  |13351 +++++++++++++++++++
246  src/conf/tcp.conf                                  | 1100 ++
247  src/conf/udp.conf                                  | 1056 ++
248  src/conf/whois.conf                                |    1 +
249  src/config.c                                       |  311 -
250  src/cpusched.c                                     |  160 +
251  src/cpusched.h                                     |   18 +
252  src/csum.h                                         |  164 +
253  src/ct_client.c                                    |  439 +
254  src/ct_server.c                                    |  822 ++
255  src/cursor.c                                       |   88 -
256  src/curve.c                                        |  294 +
257  src/curve.h                                        |  231 +
258  src/curvetun.c                                     |  768 ++
259  src/curvetun.h                                     |   44 +
260  src/curvetun/.gitignore                            |    5 +
261  src/curvetun/CMakeLists.txt                        |   36 +
262  src/curvetun/abiname.c                             |   46 +
263  src/curvetun/build_nacl.sh                         |   77 +
264  src/curvetun/nacl_path.sh                          |   51 +
265  src/definitions.mk                                 |   54 -
266  src/die.h                                          |   52 +
267  src/dissector.c                                    |  110 +
268  src/dissector.h                                    |   39 +
269  src/dissector_eth.c                                |  342 +
270  src/dissector_eth.h                                |   41 +
271  src/dump.c                                         |   79 -
272  src/examples/bpfc/all_traffic.bpf                  |    1 +
273  src/examples/bpfc/arp.bpf                          |    4 +
274  src/examples/bpfc/atalk.bpf                        |    9 +
275  src/examples/bpfc/broadcast.bpf                    |    6 +
276  src/examples/bpfc/ftp.bpf                          |   15 +
277  src/examples/bpfc/http.bpf                         |   15 +
278  src/examples/bpfc/icmp.bpf                         |    6 +
279  src/examples/bpfc/icq.bpf                          |   15 +
280  src/examples/bpfc/imap.bpf                         |   17 +
281  src/examples/bpfc/ip_broadcast.bpf                 |    8 +
282  src/examples/bpfc/ip_multicast.bpf                 |    6 +
283  src/examples/bpfc/multicast.bpf                    |    4 +
284  src/examples/bpfc/not_ip.bpf                       |    5 +
285  src/examples/bpfc/not_ssh.bpf                      |   24 +
286  src/examples/bpfc/pop3.bpf                         |   15 +
287  src/examples/bpfc/rarp.bpf                         |    4 +
288  src/examples/bpfc/rsync.bpf                        |   15 +
289  src/examples/bpfc/skype_pre.bpf                    |   13 +
290  src/examples/bpfc/smtp.bpf                         |   15 +
291  src/examples/bpfc/ssh.bpf                          |   15 +
292  src/examples/bpfc/vlan1000.bpf                     |    7 +
293  src/examples/trafgen/trafgen.txf                   |   48 +
294  src/examples/trafgen/trafgen2.txf                  |   18 +
295  src/flowtop.c                                      | 1002 ++
296  src/flowtop/.gitignore                             |    5 +
297  src/flowtop/CMakeLists.txt                         |   35 +
298  src/hash.c                                         |  515 +-
299  src/hash.h                                         |   87 +
300  src/ifpps.c                                        |  894 ++
301  src/ifpps/.gitignore                               |    5 +
302  src/ifpps/CMakeLists.txt                           |   22 +
303  src/include/bootstrap.h                            |   31 -
304  src/include/bpf.h                                  |   31 -
305  src/include/config.h                               |   83 -
306  src/include/cursor.h                               |   43 -
307  src/include/dump.h                                 |   30 -
308  src/include/ether_types.h                          |  330 -
309  src/include/hash.h                                 |   84 -
310  src/include/macros.h                               |  160 -
311  src/include/misc.h                                 |   56 -
312  src/include/netdev.h                               |   81 -
313  src/include/nsignal.h                              |  234 -
314  src/include/oui.h                                  |13420 --------------------
315  src/include/packet.h                               |  125 -
316  src/include/pcap.h                                 |   93 -
317  src/include/ports_tcp.h                            | 1134 --
318  src/include/ports_udp.h                            | 1089 --
319  src/include/print.h                                |   60 -
320  src/include/protocols/arp.h                        |  134 -
321  src/include/protocols/csum.h                       |  157 -
322  src/include/protocols/ethernet.h                   |   89 -
323  src/include/protocols/icmp.h                       |   88 -
324  src/include/protocols/ip.h                         |  128 -
325  src/include/protocols/ipv6.h                       |  129 -
326  src/include/protocols/layers_2.h                   |   27 -
327  src/include/protocols/layers_3.h                   |   26 -
328  src/include/protocols/layers_4.h                   |   27 -
329  src/include/protocols/layers_all.h                 |   27 -
330  src/include/protocols/tcp.h                        |  174 -
331  src/include/protocols/udp.h                        |  138 -
332  src/include/protocols/vlan.h                       |   85 -
333  src/include/read.h                                 |   28 -
334  src/include/replay.h                               |   33 -
335  src/include/rx_ring.h                              |   72 -
336  src/include/rxtx_common.h                          |   78 -
337  src/include/strlcpy.h                              |   25 -
338  src/include/system.h                               |   68 -
339  src/include/ticks.h                                |  173 -
340  src/include/tx_ring.h                              |   67 -
341  src/include/types.h                                |   68 -
342  src/include/version.h                              |   36 -
343  src/include/xmalloc.h                              |   46 -
344  src/locking.h                                      |   90 +
345  src/man/netsniff-ng.txt                            |  574 -
346  src/misc.c                                         |  132 -
347  src/mtrand.c                                       |  163 +
348  src/mtrand.h                                       |   23 +
349  src/netdev.c                                       |  910 --
350  src/netsniff-ng.c                                  | 1282 ++-
351  src/netsniff-ng/.gitignore                         |    5 +
352  src/netsniff-ng/CMakeLists.txt                     |   41 +
353  src/opt_memcpy.c                                   |  302 +
354  src/opt_memcpy.h                                   |   81 +
355  src/patricia.c                                     |  333 +
356  src/patricia.h                                     |   52 +
357  src/pcap.c                                         |   28 +
358  src/pcap.h                                         |  170 +
359  src/pcap_mmap.c                                    |  227 +
360  src/pcap_rw.c                                      |  107 +
361  src/pcap_sg.c                                      |  217 +
362  src/print.c                                        |  481 -
363  src/proto_arp.h                                    |  132 +
364  src/proto_esp.h                                    |   67 +
365  src/proto_ethernet.h                               |  100 +
366  src/proto_hex.h                                    |   76 +
367  src/proto_icmp.h                                   |   71 +
368  src/proto_ip_authentication_hdr.h                  |   87 +
369  src/proto_ipv4.h                                   |  128 +
370  src/proto_ipv6.h                                   |  118 +
371  src/proto_ipv6_dest_opts.h                         |   83 +
372  src/proto_ipv6_fragm.h                             |   82 +
373  src/proto_ipv6_hop_by_hop.h                        |   83 +
374  src/proto_ipv6_in_ipv4.h                           |   30 +
375  src/proto_ipv6_mobility_hdr.h                      |   87 +
376  src/proto_ipv6_no_nxt_hdr.h                        |   39 +
377  src/proto_ipv6_routing.h                           |   97 +
378  src/proto_struct.h                                 |   43 +
379  src/proto_tcp.h                                    |  180 +
380  src/proto_udp.h                                    |  110 +
381  src/proto_vlan.h                                   |   81 +
382  src/protos.h                                       |   30 +
383  src/replay.c                                       |  126 -
384  src/ring.h                                         |  139 +
385  src/ring_rx.c                                      |  117 +
386  src/ring_rx.h                                      |   32 +
387  src/ring_tx.c                                      |  126 +
388  src/ring_tx.h                                      |   37 +
389  src/rules/all_traffic.bpf                          |   21 -
390  src/rules/arp.bpf                                  |   24 -
391  src/rules/atalk.bpf                                |   29 -
392  src/rules/broadcast.bpf                            |   26 -
393  src/rules/ftp.bpf                                  |   35 -
394  src/rules/http.bpf                                 |   35 -
395  src/rules/icmp.bpf                                 |   26 -
396  src/rules/icq.bpf                                  |   35 -
397  src/rules/imap.bpf                                 |   37 -
398  src/rules/ip_broadcast.bpf                         |   28 -
399  src/rules/ip_multicast.bpf                         |   26 -
400  src/rules/multicast.bpf                            |   24 -
401  src/rules/not_ip.bpf                               |   25 -
402  src/rules/not_ssh.bpf                              |   44 -
403  src/rules/pop3.bpf                                 |   35 -
404  src/rules/rarp.bpf                                 |   24 -
405  src/rules/rsync.bpf                                |   35 -
406  src/rules/skype_pre.bpf                            |   33 -
407  src/rules/smtp.bpf                                 |   35 -
408  src/rules/ssh.bpf                                  |   35 -
409  src/rules/vlan1000.bpf                             |   27 -
410  src/rx_ring.c                                      |  449 -
411  src/servmgmt.c                                     |  285 +
412  src/servmgmt.h                                     |   24 +
413  src/strlcpy.c                                      |   54 -
414  src/stun.c                                         |  235 +
415  src/stun.h                                         |   15 +
416  src/system.c                                       |  291 -
417  src/tprintf.c                                      |  112 +
418  src/tprintf.h                                      |   17 +
419  src/trafgen.c                                      |  933 ++
420  src/trafgen/.gitignore                             |    5 +
421  src/trafgen/CMakeLists.txt                         |   19 +
422  src/trie.c                                         |  153 +
423  src/trie.h                                         |   22 +
424  src/tx_ring.c                                      |  347 -
425  src/usermgmt.c                                     |  689 +
426  src/usermgmt.h                                     |   50 +
427  src/xio.c                                          |  127 +
428  src/xio.h                                          |   19 +
429  src/xmalloc.c                                      |  193 +-
430  src/xmalloc.h                                      |   35 +
431  src/xstring.c                                      |   97 +
432  src/xstring.h                                      |   51 +
433  src/xsys.c                                         |  634 +
434  src/xsys.h                                         |  143 +
435  290 files changed, 49579 insertions(+), 28908 deletions(-)
438                 ,---------------------,
439                 < Y U NO LUV PACKETZ? >
440                 '---------------------'
441                          O
442                           o
443                             ^__^ 
444                     _______/(oo)
445                 /\/(       /(_o)
446                    | W----||  _
447                    ||     || |~|  ~~
448                              |~|  ~
449                              |_| o
450                              |#|/
451                             _+#+_