From 9b9968f68c2fb7f31d840ea2c03154a81ae42720 Mon Sep 17 00:00:00 2001 From: Daniel Borkmann Date: Wed, 28 Mar 2012 23:59:57 +0200 Subject: [PATCH] docs: added notes for tag of version 0.5.6 Added notes for tag of version 0.5.6. Signed-off-by: Daniel Borkmann --- Documentation/Notes-0.5.6 | 451 ++++++++++++++++++++++++++++++++++++++++++++++ README | 2 +- 2 files changed, 452 insertions(+), 1 deletion(-) create mode 100644 Documentation/Notes-0.5.6 diff --git a/Documentation/Notes-0.5.6 b/Documentation/Notes-0.5.6 new file mode 100644 index 00000000..0f039e44 --- /dev/null +++ b/Documentation/Notes-0.5.6 @@ -0,0 +1,451 @@ +netsniff-ng, release 0.5.6: +/////////////////////////// + +Date: 29.03.2012 + +We are pleased to announce the immediate and free availability of netsniff-ng +in version 0.5.6! This is a major release with lots of new features. If you +are using netsniff-ng 0.5.5, we highly recommend upgrading! + +So 18 months with late-night spare time hacking have passed. Promised, the next +timespan will be shorter. There are still a lot of things to be done in future +as our projects file suggests, so keep in mind that the version number 0.5.6 +indicates that this is not a mature product yet. + +No Linux kernel patch is required to make usage of the zero-copy facilities in +the kernel. And, when we speak of zero-copy, we mean that network packets are +not copied between user space and kernel space. Internally, we are using the +built-in RX_RING and TX_RING functionality, especially in netsniff-ng and +trafgen. And yes, you don't need to have PF_RING for that [1]! Netsniff-ng +users have reported performance numbers to us that indicate that the packet +per second performance has no significant differences. Own measurements agree +to that. So out of the box, RX_RING and TX_RING is the fastest you can get. + +Please find documentation about the individual tools in the Documentation/ +folder. The netsniff-ng toolkit is purely non-profit and provided in the hope, +that it is found useful. + + [1] e.g. http://www.spinics.net/lists/netfilter-devel/msg20212.html + +Obtaining the sources: + +- Via Git: + - git clone git://github.com/gnumaniacs/netsniff-ng.git + - git checkout 0.5.6 +- Via HTTP: + - wget http://pub.netsniff-ng.org/netsniff-ng/netsniff-ng-0.5.6.tar.gz + +Highlights: + +- We have thrown away the old netsniff-ng 0.5.5 code and have rewritten + netsniff-ng from scratch. It has even grown into a toolkit. Thus, next to + netsniff-ng, the tools trafgen, bpfc, ifpps, flowtop, curvetun and ashunt + are available: + + - netsniff-ng: a zero-copy protocol analyzer and traffic capturing utility. + It can record and also replay pcap files with different file I/O techniques + such as memory mapped I/O or scatter gather I/O. netsniff-ng supports packet + filtering with Berkeley Packet Filters. The dissector has also been improved + with further IPv6 functionality. + + - trafgen: is a zero-copy network packet generator. It uses the Linux' TX_RING + for high-speed transmissions, but also has a slower transmission mode where + inter-departure gaps are possible. Packets can be easily defined in a + text-based configuration file that is passed to trafgen. Note that + netsniff-ng also has a possibility of transforming pcap files into txf files + for usage with trafgen. + + - bpfc: a Berkeley Packet Filter compiler that speaks Steven McCanne and + Van Jacobson's filter language that is defined in "The BSD packet filter: + a new architecture for user-level packet capture", from Proceedings of the + USENIX Winter 1993 Conference Proceedings on USENIX Winter 1993 Conference + Proceedings. It also supports undocumented Linux kernel extensions. We think + it is useful to also have the possibility to experiment with filters on a + lower level that gives you _full control_ over filtering and haven't found + an implementation of that language yet. The output of bpfc can be used in + netsniff-ng. + + - ifpps: For measurement purposes, we have implemented a tool called ifpps, + which periodically provides top-like networking and system statistics from + the kernel. ifpps gathers its data directly from procfs files and does not + apply any user space monitoring libraries such as libpcap which is used in + tools like iptraf, for instance. Hence, no statistical distortion will come + up on high packet loads. ifpps presents what i.e. the network driver + calculates in kernel space. + + - flowtop: flowtop is a top-like connection tracking tool that can run on an + end host or router. It is able to present TCP or UDP flows that have been + collected by the kernel space netfilter framework. Next to reverse DNS data, + connection states and ports, geographical information about the connection + end points are supplied. If flowtop runs on an end host, it is able to + detect the corresponding user space application of a particular flow. For + instance, it will output 'chromium-browser' with its process id, if you + surf the web from this machine with (guess what?!) chromium. + + - curvetun: curvetun is a lightweight, high-speed ECDH multiuser IP tunnel + for Linux that is based on epoll(2). curvetun uses the Linux TUN/TAP + interface and supports {IPv4,IPv6} over {IPv4,IPv6} with UDP or TCP as + carrier protocols. As key management, public-key cryptography based on + elliptic curves are being used and packets are encrypted by a symmetric + stream cipher (Salsa20) and authenticated by a MAC (Poly1305), where + session keys have previously been computed with the ECDH key agreement + protocol (Curve25519). Cryptography is based on Daniel J. Bernsteins + Networking and Cryptography library (NaCl). We also provide a small script + for generating a user-pubkey text configuration file for curvetun servers + with information supplied from LDAP. + + - ashunt: is an autonomous system trace route utility. It uses TCP- and also + ICMP-based probes to detect intermediate nodes. Next to reverse DNS + information that is also gathered by traceroute, information about the + autonomous system of that hop is presented. Furthermore, geographical data + such as country and city that is connected to a hop is supplied as well as + some other data. For experimenting, ashunt gives full control to the TCP/IP + header fields to the user. Also, sending a probe with a specified ASCII + cleartext payload is possible. + +Summary: + +- 18 months timespan +- Git commits excluding merges since 0.5.5: + + 1107 Daniel Borkmann + 126 Emmanuel Roullit + + Plus further contributions by: + + Markus Amend + Ronald W. Henderson + James S. Binder + Markus Kötter + + Plus distribution maintenance by: + + Kartik Mistry + Jiří Skála + Can Celasun + Michael Weber + Corrado Franco + Pascal Bleser + Guillaume Rousse + Michael Prokop + Ronald W. Henderson + Fabian Affolter + +Reporting bugs: + +- E-mail to +- Online bug tracker: http://bugs.netsniff-ng.org/ + +Git short log: + + (see Git history, too long to put here this time) + +Detailed file changes: + + .gitattributes | 3 + + .gitignore | 2 - + .mailmap | 14 + + CHANGELOG | 2137 ---- + CODING | 831 -- + COPYING | 10 + + CREDITS | 113 - + Documentation/Ashunt | 86 + + Documentation/Bpfc | 236 + + Documentation/ChangeLog | 79 + + Documentation/CodingStyle | 831 ++ + Documentation/Curvetun | 236 + + Documentation/Flowtop | 74 + + Documentation/Ifpps | 90 + + Documentation/Manpages | 2 + + Documentation/Netsniff-ng | 99 + + Documentation/Performance | 286 + + Documentation/SubmittingPatches | 121 + + Documentation/Trafgen | 129 + + Documentation/logo.png | Bin 0 -> 12215 bytes + Documentation/logo.txt | 3 + + HACKING | 67 - + INSTALL | 100 +- + MAINTAINER | 100 + + MIRRORS | 9 + + PROJECTS | 204 + + README | 151 +- + REPORTING-BUGS | 8 + + THANKS | 55 + + TODO | 12 - + VERSION | 2 +- + contrib/art/logo.png | Bin 0 -> 12215 bytes + contrib/art/logo_only.png | Bin 0 -> 3928 bytes + contrib/art/netsniff_Tshirt.jpg | Bin 0 -> 1351501 bytes + contrib/art/netsniff_logo.ai | 4168 ++++++ + contrib/art/netsniff_logo2.svg | 156 + + contrib/art/netsniff_logo2_paths.pdf | Bin 0 -> 17198 bytes + contrib/art/netsniff_logo2_paths.svg | 278 + + contrib/art/netsniff_logo2_paths_white.pdf | Bin 0 -> 17203 bytes + contrib/art/netsniff_logo3.svg | 143 + + contrib/art/netsniff_logo3_paths.pdf | Bin 0 -> 8764 bytes + contrib/art/netsniff_logo3_paths.svg | 181 + + contrib/art/netsniff_logo3_paths_white.pdf | Bin 0 -> 8769 bytes + contrib/art/qr_netsniff_ng2_black_corner.pdf | 1462 +++ + contrib/art/qr_netsniff_ng2_black_corner.svg | 121 + + contrib/art/qr_netsniff_ng_black_corner.eps | 1527 +++ + contrib/html/bpf.pdf | Bin 0 -> 135803 bytes + contrib/html/faq.html | 516 + + contrib/html/img/debian.png | Bin 0 -> 2761 bytes + contrib/html/img/logo.png | Bin 0 -> 3928 bytes + contrib/html/img/logo2.png | Bin 0 -> 7349 bytes + contrib/html/img/no_epatent.png | Bin 0 -> 2267 bytes + contrib/html/img/osmc.jpg | Bin 0 -> 120430 bytes + contrib/html/img/qdn.png | Bin 0 -> 1908 bytes + contrib/html/img/tiny-logo.png | Bin 0 -> 449 bytes + contrib/html/img/vim.png | Bin 0 -> 3109 bytes + contrib/html/img/vt100.gif | Bin 0 -> 409 bytes + contrib/html/index.html | 230 + + .../Sending_and_receiving_zero-copy_networking.png | Bin 0 -> 6787 bytes + .../Sending_and_receiving_zero-copy_networking.txt | 3527 +++++ + contrib/html/pub/netsniff-ng/MD5SUMS | 5 + + contrib/html/pub/netsniff-ng/SHA256SUMS | 5 + + .../pub/netsniff-ng/netsniff-ng-0.5.3.0.tar.gz | Bin 0 -> 21469 bytes + .../pub/netsniff-ng/netsniff-ng-0.5.4.0.tar.gz | Bin 0 -> 137012 bytes + .../pub/netsniff-ng/netsniff-ng-0.5.4.1.tar.gz | Bin 0 -> 137477 bytes + .../pub/netsniff-ng/netsniff-ng-0.5.4.2.tar.gz | Bin 0 -> 141979 bytes + .../pub/netsniff-ng/netsniff-ng-0.5.5.0.tar.gz | Bin 0 -> 265313 bytes + contrib/html/style.css | 149 + + contrib/nacl/nacl-20110221.tar.bz2 | Bin 0 -> 163415 bytes + netsniff-ng.8 | 692 - + scripts/bpf.vim | 45 + + scripts/curvetun-ldap | 98 + + scripts/geoip-database-update | 42 + + src/.gitattributes | 3 + + src/CMakeLists.txt | 77 + + src/Makefile | 69 - + src/ashunt.c | 1116 ++ + src/ashunt/.gitignore | 5 + + src/ashunt/CMakeLists.txt | 29 + + src/aslookup.c | 184 + + src/aslookup.h | 24 + + src/bootstrap.c | 549 - + src/bpf.c | 327 +- + src/bpf.h | 144 + + src/bpf_lexer.l | 110 + + src/bpf_parser.y | 542 + + src/bpfc.c | 186 + + src/bpfc/.gitignore | 5 + + src/bpfc/CMakeLists.txt | 25 + + src/built_in.h | 82 + + src/cmake/modules/CheckBPFAttach.cmake | 39 + + src/cmake/modules/CheckPFPacket.cmake | 67 + + src/cmake/modules/CheckStrictAlign.cmake | 95 + + src/cmake/modules/CheckTxRing.cmake | 67 + + src/cmake/modules/FindLibGeoIP.cmake | 52 + + src/cmake/modules/FindLibNaCl.cmake | 25 + + src/cmake/modules/FindLibNetFilterConnTrack.cmake | 44 + + src/cmake/modules/FindLibURCU.cmake | 44 + + src/cmake/modules/Pod2Man.cmake | 59 + + src/conf/ether.conf | 290 + + src/conf/oui.conf |13351 +++++++++++++++++++ + src/conf/tcp.conf | 1100 ++ + src/conf/udp.conf | 1056 ++ + src/conf/whois.conf | 1 + + src/config.c | 311 - + src/cpusched.c | 160 + + src/cpusched.h | 18 + + src/csum.h | 164 + + src/ct_client.c | 439 + + src/ct_server.c | 822 ++ + src/cursor.c | 88 - + src/curve.c | 294 + + src/curve.h | 231 + + src/curvetun.c | 768 ++ + src/curvetun.h | 44 + + src/curvetun/.gitignore | 5 + + src/curvetun/CMakeLists.txt | 36 + + src/curvetun/abiname.c | 46 + + src/curvetun/build_nacl.sh | 77 + + src/curvetun/nacl_path.sh | 51 + + src/definitions.mk | 54 - + src/die.h | 52 + + src/dissector.c | 110 + + src/dissector.h | 39 + + src/dissector_eth.c | 342 + + src/dissector_eth.h | 41 + + src/dump.c | 79 - + src/examples/bpfc/all_traffic.bpf | 1 + + src/examples/bpfc/arp.bpf | 4 + + src/examples/bpfc/atalk.bpf | 9 + + src/examples/bpfc/broadcast.bpf | 6 + + src/examples/bpfc/ftp.bpf | 15 + + src/examples/bpfc/http.bpf | 15 + + src/examples/bpfc/icmp.bpf | 6 + + src/examples/bpfc/icq.bpf | 15 + + src/examples/bpfc/imap.bpf | 17 + + src/examples/bpfc/ip_broadcast.bpf | 8 + + src/examples/bpfc/ip_multicast.bpf | 6 + + src/examples/bpfc/multicast.bpf | 4 + + src/examples/bpfc/not_ip.bpf | 5 + + src/examples/bpfc/not_ssh.bpf | 24 + + src/examples/bpfc/pop3.bpf | 15 + + src/examples/bpfc/rarp.bpf | 4 + + src/examples/bpfc/rsync.bpf | 15 + + src/examples/bpfc/skype_pre.bpf | 13 + + src/examples/bpfc/smtp.bpf | 15 + + src/examples/bpfc/ssh.bpf | 15 + + src/examples/bpfc/vlan1000.bpf | 7 + + src/examples/trafgen/trafgen.txf | 48 + + src/examples/trafgen/trafgen2.txf | 18 + + src/flowtop.c | 1002 ++ + src/flowtop/.gitignore | 5 + + src/flowtop/CMakeLists.txt | 35 + + src/hash.c | 515 +- + src/hash.h | 87 + + src/ifpps.c | 894 ++ + src/ifpps/.gitignore | 5 + + src/ifpps/CMakeLists.txt | 22 + + src/include/bootstrap.h | 31 - + src/include/bpf.h | 31 - + src/include/config.h | 83 - + src/include/cursor.h | 43 - + src/include/dump.h | 30 - + src/include/ether_types.h | 330 - + src/include/hash.h | 84 - + src/include/macros.h | 160 - + src/include/misc.h | 56 - + src/include/netdev.h | 81 - + src/include/nsignal.h | 234 - + src/include/oui.h |13420 -------------------- + src/include/packet.h | 125 - + src/include/pcap.h | 93 - + src/include/ports_tcp.h | 1134 -- + src/include/ports_udp.h | 1089 -- + src/include/print.h | 60 - + src/include/protocols/arp.h | 134 - + src/include/protocols/csum.h | 157 - + src/include/protocols/ethernet.h | 89 - + src/include/protocols/icmp.h | 88 - + src/include/protocols/ip.h | 128 - + src/include/protocols/ipv6.h | 129 - + src/include/protocols/layers_2.h | 27 - + src/include/protocols/layers_3.h | 26 - + src/include/protocols/layers_4.h | 27 - + src/include/protocols/layers_all.h | 27 - + src/include/protocols/tcp.h | 174 - + src/include/protocols/udp.h | 138 - + src/include/protocols/vlan.h | 85 - + src/include/read.h | 28 - + src/include/replay.h | 33 - + src/include/rx_ring.h | 72 - + src/include/rxtx_common.h | 78 - + src/include/strlcpy.h | 25 - + src/include/system.h | 68 - + src/include/ticks.h | 173 - + src/include/tx_ring.h | 67 - + src/include/types.h | 68 - + src/include/version.h | 36 - + src/include/xmalloc.h | 46 - + src/locking.h | 90 + + src/man/netsniff-ng.txt | 574 - + src/misc.c | 132 - + src/mtrand.c | 163 + + src/mtrand.h | 23 + + src/netdev.c | 910 -- + src/netsniff-ng.c | 1282 ++- + src/netsniff-ng/.gitignore | 5 + + src/netsniff-ng/CMakeLists.txt | 41 + + src/opt_memcpy.c | 302 + + src/opt_memcpy.h | 81 + + src/patricia.c | 333 + + src/patricia.h | 52 + + src/pcap.c | 28 + + src/pcap.h | 170 + + src/pcap_mmap.c | 227 + + src/pcap_rw.c | 107 + + src/pcap_sg.c | 217 + + src/print.c | 481 - + src/proto_arp.h | 132 + + src/proto_esp.h | 67 + + src/proto_ethernet.h | 100 + + src/proto_hex.h | 76 + + src/proto_icmp.h | 71 + + src/proto_ip_authentication_hdr.h | 87 + + src/proto_ipv4.h | 128 + + src/proto_ipv6.h | 118 + + src/proto_ipv6_dest_opts.h | 83 + + src/proto_ipv6_fragm.h | 82 + + src/proto_ipv6_hop_by_hop.h | 83 + + src/proto_ipv6_in_ipv4.h | 30 + + src/proto_ipv6_mobility_hdr.h | 87 + + src/proto_ipv6_no_nxt_hdr.h | 39 + + src/proto_ipv6_routing.h | 97 + + src/proto_struct.h | 43 + + src/proto_tcp.h | 180 + + src/proto_udp.h | 110 + + src/proto_vlan.h | 81 + + src/protos.h | 30 + + src/replay.c | 126 - + src/ring.h | 139 + + src/ring_rx.c | 117 + + src/ring_rx.h | 32 + + src/ring_tx.c | 126 + + src/ring_tx.h | 37 + + src/rules/all_traffic.bpf | 21 - + src/rules/arp.bpf | 24 - + src/rules/atalk.bpf | 29 - + src/rules/broadcast.bpf | 26 - + src/rules/ftp.bpf | 35 - + src/rules/http.bpf | 35 - + src/rules/icmp.bpf | 26 - + src/rules/icq.bpf | 35 - + src/rules/imap.bpf | 37 - + src/rules/ip_broadcast.bpf | 28 - + src/rules/ip_multicast.bpf | 26 - + src/rules/multicast.bpf | 24 - + src/rules/not_ip.bpf | 25 - + src/rules/not_ssh.bpf | 44 - + src/rules/pop3.bpf | 35 - + src/rules/rarp.bpf | 24 - + src/rules/rsync.bpf | 35 - + src/rules/skype_pre.bpf | 33 - + src/rules/smtp.bpf | 35 - + src/rules/ssh.bpf | 35 - + src/rules/vlan1000.bpf | 27 - + src/rx_ring.c | 449 - + src/servmgmt.c | 285 + + src/servmgmt.h | 24 + + src/strlcpy.c | 54 - + src/stun.c | 235 + + src/stun.h | 15 + + src/system.c | 291 - + src/tprintf.c | 112 + + src/tprintf.h | 17 + + src/trafgen.c | 933 ++ + src/trafgen/.gitignore | 5 + + src/trafgen/CMakeLists.txt | 19 + + src/trie.c | 153 + + src/trie.h | 22 + + src/tx_ring.c | 347 - + src/usermgmt.c | 689 + + src/usermgmt.h | 50 + + src/xio.c | 127 + + src/xio.h | 19 + + src/xmalloc.c | 193 +- + src/xmalloc.h | 35 + + src/xstring.c | 97 + + src/xstring.h | 51 + + src/xsys.c | 634 + + src/xsys.h | 143 + + 290 files changed, 49579 insertions(+), 28908 deletions(-) + +--- + ,---------------------, + < Y U NO LUV PACKETZ? > + '---------------------' + O + o + ^__^ + _______/(oo) + /\/( /(_o) + | W----|| _ + || || |~| ~~ + |~| ~ + |_| o + |#|/ + _+#+_ diff --git a/README b/README index d690e06f..b8f6d218 100644 --- a/README +++ b/README @@ -12,7 +12,7 @@ Linux network analyzer and > `' '` < / networking toolkit. ) ,.==., ( | (|/--~~--\|)-' - Release: 2012-03-xx / + Release: 2012-03-29 / ( ___ Web: http://netsniff-ng.org \__.=|___E -- 2.11.4.GIT