From 028c835168acdb2dc660d128514f0b0025e4dc44 Mon Sep 17 00:00:00 2001 From: Bert Hubert Date: Sat, 14 Dec 2002 13:28:37 +0000 Subject: [PATCH] lots --- index.php3 | 18 +- lartc.db | 774 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++--- 2 files changed, 753 insertions(+), 39 deletions(-) diff --git a/index.php3 b/index.php3 index 844423d..cb78261 100755 --- a/index.php3 +++ b/index.php3 @@ -10,15 +10,15 @@ src=http://ds9a.nl/pub/pdns88x33c.gif>

bert hubert (PowerDNS.COM BV) <bert.hubert@netherlabs.nl>,
+href=http://www.powerdns.com>PowerDNS.COM BV) ,
Greg Maxwell <greg@linuxpower.cx> and
+href=mailto:greg@linuxpower.cx> and
Remco van Mook (Virtu Secure Webservices) <remco@virtu.nl>
-Martijn van Oosterhout <kleptog@cupid.suninternet.com>
-Paul B Schroeder <paulsch@us.ibm.com>
-Jasper Spaans <jasper@spaans.ds9a.nl>
-HOWTO@ds9a.nl (HOWTO related only, do +href=http://www.virtu.nl>Virtu Secure Webservices)
+Martijn van Oosterhout
+Paul B Schroeder
+Jasper Spaans
+HOWTO@ds9a.nl (HOWTO related only, do not send questions)
lartc@mailman.ds9a.nl (mailing list/

News

+ +
2002-11-00 +A spanish translation is appearing here.
2002-11-08IPSEC IN LINUX 2.5.47! Read all about +it here.
2002-07-20Hate so called 'opt-in' spam? Visit this page or the local mirror. Even better, submit diff --git a/lartc.db b/lartc.db index 27735bd..9d864cc 100644 --- a/lartc.db +++ b/lartc.db @@ -1595,7 +1595,6 @@ interface Tunnel1 description IPv6 tunnel no ip address no ip directed-broadcast -ipv6 enable ipv6 address 3FFE:604:6:7::1/126 tunnel source Serial0 tunnel destination 145.100.24.181 @@ -1617,7 +1616,10 @@ web interface. Search for "ipv6 tunnel broker" on your favourite search engine. IPSEC: secure IP over the Internet - There are two kinds of IPSEC available for Linux these days. For 2.2 and 2.4, there is FreeS/WAN. They + + There are two kinds of IPSEC available for Linux these days. For 2.2 + and 2.4, there is FreeS/WAN, which was the first major implementation. They + have an official site and an unofficial one that is actually maintained. FreeS/WAN has traditionally not been merged with the mainline kernel for a number of reasons. Most often mentioned are 'political' issues with Americans @@ -1625,7 +1627,13 @@ web interface. Search for "ipv6 tunnel broker" on your favourite search engine. leading it to be a bad candidate for actual merging. - Additionally, many parties have voiced worries about the quality of the code. + Additionally, many parties have voiced +worries about the quality of the code. To setup FreeS/WAN, a lot of +documentation +is available. As of Linux 2.5.47, there is a native IPSEC implementation in the kernel. It was written by Alexey Kuznetsov and @@ -1637,24 +1645,42 @@ web interface. Search for "ipv6 tunnel broker" on your favourite search engine. that its configuration wil differ from the native IPSEC. + As of 2.5.49, IPSEC works without further patches. + + - As this is written, 2.5.47 has not even been released! A lot of what follows is likely to change. To try this with 2.5.46, - fetch patches from the linux kernel mailinglist. Crude userspace utilities are - here. Compiling these userspace utilities requires editing the GNUMakefiles in there to point them at your + I've collected patches released by Alexey or Dave Miller here. Apply all of them to 2.5.48 before +reporting problems! (as yet, there are none for 2.5.49). Crude userspace utilities are + here (pre-compiled +binary & manpage). Compiling these userspace utilities requires editing the Makefiles in there to point them at your 2.5.x kernel. This situation is expected to improve rapidly however. + + When compiling your kernel, be sure to turn on 'PF_KEY', 'AH', 'ESP' and +everything in the CryptoAPI! TCP_MSS netfilter target is currently broken, + you need to turn it off. + - The author of this chapter is a complete DNSSEC nitwit! If you find the inevitable mistakes, please email - bert hubert ahu@ds9a.nl. + The author of this chapter is a complete IPSEC nitwit! If you find the inevitable mistakes, please email + bert hubert ahu@ds9a.nl. - First, we'll show how to setup secure communication between two hosts. This sample can be made much - more complicated and powerful and we'll only do that, but only in a later section. + First, we'll show how to manually setup secure communication between + two hosts. A large part of this process can also be automated, but +here we'll do it by hand so as to acquaint ourselves with what is going on +'under the hood'. + + + Feel free to skip the following section if you are only interested +in automatic keying but be aware that some understanding of manual keying is +useful. Intro with Manual Keying @@ -1664,7 +1690,7 @@ web interface. Search for "ipv6 tunnel broker" on your favourite search engine. - Many iptables configurations drop IPSEC packets! To pass IPSEC, use: 'iptables -A xxx -p 50' and 'iptables -A xxx -p 51' + Many iptables configurations drop IPSEC packets! To pass IPSEC, use: 'iptables -A xxx -p 50 -j ACCEPT' and 'iptables -A xxx -p 51 -j ACCEPT' @@ -1713,10 +1739,16 @@ spdadd 10.0.0.216 10.0.0.11 any -P out ipsec and be wrapped in an AH authenticating header. Note that this does not describe which SA is to be used, that is left as an exercise for the kernel to determine. + + In other words, a Security Policy specifies WHAT we want; a Security +Association describes HOW we want it. + - Outgoing packets are however labelled with the SA SPI ids the kernel used for encryption and authentication - so the remote can lookup the corresponding verification and decryption instruction. + Outgoing packets are labelled with the SA SPI ('the how') which the + kernel used for encryption and authentication so the remote can + lookup the corresponding verification and decryption instruction. + What follows is a very simple configuration for talking from host 10.0.0.216 to 10.0.0.11 using encryption and authentication. Note that the reverse path is plaintext in this first version and that @@ -1730,8 +1762,8 @@ add 10.0.0.216 10.0.0.11 ah 24500 -A hmac-md5 "1234567890123456"; add 10.0.0.216 10.0.0.11 esp 24501 -E 3des-cbc "123456789012123456789012"; spdadd 10.0.0.216 10.0.0.11 any -P out ipsec - esp/transport//required - ah/transport//required; + esp/transport//require + ah/transport//require; @@ -1855,12 +1887,416 @@ spdadd 10.0.0.216 10.0.0.11 any -P in ipsec transmitted using asymmetric encryption technology, according to negotiated algorithm details. - The Linux 2.5 IPSEC implementation works with the KAME 'racoon' IKE daemon. As of 8 November, the Racoon version in Alexey's kametools - distribution is non-functional. - - - FIXME: write more + The Linux 2.5 IPSEC implementation works with the KAME 'racoon' IKE + daemon. As of 9 November, the racoon version in Alexey's iptools + distribution can be compiled, although you may need to remove +#include <net/route.h> in two files. Alternatively, I've supplied a +precompiled version. + + + + IKE needs access to UDP port 500, be sure that iptables does +not block it. + + + + Theory + + As explained before, automatic keying does a lot of the work +for us. Specifically, it creates Security Associations on the fly. It does +not however set policy for us, which is as it should be. + + + So, to benefit from IKE, setup a policy, but do not supply any +SAs. If the kernel discovers that there is an IPSEC policy, but no Security +Association, it will notify the IKE daemon, which then goes to work on +trying to negotiate one. + + + Reiterating, a Security Policy specifies WHAT we want; a Security +Association describes HOW we want it. Using automatic keying lets us get +away with only specifying what we want. + + + Example + + Kame racoon comes with a grand host of options, most of which have +very fine default values, so we don't need to touch them. As described +above, the operator needs to define a Security Policy, but no Security +Associations. We leave their negotiation to the IKE daemon. + + + In this example, 10.0.0.11 and 10.0.0.216 are once again going to +setup secure communications, but this time with help from racoon. For +simplicity this configuration will be using pre-shared keys, the +dreaded 'shared secrets'. X.509 certificates are discussed in a separate +section, see . + + We're +going to stick to almost the default configuration, identical on both hosts: + + + +path pre_shared_key "/usr/local/etc/racoon/psk.txt"; + +remote anonymous +{ + exchange_mode aggressive,main; + doi ipsec_doi; + situation identity_only; + + my_identifier address; + + lifetime time 2 min; # sec,min,hour + initial_contact on; + proposal_check obey; # obey, strict or claim + + proposal { + encryption_algorithm 3des; + hash_algorithm sha1; + authentication_method pre_shared_key; + dh_group 2 ; + } +} + +sainfo anonymous +{ + pfs_group 1; + lifetime time 2 min; + encryption_algorithm 3des ; + authentication_algorithm hmac_sha1; + compression_algorithm deflate ; +} + + + + Lots of settings - I think yet more can be removed to get closer to +the default configuration. A few noteworthy things. We've configured two +anonymous settings which hold for all remotes, making further configuration +easy. There is no need for per-host stanzas here, unless we really want +them. + + + Furthermore, we've set it up such that we identify ourselves based +on our IP address ('my_identifier address'), and declare that we can do +3des, sha1, and that we will be using a pre-shared key, located in psk.txt. + + + In psk.txt, we now setup two entries, which do differ on both hosts. +On 10.0.0.11: + +10.0.0.216 password2 + +On 10.0.0.216: + +10.0.0.11 password2 + + Make sure these files are owned by root, and set to mode 0600, +racoon will not trust their contents otherwise. Note that these files are +mirrors from eachother. + + + Now we are ready to setup our desired policy, which is simple +enough. On host 10.0.0.216: + +#!/sbin/setkey -f +flush; +spdflush; + +spdadd 10.0.0.216 10.0.0.11 any -P out ipsec + esp/transport//require; + +spdadd 10.0.0.11 10.0.0.216 any -P in ipsec + esp/transport//require; + +And on 10.0.0.11: + +#!/sbin/setkey -f +flush; +spdflush; + +spdadd 10.0.0.11 10.0.0.216 any -P out ipsec + esp/transport//require; + +spdadd 10.0.0.216 10.0.0.11 any -P in ipsec + esp/transport//require; + +Note how again these policies are mirrored. + + + We are now ready to launch racoon! Once launched, the moment we try +to telnet from 10.0.0.11 to 10.0.0.216, or the other way around, racoon +will start negotiating: + +12:18:44: INFO: isakmp.c:1689:isakmp_post_acquire(): IPsec-SA + request for 10.0.0.11 queued due to no phase1 found. +12:18:44: INFO: isakmp.c:794:isakmp_ph1begin_i(): initiate new + phase 1 negotiation: 10.0.0.216[500]<=>10.0.0.11[500] +12:18:44: INFO: isakmp.c:799:isakmp_ph1begin_i(): begin Aggressive mode. +12:18:44: INFO: vendorid.c:128:check_vendorid(): received Vendor ID: + KAME/racoon +12:18:44: NOTIFY: oakley.c:2037:oakley_skeyid(): couldn't find + the proper pskey, try to get one by the peer's address. +12:18:44: INFO: isakmp.c:2417:log_ph1established(): ISAKMP-SA + established 10.0.0.216[500]-10.0.0.11[500] spi:044d25dede78a4d1:ff01e5b4804f0680 +12:18:45: INFO: isakmp.c:938:isakmp_ph2begin_i(): initiate new phase 2 + negotiation: 10.0.0.216[0]<=>10.0.0.11[0] +12:18:45: INFO: pfkey.c:1106:pk_recvupdate(): IPsec-SA established: + ESP/Transport 10.0.0.11->10.0.0.216 spi=44556347(0x2a7e03b) +12:18:45: INFO: pfkey.c:1318:pk_recvadd(): IPsec-SA established: + ESP/Transport 10.0.0.216->10.0.0.11 spi=15863890(0xf21052) + + + + If we now run setkey -D, which shows the Security Associations, they +are indeed there: + +10.0.0.216 10.0.0.11 + esp mode=transport spi=224162611(0x0d5c7333) reqid=0(0x00000000) + E: 3des-cbc 5d421c1b d33b2a9f 4e9055e3 857db9fc 211d9c95 ebaead04 + A: hmac-sha1 c5537d66 f3c5d869 bd736ae2 08d22133 27f7aa99 + seq=0x00000000 replay=4 flags=0x00000000 state=mature + created: Nov 11 12:28:45 2002 current: Nov 11 12:29:16 2002 + diff: 31(s) hard: 600(s) soft: 480(s) + last: Nov 11 12:29:12 2002 hard: 0(s) soft: 0(s) + current: 304(bytes) hard: 0(bytes) soft: 0(bytes) + allocated: 3 hard: 0 soft: 0 + sadb_seq=1 pid=17112 refcnt=0 +10.0.0.11 10.0.0.216 + esp mode=transport spi=165123736(0x09d79698) reqid=0(0x00000000) + E: 3des-cbc d7af8466 acd4f14c 872c5443 ec45a719 d4b3fde1 8d239d6a + A: hmac-sha1 41ccc388 4568ac49 19e4e024 628e240c 141ffe2f + seq=0x00000000 replay=4 flags=0x00000000 state=mature + created: Nov 11 12:28:45 2002 current: Nov 11 12:29:16 2002 + diff: 31(s) hard: 600(s) soft: 480(s) + last: hard: 0(s) soft: 0(s) + current: 231(bytes) hard: 0(bytes) soft: 0(bytes) + allocated: 2 hard: 0 soft: 0 + sadb_seq=0 pid=17112 refcnt=0 + +As are the Security Policies we configured ourselves: + +10.0.0.11[any] 10.0.0.216[any] tcp + in ipsec + esp/transport//require + created:Nov 11 12:28:28 2002 lastused:Nov 11 12:29:12 2002 + lifetime:0(s) validtime:0(s) + spid=3616 seq=5 pid=17134 + refcnt=3 +10.0.0.216[any] 10.0.0.11[any] tcp + out ipsec + esp/transport//require + created:Nov 11 12:28:28 2002 lastused:Nov 11 12:28:44 2002 + lifetime:0(s) validtime:0(s) + spid=3609 seq=4 pid=17134 + refcnt=3 + + Problems and known defects + + If this does not work, check that all configuration files +are owned by root, and can only be read by root. To start racoon on the +foreground, use '-F'. To force it to read a certain configuration file, +instead of at the compiled location, use '-f'. For staggering amounts of +detail, add a 'log debug;' statement to racoon.conf. + + + Automatic keying using X.509 certificates + + As mentioned before, the use of shared secrets is hard because they +aren't easily shared and once shared, are no longer secret. Luckily, there +is asymmetric encryption technology to help resolve this. + + + If each IPSEC participant makes a public and a private key, secure +communications can be setup by both parties publishing their public key, and +configuring policy. + + + Building a key is relatively easy, although it requires some work. +The following is based on the 'openssl' tool. + + Building an X.509 certificate for your host + + OpenSSL has a lot of infrastructure for keys that may or may not be +signed by certificate authorities. Right now, we need to circumvent all that +infrastructure and practice some good old Snake Oil security, and do without +a certificate authority. + + + First we issue a 'certificate request' for our host, called +'laptop': + +$ openssl req -new -nodes -newkey rsa:1024 -sha1 -keyform PEM -keyout \ + laptop.private -outform PEM -out request.pem + +This asks us some questions: + +Country Name (2 letter code) [AU]:NL +State or Province Name (full name) [Some-State]:. +Locality Name (eg, city) []:Delft +Organization Name (eg, company) [Internet Widgits Pty Ltd]:Linux Advanced +Routing & Traffic Control +Organizational Unit Name (eg, section) []:laptop +Common Name (eg, YOUR name) []:bert hubert +Email Address []:ahu@ds9a.nl + +Please enter the following 'extra' attributes +to be sent with your certificate request +A challenge password []: +An optional company name []: + + It is left to your own discretion how completely you want to fill +this out. You may or may not want to put your hostname in there, depending +on your security needs. In this example, we have. + + + We'll now 'self sign' this request: + +$ openssl x509 -req -in request.pem -signkey laptop.private -out \ + laptop.public +Signature ok +subject=/C=NL/L=Delft/O=Linux Advanced Routing & Traffic \ + Control/OU=laptop/CN=bert hubert/Email=ahu@ds9a.nl +Getting Private key + + The 'request.pem' file can now be discarded. + + + Repeat this procedure for all hosts you need a key for. You can +distribute the '.public' file with impunity, but keep the '.private' one +private! + + + Setting up and launching + + Once we have a public and a private key for our hosts we can tell + racoon to use them. + + + We return to our previous configuration and the two hosts, 10.0.0.11 +('upstairs') and 10.0.0.216 ('laptop'). + + + To the racoon.conf file on 10.0.0.11, we add: + +path certificate "/usr/local/etc/racoon/certs"; + +remote 10.0.0.216 +{ + exchange_mode aggressive,main; + my_identifier asn1dn; + peers_identifier asn1dn; + + certificate_type x509 "upstairs.public" "upstairs.private"; + + peers_certfile "laptop.public"; + proposal { + encryption_algorithm 3des; + hash_algorithm sha1; + authentication_method rsasig; + dh_group 2 ; + } +} + + This tells racoon that certificates are to be found in +/usr/local/etc/racoon/certs/. Furthermore, it contains +configuration items specific for remote 10.0.0.216. + + + The 'asn1dn' lines tell racoon that the identifier for both the +local and remote ends are to be extracted from the public keys. This is the +'subject=/C=NL/L=Delft/O=Linux Advanced Routing & Traffic + Control/OU=laptop/CN=bert hubert/Email=ahu@ds9a.nl' output from above. + + + The certificate_type line configures the local +public and private key. The peers_certfile statement +configures racoon to read the public key of the remote peer from the file +laptop.public. + + + The proposal stanza is unchanged from what we've +seen earlier, with the exception that the +authentication_method is now rsasig, +indicating the use of RSA public/private keys for authentication. + + + The addition to the configuration of 10.0.0.216 is nearly identical, except for the +usual mirroring: + +path certificate "/usr/local/etc/racoon/certs"; + +remote 10.0.0.11 +{ + exchange_mode aggressive,main; + my_identifier asn1dn; + peers_identifier asn1dn; + + certificate_type x509 "laptop.public" "laptop.private"; + + peers_certfile "upstairs.public"; + + proposal { + encryption_algorithm 3des; + hash_algorithm sha1; + authentication_method rsasig; + dh_group 2 ; + } +} + + + + Now that we've added these statements to both hosts, we only need to +move the key files in place. The 'upstairs' machine needs +upstairs.private, upstairs.public, +and laptop.public in +/usr/local/etc/racoon/certs. Make sure that this +directory is owned by root and has mode 0600 or racoon may refuse to read +it! + + +The 'laptop' machine needs +laptop.private, laptop.public, +and upstairs.public in +/usr/local/etc/racoon/certs. In other words, each host +needs its own public and private key and additionally, the public key of the +remote. + + + Verify that a Security Policy is in place (execute the 'spdadd' lines in +). Then launch racoon and everything should +work. + + How to setup tunnels securely + + To setup secure communications with a remote party, we must exchange +public keys. While the public key does not need to be kept a secret, on the +contrary, it is very important to be sure that it is in fact the unaltered +key. In other words, you need to be certain there is no 'man in the middle'. + + +To make this easy, OpenSSL provides the 'digest' command: + +$ openssl dgst upstairs.public +MD5(upstairs.public)= 78a3bddafb4d681c1ca8ed4d23da4ff1 + + + + Now all we need to do is verify if our remote partner sees the same +digest. This might be done by meeting in real life or perhaps over the +phone, making sure the number of the remote party was not in fact sent over +the same email containing the key! + + + Another way of doing this is the use of a Trusted Third Party which +runs a Certificate Authority. This CA would then sign your key, which we've +done ourselves above. + + + + + IPSEC tunnels @@ -4328,13 +4764,6 @@ as well: -FIXME: does this need something like 'nobroadcast'? A /31 is too small to -house a network address and a broadcast address - if this doesn't work as -planned, try a /30, and adjust the ip addresses accordingly. You might even -try to make eth1 and eth2 do without an IP address! - - - On router A: @@ -4381,8 +4810,8 @@ packets destined for ip addresses other than their own: -# echo 0 > /proc/net/ipv4/conf/eth1/rp_filter -# echo 0 > /proc/net/ipv4/conf/eth2/rp_filter +# echo 0 > /proc/sys/net/ipv4/conf/eth1/rp_filter +# echo 0 > /proc/sys/net/ipv4/conf/eth2/rp_filter @@ -8253,8 +8682,259 @@ If the last two lines give an error, update your tc tool to a newer version! for details on what this does. + + Example of a full nat solution with QoS + + I'm Pedro Larroy
piotr@omega.resa.es
Here I'm describing a common set up where we have lots of users in a private network connected to the Internet trough a Linux router with a public ip address that is doing network address translation (NAT). I use this QoS setup to give access to the Internet to 198 users in a university dorm, in which I live and I'm netadmin of. The users here do heavy use of peer to peer programs, so proper traffic control is a must. I hope this serves as a practical example for all interested lartc readers. + + + + At first I make a practical approach with step by step configuration, and in the end I explain how to make the process automatic at bootime. The network to which this example applies is a private LAN connected to the Internet through a Linux router which has one public ip address. Extending it to several public ip address should be very easy, a couple of iptables rules should be added. + In order to get things working we need: + + + Linux 2.4.18 or higher kernel version installed + + + If you use 2.4.18 you will have to apply HTB patch available here. + + + + + iproute + + + Also ensure the "tc" binary is HTB ready, a precompiled binary is distributed with HTB. + + + + + iptables + + + + + + + + + + + Let's begin optimizing that scarce bandwidth + + First we set up some qdiscs in which we will classify the traffic. We create a htb qdisc with 6 classes with ascending priority. Then we have classes that will always get allocated rate, but can use the unused bandwidth that other classes don't need. Recall that classes with higher priority ( i.e with a lower prio number ) will get excess of bandwith allocated first. Our connection is 2Mb down 300kbits/s up Adsl. I use 240kbit/s as ceil rate just because it's the higher I can set it before latency starts to grow, due to buffer filling in whatever place between us and remote hosts. This parameter should be timed experimentally, raising it and lowering while observing latency between some near hosts. + + + Adjust CEIL to 75% of your upstream bandwith limit by now, and where I use eth0, you should use the interface which has a public Internet address. To begin our example execute the following in a root shell: + + CEIL=240 + tc qdisc add dev eth0 root handle 1: htb default 15 + tc class add dev eth0 parent 1: classid 1:1 htb rate ${CEIL}kbit ceil ${CEIL}kbit + tc class add dev eth0 parent 1:1 classid 1:10 htb rate 80kbit ceil 80kbit prio 0 + tc class add dev eth0 parent 1:1 classid 1:11 htb rate 80kbit ceil ${CEIL}kbit prio 1 + tc class add dev eth0 parent 1:1 classid 1:12 htb rate 20kbit ceil ${CEIL}kbit prio 2 + tc class add dev eth0 parent 1:1 classid 1:13 htb rate 20kbit ceil ${CEIL}kbit prio 2 + tc class add dev eth0 parent 1:1 classid 1:14 htb rate 10kbit ceil ${CEIL}kbit prio 3 + tc class add dev eth0 parent 1:1 classid 1:15 htb rate 30kbit ceil ${CEIL}kbit prio 3 + tc qdisc add dev eth0 parent 1:12 handle 120: sfq perturb 10 + tc qdisc add dev eth0 parent 1:13 handle 130: sfq perturb 10 + tc qdisc add dev eth0 parent 1:14 handle 140: sfq perturb 10 + tc qdisc add dev eth0 parent 1:15 handle 150: sfq perturb 10 + + We have just created a htb tree with one level depth. Something like this: + + + +---------+ + | root 1: | + +---------+ + | + +---------------------------------------+ + | class 1:1 | + +---------------------------------------+ + | | | | | | + +----+ +----+ +----+ +----+ +----+ +----+ + |1:10| |1:11| |1:12| |1:13| |1:14| |1:15| + +----+ +----+ +----+ +----+ +----+ +----+ + + + + + classid 1:10 htb rate 80kbit ceil 80kbit prio 0 + + + This is the higher priority class. The packets in this class will have the lowest delay and would get the excess of bandwith first so it's a good idea to limit the ceil rate to this class. We will send through this class the following packets that benefit from low delay, such as interactive traffic: ssh, telnet, dns, quake3, irc, and packets with the SYN flag. + + + + + + classid 1:11 htb rate 80kbit ceil ${CEIL}kbit prio 1 + + + Here we have the first class in which we can start to put bulk traffic. In my example I have traffic from the local web server and requests for web pages: source port 80, and destination port 80 respectively. + + + + + classid 1:12 htb rate 20kbit ceil ${CEIL}kbit prio 2 + + + In this class I will put traffic with Maximize-Throughput TOS bit set and the rest of the traffic that goes from local processes on the router to the Internet. So the following classes will only have traffic that is routed through the box. + + + + + classid 1:13 htb rate 20kbit ceil ${CEIL}kbit prio 2 + + + This class is for the traffic of other NATed machines that need higher priority in their bulk traffic. + + + + + + classid 1:14 htb rate 10kbit ceil ${CEIL}kbit prio 3 + + + Here goes mail traffic (SMTP,pop3...) and packets with Minimize-Cost TOS bit set. + + + + + classid 1:15 htb rate 30kbit ceil ${CEIL}kbit prio 3 + + + And finally here we have bulk traffic from the NATed machines behind the router. All kazaa, edonkey, and others will go here, in order to not interfere with other services. + + + + + + + + + + + + + Classifying packets + + We have created the qdisc setup but no packet classification has been made, so now all outgoing packets are going out in class 1:15 ( because we used: tc qdisc add dev eth0 root handle 1: htb default 15 ). Now we need to tell which packets go where. This is the most important part. + + + + Now we set the filters so we can classify the packets with iptables. I really prefer to do it with iptables, because they are very flexible and you have packet count for each rule. Also with the RETURN target packets don't need to traverse all rules. We execute the following commands: + + tc filter add dev eth0 parent 1:0 protocol ip prio 1 handle 1 fw classid 1:10 + tc filter add dev eth0 parent 1:0 protocol ip prio 2 handle 2 fw classid 1:11 + tc filter add dev eth0 parent 1:0 protocol ip prio 3 handle 3 fw classid 1:12 + tc filter add dev eth0 parent 1:0 protocol ip prio 4 handle 4 fw classid 1:13 + tc filter add dev eth0 parent 1:0 protocol ip prio 5 handle 5 fw classid 1:14 + tc filter add dev eth0 parent 1:0 protocol ip prio 6 handle 6 fw classid 1:15 + + We have just told the kernel that packets that has a specific FWMARK value ( hanlde x fw ) go in the specified class ( classid x:x). Next you will see how to mark packets with iptables. + + + + First you have to understand how packet traverse the filters with iptables: + + +------------+ +---------+ +-------------+ + Packet -| PREROUTING |--- routing-----| FORWARD |-------+-------| POSTROUTING |- Packets + input +------------+ decision +-­-------+ | +-------------+ out + | | + +-------+ +--------+ + | INPUT |---- Local process -| OUTPUT | + +-------+ +--------+ + + + I assume you have all your tables creak and with default policy ACCEPT ( -P ACCEPT ) if you haven't poked with iptables yet, It should be ok by default. Ours private network is a class B with address 172.17.0.0/16 and public ip is 212.170.21.172 + + + + Next we instruct the kernel to actually do NAT, so clients in the private network can start talking to the outside. + + echo 1 > /proc/sys/net/ipv4/ip_forward + iptables -t nat -A POSTROUTING -s 172.17.0.0/255.255.0.0 -o eth0 -j SNAT --to-source 212.170.21.172 + + Now check that packets are flowing through 1:15: + + tc -s class show dev eth0 + + + + + You can start marking packets adding rules to the PREROUTING chain in the mangle table. + + iptables -t mangle -A PREROUTING -p icmp -j MARK --set-mark 0x1 + iptables -t mangle -A PREROUTING -p icmp -j RETURN + + Now you should be able to see packet count increasing when pinging from machines within the private network to some site on the Internet. Check packet count increasing in 1:10 + + tc -s class show dev eth0 + + We have done a -j RETURN so packets don't traverse all rules. Icmp packets won't match other rules below RETURN. Keep that in mind. + Now we can start adding more rules, lets do proper TOS handling: + + iptables -t mangle -A PREROUTING -m tos --tos Minimize-Delay -j MARK --set-mark 0x1 + iptables -t mangle -A PREROUTING -m tos --tos Minimize-Delay -j RETURN + iptables -t mangle -A PREROUTING -m tos --tos Minimize-Cost -j MARK --set-mark 0x5 + iptables -t mangle -A PREROUTING -m tos --tos Minimize-Cost -j RETURN + iptables -t mangle -A PREROUTING -m tos --tos Maximize-Throughput -j MARK --set-mark 0x6 + iptables -t mangle -A PREROUTING -m tos --tos Maximize-Throughput -j RETURN + + + Now prioritize ssh packets: + + iptables -t mangle -A PREROUTING -p tcp -m tcp --sport 22 -j MARK --set-mark 0x1 + iptables -t mangle -A PREROUTING -p tcp -m tcp --sport 22 -j RETURN + + A good idea is to prioritize packets to begin tcp connections, those with SYN flag set: + + iptables -t mangle -I PREROUTING -p tcp -m tcp --tcp-flags SYN,RST,ACK SYN -j MARK --set-mark 0x1 + iptables -t mangle -I PREROUTING -p tcp -m tcp --tcp-flags SYN,RST,ACK SYN -j RETURN + + And so on. + + When we are done adding rules to PREROUTING in mangle, we terminate the PREROUTING table with: + + iptables -t mangle -A PREROUTING -j MARK --set-mark 0x6 + + So previously unmarked traffic goes in 1:15. In fact this last step is unnecessary since default class was 1:15, but I will mark them in order to be consistent with the whole setup, and furthermore it's useful to see the counter in that rule. + + + + It will be a good idea to do the same in the OUTPUT rule, so repeat those commands with -A OUTPUT instead of PREROUTING. ( s/PREROUTING/OUTPUT/ ). Then traffic generated locally (on the Linux router) will also be classified. I finish OUTPUT chain with -j MARK --set-mark 0x3 so local traffic has higher priority. + + + + + + Improving our setup + + Now we have all our setup working. Take time looking at the graphs, and watching where your bandwith is spent and how do you want it. Doing that for lots of hours, I finally got the Internet connection working really well. Otherwise continuous timeouts and nearly zero allotment of bandwith to newly created tcp connections will occur. + + + If you find that some classes are full most of the time it would be a good idea to attach another queueing discipline to them so bandwith sharing is more fair: + + tc qdisc add dev eth0 parent 1:13 handle 130: sfq perturb 10 + tc qdisc add dev eth0 parent 1:14 handle 140: sfq perturb 10 + tc qdisc add dev eth0 parent 1:15 handle 150: sfq perturb 10 + + + + + Making all of the above start at boot + + It sure can be done in many ways. In mine, I have a shell script in /etc/init.d/packetfilter that accepts [start | stop | stop-tables | start-tables | reload-tables] it configures qdiscs and loads needed kernel modules, so it behaves much like a daemon. The same script loads iptables rules from /etc/network/iptables-rules. I will beautify it a little and will make it available on my web page here + + + + + + + Building bridges, and pseudo-bridges with Proxy ARP @@ -9025,6 +9705,22 @@ helping. + + + HannesEbner + +
he%fli4l.de
+ + + + + + DerekFawcus + +
dfawcus%cisco.com
+ + + @@ -9194,14 +9890,19 @@ Patrick McHardy <kaber@trash.net> Andreas Mohr <andi%lisas.de> - + +James Morris <jmorris@intercode.com.au> + + + + -Andrew Morton <akpm@zip.com.au> +Andrew Morton <akpm%zip.com.au> - + Wim van der Most @@ -9220,6 +9921,7 @@ Togan Muftuoglu <toganm%yahoo.com> + Chris Murray <cmurray@stargate.ca> @@ -9366,6 +10068,14 @@ Song Wang <wsong@ece.uci.edu> + ChrisWilson +
chris@netservers.co.uk
+ + + + + + LazarYanackiev
Lyanackiev%gmx.net
 -- 2.11.4.GIT