QoS Connection Tuning HOWTO <author>Emmanuel Roger, <tt>fly4@prout.be</tt> <date>$Revision: 0.61 $ $Date: 2002/04/03 22:26:52 $ <abstract> This quick howto describes how to enhance your connection speed and response time with QoS, iptables and iproute2. </abstract>  <toc>  <sect>Introduction This paper was written because I wanted to listen to shoutcast mp3 streams without being cut and wanted to have a good response time with ssh even when uploading lots of data via ftp or dcc. Basically it explains how to setup 2 classes of traffic for your upstream to get a more "interactive" connection and why to do it this way. (C) 2001 Emmanuel Roger. Licensed under the GNU Free Documentation License. <sect1>The problem: When you upload data on a slow link, a long queue of packets may form in your local buffer. Thus, if you happen to be uploading a file with ftp, any telnet, chat, or ack packets may get caught behind the queued ftp packets and will be delayed. Telnet and chat will become sluggish, or unusable due to long queing delay. Delaying acks may adversely affect downstream services. A better Quality of Experience can be achieved by expediting (giving priority to) certain types of upstream traffic for services that require a high level of interactivity. <sect1> A solution: Much of the Internet is still based on 'best effort' delivery in which all packets are treated roughly equally. But human requirements for services delivered over the Internet, don't work that way: A few ftp packets delayed for a second or two will most likely have no impact on the user. A telnet packet delayed as little as 400 milliseconds can result in a typing error by the user. Therefore, giving priority to packets whose delay _will_ directly be experienced by the user at the expense of those packets whose delay will be invisible to the user is a good strategy to improve the Quality of Experience for the user. A similar argument can be made in the downstream direction (e.g., don't starve real audio for the sake of email downloads which can tolerate more delay than real audio, telnet, etc.). However, we users have little control in that direction. In a perfect world, the OS would detect which traffic needs higher output priority. But we aren't in a perfect world. So we just divide the traffic into two classes: <descrip> <tag/Interactive or real-time/ This is the kind of traffic which will be given the higher priority, but the less bandwidth. It is composed of small packets ( tcp ack, irc, ssh, telnet, http requests etc...) <tag/Data/ This kind of traffic is given the lower priority, but most of the bandwidth. It is composed of large packets (ftp data, http responses, email with large attachments, pr0n ...) </descrip> In opposition to traffic classification by port, ip, or tcp flags, classification by size of packets doesn't need to be redefined every time you add a service to your machine or network and suits well an adsl or cable modem connection. There are better ways to class different kinds of traffic, but this is easy and seems to work well for the average user. <sect>Requirements Some software is needed to perform this enhancement: <itemize> <item><url url="http://www.kernel.org" name="Kernel"> You will need the sources of a 2.4 kernel, the latest ;) <item><url url="http://netfilter.samba.org" name="iptables"> You need the source to get the <tt>length</tt> match to mark packets so the kernel knows in which class to put them. <item><url url="ftp://ftp.inr.ac.ru/ip-routing/iproute2-current.tar.gz" name="iproute2"> This tool allows you to control the advanced routing and queueing disciplines of the linux kernel. </itemize> <sect>Compiling and installing I won't explain compilation process in details, just read the README's and various HOWTO's. Untar every package, first compile iproute2+tc. Go into the iptables source directory and "make patch-o-matic". You will be prompted with a list of kernel patches, the one we are interested in is the "length match". Note that since kernel version 2.4.16 you don't need this step, the length match is included in the official kernel. After patching, configure your kernel, including: <tscreen><verb> Networking options ---> Kernel/User netlink socket Network packet filtering (replaces ipchains) IP: Netfilter Configuration ---> IP tables support (required for filtering/masq/NAT) LENGTH match support MARK target support QoS and/or fair queueing ---> select everything here. </verb></tscreen> Either builtin or as modules. Compile and install as usual. Next compile iptables and install it too. reboot. <sect>Configure This configuration example is for an adsl link with 1024 kbits downstream and 512 Kbits upstream. Perhaps you will need fine tuning depending on your connection speed or bandwidth needs. We will match the packets by their size and mark them acordingly. A packet of size 0 -> 500 will be considered interactive traffic and marked with fwmark value "3" <tscreen><verb> iptables -t mangle -A OUTPUT -m length --length 0:500 -j MARK --set-mark 3 </verb></tscreen> Next packets of size 500 -> 1500 will be marked as data traffic with value "4". <tscreen><verb> iptables -t mangle -A OUTPUT -m length --length 500:1500 -j MARK --set-mark 4 </verb></tscreen> If you want to also apply these settings to masqueraded machines, you should use: <tscreen><verb> iptables -t mangle -A PREROUTING -m length --length 0:500 -j MARK --set-mark 3 </verb></tscreen> <tscreen><verb> iptables -t mangle -A PREROUTING -m length --length 500:1500 -j MARK --set-mark 4 </verb></tscreen> We will create a root cbq qdisc for our device ppp0: <tscreen><verb> tc qdisc add dev ppp0 root handle 10: cbq bandwidth 10Mbit avpkt 1000 mpu 64 </tscreen></verb> This creates a root queuing discipline with handle 10: for the device ppp0 of type cbq, with maximum bandwidth of 10Mbits (pppoe), average packet size will be of 1000 bytes and minimum packet size of 64 (ethernet). The 2 classes: <itemize> <item>first one will get low initial bandwidth but the higher priority and ability to steal bandwidth from the second. <item>second one will limit the rate of the outgoing data to 461 kbits with the less priority. </itemize> As a general rule for a workstation connected to internet you would calculate the rate values of the two classes like this: <tscreen><verb> Download bandwidth: 1024 kbits Upload bandwidth: 512 kbits Interactive class rate = Download bandwidth / 20 Data class rate = Upload bandwidth - Interactive class rate </verb></tscreen> So 51 and 461. These values are working well here, perhaps you'll need to fine tune to suit your needs, just make sure to have enough bandwidth in the interactive class to acknowledge all incoming tcp data. <tscreen><verb> tc class add dev ppp0 parent 10:0 classid 10:1 cbq bandwidth 10Mbit \ rate 51Kbit allot 1514 prio 1 maxburst 10 avpkt 100 isolated tc class add dev ppp0 parent 10:0 classid 10:2 cbq bandwidth 10Mbit \ rate 461Kbit allot 1514 prio 8 maxburst 2 avpkt 1500 bounded </verb></tscreen> At least we need to tell the kernel which class we want to send packets to: <tscreen><verb> tc filter add dev ppp0 parent 10:0 protocol ip handle 3 fw flowid 10:1 tc filter add dev ppp0 parent 10:0 protocol ip handle 4 fw flowid 10:2 </verb></tscreen> Well that's done. <sect>Testing Testing was realised easily: uploading 2 files on a host with more bandwidth, listening to two mp3 streams simultaneously, and downloading latest kernel from local mirror. ping time never gets more than 200ms, ssh is interactive ( ever tried to type with a 2 sec ping ? ) and online gaming works well too. <sect>Example script <tscreen><verb> #!/bin/bash #Example config for an adsl link with 1024Kbit downstream and 128Kbit upstream iptables -t mangle -A OUTPUT -m length --length 0:500 -j MARK --set-mark 3 iptables -t mangle -A OUTPUT -m length --length 500:1500 -j MARK --set-mark 4 #We assume masqueraded machines on eth0 should also have their upload tuned iptables -t mangle -A PREROUTING -i eth0 -m length --length 0:500 -j MARK --set-mark 3 iptables -t mangle -A PREROUTING -i eth0 -m length --length 500:1500 -j MARK --set-mark 4 tc qdisc add dev ppp0 root handle 10: cbq bandwidth 10Mbit avpkt 1000 mpu 64 tc class add dev ppp0 parent 10:0 classid 10:1 cbq bandwidth 10Mbit \ rate 51Kbit allot 1514 prio 1 maxburst 10 avpkt 100 isolated tc class add dev ppp0 parent 10:0 classid 10:2 cbq bandwidth 10Mbit \ rate 77Kbit allot 1514 prio 8 maxburst 2 avpkt 1500 bounded tc filter add dev ppp0 parent 10:0 protocol ip handle 3 fw flowid 10:1 tc filter add dev ppp0 parent 10:0 protocol ip handle 4 fw flowid 10:2 </verb></tscreen> <sect>Further information <url url="http://www.ds9a.nl/2.4Routing/" name="Linux 2.4 Advanced Routing & Traffic Control HOWTO"> Sally Floyd <url url="http://www.aciri.org/floyd/cbq.html" name="CBQ"> papers. <url url="http://diffserv.sf.net" name="Differentiated Services on Linux">. <sect>Changes <itemize> <item>tip for masqueraded machines <item>again some typos and language faults. <item>add example script. <item>corrected formula. <item>fixed some typos. <item>added a generic formula to calculate classes rates. <item>explanation of the qdisc line. <item>patching not needed with 2.4.16. <item>added Changelog ;) <item>correction of kernel options. <item>added diffserv url. <item>license changed to FDL. </itemize> <sect>Thanks Thanks first to the netfilter and kernel teams for providing great software. Thanks to Ben Bauer from Nortel Networks for providing corrections. Greats also to everyone at <url url="http://www.adsl-bc.org" name="adsl-bc.org"> who provided feedback. And to everyone on <url url="http://www.undernet.org" name="undernet"> #linuxbe and #linuxfr channels. Emmanuel Roger </article>