TCPLAG

Here is my brief homepage for the TCPLAG product.

What Does it Do?

In simple terms, TCPLAG monitors the round-trip time of TCP packets and sends the output to syslog. Big times mean you network is lagged and interactive applications will be difficult to use.

Why Did I Write This?

Initially for two reasons, most importantly because I realised that it was possible to easily monitor network response times with a completely passive system and I further realised that no one in the Linux world was doing this. Having a background in MUD gaming, I am well aware of what lag can do to make interactive applications difficult to use so the benefits of such a system seemed obvious.

But What Use is It?

Not everyone uses the Internet for browsing the "World Wide Web". Some people use interactive applications that involve making a connection to a remote machine and keeping that connection open. The most common are interactive text applications, for example a remote Linux shell (probably bash) or maybe a text editor. If you press a key, you expect to see a response, but in reality there is some delay as a network packet goes from your machine, to the remote machine, through the application, and a reply is sent back to your screen.

Humans have a reaction time of approximately 1/3 of a second. If the round-trip of the packet is less than 1/3 of a second then the interactive application "feels" like it is running in real time. With a round trip of 1 second it feels a bit sluggish but you get used to it and you find it workable. Anything longer than 2 seconds gets pretty nasty.

As bad luck would have it, the packet-switched nature of the Internet is not a good design for efficiently handling low-latency, real-time event data. The problem is that there is no guarantee of delivery time nor any planned time schedule for data delivery. Switches and routers contain queues which level out local congestion at a cost of delaying packet delivery. This system works well for bulk data transfer but for interactive applications it means that you get random delays depending on what else might be happening on the network. Because of the random nature of traffic congestion there will always be some instances where the delays get annoyingly long.

Most Internet service providers concentrate on making sure their links are reliable, their servers are secure and don't crash too often and that their customers pay their bills. They often look at ping times to estimate network congestion and they look at average traffic volume to estimate their link capacity and utilisation. None of these things are very useful measurements for interactive applications. Generally, interactive text applications are low bandwidth so link capacity is not a big issue, however, because of the design limitations of TCP/IP you probably need a link that is of much larger capacity that what you actually use -- otherwise you get lagged.

Here I bring you a tool that will monitor and log the actual interactive delay experienced by the user at the keyboard. It gives you the ability to measure which ISPs provide the best service for interactive users, helps you track down where and when the congestion occurs and give you hard evidence to prove that your link isn't delivering what you expect.

License

It's GPL, it heavily depends on the iptables package and the Linux kernel both of which are also GPL.

Installation

Probably the easiest way is to compile the SRPM file on your RedHat machine. You will need to have the redhat kernel-source package installed and the source code of iptables unpacked in the /usr/src directory because I use some header files. I try to match my version to the currently installed iptables and I presume that is the same as the unpacked source code. If you have that under control then use a command like:
   rpmbuild --rebuild tcplag-2004.01.20-1.src.rpm
Then install the resulting binary with:
   rpm --install tcplag-2004.01.20-1.i386.rpm

If you have trouble building the package then you can use my already compiled binary module but your version of iptables may be different or your kernel may be incompatible. I build three different kernel modules to cover the three most common redhat kernels but still many other systems will not be compatible with my binary.

If you don't use RedHat then the plain source is available in the zip file. The Makefile may need adjusting for your system and you still need iptables source and a kernel source tree that matches your kernel (preferably the one that built your kernel).

Once installed you still need one more step which is:

   depmod -a -e
If no errors come up then you are probably in business. To get it running now that you have it installed you want to put it into your iptables firewall with a comand like:
   iptables --insert INPUT --jump TCPLAG
   iptables --insert OUTPUT --jump TCPLAG
   iptables --insert FORWARD --jump TCPLAG
Suddenly you get log messages at the console for how lagged your connections are at the moment. You can use something like --log-level=5 to make it not come up on the console and you can play around with the log prefix. It should be compatible with other firewall scripts and with:
   service iptables save
You can use all the normal matching constraints if you want to only do lag monitoring on particular networks or particular ports (for example you may only wish to measure lag on port 22 and port 23 and you may have deliberately put throttles on other traffic such as port 25). The TCPLAG target is passive with respect to the packets so it will never influence the behaviour of the firewall (other than use some CPU and memory). In addition, TCPLAG harmlessly ignores non-TCP packets so you can safely put it right at the top of the chains if you just want to track as much as possible.

You could put it into the PREROUTING or POSTROUTING chains if you wanted to instead of putting it into the normal chains, I haven't tested this but in theory it should do the same thing but show the externally-visible IP numbers. If you are putting it onto a masquerading firewall or other packet-mangling device then results obtained in the FORWARD chain will be the most useful.

The current implementation of TCPLAG pretty much ignores ports but because Linux uses random SEQ starting values the probability of two streams getting confused is very small and will have a minimal impact on your network statistics (on average, one packet in two billion will get wrongly measured and if you put two billion packets through your network on a regular basis then give me a call ;-).

Download

Available versions are:

Warning and Disclaimer

This is kernel-level mojo. Feel free to inspect the source, do plenty of testing by all means but do NOT just throw it straight onto an important production machine without taking plenty of backups. The crash potential for a bug in a kernel module is pretty huge and I haven't tested this nearly as much as I would like to. If you don't really understand what is going on here and you think this software might be useful then either hire someone who knows what they are doing or do a lot of reading and then come back here or get a crappy test machine and practice on that.

    This program is distributed in the hope that it will be useful,
    but WITHOUT ANY WARRANTY; without even the implied warranty of
    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
    GNU General Public License for more details.

Linux Bespoke Homepage