TCPKali Part II

I've spent about a week testing with this tcpkali tool now.

This is the test case I initially put together, which is to test connections between two CentOS 7 VMs via an Ubuntu VM (IP Forwarder / Router).

The tool spawns worker threads that test connection establishment and bandwidth on each of the connections, both based on an individual thread and collectively.

One thing I did run into with using Lighttpd, is that at high volume levels I kept getting invalid request headers. I wasn't sure if this was tcpkali messing up the headers, or if it was Lighttpd.

So I switched the architecture a bit (based on the suggestion of one of the other engineers in here). Instead of using Lighttd as the web server, I put tcpkali on both sides (left and right) - in client and server mode respectively.

This worked much better than using Lighttpd as a web server. You can send messages and run the server in active or silent mode, returning responses or discarding them.

The tcpkali has some great features. One HUGE feature is the ability to send requests per second with the -r option. The tool attempts to ramp up to the specified number of connections, and if it cannot do this, it will bail without examining throughput measurements. I did notice that it sends data with the -r option DURING the ramp-up period, in addition to after. The tool also has other features like the connect rate, so that if you are establishing 100K connections you can speed up that process by doing 10K or 20K a clip rather than smaller default limits.

What I discovered in the testing I was doing, was that the connection rate had a profound effect on the number of connections you could establish. The default connection rate is 100. And this works fine for up to about 10,000 connections. But if you try to go to higher than that, the rate starts to inhibit your ability to scale the connections and you need to tweak it higher - just not too much higher.
The sweet spot I discovered was a 1,000 : 1 ratio. So 10,000 connections would use a rate of 100 connections per second, 20,000 would use a rate of 200 connections per second, and so forth.