Stress Testing with Siege and Bombard, know your limits

I couldn't find a clear quick intro on getting siege and bombard in action, so I'm writing one here.

Siege is a load testing and benchmarking utility that has been available for quite a while, It allows you to hit your web application on a specific url (or a set of urls in a file) with specific concurrency and size settings.  siege summarizes the measures of the test outcome including:

• Transaction rate (requests/sec)
• Actual concurrency (even if you hit with 200 concurrent connections, your server might be responding with just 80)
• Average, longest and shortest response time

example:

triggering 200 concurrent users, each hitting the url twice

siege -c200 -r2 http://www.modsaid.com/

summary:

Transactions:         400 hits

Availability:      100.00 %

Elapsed time:       38.44 secs

Data transferred:        0.26 MB

Response time:       10.09 secs

Transaction rate:       10.41 trans/sec

Throughput:        0.01 MB/sec

Concurrency:      104.96

Successful transactions:         400

Failed transactions:           0

Longest transaction:       34.19

Shortest transaction:        0.52

Siege is available in most repos and can be installed directly on ubuntu/debian via

apt-get install siege

However, in order for POST requests to work normally, I'd recommend installing the latest version (currently 3.0.5) from source

wget http://www.joedog.org/pub/siege/siege-3.0.5.tar.gz
tar zxvf siege-3.0.5.tar.gz
cd siege-3.0.5/
./configure && make && sudo make install

Similar tools exist, mainly ab of apache,  but ab is very basic compared to the flexibility of siege

Using siege is great for a single test, but you'll need to run it several times with different parameters in order to be able to understand your limits. and there comes the handy wrapper, bombard.

Bombard is a wrapper for siege that allows you to start your load testing with certain parameters and increase the load (concurrency) incrementally. and it draws charts that let's you see how your server reacts.  plotting makes things more clear and gives you a better sense of your server current limits.

Installing bombard requires dependencies:

• GD2 perl module
  sudo apt-get install libgd-graph-perl
• Chart-2.x
  wget http://www.cpan.org/authors/id/C/CH/CHARTGRP/Chart-2.4.6.tar.gz
  tar zxf  Chart-2.4.6.tar.gz && cd Chart-2.4.6 && \
  perl Makefile.PL && make && make test  && sudo make install

Then you can install bombard from the source

git clone git@github.com:allardhoeve/bombard.git

cd bombard

./configure

make && sudo make install

Now you can check all options through bombard -h

let's try experimenting out site:

Let's start with:

• starting concurrency 50
• increment 10
• runs 10 times  (so it will try 50, 60, 70, 80, ..., 140)
• each run will take 1 minute

Note: Bombard expects the absolute path of the file, it will not work with relative paths

bombard  -f /home/me/bombard-test-urls.txt -i10 -r10 -s50 -t1

From both graphs we can see that the transactions rate increases as we increase test concurrency. but it nearly saturates at concurrency 80.  The response time starts at nearly half a second, and remains so as we increase the load. until we reach 80 concurrency.  then it increases. the increase here is not from our web application itself, It is actually from having the requests buffer at the web server and application.

The two graphs indicate that our setup is able to handle up to 80 concurrent requests decently. then the performance will degrade as the load increases.

If you find this useful, please share your experience via comments, or get it touch with @modsaid