Testing Web Services using ApacheBench

ApacheBench (ab) is a tool for benchmarking an Apache Hypertext Transfer Protocol (HTTP) server. This shows how many requests per second the server is capable of handling.

A point to note is that ApacheBench will only use one operating system thread regardless of the concurrency level; specified by the -c parameter. In some cases, especially when benchmarking high-capacity servers, a single instance of ApacheBench can itself be a bottleneck. To overcome this, additional instances of ApacheBench may be used in parallel to more fully saturate the target URL.

ApacheBench was recently used to test the capability of the Caleum server, to find the threshold of the total number of web requests it can concurrently serve, in its current configuration.

Working with ApacheBench

Installing on a Windows machine

  1. Download the software from the link http://www.apache.org/dist/httpd/binaries/win32/ by selecting any mirrors on the site.
  2. Select the latest version of the software or later
  3. Double-click and install the software. While installing provide the information

Network Domain: localhost

Server Name: localhost

Admin Email: provide a real or fake email

Leave all default checkboxes checked

  1. After installation, an icon will be displayed in the system tray. This means Apache2.2 has been installed and started.
  2. To verify further type http://localhost/ in the browser. If Apache 2.2 has been started the message “It works!” in bold will be loaded in the browser.
  3. To stop/restart the server click on the icon ->Apache 2.2->Stop/Restart.

To measure the performance of a server you may need to point your files to Apache. Since we are doing a web service testing this step is optional.

Execution:

  1. Open the command prompt and go to the path where ApacheBench is installed say “C:\Program Files\Apache Software Foundation\Apache2.2\bin”
  2. Type ab –n 100 –c 10 http://{webserver hostname:port}/{document path}

You can also provide the authentication details as the parameters in the document path.

Other options that can be used are

Options are:

-n requests Number of requests to perform
-t timelimit Seconds to max. wait for responses
-v verbosity How much troubleshooting info to print
-b windowsize Size of TCP send/receive buffer, in bytes
-C attribute Add cookie, eg. ‘Apache=1234. (repeatable)
-H attribute Add Arbitrary header line, eg. ‘Accept-Encoding: gzip’ Inserted after all normal header lines. (repeatable)
-A attribute Add Basic WWW Authentication, the attributes are a colon-separated username and password.
-P attribute Add Basic Proxy Authentication, the attributes are a colon-separated username and password.
-x attributes String to insert as table attributes
-y attributes String to insert as tr attributes
-z attributes String to insert as td or th attributes
-Z ciphersuite Specify SSL/TLS cipher suite (See openssl ciphers)
-c concurrency Number of multiple requests to make
-T content-type Content-type header for POSTing, eg. ‘application/x-www-form-urlencoded’. The default is ‘text/plain’
-g filename Output collected data to gnuplot format file.
-e filename Output CSV file with percentages served
-p postfile The file containing data to POST. Remember also to set -T
-f protocol Specify SSL/TLS protocol (SSL2, SSL3, TLS1, or ALL)
-X proxy:port Proxy server and port number to use
-i Use HEAD instead of GET
-V Print the version number and exit
-k Use the HTTP KeepAlive feature
-d Do not show the percentiles served table.
-S Do not show confidence in estimators and warnings.
-r Don’t exit on socket receive errors.
-h Display usage information (this message)
-w Print out results in HTML tables

Output as below is displayed in the cmd prompt after the execution

Concurrency Level: 10
Time taken for tests: 321.212 sec
Complete requests: 1000
Failed requests: 11 (Connect: 0, Receive: 0, Length: 11, Exceptions: 0)
Write errors: 0
Document length 21bytes
Total transferred: 22124 bytes
HTML transferred: 11994 bytes
Requests per second: 1.01 [#/sec] (mean)
Time per request: 1216.319 [ms] (mean)
Time per request: 156.272 [ms] (mean, across all concurrent requests)
Transfer rate: 1.81 [Kbytes/sec] received

1.61 kb/s sent

0.42 kb/s total

Connection Times (ms) min mean [+/-sd] median max
Connect: 200 200 121 212 3000
Processing: 301 2121 612.8 1921 3267
Waiting: 211 2112 21 121 1211
Total: 711 3546 799.3 3281 6547

Percentage of the requests served within a certain time (ms)

50% 1212
66% 3823
75% 2211
80% 4555
90% 5555
95% 6666
98% 7777
99% 8888
100% 8899 (longest request)

It shows the total time to complete the entire test and the number of completed requests and failed requests. If there is any fail an additional line will be displayed. Connect:, Receive:, Length:, Exceptions:

While testing the web server, we mainly focus on the fails in Connect and Receive. The failure in the length is due to the content length not being specified or some additional data like ads coming up in the page which goes beyond the specified length.

Have questions? Contact the software testing experts at InApp to learn more.