Benchmarks
Cherokee is really really fast.
The speed at which any web server can serve requests for content is
both directly tied to and limited by the I/O speed of underlying
hardware and operating system. In this regard, a web server's
performance is measured by the latency incurred after an I/O request to
the underlying system has completed.
The primary objective of the Cherokee project is to reduce to zero the latency incurred between the time a dynamic or static system I/O request has completed and the time the resulting content is served to the requesting client. Admittedly, a very tough goal to reach.
In fact, it might be impossible. But achieving that which was once seen as impossible
is what drives innovation. And it's innovation that drives the ongoing
development of the Cherokee project, bringing us closer to the impossible
with each new release.
To show the project's progression towards the ultimate goal, whenever a benchmark is performed it will be published right here. Older releases had impressive but not very thorough benchmarks. Whenever it is possible the conditions of the benchmark will be provided so that anyone can replicate the results. That is, after all, an essential basis of the Scientific method.
Cherokee 0.6
This benchmark was performed by Miguel Angel Ajo Pelayo with Django running on Cherokee 0.6.0 and Apache 2.0.40. It shows Cherokee as a clear winner serving dynamic content. Specifically three times as much! And you haven't yet seen static results. The advantage is a lot bigger.
Date: 2008-03-10 23:24 Subject: [cherokee-dev] Django & Apache migration to Django & Cherokee 0.6 x3 speed!!! :D Phewww, incredible: This was Apache: ab -n 200 -c 50 http://leemelo.com/feeds/ This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0 Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Copyright 2006 The Apache Software Foundation, http://www.apache.org/ Benchmarking leemelo.com (be patient) Completed 100 requests Finished 200 requests Server Software: Apache/2.2.3 Server Hostname: leemelo.com Server Port: 80 Document Path: /feeds/ Document Length: 13803 bytes Concurrency Level: 50 Time taken for tests: 6.581932 seconds Complete requests: 200 Failed requests: 0 Write errors: 0 Total transferred: 2811600 bytes HTML transferred: 2760600 bytes Requests per second: 30.39 [#/sec] (mean) Time per request: 1645.483 [ms] (mean) Time per request: 32.910 [ms] (mean, across all concurrent requests) Transfer rate: 417.05 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 0.0 0 0 Processing: 222 1369 1660.9 474 6581 Waiting: 222 1368 1660.8 474 6580 Total: 222 1369 1660.9 474 6581 Percentage of the requests served within a certain time (ms) 50% 474 66% 821 75% 2099 80% 3247 90% 3726 95% 4657 98% 6401 99% 6492 100% 6581 (longest request) And this is Cherokee: ab -n 200 -c 50 http://leemelo.com/feeds/ This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 > apache-2.0 Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Copyright 2006 The Apache Software Foundation, http://www.apache.org/ Benchmarking leemelo.com (be patient) Completed 100 requests Finished 200 requests Server Software: Cherokee Server Hostname: leemelo.com Server Port: 80 Document Path: /feeds/ Document Length: 13803 bytes Concurrency Level: 50 Time taken for tests: 2.247276 seconds Complete requests: 200 Failed requests: 0 Write errors: 0 Total transferred: 2787200 bytes HTML transferred: 2760600 bytes Requests per second: 89.00 [#/sec] (mean) Time per request: 561.819 [ms] (mean) Time per request: 11.236 [ms] (mean, across all concurrent requests) Transfer rate: 1210.80 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 0.2 0 1 Processing: 14 499 158.8 527 825 Waiting: 13 497 159.0 523 825 Total: 15 499 158.7 527 825 Percentage of the requests served within a certain time (ms) 50% 527 66% 559 75% 578 80% 600 90% 667 95% 725 98% 769 99% 818 100% 825 (longest request)
Other 0.6 benchmarks
Cherokee 0.5
Benchmark of Cherokee 0.5.4 under Windows, contributed to the main mailing list. Here is the link to the non abridged post.
Cherokee had one million static requests and one million scgi dynamic requests with two ab commands started at about the same time. More info follows:
The scgi performance was quite impressive. I'm not sure about the
details of Cherokee's scgi and fcgi implementation. In my testing, a
multi-threaded scgi back-end written in Python was much faster
handling concurrent requests than a single-threaded fcgi backend
written in a compiled language, when the number of concurrent requests
was more than 5. But dealing with continuous but non-concurrent
requests, the natively compiled fastcgi takes about 40% of the time of
what it takes the scgi Python app to handle a request. This tells me
that the Python scgi backend (web.py) is able to cope with pretty
heavy traffic, which will be a bit less than half of the ultimate
capacity that could be achieved by running a server pool of
fastcgi/scgi written in a compiled language. (I'm assuming the
overhead of fcgi and scgi are similar, or the difference ignorable.)
The test is done on a single-CPU vmware guest OS running on a powerful
core 2 dual host. It was taking most CPU time of a core during the
test. No other vms or heavy tasks were run during the tests.
C:\Internet>ab -n 1000000 -c 1 http://localhost/image/google.html
Server Software: Cherokee/0.5.4
Server Hostname: localhost
Server Port: 80
Document Path: /image/static.html
Document Length: 67859 bytes
Concurrency Level: 1
Time taken for tests: 6012.703125 seconds
Complete requests: 1000000
Failed requests: 5
(Connect: 5, Length: 0, Exceptions: 0)
Write errors: 0
Total transferred: -681476736 bytes
HTML transferred: -860476736 bytes
Requests per second: 166.31 [#/sec] (mean)
Time per request: 6.013 [ms] (mean)
Time per request: 6.013 [ms] (mean, across all concurrent requests)
Transfer rate: -110.68 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 1 12.0 0 3015
Processing: 0 2 6.1 0 203
Waiting: 0 0 3.0 0 156
Total: 0 4 13.2 0 3046
Percentage of the requests served within a certain time (ms)
50% 0
66% 0
75% 15
80% 15
90% 15
95% 15
98% 15
99% 15
100% 3046 (longest request)
C:\Internet>ab -n 1000000 -c 1 http://localhost/s/scgi/
Benchmarking localhost (be patient)
Completed 100000 requests
Completed 200000 requests
Completed 300000 requests
Completed 400000 requests
Completed 500000 requests
Completed 600000 requests
Completed 700000 requests
Completed 800000 requests
Completed 900000 requests
Finished 1000000 requests
Server Software: Cherokee/0.5.4
Server Hostname: localhost
Server Port: 80
Document Path: /s/scgi/
Document Length: 693 bytes
Concurrency Level: 1
Time taken for tests: 8875.62500 seconds
Complete requests: 1000000
Failed requests: 5
(Connect: 5, Length: 0, Exceptions: 0)
Write errors: 0
Total transferred: 803000000 bytes
HTML transferred: 693000000 bytes
Requests per second: 112.68 [#/sec] (mean)
Time per request: 8.875 [ms] (mean)
Time per request: 8.875 [ms] (mean, across all concurrent requests)
Transfer rate: 88.36 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 6.9 0 2984
Processing: 0 8 8.1 15 281
Waiting: 0 7 8.0 0 281
Total: 0 8 10.5 15 3031
Percentage of the requests served within a certain time (ms)
50% 15
66% 15
75% 15
80% 15
90% 15
95% 15
98% 15
99% 31
100% 3031 (longest request)
Other 0.5.4 benchmarks:
Older benchmarks
- Cherokee 0.4.3 pre6 benchmark:
Cherokee: 20426.30, Boa: 12287.60, thttpd: 4347.81, Apache: 3953.4 reqs/sec - Cherokee 0.3.0 Beta14 benchmark:
Cherokee: 3785.01, Boa: 3328.89, thttpd: 3167.56, Apache: 732.12 reqs/sec