nginx,uwsgi,bottle,virtualenv在centos6上安装及性能测试

终于决定废弃java而使用python来开发网站，测试web.py的性能有点失望，看到bottle性能不错，也满足我们的需求，符合pythonsimple的理念。测试一下生产环境的性能。

参照http://studio.zeuik.com/?p=791

测试机器:X5650X2,48G内存，centos6，包括nginx,uwsgi

压测机器:X5680X2,96G内存

web.py把cpu几乎用尽的情况下也就5000req/s左右。

1、获取安装文件

nginx-1.0.10.tar.gz

uwsgi-0.9.9.2.tar.gz

virtualenv-1.6.4.tar.gz

bottle-0.10.1.tar.gz

2、安装nginx

tar zxvf nginx-1.0.10.tar.gz 
cd nginx-1.0.10
./configure --prefix=/usr/local/nginx --with-select_module --with-poll_module --with-http_ssl_module --with-http_gzip_static_module --with-http_flv_module --with-http_sub_module --with-http_stub_status_module --with-http_secure_link_module
make && make install

配置nginx配置文件

/usr/local/nginx/conf/nginx.conf

worker_processes 8; #24线程cpu，8个nginx，16个uwsgi
events {
    worker_connections  10240;
}
http {
    include       mime.types;
    default_type  application/octet-stream;
    sendfile        on;
    keepalive_timeout  65;

    server {
        listen      80;
        charset     utf-8;
        root        /application/venv/bottletest;
        server_name cow.com;
 
        location / {
            include uwsgi_params;
            uwsgi_param UWSGI_PYHOME /application/venv/bottletest;
            uwsgi_param UWSGI_CHDIR /application/venv/bottletest;
            uwsgi_param UWSGI_SCRIPT index; # 对应index.py
            uwsgi_pass  127.0.0.1:8888;
        }   
 
        location ^~ /static {
            root /application/venv/bottletest;
            access_log off;
        }   
 
        location /status {
            stub_status on;
            access_log  off;
        }
    }  
}

3、安装uwsgi

tar zxvf uwsgi-0.9.9.2.tar.gz 
cd uwsgi-0.9.9.2
mkdir /usr/local/uwsgi
cp uwsgi /usr/local/uwsgi/
cp uwsgi.xml /usr/local/uwsgi/bottletest.xml
chown root:wheel /usr/local/uwsgi/logs

配置uwsgi的xml格式配置文件

/usr/local/uwsgi/bottletest.xml

<uwsgi>
    <socket>127.0.0.1:8888</socket>
    <home>/application/venv/bottletest</home>
    <chdir>/application/venv/bottletest</chdir>
    <python-path>/application/venv/bottletest</python-path>
    <module>[WSGI Script (index)]</module>
    <limit-as>256MB</limit-as>
    <processes>16</processes> <!-- 进程数 -->
    <master/>
    <memory/>
    <logto>/usr/local/uwsgi/logs/bottletest.log</logto>
    <daemonize>/var/log/uwsgi.log</daemonize>
    <max-requests>10000</max-requests>
</uwsgi>

4、安装virtualenv并创建bottletest应用

tar zxvf virtualenv-1.6.4.tar.gz 
cd virtualenv-1.6.4
python setup.py install
/usr/bin/virtualenv /application/venv/bottletest
cd /application/venv/bottletest
source bin/activate

5、编辑python文件

/application/venv/bottletest/index.py

from bottle import route, run, default_app
@route('/')
def index():
        return "hello world"
if __name__ == "__main__":
        run(host="localhost", port=8888)
else:
        application = default_app()

6、启动nginx和uwsgi

/usr/local/nginx/sbin/nginx
/usr/local/uwsgi/uwsgi -x /usr/local/uwsgi/bottletest.xml

7、ab测试结果

100client

ab -c 100 -n 100000 http://192.168.0.90/
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 192.168.0.90 (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        nginx/1.0.10
Server Hostname:        192.168.0.90
Server Port:            80

Document Path:          /
Document Length:        11 bytes

Concurrency Level:      100
Time taken for tests:   4.891 seconds
Complete requests:      100000
Failed requests:        0
Write errors:           0
Total transferred:      16800336 bytes
HTML transferred:       1100022 bytes
Requests per second:    20443.76 [#/sec] (mean)
Time per request:       4.891 [ms] (mean)
Time per request:       0.049 [ms] (mean, across all concurrent requests)
Transfer rate:          3354.12 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    1   0.7      1      15
Processing:     1    4   1.8      4     134
Waiting:        1    4   1.9      4     134
Total:          1    5   1.8      5     135

Percentage of the requests served within a certain time (ms)
  50%      5
  66%      5
  75%      5
  80%      5
  90%      6
  95%      6
  98%      8
  99%     13
 100%    135 (longest request)

1000client

ab -c 1000 -n 100000 http://192.168.0.90/
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 192.168.0.90 (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        nginx/1.0.10
Server Hostname:        192.168.0.90
Server Port:            80

Document Path:          /
Document Length:        11 bytes

Concurrency Level:      1000
Time taken for tests:   9.057 seconds
Complete requests:      100000
Failed requests:        0
Write errors:           0
Total transferred:      16800000 bytes
HTML transferred:       1100000 bytes
Requests per second:    11040.71 [#/sec] (mean)
Time per request:       90.574 [ms] (mean)
Time per request:       0.091 [ms] (mean, across all concurrent requests)
Transfer rate:          1811.37 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    9 114.9      4    3007
Processing:     1   51 372.0     12    9028
Waiting:        0   50 372.1     10    9028
Total:          3   60 389.4     16    9043

Percentage of the requests served within a certain time (ms)
  50%     16
  66%     19
  75%     21
  80%     22
  90%     26
  95%     31
  98%     50
  99%   3006
 100%   9043 (longest request)

webbench模拟1000和5000客户端测试，系统负载都在4左右

1000client

./webbench -t 30 -c 1000 http://192.168.0.90/
Webbench - Simple Web Benchmark 1.5
Copyright (c) Radim Kolar 1997-2004, GPL Open Source Software.

Benchmarking: GET http://192.168.0.90/
1000 clients, running 30 sec.

Speed=1195526 pages/min, 3347411 bytes/sec.
Requests: 597753 susceed, 10 failed.

5000client

./webbench -t 30 -c 5000 http://192.168.0.90/ 
Webbench - Simple Web Benchmark 1.5
Copyright (c) Radim Kolar 1997-2004, GPL Open Source Software.

Benchmarking: GET http://192.168.0.90/
5000 clients, running 30 sec.

Speed=1209354 pages/min, 3383884 bytes/sec.
Requests: 604269 susceed, 408 failed.

总结

性能基本满意，回头测试一下模板cheetah和mako的性能。

再测试一下nginxupstreamingtornado的性能，看是否能作为nginxcmodule某些功能的替代品