环境CentOS 6.5，Nginx 1.7.2 ，Memcached 1.4.20

一.导出lib

1.利用maven，构建并导出基于kryo-serializer的lib，可以将下面的依赖版本和配置如下

<properties>
        <memcached-session-manager.version>1.8.2</memcached-session-manager.version>
        <spymemcached.version>2.11.4</spymemcached.version>
        <kryo-serializers.version>0.11</kryo-serializers.version>
        <kryo.version>1.04</kryo.version>
        <minlog.version>1.2</minlog.version>
        <reflectasm.version>1.01</reflectasm.version>
        <asm.version>3.2</asm.version>
    </properties>

    <dependencies>
        <!-- memcached-session-manager -->
        <dependency>
            <groupId>de.javakaffee.msm</groupId>
            <artifactId>memcached-session-manager</artifactId>
            <version>${memcached-session-manager.version}</version>
            <exclusions>
                <exclusion>
                    <groupId>net.spy</groupId>
                    <artifactId>spymemcached</artifactId>
                </exclusion>
                <exclusion>
                    <groupId>com.couchbase.client</groupId>
                    <artifactId>couchbase-client</artifactId>
                </exclusion>
                <exclusion>
                    <groupId>com.google.code.findbugs</groupId>
                    <artifactId>jsr305</artifactId>
                </exclusion>
                <exclusion>
                    <groupId>com.google.code.findbugs</groupId>
                    <artifactId>annotations</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
        <dependency>
            <groupId>de.javakaffee.msm</groupId>
            <artifactId>memcached-session-manager-tc7</artifactId>
            <version>${memcached-session-manager.version}</version>
            <exclusions>
                <exclusion>
                    <groupId>com.google.code.findbugs</groupId>
                    <artifactId>jsr305</artifactId>
                </exclusion>
                <exclusion>
                    <groupId>com.google.code.findbugs</groupId>
                    <artifactId>annotations</artifactId>
                </exclusion>
            </exclusions>
        </dependency>

        <!-- spymemcached -->
        <dependency>
            <groupId>net.spy</groupId>
            <artifactId>spymemcached</artifactId>
            <version>${spymemcached.version}</version>
        </dependency>

        <!-- kryo-serializer -->
        <dependency>
            <groupId>de.javakaffee.msm</groupId>
            <artifactId>msm-kryo-serializer</artifactId>
            <version>${memcached-session-manager.version}</version>
            <exclusions>
                <exclusion>
                    <groupId>com.google.code.findbugs</groupId>
                    <artifactId>jsr305</artifactId>
                </exclusion>
                <exclusion>
                    <groupId>com.google.code.findbugs</groupId>
                    <artifactId>annotations</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
        <dependency>
            <groupId>de.javakaffee</groupId>
            <artifactId>kryo-serializers</artifactId>
            <version>${kryo-serializers.version}</version>
        </dependency>
        <dependency>
            <groupId>com.googlecode</groupId>
            <artifactId>kryo</artifactId>
            <version>${kryo.version}</version>
        </dependency>
        <dependency>
            <groupId>com.googlecode</groupId>
            <artifactId>minlog</artifactId>
            <version>${minlog.version}</version>
        </dependency>
        <dependency>
            <groupId>com.googlecode</groupId>
            <artifactId>reflectasm</artifactId>
            <version>${reflectasm.version}</version>
        </dependency>
        <dependency>
            <groupId>asm</groupId>
            <artifactId>asm</artifactId>
            <version>${asm.version}</version>
        </dependency>
    </dependencies>

2.执行下面的命令下载lib

mvn dependency:copy-dependencies -DoutputDirectory=./target/lib

lib就会被自动下载到/target/lib目录下，内容如下图。

2.基于memcached-session-manager的负载均衡

二.配置tomcat

配置方式主要见https://code.google.com/p/memcached-session-manager/wiki/SetupAndConfiguration

1.拷贝LIB

将上面导出的lib拷贝到 $CATALINA_HOME/lib/ 目录下

2.配置$CATALINA_HOME/conf/context.xml

以下两种配置选择其中一种，推荐第二种

第一种: sticky sessions + kryo

<Context>
  ...
  <Manager className="de.javakaffee.web.msm.MemcachedBackupSessionManager"
    memcachedNodes="n1:host1.yourdomain.com:11211,n2:host2.yourdomain.com:11211"
    failoverNodes="n1"
    requestUriIgnorePattern=".*\.(ico|png|gif|jpg|css|js)$"
    transcoderFactoryClass="de.javakaffee.web.msm.serializer.kryo.KryoTranscoderFactory"
    />
</Context>

注意:

(1) sticky模式时，配置jvmroute参数，每台tomcat的jvmroute参数都不能一样 (测试这种配置方式不适用于多台Servlet容器，或适用于反向代理服务器可以把相同来源的请求映射到同一台容器上的情况下，不推荐；另网上看到该配置修改transcoderFactoryclass="de.javakaffee.web.msm.serializer.javolution.JavolutionTranscoderFactory" 未验证)

$CATALINA_HOME/conf/server.xml

<Engine name="Catalina" defaultHost="localhost" jvmRoute="jvm1">

(2) 必须启动两个Memcached实例，比如

/usr/local/memcached/bin/memcached -d -m 256 -u root -p 11211 -c 128 -P /tmp/memcached1.pid     
/usr/local/memcached/bin/memcached -d -m 256 -u root -p 11212 -c 128 -P /tmp/memcached2.pid

第二种:non-sticky sessions + kryo

<Context>
  ...
  <Manager className="de.javakaffee.web.msm.MemcachedBackupSessionManager"
    memcachedNodes="n1:host1.yourdomain.com:11211,n2:host2.yourdomain.com:11211"
    sticky="false"
    sessionBackupAsync="false"
    lockingMode="uriPattern:/path1|/path2"
    requestUriIgnorePattern=".*\.(ico|png|gif|jpg|css|js)$"
    transcoderFactoryClass="de.javakaffee.web.msm.serializer.kryo.KryoTranscoderFactory"
    />
</Context>

3.可选配置$CATALINA_HOME/conf/logging.properties

如果需要调试日志，可以在logging.properties中添加如下内容

de.javakaffee.web.msm.level=FINE

4. 多台Tomcat配置需要注意下面内容

1) 修改http访问端口（默认为8080端口）

2) 修改Shutdown端口（默认为8005端口）

3) 修改AJP 访问端口（默认为8009端口）

4) 修改http访问（默认 Connector port="8080"）,添加 URIEncoding="UTF-8"

三.配置Nginx

修改配置文件 /usr/local/nginx/conf/nginx.conf

下面是Nginx 1.7.2完整事例

#user  nobody;
user gqshao;
worker_processes  1;

#error_log  logs/error.log;
#error_log  logs/error.log  notice;
#error_log  logs/error.log  info;

#pid        logs/nginx.pid;


events {
    // windows系统下不要 use epoll
    use epoll;
    worker_connections  1024;
}


http {
    include       mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  logs/access.log  main;

    sendfile        on;
    #tcp_nopush     on;

    #keepalive_timeout  0;
    keepalive_timeout  65;

    #gzip  on;
    gzip  on;
    gzip_types       text/javascript text/plain text/css application/xml application/x-javascript;
    gzip_disable "MSIE [1-6].";

    # 配置tomcat服务器
    upstream web_server {
        #ip_hash;
        server 192.168.158.131:8080 max_fails=3  fail_timeout=30s;
        server 192.168.158.131:8081 max_fails=3  fail_timeout=30s;
        server 192.168.158.131:8082 max_fails=3  fail_timeout=30s;
    }

    server {
        # 侦听192.168.158.131的80端口
        listen       80;
        server_name  192.168.158.131;

        #charset koi8-r;
        charset utf-8;

        access_log  logs/host.access.log  main;

        location / {
            proxy_pass    http://web_server ;
            proxy_redirect off ;

            #后端的Web服务器可以通过X-Forwarded-For获取用户真实IP
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        }

        #error_page  404              /404.html;

        # redirect server error pages to the static page /50x.html
        #
        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }
    }
}

四.启动和停止

启动

# 启动 memcached
sudo /usr/local/memcached/bin/memcached -d -m 256 -u gqshao -p 11211 -c 128 -P /tmp/memcached1.pid
sudo /usr/local/memcached/bin/memcached -d -m 256 -u gqshao -p 11212 -c 128 -P /tmp/memcached2.pid

# 启动 Tomcat
./startup.sh

# 启动 Nginx
sudo /usr/local/nginx/sbin/nginx