OpenVPN异地机房互连以及负载均衡高可用解决方案
keepon 2015-07-22
构方案如下：
---OpenVPN server 搭建部署
1、在OpenVPN-1 server上安装流程
（1.添加epel源
[root@ShangHai-VPN-1 ~]# yum install epel-release
（2.安装OpenVPN
[root@ShangHai-VPN-1 ~]# yum install openvpn lzo-devel easy-rsa -y
（3.复制server示例文件
[root@ShangHai-VPN-1 ~]# cp /usr/share/doc/openvpn-*/sample/sample-config-files/server.conf /etc/openvpn
（4.使用easy-rsa生成服务器证书以及密钥
[root@ShangHai-VPN-1 ~]# cp -R /usr/share/easy-rsa/ /etc/openvpn
[root@ShangHai-VPN-1 ~]# cd /etc/openvpn/easy-rsa/2.0/
#vars文件修改如下
[root@ShangHai-VPN-1 2.0]# egrep -v '^$|^#' vars
export EASY_RSA="`pwd`"
export OPENSSL="openssl"
export PKCS11TOOL="pkcs11-tool"
export GREP="grep"
export KEY_CONFIG=`$EASY_RSA/whichopensslcnf $EASY_RSA`
export KEY_DIR="$EASY_RSA/keys"
echo NOTE: If you run ./clean-all, I will be doing a rm -rf on $KEY_DIR
export PKCS11_MODULE_PATH="dummy"
export PKCS11_PIN="dummy"
export KEY_SIZE=2048
export CA_EXPIRE=3650
export KEY_EXPIRE=3650
export KEY_COUNTRY="CN"
export KEY_PROVINCE="BJ"
export KEY_CITY="BeiJing"
export KEY_ORG="example.com"
export KEY_EMAIL="[email protected]"
export KEY_OU="www.example.com"
export KEY_name="EasyRSA"
#初始化配置
[root@ShangHai-VPN-1 2.0]# source vars
[root@ShangHai-VPN-1 2.0]# ./clean-all
#生成CA证书
[root@linuxidc-VPN-1 2.0]# ./build-ca
Generating a 2048 bit RSA private key
..............................+++
................................+++
writing new private key to 'ca.key'
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [CN]:
State or Province Name (full name) [BJ]:
Locality Name (eg, city) [BeiJing]:
Organization Name (eg, company) [example.com]:
Organizational Unit Name (eg, section) [www.example.com]:
Common Name (eg, your name or your server's hostname) [example.com CA]:
Name [EasyRSA]:
Email Address [[email protected]]:
#生成服务器证书
[root@linuxidc-VPN-1 2.0]# ./build-key-server server
Generating a 2048 bit RSA private key
....................+++
....................................................................................+++
writing new private key to 'server.key'
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [CN]: [直接回车]
State or Province Name (full name) [BJ]:[直接回车]
Locality Name (eg, city) [BeiJing]:[直接回车]
Organization Name (eg, company) [example.com]:[直接回车]
Organizational Unit Name (eg, section) [www.example.com]:[直接回车]
Common Name (eg, your name or your server's hostname) [server]:[直接回车]
Name [EasyRSA]:[直接回车]
Email Address [[email protected]]:[直接回车]
Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:[直接回车]
An optional company name []:[直接回车]
Using configuration from /etc/openvpn/easy-rsa/2.0/openssl-1.0.0.cnf
Check that the request matches the signature
Signature ok
The Subject's Distinguished Name is as follows
countryName :PRINTABLE:'CN'
stateOrProvinceName :PRINTABLE:'BJ'
localityName :PRINTABLE:'BeiJing'
organizationName :PRINTABLE:'example.com'
organizationalUnitName:PRINTABLE:'www.example.com'
commonName :PRINTABLE:'server'
name :PRINTABLE:'EasyRSA'
emailAddress :IA5STRING:'[email protected]'
Certificate is to be certified until Jul 18 03:07:04 2025 GMT (3650 days)
Sign the certificate? [y/n]:y
1 out of 1 certificate requests certified, commit? [y/n]y
Write out database with 1 new entries
Data Base Updated
#生成服务器密钥
[root@linuxidc-VPN-1 2.0]# ./build-dh
Generating DH parameters, 2048 bit long safe prime, generator 2
This is going to take a long time
........................+............................................................................................+...........................+.............................................+........................+...................................................................................................................+........+.............................................+...............................................................................................................+..............++*++*
（5.生成客户端证书和密钥
[root@ShangHai-VPN-1 2.0]# ./build-key client
Generating a 2048 bit RSA private key
......+++
.......................+++
writing new private key to 'client.key'
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [CN]:[直接回车]
State or Province Name (full name) [BJ]:[直接回车]
Locality Name (eg, city) [BeiJing]:[直接回车]
Organization Name (eg, company) [example.com]:[直接回车]
Organizational Unit Name (eg, section) [www.example.com]:[直接回车]
Common Name (eg, your name or your server's hostname) [client]:[直接回车]
Name [EasyRSA]:[直接回车]
Email Address [[email protected]]:[直接回车]
Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:[直接回车]
An optional company name []:[直接回车]
Using configuration from /etc/openvpn/easy-rsa/2.0/openssl-1.0.0.cnf
Check that the request matches the signature
Signature ok
The Subject's Distinguished Name is as follows
countryName :PRINTABLE:'CN'
stateOrProvinceName :PRINTABLE:'BJ'
localityName :PRINTABLE:'BeiJing'
organizationName :PRINTABLE:'example.com'
organizationalUnitName:PRINTABLE:'www.example.com'
commonName :PRINTABLE:'client'
name :PRINTABLE:'EasyRSA'
emailAddress :IA5STRING:'[email protected]'
Certificate is to be certified until Jul 18 03:09:14 2025 GMT (3650 days)
Sign the certificate? [y/n]:y
1 out of 1 certificate requests certified, commit? [y/n]y
Write out database with 1 new entries
Data Base Updated
（6.将四个所需文件复制到OpenVPN配置目录
[root@ShangHai-VPN-1 2.0]# [root@linuxidc-VPN-1 2.0]# cp keys/{dh2048.pem,ca.crt,server.crt,server.key} /etc/openvpn/
（7.修改server配置文件
[root@ShangHai-VPN-1 2.0 ~]# egrep -v '^;|^#|^$' /etc/openvpn/server.conf
#公网IP
local 192.168.64.129
port 11194
proto tcp
dev tun
ca ca.crt
cert server.crt
key server.key # This file should be kept secret
dh dh2048.pem
#VPN获取的IP网段
server 10.10.10.0 255.255.255.0
ifconfig-pool-persist ipp.txt
#允许VPN访问的本地私网网段
push "route 172.16.65.0 255.255.255.0"
push "dhcp-option DNS 8.8.8.8"
push "dhcp-option DNS 8.8.4.4"
client-to-client
duplicate-cn
keepalive 10 120
comp-lzo
max-clients 300
user nobody
group nobody
persist-key
persist-tun
status /var/log/openvpn-status.log
log-append /var/log/openvpn.log
verb 3
mute 10
（8.配置路由转发
[root@linuxidc-VPN-1 2.0]# sed -i s'/net.ipv4.ip_forward = 0/net.ipv4.ip_forward = 1/' /etc/sysctl.conf
（9.配置防火墙开启端口并做NAT
[root@linuxidc-VPN-1 2.0]# iptables -t nat -A POSTROUTING -s 10.10.10.0/24 -j SNAT --to-source 172.16.65.128
[root@linuxidc-VPN-1 2.0]# iptables -A INPUT -m state --state NEW -m tcp -p tcp --dport 11194 -j ACCEPT
[root@linuxidc-VPN-1 2.0]# /etc/init.d/iptables save
iptables：将防火墙规则保存到 /etc/sysconfig/iptables： [确定]
[root@linuxidc-VPN-1 2.0]# /etc/init.d/iptables restart
iptables：将链设置为政策 ACCEPT：filter nat [确定]
iptables：清除防火墙规则： [确定]
iptables：正在卸载模块： [确定]
iptables：应用防火墙规则： [确定]
（10.启动OpenVPN [如果启动失败,请查看日志]
[root@linuxidc-VPN-1 2.0]# /etc/init.d/openvpn start
正在启动 openvpn： [确定]
[root@linuxidc-VPN-1 2.0]# netstat -antup |grep openvpn
tcp 0 0 192.168.64.128:11194 0.0.0.0:* LISTEN 2419/openvpn
2、在OpenVPN-2 server上安装流程
（1.添加epel源
[root@ShangHai-VPN-2 ~]# yum install epel-release
（2.安装OpenVPN
[root@ShangHai-VPN-2 ~]# yum install openvpn lzo-devel easy-rsa -y
（3.从OpenVPN-1 server上拷贝配置文件和证书密钥
[root@ShangHai-VPN-2 ~]# scp -r 192.168.64.128:/etc/openvpn/* /etc/openvpn/
（4.修改server配置文件
[root@ShangHai-VPN-1 2.0 ~]# egrep -v '^;|^#|^$' /etc/openvpn/server.conf
#本机公网IP [和OpenVPN-1 server 有区别]
local 192.168.64.129
port 11194
proto tcp
dev tun
ca ca.crt
cert server.crt
key server.key # This file should be kept secret
dh dh2048.pem
#VPN获取的IP网段 [和OpenVPN-1 server 有区别]
server 10.10.11.0 255.255.255.0
ifconfig-pool-persist ipp.txt
#允许VPN访问的本地私网网段
push "route 172.16.65.0 255.255.255.0"
push "dhcp-option DNS 8.8.8.8"
push "dhcp-option DNS 8.8.4.4"
client-to-client
duplicate-cn
keepalive 10 120
comp-lzo
max-clients 300
user nobody
group nobody
persist-key
persist-tun
status /var/log/openvpn-status.log
log-append /var/log/openvpn.log
verb 3
mute 10
（5.配置路由转发
[root@linuxidc-VPN-2 ~]# sed -i s'/net.ipv4.ip_forward = 0/net.ipv4.ip_forward = 1/' /etc/sysctl.conf
（6.配置防火墙开启端口并做NAT [和OpenVPN-1 server 有区别]
[root@linuxidc-VPN-2 ~]# iptables -t nat -A POSTROUTING -s 10.10.11.0/24 -j SNAT --to-source 172.16.65.129
[root@linuxidc-VPN-2 ~]# iptables -A INPUT -m state --state NEW -m tcp -p tcp --dport 11194 -j ACCEPT
[root@linuxidc-VPN-2 ~]# /etc/init.d/iptables save
iptables：将防火墙规则保存到 /etc/sysconfig/iptables： [确定]
[root@linuxidc-VPN-2 2.0]# /etc/init.d/iptables restart
iptables：将链设置为政策 ACCEPT：filter nat [确定]
iptables：清除防火墙规则： [确定]
iptables：正在卸载模块： [确定]
iptables：应用防火墙规则： [确定]
（8.启动OpenVPN [如果启动失败,请查看日志]
[root@ShangHai-VPN-2 ~]# /etc/init.d/openvpn start
正在启动 openvpn： [确定]
[root@ShangHai-VPN-2 ~]# netstat -antup |grep openvpn
tcp 0 0 192.168.64.129:11194 0.0.0.0:* LISTEN 2419/openvpn
3、安装OpenVPN client端
（1.添加epel源
[root@BeiJing-Client ~]# yum install epel-release
（2.安装OpenVPN
[root@BeiJing-Client ~]# yum install openvpn lzo-devel easy-rsa -y
（3.复制client示例文件
[root@BeiJing-Client ~]# cp /usr/share/doc/openvpn-*/sample/sample-config-files/client.conf /etc/openvpn
（4.修改client配置文件
[root@BeiJing-Client ~]# egrep -v '^;|^#|^$' /etc/openvpn/client.conf
client
dev tun
proto tcp
#OpenVPN-1 server
remote 192.168.64.128 11194
#OpenVPN-2 server
remote 192.168.64.129 11194
resolv-retry infinite
nobind
persist-key
persist-tun
comp-lzo
verb 3
ca ca.crt
cert client.crt
key client.key
（5.从OpenVPN server端拷贝client证书
[root@BeiJing-Client ~]# scp 192.168.64.128:/etc/openvpn/easy-rsa/2.0/keys/{ca.crt,client.crt,client.key} /etc/openvpn/
（6.启动OpenVPN Client
[root@ShangHai-VPN-2 ~]# /etc/init.d/openvpn start
正在启动 openvpn： [确定]
#查看是否获取到IP [获取到的是OpenVPN-1 server的虚拟IP段]
[root@BeiJing-Client ~]# ifconfig tun0
tun0 Link encap:UNSPEC HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
inet addr:10.10.10.6 P-t-P:10.10.10.5 Mask:255.255.255.255
UP POINTOPOINT RUNNING NOARP MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:100
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
（7.登录 OpenVPN-1 server 查看OpenVPN日志信息
[root@ShangHai-VPN-1 network-scripts]# tail -f /var/log/openvpn.log
Tue Jul 21 15:55:41 2015 192.168.64.138:52258 Data Channel Decrypt: Cipher 'BF-CBC' initialized with 128 bit key
Tue Jul 21 15:55:41 2015 192.168.64.138:52258 Data Channel Decrypt: Using 160 bit message hash 'SHA1' for HMAC authentication
Tue Jul 21 15:55:41 2015 192.168.64.138:52258 Control Channel: TLSv1.2, cipher TLSv1/SSLv3 DHE-RSA-AES256-GCM-SHA384, 2048 bit RSA
Tue Jul 21 15:55:41 2015 192.168.64.138:52258 [client] Peer Connection Initiated with [AF_INET]192.168.64.138:52258
Tue Jul 21 15:55:41 2015 client/192.168.64.138:52258 MULTI_sva: pool returned IPv4=10.10.10.6, IPv6=(Not enabled)
Tue Jul 21 15:55:41 2015 client/192.168.64.138:52258 MULTI: Learn: 10.10.10.6 -> client/192.168.64.138:52258 #显示将10.10.10.6分配给192.168.64.138 client
Tue Jul 21 15:55:41 2015 client/192.168.64.138:52258 MULTI: primary virtual IP for client/192.168.64.138:52258: 10.10.10.6
Tue Jul 21 15:55:43 2015 client/192.168.64.138:52258 PUSH: Received control message: 'PUSH_REQUEST'
Tue Jul 21 15:55:43 2015 client/192.168.64.138:52258 send_push_reply(): safe_cap=940
Tue Jul 21 15:55:43 2015 client/192.168.64.138:52258 SENT CONTROL [client]: 'PUSH_REPLY,route 172.16.64.0 255.255.255.0,dhcp-option DNS 8.8.8.8,dhcp-option DNS 8.8.4.4,route 10.10.10.0 255.255.255.0,topology net30,ping 10,ping-restart 120,ifconfig 10.10.10.6 10.10.10.5' (status=1)
4、测试OpenVPN的连通性和OpenVPN的fuzaijun高可用性能
（1.获取到虚拟IP后，测试ping上海机房的内部server
（2.关闭OpenVPN-1 server
[root@ShangHai-VPN-1 ~]# /etc/init.d/openvpn stop
正在关闭openvpn： [确定]
[root@ShangHai-VPN-1 ~]# netstat -antup |grep openvpn
（3.查看OpenVPN-2 server OpenVPN日志信息
[root@ShangHai-VPN-2 ~]# tail -f /var/log/openvpn.log
Tue Jul 21 16:29:07 2015 192.168.64.138:40636 Data Channel Decrypt: Cipher 'BF-CBC' initialized with 128 bit key
Tue Jul 21 16:29:07 2015 192.168.64.138:40636 Data Channel Decrypt: Using 160 bit message hash 'SHA1' for HMAC authentication
Tue Jul 21 16:29:07 2015 192.168.64.138:40636 Control Channel: TLSv1.2, cipher TLSv1/SSLv3 DHE-RSA-AES256-GCM-SHA384, 2048 bit RSA
Tue Jul 21 16:29:07 2015 192.168.64.138:40636 [client] Peer Connection Initiated with [AF_INET]192.168.64.138:40636
Tue Jul 21 16:29:07 2015 client/192.168.64.138:40636 MULTI_sva: pool returned IPv4=10.10.11.6, IPv6=(Not enabled)
Tue Jul 21 16:29:07 2015 client/192.168.64.138:40636 MULTI: Learn: 10.10.11.6 -> client/192.168.64.138:40636 #显示将10.10.11.6分配给192.168.64.138 client
Tue Jul 21 16:29:07 2015 client/192.168.64.138:40636 MULTI: primary virtual IP for client/192.168.64.138:40636: 10.10.11.6
Tue Jul 21 16:29:09 2015 client/192.168.64.138:40636 PUSH: Received control message: 'PUSH_REQUEST'
Tue Jul 21 16:29:09 2015 client/192.168.64.138:40636 send_push_reply(): safe_cap=940
Tue Jul 21 16:29:09 2015 client/192.168.64.138:40636 SENT CONTROL [client]: 'PUSH_REPLY,dhcp-option DNS 8.8.8.8,dhcp-option DNS 8.8.4.4,route 10.10.11.0 255.255.255.0,topology net30,ping 10,ping-restart 120,ifconfig 10.10.11.6 10.10.11.5' (status=1)
（4.查看是否获取新的IP，再次测试ping上海机房的内部server
至此,OpenVPN异地跨机房以及负载均衡高可用方案以构建完成，不足至此请大家指出，相互学习，共同进步。
