xcznb 2020-02-28
比如我们在使用rabbitmq的时候做了这么一个集群,一般做了集群的话,那肯定就需要这么一个高可用的负载均衡器来实现流量的分发,如果使用的是haproxy,比如一台rebbitmq的节点突然宕机或者网卡失效,那么虽然RabbitMQ集群没有任何故障,但是对于外界的客户端来说所有的连接都会被断开,结果将是灾难性的。确保负载均衡服务的可靠性同样显得十分的重要。这里就引入Keepalived工具,它能够通过自身健康检查、资源接管功能做高可用(双机热备),实现故障转移。
Keepalived采用VRRP(Virtual Router Redundancy Protocol,虚拟路由冗余协议),以软件的形式实现服务器热备功能。通常情况下是将两台Linux服务器组成一个热备组(Master和Backup),同一时间热备组内只有一台主服务器Master提供服务,同时Master会虚拟出一个公用的虚拟IP地址,简称VIP。这个VIP只存在在Master上并对外提供服务。如果Keepalived检测到Master宕机或者服务故障,备份服务器Backup会自动接管VIP称为Master,Keepalived并将原Master从热备组中移除。当原Master恢复后,会自动加入到热备组,默认再抢占称为Master,起到故障转移的功能。
Keepalived工作在OSI模型中的第3层、第4层和第7层。
工作在第3层是指Keepalived会定期向热备组中的服务器发送一个ICMP数据包来判断某台服务器是否故障,如果故障则将这台服务器从热备组移除。
工作在第4层是指Keepalived以TCP端口的状态判断服务器是否故障,比如检测RabbitMQ的5672端口,如果故障则将这台服务器从热备组中移除。
工作在第7层是指Keepalived根据用户设定的策略(通常是一个自定义的检测脚本)判断服务器上的程序是否正常运行,如果故障将这台服务器从热备组移除。
首先需要去Keepalived的官网下载Keepalived的安装文件,目前最新的版本为:keepalived-2.0.20.tar.gz,下载地址为http://www.keepalived.org/download.html
可以找到最新版
[_0_8_centos ~]# mkdir keepalived [_0_8_centos ~]# cd keepalived/ [_0_8_centos keepalived]# wget https://www.keepalived.org/software/keepalived-2.0.20.tar.gz --2020-02-27 10:46:37-- https://www.keepalived.org/software/keepalived-2.0.20.tar.gz Resolving www.keepalived.org (www.keepalived.org)... 37.59.63.157, 2001:41d0:8:7a9d::1 Connecting to www.keepalived.org (www.keepalived.org)|37.59.63.157|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 1036063 (1012K) [application/x-gzip] Saving to: ‘keepalived-2.0.20.tar.gz’ 100%[==================================================================================================================================>] 1,036,063 13.7KB/s in 1m 43s 2020-02-27 10:48:37 (9.80 KB/s) - ‘keepalived-2.0.20.tar.gz’ saved [1036063/1036063]
解压
[_0_8_centos keepalived]# ls keepalived-2.0.20.tar.gz [_0_8_centos keepalived]# tar xf keepalived-2.0.20.tar.gz [_0_8_centos keepalived]# cd keepalived-2.0.20/ [_0_8_centos keepalived-2.0.20]# ls aclocal.m4 AUTHOR build_setup compile configure.ac COPYING doc INSTALL keepalived lib Makefile.in README.md TODO ar-lib bin_install ChangeLog configure CONTRIBUTORS depcomp genhash install-sh keepalived.spec.in Makefile.am missing snap [_0_8_centos keepalived-2.0.20]# ./configure --prefix=/opt/keepalived --with-init=SYSV checking for a BSD-compatible install... /usr/bin/install -c checking whether build environment is sane... yes checking for a thread-safe mkdir -p... /usr/bin/mkdir -p checking for gawk... gawk checking whether make sets $(MAKE)... yes checking whether make supports nested variables... yes checking whether make supports nested variables... (cached) yes checking for pkg-config... /usr/bin/pkg-config checking pkg-config is at least version 0.9.0... yes checking for gcc... gcc checking whether the C compiler works... yes checking for C compiler default output file name... a.out
出现报错:缺少openssl-devel
configure: error: !!! OpenSSL is not properly installed on your system. !!! !!! Can not include OpenSSL headers files. !!! [_0_8_centos keepalived-2.0.20]# yum -y install openssl-devel Loaded plugins: fastestmirror, langpacks Determining fastest mirrors Resolving Dependencies --> Running transaction check ---> Package openssl-devel.x86_64 1:1.0.2k-19.el7 will be installed
安装后重新再执行[_0_8_centos keepalived-2.0.20]# ./configure --prefix=/opt/keepalived --with-init=SYSV
再次报错,说这个版本不支持ipv6,需要安装依赖包
*** WARNING - this build will not support IPVS with IPv6. Please install libnl/libnl-3 dev libraries to support IPv6 with IPVS.
[_0_8_centos keepalived-2.0.20]# yum -y install libnl libnl-devel Loaded plugins: fastestmirror, langpacks Loading mirror speeds from cached hostfile Package libnl-1.1.4-3.el7.x86_64 already installed and latest version
安装完依赖包再次执行
[_0_8_centos keepalived-2.0.20]# ./configure --prefix=/opt/keepalived --with-init=SYSV
编译
[_0_8_centos keepalived-2.0.20]# make && make install
将启动文件放到/etc/init.d下,可以使用service来启动keepalived
[_0_8_centos keepalived-2.0.20]# cp /opt/keepalived/etc/rc.d/init.d/keepalived /etc/init.d/ [_0_8_centos keepalived-2.0.20]# cp /opt/keepalived/etc/sysconfig/keepalived /etc/sysconfig [_0_8_centos keepalived-2.0.20]# cp /opt/keepalived/sbin/keepalived /usr/sbin [_0_8_centos keepalived-2.0.20]# chmod +x /etc/init.d/keepalived [_0_8_centos keepalived-2.0.20]# chkconfig --add keepalived [_0_8_centos keepalived-2.0.20]# chkconfig keepalived on [_0_8_centos keepalived-2.0.20]# cp /opt/keepalived/etc/keepalived/keepalived.conf /etc/keepalived.conf
启动失败,查看报错,这里提示 Unable to find configuration file /etc/keepalived/keepalived.conf,没有发现我们的配置文件的地址,这里是因为我们在安装的时候指定了自己的安装路径
[_0_8_centos keepalived-2.0.20]# service keepalived start Starting keepalived (via systemctl): Job for keepalived.service failed because the control process exited with error code. See "systemctl status keepalived.service" and "journalctl -xe" for details. [FAILED] [_0_8_centos keepalived-2.0.20]# journalctl -xe
找出放置我们配置文件的地方,修改文件地址
[_0_8_centos ~]# find / -name keepalived.conf /opt/keepalived/etc/keepalived/keepalived.conf /etc/keepalived.conf /root/keepalived/keepalived-2.0.20/keepalived/etc/keepalived/keepalived.conf
在最后一行添加我们配置文件的地址
[_0_8_centos ~]# vim /etc/sysconfig/keepalived [_0_8_centos ~]# tail -1 /etc/sysconfig/keepalived KEEPALIVED_OPTIONS="-f /etc/keepalived.conf -D -S 0"
使用systemctl启动成功
[_0_8_centos ~]# systemctl restart keepalived [_0_8_centos ~]# systemctl status keepalived ● keepalived.service - SYSV: Start and stop Keepalived Loaded: loaded (/etc/rc.d/init.d/keepalived; bad; vendor preset: disabled) Active: active (running) since Thu 2020-02-27 11:02:22 CST; 10s ago Docs: man:systemd-sysv-generator(8) Process: 30129 ExecStart=/etc/rc.d/init.d/keepalived start (code=exited, status=0/SUCCESS) Main PID: 30136 (keepalived) CGroup: /system.slice/keepalived.service ├─30136 keepalived -f /etc/keepalived.conf -D -S 0 ├─30138 keepalived -f /etc/keepalived.conf -D -S 0 └─30139 keepalived -f /etc/keepalived.conf -D -S 0 Feb 27 11:02:31 VM_0_8_centos Keepalived_vrrp[30139]: Sending gratuitous ARP on eth0 for 192.168.200.18 Feb 27 11:02:31 VM_0_8_centos Keepalived_vrrp[30139]: Sending gratuitous ARP on eth0 for 192.168.200.16 Feb 27 11:02:31 VM_0_8_centos Keepalived_vrrp[30139]: Sending gratuitous ARP on eth0 for 192.168.200.17 Feb 27 11:02:31 VM_0_8_centos Keepalived_vrrp[30139]: Sending gratuitous ARP on eth0 for 192.168.200.18 Feb 27 11:02:31 VM_0_8_centos Keepalived_healthcheckers[30138]: HTTP_CHECK on service [192.168.200.3]:tcp:1358 failed after 3 retry. Feb 27 11:02:31 VM_0_8_centos Keepalived_healthcheckers[30138]: Removing service [192.168.200.3]:tcp:1358 to VS [10.10.10.2]:tcp:1358 Feb 27 11:02:31 VM_0_8_centos Keepalived_healthcheckers[30138]: Lost quorum 1-0=1 > 0 for VS [10.10.10.2]:tcp:1358 Feb 27 11:02:31 VM_0_8_centos Keepalived_healthcheckers[30138]: Adding sorry server [192.168.200.200]:tcp:1358 to VS [10.10.10.2]:tcp:1358 Feb 27 11:02:31 VM_0_8_centos Keepalived_healthcheckers[30138]: Removing alive servers from the pool for VS [10.10.10.2]:tcp:1358 Feb 27 11:02:31 VM_0_8_centos Keepalived_healthcheckers[30138]: Remote SMTP server [192.168.200.1]:25 connected.
使用service进行启动停止
[_0_8_centos ~]# service keepalived stop Stopping keepalived (via systemctl): [ OK ] [_0_8_centos ~]# service keepalived start Starting keepalived (via systemctl): [ OK ] [_0_8_centos ~]# service keepalived status ● keepalived.service - SYSV: Start and stop Keepalived Loaded: loaded (/etc/rc.d/init.d/keepalived; bad; vendor preset: disabled) Active: active (running) since Thu 2020-02-27 11:03:27 CST; 10s ago Docs: man:systemd-sysv-generator(8) Process: 30394 ExecStop=/etc/rc.d/init.d/keepalived stop (code=exited, status=0/SUCCESS) Process: 30469 ExecStart=/etc/rc.d/init.d/keepalived start (code=exited, status=0/SUCCESS) Main PID: 30476 (keepalived) CGroup: /system.slice/keepalived.service ├─30476 keepalived -f /etc/keepalived.conf -D -S 0 ├─30478 keepalived -f /etc/keepalived.conf -D -S 0 └─30479 keepalived -f /etc/keepalived.conf -D -S 0 Feb 27 11:03:36 VM_0_8_centos Keepalived_vrrp[30479]: Sending gratuitous ARP on eth0 for 192.168.200.18 Feb 27 11:03:36 VM_0_8_centos Keepalived_vrrp[30479]: Sending gratuitous ARP on eth0 for 192.168.200.16 Feb 27 11:03:36 VM_0_8_centos Keepalived_vrrp[30479]: Sending gratuitous ARP on eth0 for 192.168.200.17 Feb 27 11:03:36 VM_0_8_centos Keepalived_vrrp[30479]: Sending gratuitous ARP on eth0 for 192.168.200.18 Feb 27 11:03:37 VM_0_8_centos Keepalived_healthcheckers[30478]: HTTP_CHECK on service [192.168.200.3]:tcp:1358 failed after 3 retry. Feb 27 11:03:37 VM_0_8_centos Keepalived_healthcheckers[30478]: Removing service [192.168.200.3]:tcp:1358 to VS [10.10.10.2]:tcp:1358 Feb 27 11:03:37 VM_0_8_centos Keepalived_healthcheckers[30478]: Lost quorum 1-0=1 > 0 for VS [10.10.10.2]:tcp:1358 Feb 27 11:03:37 VM_0_8_centos Keepalived_healthcheckers[30478]: Adding sorry server [192.168.200.200]:tcp:1358 to VS [10.10.10.2]:tcp:1358 Feb 27 11:03:37 VM_0_8_centos Keepalived_healthcheckers[30478]: Removing alive servers from the pool for VS [10.10.10.2]:tcp:1358 Feb 27 11:03:37 VM_0_8_centos Keepalived_healthcheckers[30478]: Remote SMTP server [192.168.200.1]:25 connected.