217, k8s 总章

1，安装 下载生成证书工具
wget https://pkg.cfssl.org/R1.2/cfssl_linux-amd64
wget https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64
wget https://pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64
chmod +x cfssl*
mv cfssl_linux-amd64 /usr/local/bin/cfssl
mv cfssljson_linux-amd64 /usr/local/bin/cfssljson
mv cfssl-certinfo_linux-amd64 /usr/bin/cfssl-certinfo
2，生成证书：
执行：./etcd-cert.sh
tar xf etcd-v3.3.10-linux-amd64.tar.gz  && cd  etcd-v3.3.10-linux-amd64
cp etcd etcdctl /opt/etcd/bin
cp ca.pem  server*pem /opt/etcd/ssl/
./etcd.sh  etcd01 192.168.132.166 etcd02=https://192.168.132.167:2380,etcd03=https://192.168.132.168:2380
//黄色的是本机IP，执行脚本的这个机器IP。
systemctl start etcd  (启动前先改下各自的/opt/etcd/cfg/etcd 这个文件的IP)
完成etcd启动！
3，
scp -r  /opt/etcd/*   :/opt/etcd/
scp -r  /opt/etcd/*   :/opt/etcd/
 [ opt]# scp /usr/lib/systemd/system/etcd.service :/usr/lib/systemd/system/
到这etcd 集群搭建完成！！（直接scp完，修改cfg/etcd 文件的IP 就可以直接启动）

二：
node安装docker
yum install -y yum-utils device-mapper-persistent-data lvm2
yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
yum makecache fast
yum -y install docker-ce
systemctl start docker

node 部署flanneld:
写入分配的子网段到etcd，供flanneld使用。（这句在master端 操作一次就可以了）
/opt/etcd/bin/etcdctl --ca-file=/opt/etcd/ssl/ca.pem --cert-file=/opt/etcd/ssl/server.pem --key-file=/opt/etcd/ssl/server-key.pem --endpoints="https://192.168.132.166:2379,https://192.168.132.167:2379,https://192.168.132.168:2379" set /coreos.com/network/config ‘{ "Network": "172.17.0.0/16", "Backend": {"Type": "vxlan"}}‘
//endpoints 这个为etcd集群的地址

tar xf flannel-v0.10.0-linux-amd64.tar.gz
mkdir -p /opt/kubernetes/{ssl,bin,cfg}
scp flanneld mk-docker-opts.sh  /opt/kubernetes/bin

cat <<EOF >/opt/kubernetes/cfg/flanneld

FLANNEL_OPTIONS="--etcd-endpoints=${ETCD_ENDPOINTS} \
-etcd-cafile=/opt/etcd/ssl/ca.pem -etcd-certfile=/opt/etcd/ssl/server.pem -etcd-keyfile=/opt/etcd/ssl/server-key.pem"

EOF

红色部分改为如上截图
cat <<EOF >/usr/lib/systemd/system/flanneld.service
[Unit]
Description=Flanneld overlay address etcd agent
After=network-online.target network.target
Before=docker.service

[Service]
Type=notify
EnvironmentFile=/opt/kubernetes/cfg/flanneld
ExecStart=/opt/kubernetes/bin/flanneld --ip-masq \$FLANNEL_OPTIONS
ExecStartPost=/opt/kubernetes/bin/mk-docker-opts.sh -k DOCKER_NETWORK_OPTIONS -d /run/flannel/subnet.env
Restart=on-failure

[Install]
WantedBy=multi-user.target

EOF

mkdir /opt/etcd/ssl -p
从master端：
scp ca.pem  server*pem  /opt/etcd/ssl

systemctl start flanneld

在修改docker的service 如下：
EnvironmentFile=/run/flannel/subnet.env
ExecStart=/usr/bin/dockerd $DOCKER_NETWORK_OPTIONS

systemctl daemon-relaod
systemctl restart docker

三：
Master 安装APIserver：
    kubernetes-server-linux-amd64.tar.gz
    master.zip
unzip master.zip
tar xf  kubernetes-server-linux-amd64.tar.gz
[ bin]# pwd
/root/kubernetes/server/bin
mkdir -p /opt/kubernetes/{bin,ssl,cfg}
# cp kube-apiserver kubectl kube-controller-manager kube-scheduler /opt/kubernetes/bin/
创建k8s-cert 证书：
# cp ca.pem server*.pem /opt/kubernetes/ssl/
[ ~]# ./apiserver.sh 192.168.132.166 https://192.168.132.166:2379,https://192.168.132.167:2379,https://192.168.132.166:2379
[ ~]# head -c 16 /dev/urandom |od -An -t x|tr -d " "
c119666bfb3c18c8da8130a0a1c707a2
[ ~]# vim  /opt/kubernetes/cfg/token.csv
c119666bfb3c18c8da8130a0a1c707a2,kubelet-bootstrap,10001,system:kubelet-bootstrap
[ ~]#
[ ssl]# cp ca-key.pem  /opt/kubernetes/ssl/
[ ~]# systemctl start kube-apiserver


Master  controller-manager安装
[ ~]# ./controller-manager.sh 127.0.0.1

Matser  scheduler安装：
[ ~]# ./ scheduler.sh 127.0.0.1

Node 节点的安装：（必须关闭swap）
1,先拷贝node包到node节点
2,在scp  soft/kubernets/server/bin/kubelet  kube-proxy  :/opt/kubernetes/bin/
2,在master生成kubeconfig
# ./kubeconfig.sh 192.168.132.176  /root/k8s-cret/   //备注：176为本机IP
scp bootstrap.kubeconfig  kube-proxy.kubeconfig  node节点的 /opt/kubernetes/cfg/
# ./kubelet.sh 192.168.132.179 10.0.0.2  //备注：192.168.132.179为你当前节点的IP   10.10.10.2为你的DNS地址
查看kubelete是否启动
发现有错误日志,创建证权限拒绝
error: failed to run Kubelet: cannot create certificate signing request: certificatesigningrequests.certificates.k8s.io is forbidden: User "kubelet-bootstrap" cannot create certificatesigningrequests.certificates.k8s.io at the cluster scope: clusterrole.rbac.authorization.k8s.io "system:node-bootstrap" not found
解决方法
在master端创建权限分配角色
[ ssl]# kubectl create clusterrolebinding kubelet-bootstrap --clusterrole=system:node-bootstrapper --user=kubelet-bootstrap

# ./proxy.sh 192.168.132.179   //179为当前节点的名字
[ bin]# kubectl get csr
NAME                                                   AGE   REQUESTOR           CONDITION
node-csr-BYEi6huAGgEwODnaAWr6DacXHUkZZgikJae6P9KoVu8   11m   kubelet-bootstrap   Pending
[ bin]#
[ bin]# kubectl certificate approve node-csr-BYEi6huAGgEwODnaAWr6DacXHUkZZgikJae6P9KoVu8
certificatesigningrequest.certificates.k8s.io/node-csr-BYEi6huAGgEwODnaAWr6DacXHUkZZgikJae6P9KoVu8 approved
[ bin]#
[ bin]# kubectl get node
NAME              STATUS   ROLES    AGE   VERSION
192.168.132.179   Ready    <none>   41s   v1.12.1
[ bin]#
到此单节点的master----node部署完成。

所以此时执行 kubelet get node 是没有节点的。

2，现在将另外一个node 添加到集群。

[ cfg]# rm -f kubelet.kubeconfig 
[ cfg]# pwd
/opt/kubernetes/cfg
# vim kube-proxy   //备注：修改这个文件的IP为本机node的IP
# vim kubelet  //备注：修改这个文件的IP为本机node的IP
 
[ cfg]# cd /opt/kubernetes/ssl/
[ ssl]# 
[ ssl]# ll
总用量 16
-rw-------. 1 root root 1277 10月 30 22:43 kubelet-client-2018-10-30-22-18-34.pem
-rw-------. 1 root root 1277 10月 30 22:43 kubelet-client-current.pem
-rw-r--r--. 1 root root 2197 10月 30 22:43 kubelet.crt
-rw-------. 1 root root 1675 10月 30 22:43 kubelet.key
[ ssl]# rm -f *
[ ssl]# ll
总用量 0
[ ssl]#
[ ~]# scp /usr/lib/systemd/system/{kubelet,kube-proxy}.service  192.168.132.178:/usr/lib/systemd/system/
systemctl start kubelet
systemctl start kube-proxy

添加node2完成，现在的架构变成了，一个master，2个node 的架构。Master-node-node.