kubernetes(十三) k8s 业务上线流程（手动版）

• 备注： 用到的源代码可以联系QQ：122725501 索取即可

传统部署与k8s部署的区别

• 传统部署

• k8s 部署架构

• 项目迁移到k8s的流程

制作镜像

• 镜像分类
  • 基础镜像
  • 环境镜像
  • 项目镜像

控制器管理POD

• Deployment：无状态部署，例如Web，微服务，API
• StatefulSet：有状态部署，例如数据库，ZK，ETCD
• DaemonSet：守护进程部署，例如监控Agent、日志Agent
• Job & CronJob：批处理，例如数据库备份，邮件通知

Pod数据持久化

• 容器部署过程中一般有如下的三种数据
  • 启动时需要的初始数据，可以是配置文件
  • 启动过程中产生的初始化数据，该临时数据需要多个容器间共享
  • 启动过程中产生的业务数据

暴露应用

使用Service ClusterIP类型暴露集群内部的应用访问

• service定义了Pod逻辑集合和访问这个集合的策略
• service引入为了解决Pod的动态变化，提供服务发现和负载均衡
• 使用coreDNS解析Service名称

对外发布应用

• 使用ingress对外暴露应用
  • 通过Service关联Pod
  • 基于域名访问
  • 通过Ingress Controller实现pod的负载均衡
  • 支持TCP/UDP四层和Http七层

部署Java/PHP项目

部署Java项目

• 构建环境镜像

$ mkdir ~/base_env/ && cd ~/base_env
$ vim Dockerfile
FROM centos:7
LABEL maintainer=""
ENV tomcat_version="8.5.57"

• 代码提交至代码仓库： http://192.168.56.19:9999/root/java-demo.git （如何部署代码仓库可以联系QQ 122725501协助：）

• 代码编译

  • git clone http://192.168.56.19:9999/root/java-demo.git

  • 编译机安装好编译环境（yum install java-1.8.0-openjdk maven -y）

  • 修改代码的maven源

  $ vi /etc/maven/settings.xml
    <mirrors>
         <mirror>
           <id>central</id>
           <mirrorOf>central</mirrorOf>
           <name>aliyun maven</name>
           <url>https://maven.aliyun.com/repository/public</url>
         </mirror>
     </mirrors>
  $ cd java-demo
  $ mvn clean package -D maven.test.skip=true

• 构建镜像并推送到镜像仓库

$  vim Dockerfile 
FROM hub.cropy.cn/base/tomcat:v1
LABEL maintainer=""
RUN rm -rf /usr/local/tomcat/webapps/*
COPY target/*.war /usr/local/tomcat/webapps/ROOT.war

$ docker build -t hub.cropy.cn/demo/java-demo:v1 .
$ docker push hub.cropy.cn/demo/java-demo:v1

• k8s服务编排部署项目

  • 创建命名空间，资源隔离，权限控制

  $ vim namespace.yaml 
  apiVersion: v1
  kind: Namespace
  metadata:
  name: test
  $ kubectl apply -f namespace.yaml

  • 使用deploy控制器创建Pod

  # 需要创建hub仓库的secret
  $ kubectl create secret docker-registry docker-regsitry-auth --docker-server="hub.cropy.cn" --docker-username="admin" --docker-password="Harbor12345" -n test
  $ vim deployment.yaml 
  apiVersion: apps/v1
  kind: Deployment
  metadata:
  name: java-demo
  namespace: test
  spec:
  replicas: 2
  selector:
    matchLabels:
      project: www
      app: java-demo
  template:
    metadata:
      labels:
        project: www
        app: java-demo
    spec:
      imagePullSecrets:
      - name: "docker-regsitry-auth"
      containers:
      - image: hub.cropy.cn/demo/java-demo:v1
        name: java-demo
        imagePullPolicy: Always
        ports:
        - containerPort: 8080
          name: web
          protocol: TCP
        resources:
          requests:
            cpu: 0.5
            memory: 0.5Gi
          limits:
            cpu: 1
            memory: 2Gi
        livenessProbe:
          httpGet:
            path: /
            port: 8080
          initialDelaySeconds: 60
          timeoutSeconds: 20
        readinessProbe:
          httpGet:
            path: /
            port: 8080
          initialDelaySeconds: 60
          timeoutSeconds: 20
  $ kubectl apply -f deployment.yaml     
  $ kubectl get pod -n test -o wide

  • 使用service暴露应用

  $ vim service.yaml 
  apiVersion: v1
  kind: Service
  metadata:
  labels:
    app: java-demo
  name: java-demo
  namespace: test
  spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 8080
    nodePort: 30018
  selector:
    app: java-demo
    project: www
  type: NodePort
  
  $ kubectl apply -f service.yaml
  $ kubectl get ep -n test

  node节点访问测试

  • 使用ingress对外暴露

  $ vim ingress.yaml 
  apiVersion: extensions/v1beta1
  kind: Ingress
  metadata:
  name: java-demo
  namespace: test
  spec:
  rules:
  - host: java.cropy.cn
    http:
      paths:
      - path: /
        backend:
          serviceName: java-demo
          servicePort: 80
  $ kubectl apply -f ingress.yaml 
  $ kubectl  get ing -n test

  外网用户加入k8s任意集群节点的ip:java.cropy.cn 到hosts完成解析，就可以使用域名访问了

• 外部环境和中间件准备（当前准备mysql）

$ helm install java-demo-db --set persistence.storageClass="managed-nfs-storage" stable/mysql
$ MYSQL_ROOT_PASSWORD=$(kubectl get secret --namespace default java-demo-db-mysql -o jsonpath="{.data.mysql-root-password}" | base64 --decode; echo)
$ echo $MYSQL_ROOT_PASSWORD    #获取数据库密码
$ mysql -h java-demo-db-mysql -p     #-h 这里的host是mysql的svc
E2szJIvuhI
$ kubectl get pods
NAME                                      READY   STATUS    RESTARTS   AGE
java-demo-db-mysql-745b9688b8-7f5fn       1/1     Running   0          93s
$ kubectl cp db/tables_ly_tomcat.sql java-demo-db-mysql-745b9688b8-7f5fn:/  #拷贝sql文件到数据库容器,然后导入数据
$ kubectl exec -it java-demo-db-mysql-745b9688b8-7f5fn -- bash
:/# mysql -uroot -pE2szJIvuhI  
mysql> source /tables_ly_tomcat.sql;

• 修改源代码中的配置文件

$ vim src/main/resources/application.yml 
server:
  port: 8080
spring:
  datasource:
    url: jdbc:mysql://java-demo-db-mysql.default:3306/test?characterEncoding=utf-8
    username: root
    password: E2szJIvuhI
    driver-class-name: com.mysql.jdbc.Driver
  freemarker:
    allow-request-override: false
    cache: true
    check-template-location: true
    charset: UTF-8
    content-type: text/html; charset=utf-8
    expose-request-attributes: false
    expose-session-attributes: false
    expose-spring-macro-helpers: false
    suffix: .ftl
    template-loader-path:
      - classpath:/templates/
$ mvn clean package -D maven.test.skip=true     #重新打包
$ docker build -t hub.cropy.cn/demo/java-demo:v2 . #重新构建
$ docker push hub.cropy.cn/demo/java-demo:v2     #镜像上传

• 重新发布和回滚

$ vim k8s-yaml/deployment.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  name: java-demo
  namespace: test
spec:
  replicas: 3
  selector:
    matchLabels:
      project: www
      app: java-demo
  template:
    metadata:
      labels:
        project: www
        app: java-demo
    spec:
      imagePullSecrets:
      - name: "docker-regsitry-auth"
      containers:
      - image: hub.cropy.cn/demo/java-demo:v2
        name: java-demo
        imagePullPolicy: Always
        ports:
        - containerPort: 8080
          name: web
          protocol: TCP
        resources:
          requests:
            cpu: 0.5
            memory: 0.5Gi
          limits:
            cpu: 1
            memory: 2Gi
        livenessProbe:
          httpGet:
            path: /
            port: 8080
          initialDelaySeconds: 60
          timeoutSeconds: 20
        readinessProbe:
          httpGet:
            path: /
            port: 8080
          initialDelaySeconds: 60
          timeoutSeconds: 20
$ kubectl apply -f k8s-yaml/deployment.yaml
$ kubectl rollout history deploy/java-demo -n test    #查看历史
$ kubectl rollout undo deploy/java-demo -n test       #回滚至上一版本

如果项目不能正常写入数据，可以到myqsl添加授权即可

授权root有所有远程权限：

grant all on java.* to ‘root‘@‘%‘ identified by ‘E2szJIvuhI‘;

• 测试

加入客户端主机名解析，然后浏览器直接访问即可

192.168.56.14 java.cropy.cn

部署php项目

• 下载代码

$ git clone http://192.168.56.19:9999/root/php-demo.git

• 构建镜像

$ vim Dockerfile
FROM lizhenliang/nginx-php:latest
LABEL maintainer="php.cropy.cn"
ADD . /usr/local/nginx/html 
$ docker build -t hub.cropy.cn/demo/php-demo:v1 .
$ docker push hub.cropy.cn/demo/php-demo:v1

• k8s部署编排配置

  • 创建命名空间，资源隔离，权限控制

  $ vim namespace.yaml 
  apiVersion: v1
  kind: Namespace
  metadata:
  name: test
  $ kubectl apply -f namespace.yaml

  • 使用deploy控制器创建Pod

  $ vim deployment.yaml 
  apiVersion: apps/v1
  kind: Deployment
  metadata:
  name: php-demo
  namespace: test
  spec:
  replicas: 3
  selector:
    matchLabels:
      project: www
      app: php-demo
  template:
    metadata:
      labels:
        project: www
        app: php-demo
    spec:
      imagePullSecrets:
      - name: docker-regsitry-auth
      containers:
      - name: nginx
        image: hub.cropy.cn/demo/php-demo:v1
        ports:
        - containerPort: 80
          name: web
          protocol: TCP
        resources:
          requests:
            cpu: 0.5
            memory: 256Mi
          limits:
            cpu: 1
            memory: 1Gi
        livenessProbe:
          httpGet:
            path: /status.php
            port: 80
          initialDelaySeconds: 6
          timeoutSeconds: 20
        readinessProbe:
          httpGet:
            path: /status.php
            port: 80
          initialDelaySeconds: 6
          timeoutSeconds: 20
  $ kubectl apply -f deployment.yaml     
  $ kubectl get pod -n test -o wide

  • 使用service暴露应用

  $ vim service.yaml 
  apiVersion: v1
  kind: Service
  metadata:
  name: php-demo
  namespace: test
  spec:
  selector:
    project: www
    app: php-demo
  ports:
  - name: web
    port: 80
    targetPort: 80
  $ kubectl apply -f service.yaml
  $ kubectl get ep -n test

  node节点访问测试

  • 使用ingress对外暴露

  $ vim ingress.yaml 
  apiVersion: extensions/v1beta1
  kind: Ingress
  metadata:
  name: php-demo
  namespace: test
  spec:
  rules:
    - host: php.cropy.cn
      http:
        paths:
        - path: /
          backend:
            serviceName: php-demo
            servicePort: 80
  $ kubectl apply -f ingress.yaml 
  $ kubectl  get ing -n test

  上述流程就是上线php项目的流程

  • 外部环境和中间件准备（当前准备mysql）

  $ helm install php-demo-db --set persistence.storageClass="managed-nfs-storage" stable/mysql    
  $ MYSQL_ROOT_PASSWORD=$(kubectl get secret --namespace default php-demo-db-mysql -o jsonpath="{.data.mysql-root-password}" | base64 --decode; echo)
  $ echo $MYSQL_ROOT_PASSWORD
  cHDi5ikYAj
  $ kubectl get pod
  NAME                                      READY   STATUS    RESTARTS   AGE
  nfs-client-provisioner-75f9fd7b57-67jtk   1/1     Running   1          4d7h
  php-demo-db-mysql-5c7fb76866-wn8rd        1/1     Running   0          115s
  $ kubectl exec -it php-demo-db-mysql-5c7fb76866-wn8rd -- bash
  :/# mysql -uroot -pcHDi5ikYAj
  mysql> create database wp;

  • 修改代码配置文件，重新打包

  $ vim wp-config.php
  define(‘DB_NAME‘, ‘wp‘);
  
  /** MySQL数据库用户名 */
  define(‘DB_USER‘, ‘root‘);
  
  /** MySQL数据库密码 */
  define(‘DB_PASSWORD‘, ‘cHDi5ikYAj‘);
  
  /** MySQL主机 */
  define(‘DB_HOST‘, ‘php-demo-db-mysql.default‘);
  
  $ docker build -t hub.cropy.cn/demo/php-demo:v2 .
  $ docker push hub.cropy.cn/demo/php-demo:v2 .

  • 更新&回滚

  $ vim deployment.yaml   #更新到v2版本
  apiVersion: apps/v1
  kind: Deployment
  metadata:
  name: php-demo
  namespace: test
  spec:
  replicas: 3
  selector:
    matchLabels:
      project: www
      app: php-demo
  template:
    metadata:
      labels:
        project: www
        app: php-demo
    spec:
      imagePullSecrets:
      - name: docker-regsitry-auth
      containers:
      - name: nginx
        image: hub.cropy.cn/demo/php-demo:v2
        ports:
        - containerPort: 80
          name: web
          protocol: TCP
        resources:
          requests:
            cpu: 0.5
            memory: 256Mi
          limits:
            cpu: 1
            memory: 1Gi
        livenessProbe:
          httpGet:
            path: /status.php
            port: 80
          initialDelaySeconds: 6
          timeoutSeconds: 20
        readinessProbe:
          httpGet:
            path: /status.php
            port: 80
          initialDelaySeconds: 6
          timeoutSeconds: 20
  $ kubectl apply -f deployment.yaml

  • 测试

  加入客户端主机名解析，然后浏览器直接访问即可
  
  192.168.56.14 php.cropy.cn