K8S 快速部署（单master）（01）
本文适用于有linux 基础，网络基础，安全基础的童鞋阅读。
1、规划
1.1架构规划
1.2 服务器配置规划
1.3 软件版本规划
1.4 证书规划
2、安装过程
2.1、关闭关闭所有服务器的selinux和防火墙
systemctl stop firewalld.service
systemctl disable firewalld.service
vi /etc/selinux/config
2.2 修改主机名
vi /etc/hostname
reboot 重启所有服务器3、生成证书
3.1创建目录
mkdir k8s-cert
mkdir etcd-cert
cd etcd-cert/
3.2安装cfssl
wget https://tenthpinnemo.s3.cn-northwest-1.amazonaws.com.cn/Deploy/cfssl.sh
chmod +x cfssl.sh
./cfssl.sh
3.3生成证书
wget https://tenthpinnemo.s3.cn-northwest-1.amazonaws.com.cn/Deploy/etcd-cert.sh
chmod +x etcd-cert.sh
vi etcd-cert.sh
根据实际情况替换为自己规划的IP
./etcd-cert.sh
4、安装etcd
4.1创建软件安装包存放目录并下载软件
mkdir soft
cd soft/
wget https://tenthpinnemo.s3.cn-northwest-1.amazonaws.com.cn/Soft/etcd-v3.3.10-linux-amd64.tar.gz
tar zxvf etcd-v3.3.10-linux-amd64.tar.gz
cd etcd-v3.3.10-linux-amd64/
mkdir /opt/etcd/{cfg,bin,ssl} -p
mv etcd etcdctl /opt/etcd/bin/
4.2下载安装脚本并执行安装
wget https://tenthpinnemo.s3.cn-northwest-1.amazonaws.com.cn/Deploy/etcd.sh
chmod +x etcd.sh
根据实际情况替换IP
./etcd.sh etcd01 192.168.3.200 etcd02=https://192.168.3.202:2380,etcd03=https://192.168.3.203:2380
4.3拷贝证书
cp /data/etcd-cert/{ca,server-key,server}.pem /opt/etcd/ssl/
4.4拷贝配置到其他节点
拷贝etcd.service
4.5登录其它节点修改配置
cd /opt/etcd/cfg/
vi etcd
4.6 在所有节点启动etcd
systemctl daemon-reload
systemctl start etcd
systemctl enable etcd
4.7 验证集群状态，IP替换为实际IP
/opt/etcd/bin/etcdctl --ca-file=/opt/etcd/ssl/ca.pem --cert-file=/opt/etcd/ssl/server.pem --key-file=/opt/etcd/ssl/server-key.pem --endpoints="https://192.168.3.200:2379,https://192.168.3.202:2379,https://192.168.3.203:2379" cluster-health
5、在node节点安装docker
5.1安装必要的一些系统工具
yum install -y yum-utils device-mapper-persistent-data lvm2
5.2添加软件源信息
yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
5.3更新并安装 Docker-CE
yum makecache fast
yum -y install docker-ce
5.4配置 daocloud docker 加速器
curl -sSL https://get.daocloud.io/daotools/set_mirror.sh | sh -s http://f1361db2.m.daocloud.io
5.5重启docker 服务
systemctl restart docker
6、在node节点安装flanneld网络插件
6.1分配容器网络，讲数据写入etcd，IP根据实际情况修改，172.16.0.0/16是容器网络。
/opt/etcd/bin/etcdctl --ca-file=/opt/etcd/ssl/ca.pem --cert-file=/opt/etcd/ssl/server.pem --key-file=/opt/etcd/ssl/server-key.pem --endpoints="https://192.168.3.200:2379,https://192.168.3.202:2379,https://192.168.3.203:2379" set /coreos.com/network/config '{ "Network": "172.16.0.0/16", "Backend": {"Type": "vxlan"}}'
查看配置
/opt/etcd/bin/etcdctl --ca-file=/opt/etcd/ssl/ca.pem --cert-file=/opt/etcd/ssl/server.pem --key-file=/opt/etcd/ssl/server-key.pem --endpoints="https://192.168.3.200:2379,https://192.168.3.202:2379,https://192.168.3.203:2379" get /coreos.com/network/config
6.2 下载flanneld网络插件并执行安装
wget https://tenthpinnemo.s3.cn-northwest-1.amazonaws.com.cn/Soft/flannel-v0.10.0-linux-amd64.tar.gz
解压
tar -zxvf flannel-v0.10.0-linux-amd64.tar.gz
下载安装脚本
wget https://tenthpinnemo.s3.cn-northwest-1.amazonaws.com.cn/Deploy/flannel.sh
创建工作目录
mkdir /opt/kubernetes/{bin,cfg,ssl} -p
移动安装文件
mv flanneld mk-docker-opts.sh /opt/kubernetes/bin/
执行安装
./flannel.sh https://192.168.3.200:2379,https://192.168.3.202:2379,https://192.168.3.203:2379
重启docker
systemctl restart docker
查看docker0是否获取flannel分配的IP
ifconfig
7、在master01 节点部署apiserver
7.1下载安装包
wget https://tenthpinnemo.s3.cn-northwest-1.amazonaws.com.cn/Soft/kubernetes-server-linux-amd64.tar.gz
wget https://tenthpinnemo.s3.cn-northwest-1.amazonaws.com.cn/Deploy/master/apiserver.sh
wget https://tenthpinnemo.s3.cn-northwest-1.amazonaws.com.cn/Deploy/master/controller-manager.sh
wget https://tenthpinnemo.s3.cn-northwest-1.amazonaws.com.cn/Deploy/master/scheduler.sh
解压
tar -zxvf kubernetes-server-linux-amd64.tar.gz
7.2安装过程
创建安装目录
mkdir -p /opt/kubernetes/{bin,cfg,ssl} -p
进入解压目录
cd kubernetes/server/bin/
拷贝
cp kube-apiserver kube-controller-manager kube-scheduler /opt/kubernetes/bin/
执行安装
chmod + apiserver.sh
./apiserver.sh 192.168.3.200 https://192.168.3.200:2379,https://192.168.3.202:2379,https://192.168.3.203:2379
指定日志目录
mkdir /data/logs
vi /opt/kubernetes/cfg/kube-apiserver
将true替换为false
KUBE_APISERVER_OPTS="--logtostderr=true \
替换为
KUBE_APISERVER_OPTS="--logtostderr=false \
插入一行
--log-dir= /data/logs \
7.3 为k8s生成证书
cd /data/k8s-cert/
wget https://tenthpinnemo.s3.cn-northwest-1.amazonaws.com.cn/Deploy/k8s-cert.sh
chmod +x k8s-cert.sh
执行脚本
./k8s-cert.sh
拷贝证书
cp ca.pem server.pem server-key.pem ca-key.pem /opt/kubernetes/ssl/
7.4生产token文件
wget https://tenthpinnemo.s3.cn-northwest-1.amazonaws.com.cn/Deploy/kubeconfig.sh
chmod +x kubeconfig.sh
执行生产csv
./kubeconfig.sh
移动配置文件
mv token.csv /opt/kubernetes/cfg/
7.5、启动apiserver
systemctl start kube-apiserver
systemctl enable kube-apiserver
8、在master01上安装kube-controller-manager
chmod +x controller-manager.sh
./controller-manager.sh 127.0.0.1
systemctl enable kube-controller-manger
按照apiserver配置方法修改日志存放路径
vi /opt/kubernetes/cfg/kube-controller-manager
9、在master01上安装kube-scheduler
chmod +x scheduler.sh
./scheduler.sh 127.0.0.1
systemctl enable kube-controller-manger
按照apiserver配置方法修改日志存放路径
10、使用kubectl管理工具
cp kubernetes/server/bin/kubectl /usr/bin/
查看集群状态
kubectl get cs
11、部署k8s node
11.1将kubelet-bootstrap用户绑定到系统集群角色
kubectl create clusterrolebinding kubelet-bootstrap \
--clusterrole=system:node-bootstrapper \
--user=kubelet-bootstrap
11.2创建kubeconfig文件
下载配置脚本
wget https://tenthpinnemo.s3.cn-northwest-1.amazonaws.com.cn/Deploy/kubeconfig.sh
查看token
cat /opt/kubernetes/cfg/token.csv
注释掉token创建章节
vi kubeconfig.sh
# 创建 TLS Bootstrapping Token
##BOOTSTRAP_TOKEN=$(head -c 16 /dev/urandom | od -An -t x | tr -d ' ')
#BOOTSTRAP_TOKEN=0fb61c46f8991b718eb38d27b605b008
#cat > token.csv <<EOF
#${BOOTSTRAP_TOKEN},kubelet-bootstrap,10001,"system:kubelet-bootstrap"
#EOF
添加token信息
#----------------------
APISERVER=$1
SSL_DIR=$2
BOOTSTRAP_TOKEN=0fb61c46f8991b718eb38d27b605b008
执行
chmod +x kubeconfig.sh
./kubeconfig.sh 192.168.3.200 /data/k8s-cert/
11.3拷贝配置文件到node节点
scp bootstrap.kubeconfig kube-proxy.kubeconfig root@192.168.3.202:/opt/kubernetes/cfg/
scp bootstrap.kubeconfig kube-proxy.kubeconfig root@192.168.3.203:/opt/kubernetes/cfg/
cd /data/kubernetes/server/bin
scp kubelet kube-proxy root@192.168.3.202:/opt/kubernetes/bin/
scp kubelet kube-proxy root@192.168.3.203:/opt/kubernetes/bin/
11.4 下载node节点安装包
wget https://tenthpinnemo.s3.cn-northwest-1.amazonaws.com.cn/Deploy/node/kubelet.sh
wget https://tenthpinnemo.s3.cn-northwest-1.amazonaws.com.cn/Deploy/node/proxy.sh
添加执行权限
chmod +x kubelet.sh
chmod +x proxy.sh
分别在各自节点执行，如在192.168.3.202执行
./kubelet.sh 192.168.3.202
vi /opt/kubernetes/cfg/kubelet.config
在末尾添加
authentication:
anonymous:
enabled: true
绑定角色
11.5在master允许node加入集群neme为每个节点的具体值
kubectl get csr
kubectl certificate approve +name
绑定角色
kubectl create clusterrolebinding cluster-system-anonymous --clusterrole=cluster-admin --user=system:anonymous
查看节点
kubectl get node
12、安装kube-proxy，分别在各个节点执行，如在192.168.3.202执行
./proxy.sh 192.168.3.202
systemctl enable kube-proxy
13、node扩容
13.1拷贝配置文件
scp -r /opt/kubernetes/ root@目标IP：/opt/
scp /usr/lib/systemd/system/{kubelet,kube-proxy}.service root@目标IP：/usr/lib/systemd/system/
13.2 进入节点ssl目录删除文件
cd /opt/kubernetes/ssl/
rm -rf *
13.3修改IP
cd /opt/kubernetes/cfg/
修改
kubelet，kubelet.config，kube-proxy
为当前节点IP
启动
systemctl restart kubelet
systemctl restart kube-proxy
同上在msater颁发证书
14测试
14.1 登陆master创建一个nginx
kubectl run nginx --image=nginx
查看镜像
kubectl get pods
14.2创建地址映射
kubectl expose deployment nginx --port=80 --target-port=80 --type=NodePort
14.3 查看容易所在节点
kubectl get pods -o wide
14.4查看映射的端口，80右侧的端口即为外部访问端口
kubectl get svc
14.5通过任意node ip + 外部端口访问nginx
14.6查看日志
kubectl get pods
15部署UI
15.1进入安装目录解压安装包
cd /data/kubernetes/
tar -zxvf kubernetes-src.tar.gz
cd cluster/
cd addons/dashboard/
kubectl create -f dashboard-configmap.yaml
kubectl create -f dashboard-rbac.yaml
kubectl create -f dashboard-secret.yaml
修改iamge镜像源vi dashboard-controller.yaml
registry.cn-hangzhou.aliyuncs.com/google_containers/kubernetes-dashboard-amd64:v1.10.0
创建
kubectl create -f dashboard-controller.yaml
查看
kubectl get pods -n kube-system
查看日志红色部分根据实际情况替换
kubectl logs kubernetes-dashboard-6bff7dc67d-6gw78 -n kube-system
15.2 创建service
vi dashboard-service.yaml
spec:
type: NodePort
kubectl create -f dashboard-service.yaml
查看端口信息
kubectl get svc -n kube-system
15.3 配置token登陆信息
vi k8s-admin.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: dashboard-admin
namespace: kube-system
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: dashboard-admin
subjects:
- kind: ServiceAccount
name: dashboard-admin
namespace: kube-system
roleRef:
kind: ClusterRole
name: cluster-admin
apiGroup: rbac.authorization.k8s.io
创建
kubectl -f k8s-admin.yaml
15.4 获取name
kubectl get secret -n kube-system
获取密钥name 以实际情况为准
kubectl describe secret dashboard-admin-token-9rh2s -n kube-system
15.5 访问任意 https:// node ip +端口
使用token值登陆