k8s集群搭建 – cuo的小窝

本文最后更新于135 天前，其中的信息可能已经过时，如有错误请发送邮件到2639131093@qq.com

虚拟机准备

操作系统

虚拟机	centos 7.9.2009 x86_64
Docker	v24.0.6
k8s	v1.20.9
kubeadm	v1.20

节点配置

主机	角色	ip
node1	master	192.32.62.30
node2	node1	192.32.62.31
node3	node2	192.32.62.32

环境准备

以下所有操作全节点进行，这里只展示主节点

设置静态ip

主机ip映射

分发给其余两台

配置SSH免密登录

创建公钥

分发

关闭Swap分区

kubelet要求必须禁用交换分区，所以kubeadm初始化时回检测swap是否关闭，如果没有关闭会报错，如果不想关闭安装时kubelet命令需要添加–fail-swap-on=false

临时关闭，重启恢复

swapoff -a

永久关闭

vi /etc/fstab

#/dev/mapper/centos-swap swap                    swap    defaults        0 0    ##注释此行

关闭防火墙

关闭selinux

重启

修改内核参数

modprobe br_netfilter     ##加载内核模块

echo "modprobe br_netfilter" >> /etc/profile     ##导入环境配置文件

tee /etc/sysctl.d/k8s.conf << EOF                     ##允许端口转发
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF

时间同步

node1：

服务开启以及验证

node2，node3，填写node1节点来时间同步

此处只展示node2

配置yum源

基础源

参考：centos7源配置 – cuo的小窝

docker源

参考：centos7源配置 – cuo的小窝

k8s源

参考：centos7源配置 – cuo的小窝

docker安装

三台节点都需要部署

移步: docker部署 – cuo的小窝

安装K8S集群

安装三大组件-kubeadm、kubelet、kubectl

三台节点都需要

kubeadm：用来初始化k8s集群的指令。
kubelet：在集群的每个节点上用来启动 Pod 和容器等。
kubectl：用来与k8s集群通信的命令行工具，查看、创建、更新和删除各种资源。

安装

yum install -y kubelet-1.20.9 kubeadm-1.20.9 kubectl-1.20.9

展示node1

所有设置开机自启

systemctl enable kubelet

初始化k8s集群

apiserver-advertise-address：apiserver监听地址
control-plane-endpoint：控制平面的IP地址或DNS名称
image-repository：镜像仓库，此处为国内阿里云镜像仓库加速下载
service-cidr：为Service分配的IP地址段
pod-network-cidr：为pod分配的IP地址段

##所有节点添加
echo "192.32.62.30  cluster-endpoint" >> /etc/hosts           ##master，我这里是node1
# 只在主节点执行
kubeadm init \
--apiserver-advertise-address=192.32.62.30 \
--control-plane-endpoint=cluster-endpoint \
--image-repository  registry.cn-hangzhou.aliyuncs.com/google_containers \
--kubernetes-version v1.20.9 \
--service-cidr=10.96.0.0/12 \
--pod-network-cidr=172.20.0.0/16

成功

使用非root用户运行kubectl

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

root用户

export KUBECONFIG=/etc/kubernetes/admin.conf

加入节点

上述初始化文件内

两个附节点输入：

  kubeadm join cluster-endpoint:6443 --token ckvqxx.z6dg46cxn5cjbmyg \
    --discovery-token-ca-cert-hash sha256:00003dda8692a4412e44a4b187969643d9db2cb72dabe3fb48686dd31798486b \

查看token列表，可观察到每个token的剩余有效时间

kubeadm token list

默认token有效期为24小时，过期之后token失效，可重新生成token：

kubeadm token create --print-join-command

修改角色：

在master上

kubectl label node node2 node-role.kubernetes.io/worker=worker
kubectl label node node3 node-role.kubernetes.io/worker=worker

查看节点

安装网络插件

进入

cd /etc/yum.repos.d

输入

curl https://docs.projectcalico.org/v3.15/manifests/calico.yaml -O

修改其配置

提交

kubectl apply -f calico.yaml

验证是否成功

kubectl get pod -A | grep calico

查看node

k8s集群测试

创建nginx pod

##默认会在默认命名空间default中创建一个名称为mynignx的deployment，同时会创建一个名称以myniginx为前缀，叫mynginx-5b686ccd46-wshz6的Pod
kubectl create deployment mynginx --image=nginx

# 查看pod
kubectl get pod,svc

对外暴露访问

基于第一步创建的deployment再创建一个名叫mynginx的Service，资源类型由–type=ClusterIP修改为–type=NodePort，会在每个Node节点上监听30161端口，用于接收集群外部访问

[root@node1 kubernetes]# kubectl expose deployment mynginx --port=80 --type=NodePort
service/mynginx exposed
[root@node1 kubernetes]# kubectl get pod,svc
NAME                           READY   STATUS    RESTARTS   AGE
pod/mynginx-5b686ccd46-vs224   1/1     Running   0          9m26s

NAME                 TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)        AGE
service/kubernetes   ClusterIP   10.96.0.1        <none>        443/TCP        17h
service/mynginx      NodePort    10.101.190.255   <none>        80:31440/TCP   17s

安装Dashboard

cd /etc/yum.repos.d/
vi dashboard.yaml

输入以下内容
# Copyright 2017 The Kubernetes Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
 
apiVersion: v1
kind: Namespace
metadata:
  name: kubernetes-dashboard
 
---
 
apiVersion: v1
kind: ServiceAccount
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard
 
---
 
kind: Service
apiVersion: v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard
spec:
  ports:
    - port: 443
      targetPort: 8443
  selector:
    k8s-app: kubernetes-dashboard
 
---
 
apiVersion: v1
kind: Secret
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard-certs
  namespace: kubernetes-dashboard
type: Opaque
 
---
 
apiVersion: v1
kind: Secret
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard-csrf
  namespace: kubernetes-dashboard
type: Opaque
data:
  csrf: ""
 
---
 
apiVersion: v1
kind: Secret
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard-key-holder
  namespace: kubernetes-dashboard
type: Opaque
 
---
 
kind: ConfigMap
apiVersion: v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard-settings
  namespace: kubernetes-dashboard
 
---
 
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard
rules:
  # Allow Dashboard to get, update and delete Dashboard exclusive secrets.
  - apiGroups: [""]
    resources: ["secrets"]
    resourceNames: ["kubernetes-dashboard-key-holder", "kubernetes-dashboard-certs", "kubernetes-dashboard-csrf"]
    verbs: ["get", "update", "delete"]
    # Allow Dashboard to get and update 'kubernetes-dashboard-settings' config map.
  - apiGroups: [""]
    resources: ["configmaps"]
    resourceNames: ["kubernetes-dashboard-settings"]
    verbs: ["get", "update"]
    # Allow Dashboard to get metrics.
  - apiGroups: [""]
    resources: ["services"]
    resourceNames: ["heapster", "dashboard-metrics-scraper"]
    verbs: ["proxy"]
  - apiGroups: [""]
    resources: ["services/proxy"]
    resourceNames: ["heapster", "http:heapster:", "https:heapster:", "dashboard-metrics-scraper", "http:dashboard-metrics-scraper"]
    verbs: ["get"]
 
---
 
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
rules:
  # Allow Metrics Scraper to get metrics from the Metrics server
  - apiGroups: ["metrics.k8s.io"]
    resources: ["pods", "nodes"]
    verbs: ["get", "list", "watch"]
 
---
 
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: kubernetes-dashboard
subjects:
  - kind: ServiceAccount
    name: kubernetes-dashboard
    namespace: kubernetes-dashboard
 
---
 
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: kubernetes-dashboard
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: kubernetes-dashboard
subjects:
  - kind: ServiceAccount
    name: kubernetes-dashboard
    namespace: kubernetes-dashboard
 
---
 
kind: Deployment
apiVersion: apps/v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard
spec:
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      k8s-app: kubernetes-dashboard
  template:
    metadata:
      labels:
        k8s-app: kubernetes-dashboard
    spec:
      containers:
        - name: kubernetes-dashboard
          image: kubernetesui/dashboard:v2.4.0
          imagePullPolicy: Always
          ports:
            - containerPort: 8443
              protocol: TCP
          args:
            - --auto-generate-certificates
            - --namespace=kubernetes-dashboard
            # Uncomment the following line to manually specify Kubernetes API server Host
            # If not specified, Dashboard will attempt to auto discover the API server and connect
            # to it. Uncomment only if the default does not work.
            # - --apiserver-host=http://my-address:port
          volumeMounts:
            - name: kubernetes-dashboard-certs
              mountPath: /certs
              # Create on-disk volume to store exec logs
            - mountPath: /tmp
              name: tmp-volume
          livenessProbe:
            httpGet:
              scheme: HTTPS
              path: /
              port: 8443
            initialDelaySeconds: 30
            timeoutSeconds: 30
          securityContext:
            allowPrivilegeEscalation: false
            readOnlyRootFilesystem: true
            runAsUser: 1001
            runAsGroup: 2001
      volumes:
        - name: kubernetes-dashboard-certs
          secret:
            secretName: kubernetes-dashboard-certs
        - name: tmp-volume
          emptyDir: {}
      serviceAccountName: kubernetes-dashboard
      nodeSelector:
        "kubernetes.io/os": linux
      # Comment the following tolerations if Dashboard must not be deployed on master
      tolerations:
        - key: node-role.kubernetes.io/master
          effect: NoSchedule
 
---
 
kind: Service
apiVersion: v1
metadata:
  labels:
    k8s-app: dashboard-metrics-scraper
  name: dashboard-metrics-scraper
  namespace: kubernetes-dashboard
spec:
  ports:
    - port: 8000
      targetPort: 8000
  selector:
    k8s-app: dashboard-metrics-scraper
 
---
 
kind: Deployment
apiVersion: apps/v1
metadata:
  labels:
    k8s-app: dashboard-metrics-scraper
  name: dashboard-metrics-scraper
  namespace: kubernetes-dashboard
spec:
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      k8s-app: dashboard-metrics-scraper
  template:
    metadata:
      labels:
        k8s-app: dashboard-metrics-scraper
    spec:
      securityContext:
        seccompProfile:
          type: RuntimeDefault
      containers:
        - name: dashboard-metrics-scraper
          image: kubernetesui/metrics-scraper:v1.0.7
          ports:
            - containerPort: 8000
              protocol: TCP
          livenessProbe:
            httpGet:
              scheme: HTTP
              path: /
              port: 8000
            initialDelaySeconds: 30
            timeoutSeconds: 30
          volumeMounts:
          - mountPath: /tmp
            name: tmp-volume
          securityContext:
            allowPrivilegeEscalation: false
            readOnlyRootFilesystem: true
            runAsUser: 1001
            runAsGroup: 2001
      serviceAccountName: kubernetes-dashboard
      nodeSelector:
        "kubernetes.io/os": linux
      # Comment the following tolerations if Dashboard must not be deployed on master
      tolerations:
        - key: node-role.kubernetes.io/master
          effect: NoSchedule
      volumes:
        - name: tmp-volume
          emptyDir: {}

执行：

kubectl apply -f dashboard.yaml

查看服务是否启动

kubectl get pod -n kubernetes-dashboard

配置访问的token

创建相关的sa，复制下面的内容，到rbac.yaml

vim rbac.yaml

输入：
apiVersion: v1
kind: ServiceAccount
metadata:
  name: dashboard-admin
  namespace: kube-system
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1      
metadata:
  name: dashboard-admin
subjects:
  - kind: ServiceAccount
    name: dashboard-admin
    namespace: kube-system
roleRef:
  kind: ClusterRole
  name: cluster-admin
  apiGroup: rbac.authorization.k8s.io

输入：

kubectl apply -f rbac.yaml

获取token

kubectl get secret -n kubernetes-dashboard
选择第一个admin的

输入：

kubectl describe secret admin的密钥 -n kubernetes-dashboard

对外端口暴露

由–type=ClusterIP修改为–type=NodePort

kubectl edit svc kubernetes-dashboard -n kubernetes-dashboard

获取对外端口

kubectl get svc -A|grep dashboard

输入https://master的ip+端口号

一般进去是此页面

点击高级然后键盘依次输入：

thisisunsafe

输入密钥

最后成功

cuo

社畜大二学生

虚拟机准备

操作系统

节点配置

环境准备

设置静态ip

主机ip映射

配置SSH免密登录

关闭Swap分区

关闭防火墙

关闭selinux

修改内核参数

时间同步

配置yum源

docker安装

安装K8S集群

安装三大组件-kubeadm、kubelet、kubectl

安装

初始化k8s集群

加入节点

安装网络插件

k8s集群测试

创建nginx pod

对外暴露访问

安装Dashboard

发送评论编辑评论

2025 年 4 月
一	二	三	四	五	六	日
	1	2	3	4	5	6
7	8	9	10	11	12	13
14	15	16	17	18	19	20
21	22	23	24	25	26	27
28	29	30

虚拟机准备

操作系统

节点配置

环境准备

设置静态ip

主机ip映射

配置SSH免密登录

关闭Swap分区

关闭防火墙

关闭selinux

修改内核参数

时间同步

配置yum源

docker安装

安装K8S集群

安装三大组件-kubeadm、kubelet、kubectl

安装

初始化k8s集群

加入节点

安装网络插件

k8s集群测试

创建nginx pod

对外暴露访问

安装Dashboard

发送评论 编辑评论

推荐文章

发送评论编辑评论