Kubernetes完全指南：从集群搭建到生产部署

前言

当Docker容器数量从10个增长到1000个时，手动管理就变成了噩梦。我们迫切需要一个容器编排平台。

选择了Kubernetes后，我们实现了自动扩展、自我修复和灰度发布。这篇文章总结了我们的K8s实践。

一、为什么需要Kubernetes？

1.1 容器编排的需求

手动管理的痛点：❌ 容器宕机无法自动重启❌ 无法自动扩展应对流量高峰❌ 无法实现零停机更新❌ 资源分配不均❌ 网络通信复杂❌ 存储管理困难Kubernetes解决方案：✅ 自动恢复：容器故障自动重启✅ 自动扩展：根据资源自动伸缩✅ 灰度发布：零停机更新✅ 负载均衡：智能调度✅ 服务发现：自动服务注册✅ 存储编排：多种存储支持

1.2 Kubernetes架构

┌─────────────────────────────────────────┐│ Kubernetes Cluster │├─────────────────────────────────────────┤│ Master Node (Control Plane) ││ ├─ API Server ││ ├─ Scheduler ││ ├─ Controller Manager ││ └─ etcd (键值存储) │├─────────────────────────────────────────┤│ Worker Node 1 │ Worker Node 2 ││ ├─ Kubelet │ ├─ Kubelet ││ ├─ Container Runtime│ ├─ Container Runtime│ └─ Pods │ └─ Pods │└─────────────────────────────────────────┘

二、本地Kubernetes开发环境

2.1 安装Minikube

bash

# macOS安装Minikubebrew install minikube# 启动集群minikube start --cpus=4 --memory=8192# 查看集群状态kubectl cluster-infokubectl get nodes# 访问Kubernetes Dashboardminikube dashboard# 停止集群minikube stop# 删除集群minikube delete

2.2 安装kubectl

bash

# macOSbrew install kubectl# 验证安装kubectl version --client# 配置自动补全source <(kubectl completion bash)

三、Pod基础

3.1 创建第一个Pod

yaml

# pod.yamlapiVersion: v1kind: Podmetadata: name: nginx-pod labels: app: web tier: frontendspec: containers: - name: nginx image: nginx:1.21 ports: - containerPort: 80 resources: requests: cpu: 100m memory: 128Mi limits: cpu: 500m memory: 512Mi

bash

# 创建Podkubectl apply -f pod.yaml# 查看Podkubectl get podskubectl describe pod nginx-pod# 查看Pod日志kubectl logs nginx-pod# 进入Pod容器kubectl exec -it nginx-pod -- /bin/bash# 端口转发kubectl port-forward nginx-pod 8080:80# 删除Podkubectl delete pod nginx-pod

3.2 Pod生命周期

yaml

apiVersion: v1kind: Podmetadata: name: lifecycle-podspec: containers: - name: app image: myapp:latest # 启动探针：检查容器是否启动成功 startupProbe: httpGet: path: /health/startup port: 8000 initialDelaySeconds: 0 periodSeconds: 10 failureThreshold: 30 # 存活探针：检查容器是否仍在运行 livenessProbe: httpGet: path: /health/live port: 8000 initialDelaySeconds: 10 periodSeconds: 10 failureThreshold: 3 # 就绪探针：检查容器是否已准备好接收流量 readinessProbe: httpGet: path: /health/ready port: 8000 initialDelaySeconds: 5 periodSeconds: 5 failureThreshold: 1 # 启动时执行 lifecycle: postStart: exec: command: ["/bin/sh", "-c", "echo 'Pod started' > /tmp/started.txt"] # 停止前执行 preStop: exec: command: ["/bin/sh", "-c", "sleep 15"] # 优雅关闭

四、Deployment管理应用

4.1 基础Deployment

yaml

# deployment.yamlapiVersion: apps/v1kind: Deploymentmetadata: name: nginx-deployment namespace: defaultspec: replicas: 3 selector: matchLabels: app: nginx template: metadata: labels: app: nginx spec: containers: - name: nginx image: nginx:1.21 ports: - containerPort: 80 resources: requests: cpu: 100m memory: 128Mi limits: cpu: 500m memory: 512Mi volumeMounts: - name: config mountPath: /etc/nginx/conf.d volumes: - name: config configMap: name: nginx-config

bash

# 创建Deploymentkubectl apply -f deployment.yaml# 查看Deploymentkubectl get deploymentkubectl describe deployment nginx-deployment# 查看ReplicaSetkubectl get replicaset# 查看所有Podkubectl get pods -o wide# 扩展副本数kubectl scale deployment nginx-deployment --replicas=5# 更新镜像kubectl set image deployment/nginx-deployment nginx=nginx:1.22# 查看更新历史kubectl rollout history deployment/nginx-deployment# 回滚到上一个版本kubectl rollout undo deployment/nginx-deployment# 回滚到指定版本kubectl rollout undo deployment/nginx-deployment --to-revision=2

4.2 更新策略

yaml

apiVersion: apps/v1kind: Deploymentmetadata: name: app-deploymentspec: replicas: 10 strategy: # 滚动更新（默认） type: RollingUpdate rollingUpdate: maxSurge: 2 # 最多超出2个Pod maxUnavailable: 1 # 最多有1个Pod不可用 # 或者使用蓝绿部署 # type: Recreate # 先删除所有旧Pod，再启动新Pod selector: matchLabels: app: myapp template: metadata: labels: app: myapp spec: containers: - name: app image: myapp:v1

五、Service服务发现

5.1 ClusterIP服务

yaml

# service.yamlapiVersion: v1kind: Servicemetadata: name: nginx-servicespec: type: ClusterIP selector: app: nginx ports: - port: 80 # 服务端口 targetPort: 80 # Pod端口 protocol: TCP

5.2 NodePort服务

yaml

apiVersion: v1kind: Servicemetadata: name: nginx-nodeportspec: type: NodePort selector: app: nginx ports: - port: 80 # 集群内端口 targetPort: 80 # Pod端口 nodePort: 30080 # 节点端口（30000-32767）

bash

# 访问服务# 从Pod内访问kubectl exec -it pod-name -- curl http://nginx-service# 从外部访问NodePortcurl http://node-ip:30080

5.3 LoadBalancer服务

yaml

apiVersion: v1kind: Servicemetadata: name: nginx-lbspec: type: LoadBalancer selector: app: nginx ports: - port: 80 targetPort: 80

六、Ingress流量管理

6.1 安装Ingress Controller

bash

# 安装Nginx Ingress Controllerkubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.0.0/deploy/static/provider/cloud/deploy.yaml# 验证安装kubectl get pods -n ingress-nginx

6.2 配置Ingress规则

yaml

# ingress.yamlapiVersion: networking.k8s.io/v1kind: Ingressmetadata: name: app-ingressspec: ingressClassName: nginx rules: # 基于主机名的路由 - host: api.example.com http: paths: - path: / pathType: Prefix backend: service: name: api-service port: number: 8000 # 基于路径的路由 - host: example.com http: paths: - path: /api pathType: Prefix backend: service: name: api-service port: number: 8000 - path: /web pathType: Prefix backend: service: name: web-service port: number: 80 # HTTPS配置 tls: - hosts: - example.com - api.example.com secretName: example-tls

bash

# 创建TLS证书kubectl create secret tls example-tls \ --cert=path/to/tls.crt \ --key=path/to/tls.key# 查看Ingresskubectl get ingresskubectl describe ingress app-ingress

七、ConfigMap和Secret

7.1 ConfigMap配置管理

yaml

# configmap.yamlapiVersion: v1kind: ConfigMapmetadata: name: app-configdata: # 简单的键值对 DATABASE_HOST: postgres DATABASE_PORT: "5432" # 文件内容 nginx.conf: | server { listen 80; location / { proxy_pass http://backend; } }

bash

# 从文件创建ConfigMapkubectl create configmap app-config --from-file=./config/# 从键值对创建kubectl create configmap app-config \ --from-literal=DB_HOST=postgres \ --from-literal=DB_PORT=5432# 在Pod中使用ConfigMap

yaml

apiVersion: v1kind: Podmetadata: name: app-podspec: containers: - name: app image: myapp:latest # 方式1：作为环境变量 env: - name: DB_HOST valueFrom: configMapKeyRef: name: app-config key: DATABASE_HOST # 方式2：挂载为文件 volumeMounts: - name: config mountPath: /etc/config volumes: - name: config configMap: name: app-config

7.2 Secret敏感信息管理

yaml

# secret.yamlapiVersion: v1kind: Secretmetadata: name: app-secrettype: OpaquestringData: # 注意：stringData中的值会自动Base64编码 DATABASE_PASSWORD: "secure-password" API_KEY: "secret-key"

bash

# 从文件创建Secretkubectl create secret generic app-secret \ --from-file=./secrets/# 从键值对创建kubectl create secret generic app-secret \ --from-literal=db_password=secure-password# 查看Secret内容kubectl get secret app-secret -o yaml# 解码Secretkubectl get secret app-secret -o jsonpath='{.data.DATABASE_PASSWORD}' | base64 -d

yaml

apiVersion: v1kind: Podmetadata: name: app-podspec: containers: - name: app image: myapp:latest env: - name: DB_PASSWORD valueFrom: secretKeyRef: name: app-secret key: DATABASE_PASSWORD

八、持久化存储

8.1 PersistentVolume和PersistentVolumeClaim

yaml

# pv.yamlapiVersion: v1kind: PersistentVolumemetadata: name: app-pvspec: capacity: storage: 10Gi accessModes: - ReadWriteOnce storageClassName: standard hostPath: path: /data/app

yaml

# pvc.yamlapiVersion: v1kind: PersistentVolumeClaimmetadata: name: app-pvcspec: accessModes: - ReadWriteOnce storageClassName: standard resources: requests: storage: 5Gi

yaml

# pod-with-pv.yamlapiVersion: v1kind: Podmetadata: name: app-podspec: containers: - name: app image: myapp:latest volumeMounts: - name: data mountPath: /data volumes: - name: data persistentVolumeClaim: claimName: app-pvc

8.2 StatefulSet用于有状态应用

yaml

# statefulset.yamlapiVersion: apps/v1kind: StatefulSetmetadata: name: mysqlspec: serviceName: mysql replicas: 3 selector: matchLabels: app: mysql template: metadata: labels: app: mysql spec: containers: - name: mysql image: mysql:8.0 ports: - containerPort: 3306 env: - name: MYSQL_ROOT_PASSWORD valueFrom: secretKeyRef: name: mysql-secret key: password volumeMounts: - name: data mountPath: /var/lib/mysql volumeClaimTemplates: - metadata: name: data spec: accessModes: [ "ReadWriteOnce" ] resources: requests: storage: 10Gi

九、自动扩展

9.1 水平Pod自动扩展（HPA）

yaml

# hpa.yamlapiVersion: autoscaling/v2kind: HorizontalPodAutoscalermetadata: name: app-hpaspec: scaleTargetRef: apiVersion: apps/v1 kind: Deployment name: app-deployment minReplicas: 2 maxReplicas: 10 metrics: # CPU利用率 - type: Resource resource: name: cpu target: type: Utilization averageUtilization: 70 # 内存利用率 - type: Resource resource: name: memory target: type: Utilization averageUtilization: 80 behavior: scaleDown: stabilizationWindowSeconds: 300 policies: - type: Percent value: 50 periodSeconds: 60 scaleUp: stabilizationWindowSeconds: 0 policies: - type: Percent value: 100 periodSeconds: 30 - type: Pods value: 2 periodSeconds: 30 selectPolicy: Max

bash

# 查看HPA状态kubectl get hpakubectl describe hpa app-hpa# 模拟负载测试kubectl run -i --tty load-generator --rm --image=busybox --restart=Never -- /bin/sh -c "while sleep 0.01; do wget -q -O- http://app-deployment; done"

9.2 垂直Pod自动扩展（VPA）

yaml

apiVersion: autoscaling.k8s.io/v1kind: VerticalPodAutoscalermetadata: name: app-vpaspec: targetRef: apiVersion: apps/v1 kind: Deployment name: app-deployment updatePolicy: updateMode: "Auto" resourcePolicy: containerPolicies: - containerName: app minAllowed: cpu: 100m memory: 128Mi maxAllowed: cpu: 2 memory: 1Gi

十、监控和日志

10.1 部署Prometheus和Grafana

yaml

# prometheus-deployment.yamlapiVersion: apps/v1kind: Deploymentmetadata: name: prometheus namespace: monitoringspec: replicas: 1 selector: matchLabels: app: prometheus template: metadata: labels: app: prometheus spec: containers: - name: prometheus image: prom/prometheus:latest ports: - containerPort: 9090 volumeMounts: - name: config mountPath: /etc/prometheus - name: storage mountPath: /prometheus volumes: - name: config configMap: name: prometheus-config - name: storage emptyDir: {}

10.2 日志聚合

bash

# 安装EFK stack (Elasticsearch, Fluentd, Kibana)kubectl apply -f https://raw.githubusercontent.com/fluent/fluent-kubernetes-daemonset/master/fluentd-daemonset-elasticsearch-rbac.yaml# 查看Fluentd日志kubectl logs -n kube-system -l app=fluentd

十一、跨时区团队技术会议

我们公司的Kubernetes团队分布在美国、德国和中国，定期在线上进行技术讨论会议，涉及集群架构设计、部署策略和故障排查等深度技术话题。考虑到团队成员的语言差异，我们使用同言翻译（Transync AI）进行实时同声传译，该工具不仅提供准确的实时翻译，还能自动生成会议纪要，确保每位工程师都能准确理解技术细节，同时为后续参考提供了完整的会议记录。这大幅提升了全球团队的技术协作效率。

十二、生产环保最佳实践

12.1 资源配额管理

yaml

# namespace-quota.yamlapiVersion: v1kind: Namespacemetadata: name: production---apiVersion: v1kind: ResourceQuotametadata: name: prod-quota namespace: productionspec: hard: requests.cpu: "100" requests.memory: "200Gi" limits.cpu: "200" limits.memory: "400Gi" pods: "1000" services: "100"

12.2 网络策略

yaml

# network-policy.yamlapiVersion: networking.k8s.io/v1kind: NetworkPolicymetadata: name: app-network-policyspec: podSelector: matchLabels: app: web policyTypes: - Ingress - Egress ingress: - from: - namespaceSelector: matchLabels: name: production ports: - protocol: TCP port: 8080 egress: - to: - namespaceSelector: matchLabels: name: production ports: - protocol: TCP port: 5432

12.3 Pod安全策略

yaml

apiVersion: policy/v1beta1kind: PodSecurityPolicymetadata: name: restrictedspec: privileged: false allowPrivilegeEscalation: false requiredDropCapabilities: - ALL volumes: - 'configMap' - 'emptyDir' - 'projected' - 'secret' - 'downwardAPI' - 'persistentVolumeClaim' hostNetwork: false hostIPC: false hostPID: false runAsUser: rule: 'MustRunAsNonRoot' seLinux: rule: 'MustRunAs' seLinuxOptions: level: "s0:c123,c456"

十三、故障排查

13.1 常见问题诊断

bash

# 查看Pod状态kubectl get pods -o widekubectl describe pod <pod-name># 查看Pod日志kubectl logs <pod-name>kubectl logs <pod-name> --previous # 查看上一个Pod的日志# 进入容器调试kubectl exec -it <pod-name> -- /bin/bash# 查看事件kubectl get eventskubectl describe node <node-name># 查看资源使用情况kubectl top nodeskubectl top pods# 检查DNSkubectl run -it --rm debug --image=nicolaka/netshoot --restart=Never -- nslookup kubernetes.default

13.2 查看集群状态

bash

# 集群信息kubectl cluster-infokubectl get nodeskubectl describe node <node-name># API服务器状态kubectl get componentstatuses# Pod事件kubectl get events --sort-by='.lastTimestamp'# 资源使用总结kubectl describe nodes | grep -A 5 "Allocated resources"

十四、性能对比

十五、推荐工具

kubectl - CLI工具Kubeadm - 集群初始化Helm - 包管理器Kustomize - 配置管理Lens - IDE for K8skubectx - 上下文切换Stern - 日志查看Velero - 备份恢复

十六、学习资源

官方文档：https://kubernetes.io/docs/Kubernetes教程：https://kubernetes.io/docs/tutorials/Helm文档：https://helm.sh/docs/CNCF景观：https://landscape.cncf.io/

十七、结语

Kubernetes虽然学习曲线陡峭，但一旦掌握，它提供的价值是巨大的。从自动扩展、自我修复到灰度发布，它极大地简化了容器编排。

核心理念：声明式配置 → 自动化执行 → 持续优化

这正是Kubernetes的威力所在。希望这篇文章能帮助你踏上Kubernetes之旅。欢迎在评论区分享你的K8s实践经验！