Pod-Resource-Mutator基于Kind集群的端到端测试指南

本文档基于 macOS (Apple Silicon) 环境实测验证,记录从零搭建 Kind 集群到完成全部测试用例的完整流程。

一、前置准备

1.1 工具要求

kind version        # >= 0.20.0
kubectl version     # >= 1.28
helm version        # >= 3.12
docker version      # Docker Desktop 运行中
go version          # >= 1.22

macOS 安装:

brew install kind kubectl helm go

1.2 项目源码

cd /path/to/pod-resource-mutator

二、创建 Kind 集群

2.1 集群配置

创建 kind-config.yaml:

kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
  - role: control-plane
  - role: worker
  - role: worker

两个 worker 节点用于验证 HA 部署(2 replicas 分布到不同节点)。

2.2 创建集群

kind create cluster --name podmutator-test --config kind-config.yaml

2.3 导出 kubeconfig

Kind 生成的 kubeconfig 可能存在权限问题,建议导出到独立文件:

kind get kubeconfig --name podmutator-test > /tmp/podmutator-kubeconfig
export KUBECONFIG=/tmp/podmutator-kubeconfig

后续所有 kubectl / helm 命令均依赖此环境变量,请确保每个终端窗口都已 export。

验证集群就绪:

kubectl get nodes
# NAME                              STATUS   ROLES           AGE   VERSION
# podmutator-test-control-plane     Ready    control-plane   30s   v1.31.0
# podmutator-test-worker            Ready    <none>          20s   v1.31.0
# podmutator-test-worker2           Ready    <none>          20s   v1.31.0

三、构建镜像并加载到 Kind

Kind 集群使用独立的容器运行时,不能直接访问宿主机 Docker 镜像,需要将镜像显式加载进去。

3.1 构建 Docker 镜像

cd /path/to/pod-resource-mutator

# 构建机 / Linux amd64(默认)
docker build -t pod-resource-mutator:test .

# 本地 Apple Silicon Mac
docker build --build-arg GOARCH=arm64 -t pod-resource-mutator:test .

3.2 加载到 Kind 集群

kind load docker-image pod-resource-mutator:test --name podmutator-test

验证镜像已加载:

docker exec -it podmutator-test-worker crictl images | grep podmutator

四、部署 Pod Resource Mutator

4.1 Helm 安装

helm upgrade --install pod-resource-mutator chart/ \
  --namespace podmutator-system \
  --create-namespace \
  --set image.repository=pod-resource-mutator \
  --set image.tag=test \
  --set image.pullPolicy=Never

pullPolicy=Never 确保使用 Kind 中预加载的本地镜像,而不是尝试从远程仓库拉取。

4.2 验证部署

# 检查 Pod 状态(默认 2 副本)
kubectl get pods -n podmutator-system
# NAME                                      READY   STATUS    RESTARTS   AGE
# pod-resource-mutator-xxxxx                1/1     Running   0          30s
# pod-resource-mutator-yyyyy                1/1     Running   0          30s

# 检查 webhook 配置
kubectl get mutatingwebhookconfigurations
# NAME                       WEBHOOKS   AGE
# pod-resource-mutator       1          30s

# 检查 CRD 已注册
kubectl get crd podmutationrules.podmutator.io

# 检查 TLS 证书 Secret 已生成
kubectl get secret -n podmutator-system | grep tls
# pod-resource-mutator-tls   kubernetes.io/tls   3      30s

如果 Pod 未就绪,排查:

kubectl logs -n podmutator-system -l app.kubernetes.io/name=pod-resource-mutator
kubectl get events -n podmutator-system --sort-by='.lastTimestamp'

五、创建测试 namespace 和规则

5.1 创建目标 namespace

kubectl create namespace test-ci
kubectl label namespace test-ci podmutator.io/enabled=true

验证:

kubectl get namespace test-ci --show-labels
# NAME      STATUS   AGE   LABELS
# test-ci   Active   5s    podmutator.io/enabled=true,...

5.2 创建测试用 PodMutationRule

cat <<'EOF' | kubectl apply -f -
apiVersion: podmutator.io/v1alpha1
kind: PodMutationRule
metadata:
  name: test-memory-fix
spec:
  description: "测试规则:将匹配 Pod 的资源修改为指定值"

  match:
    namespaces: ["test-ci"]
    labelSelector:
      matchLabels:
        app: builder

  priority: 10
  mode: Enforce

  mutations:
    containers:
      resources:
        requests:
          memory: "4Gi"
          cpu: "500m"
        limits:
          memory: "8Gi"
          cpu: "2"
        strategy: Replace
      env:
        - name: MUTATED_BY
          value: "pod-resource-mutator"
          action: Upsert
    pod:
      annotations:
        podmutator.io/mutated: "true"
        podmutator.io/rule: "test-memory-fix"
EOF

验证规则创建:

kubectl get podmutationrules
# NAME              MODE      PRIORITY   MATCHES   AGE
# test-memory-fix   Enforce   10         0         5s

六、测试用例

6.1 测试一:匹配的 Pod 被正确 mutate

创建一个匹配规则的 Pod:

cat <<'EOF' | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
  name: test-builder-match
  namespace: test-ci
  labels:
    app: builder
spec:
  containers:
    - name: worker
      image: busybox
      command: ["sleep", "3600"]
      resources:
        requests:
          memory: "1Gi"
          cpu: "100m"
        limits:
          memory: "2Gi"
          cpu: "1"
EOF

验证资源被修改:

# 检查 resource 被 Replace 为规则定义的值
kubectl get pod test-builder-match -n test-ci -o jsonpath='{.spec.containers[0].resources}' | jq .
# {
#   "limits": { "cpu": "2", "memory": "8Gi" },
#   "requests": { "cpu": "500m", "memory": "4Gi" }
# }

# 检查 env 被注入
kubectl get pod test-builder-match -n test-ci -o jsonpath='{.spec.containers[0].env}' | jq .
# [
#   { "name": "MUTATED_BY", "value": "pod-resource-mutator" }
# ]

# 检查审计注解
kubectl get pod test-builder-match -n test-ci -o jsonpath='{.metadata.annotations}' | jq .
# {
#   "podmutator.io/mutated": "true",
#   "podmutator.io/rule": "test-memory-fix",
#   "podmutator.io/rules": "test-memory-fix"
# }

预期结果:

6.2 测试二:不匹配的 Pod 不受影响

cat <<'EOF' | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
  name: test-no-match
  namespace: test-ci
  labels:
    app: web-server
spec:
  containers:
    - name: nginx
      image: nginx
      resources:
        requests:
          memory: "256Mi"
EOF

验证资源未被修改:

kubectl get pod test-no-match -n test-ci -o jsonpath='{.spec.containers[0].resources.requests.memory}'
# 256Mi  ← 保持原值

kubectl get pod test-no-match -n test-ci -o jsonpath='{.metadata.annotations.podmutator\.io/mutated}'
# (空,无此注解)

6.3 测试三:Escape Hatch — skip label

cat <<'EOF' | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
  name: test-skip
  namespace: test-ci
  labels:
    app: builder
    podmutator.io/skip: "true"
spec:
  containers:
    - name: worker
      image: busybox
      command: ["sleep", "3600"]
      resources:
        requests:
          memory: "1Gi"
EOF

验证跳过了 mutation:

kubectl get pod test-skip -n test-ci -o jsonpath='{.spec.containers[0].resources.requests.memory}'
# 1Gi  ← 保持原值,skip 生效

6.4 测试四:非目标 namespace 不受影响

kubectl create namespace test-other

cat <<'EOF' | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
  name: test-other-ns
  namespace: test-other
  labels:
    app: builder
spec:
  containers:
    - name: worker
      image: busybox
      command: ["sleep", "3600"]
      resources:
        requests:
          memory: "1Gi"
EOF
kubectl get pod test-other-ns -n test-other -o jsonpath='{.spec.containers[0].resources.requests.memory}'
# 1Gi  ← namespace 未启用 podmutator.io/enabled label,webhook 未拦截

6.5 测试五:DryRun 模式

# 将规则切换为 DryRun
kubectl patch podmutationrule test-memory-fix --type=merge -p '{"spec":{"mode":"DryRun"}}'

# 创建新 Pod
cat <<'EOF' | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
  name: test-dryrun
  namespace: test-ci
  labels:
    app: builder
spec:
  containers:
    - name: worker
      image: busybox
      command: ["sleep", "3600"]
      resources:
        requests:
          memory: "1Gi"
EOF
# 资源不应被修改
kubectl get pod test-dryrun -n test-ci -o jsonpath='{.spec.containers[0].resources.requests.memory}'
# 1Gi  ← DryRun 不实际 patch

# webhook 日志中可以看到 dry-run 记录
kubectl logs -n podmutator-system -l app.kubernetes.io/name=pod-resource-mutator | grep dry-run

测试完恢复为 Enforce:

kubectl patch podmutationrule test-memory-fix --type=merge -p '{"spec":{"mode":"Enforce"}}'

6.6 测试六:Resource 策略 — Max

cat <<'EOF' | kubectl apply -f -
apiVersion: podmutator.io/v1alpha1
kind: PodMutationRule
metadata:
  name: test-max-strategy
spec:
  match:
    namespaces: ["test-ci"]
    labelSelector:
      matchLabels:
        app: max-test
  priority: 50
  mode: Enforce
  mutations:
    containers:
      resources:
        requests:
          memory: "4Gi"
        strategy: Max
EOF

场景 A:Pod 原始 request 低于 4Gi → 被提升

cat <<'EOF' | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
  name: test-max-low
  namespace: test-ci
  labels:
    app: max-test
spec:
  containers:
    - name: app
      image: busybox
      command: ["sleep", "3600"]
      resources:
        requests:
          memory: "1Gi"
EOF

kubectl get pod test-max-low -n test-ci -o jsonpath='{.spec.containers[0].resources.requests.memory}'
# 4Gi  ← max(1Gi, 4Gi) = 4Gi

场景 B:Pod 原始 request 高于 4Gi → 保持不变

cat <<'EOF' | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
  name: test-max-high
  namespace: test-ci
  labels:
    app: max-test
spec:
  containers:
    - name: app
      image: busybox
      command: ["sleep", "3600"]
      resources:
        requests:
          memory: "8Gi"
EOF

kubectl get pod test-max-high -n test-ci -o jsonpath='{.spec.containers[0].resources.requests.memory}'
# 8Gi  ← max(8Gi, 4Gi) = 8Gi,保持原值

6.7 测试七:Rule Status 更新

# 等待 status updater 刷新(默认 10s 间隔)
sleep 15

kubectl get podmutationrule test-memory-fix -o jsonpath='{.status.matchCount}'
# 应该 > 0

kubectl get podmutationrule test-memory-fix -o jsonpath='{.status.lastMatchTime}'
# 应该有时间戳

6.8 测试八:Webhook 高可用

# 删掉一个副本,验证另一个继续工作
POD=$(kubectl get pod -n podmutator-system -l app.kubernetes.io/name=pod-resource-mutator -o jsonpath='{.items[0].metadata.name}')
kubectl delete pod $POD -n podmutator-system

# 立即创建新 Pod 测试
cat <<'EOF' | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
  name: test-ha
  namespace: test-ci
  labels:
    app: builder
spec:
  containers:
    - name: worker
      image: busybox
      command: ["sleep", "3600"]
      resources:
        requests:
          memory: "1Gi"
EOF

kubectl get pod test-ha -n test-ci -o jsonpath='{.spec.containers[0].resources.requests.memory}'
# 4Gi  ← 另一个副本正常工作

七、检查日志和指标

7.1 Webhook 日志

kubectl logs -n podmutator-system -l app.kubernetes.io/name=pod-resource-mutator

示例输出:

{
  "level": "info",
  "msg": "mutation applied",
  "pod": "test-builder-match",
  "namespace": "test-ci",
  "matchedRules": ["test-memory-fix"],
  "patchCount": 5,
  "dryRun": false
}

7.2 Prometheus 指标

# 端口转发到 metrics 端口
kubectl port-forward -n podmutator-system svc/pod-resource-mutator 8080:8080 &

# 查看指标
curl -s localhost:8080/metrics | grep podmutator
# podmutator_requests_total{mode="enforce",namespace="test-ci",rule="test-memory-fix"} 3
# podmutator_request_duration_seconds_bucket{namespace="test-ci",le="0.005"} 3
# podmutator_rules_matched_total{namespace="test-ci",rule="test-memory-fix"} 3
# podmutator_patches_applied_total{namespace="test-ci",rule="test-memory-fix"} 2

# 结束端口转发
kill %1

八、清理

# 删除测试 Pod
kubectl delete pod --all -n test-ci
kubectl delete pod --all -n test-other

# 删除测试规则
kubectl delete podmutationrule --all

# 卸载 webhook
helm uninstall pod-resource-mutator -n podmutator-system

# 删除 namespace
kubectl delete namespace test-ci test-other podmutator-system

# 删除 Kind 集群
kind delete cluster --name podmutator-test

九、测试 Checklist

# 测试项 预期结果 通过
1 匹配的 Pod 资源被修改 memory: 1Gi → 4Gi, cpu: 100m → 500m ☐
2 匹配的 Pod env 被注入 MUTATED_BY=pod-resource-mutator ☐
3 匹配的 Pod 有审计注解 podmutator.io/mutated=true ☐
4 不匹配的 Pod 不受影响 memory 保持 256Mi ☐
5 Skip label 生效 memory 保持 1Gi ☐
6 非目标 namespace 不受影响 memory 保持 1Gi ☐
7 DryRun 模式不修改资源 memory 保持 1Gi,日志有 dry-run 记录 ☐
8 Max 策略:低值被提升 1Gi → 4Gi ☐
9 Max 策略:高值保持 8Gi 不变 ☐
10 Rule status matchCount 更新 > 0 ☐
11 删掉一个副本仍正常 mutation 继续生效 ☐
12 Prometheus 指标存在 podmutator_* 有数据 ☐