Pod-Resource-Mutator基于Kind集群的端到端测试指南
本文档基于 macOS (Apple Silicon) 环境实测验证,记录从零搭建 Kind 集群到完成全部测试用例的完整流程。
一、前置准备
1.1 工具要求
kind version # >= 0.20.0
kubectl version # >= 1.28
helm version # >= 3.12
docker version # Docker Desktop 运行中
go version # >= 1.22
macOS 安装:
brew install kind kubectl helm go
1.2 项目源码
cd /path/to/pod-resource-mutator
二、创建 Kind 集群
2.1 集群配置
创建 kind-config.yaml:
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
- role: worker
- role: worker
两个 worker 节点用于验证 HA 部署(2 replicas 分布到不同节点)。
2.2 创建集群
kind create cluster --name podmutator-test --config kind-config.yaml
2.3 导出 kubeconfig
Kind 生成的 kubeconfig 可能存在权限问题,建议导出到独立文件:
kind get kubeconfig --name podmutator-test > /tmp/podmutator-kubeconfig
export KUBECONFIG=/tmp/podmutator-kubeconfig
后续所有
kubectl/helm命令均依赖此环境变量,请确保每个终端窗口都已 export。
验证集群就绪:
kubectl get nodes
# NAME STATUS ROLES AGE VERSION
# podmutator-test-control-plane Ready control-plane 30s v1.31.0
# podmutator-test-worker Ready <none> 20s v1.31.0
# podmutator-test-worker2 Ready <none> 20s v1.31.0
三、构建镜像并加载到 Kind
Kind 集群使用独立的容器运行时,不能直接访问宿主机 Docker 镜像,需要将镜像显式加载进去。
3.1 构建 Docker 镜像
cd /path/to/pod-resource-mutator
# 构建机 / Linux amd64(默认)
docker build -t pod-resource-mutator:test .
# 本地 Apple Silicon Mac
docker build --build-arg GOARCH=arm64 -t pod-resource-mutator:test .
3.2 加载到 Kind 集群
kind load docker-image pod-resource-mutator:test --name podmutator-test
验证镜像已加载:
docker exec -it podmutator-test-worker crictl images | grep podmutator
四、部署 Pod Resource Mutator
4.1 Helm 安装
helm upgrade --install pod-resource-mutator chart/ \
--namespace podmutator-system \
--create-namespace \
--set image.repository=pod-resource-mutator \
--set image.tag=test \
--set image.pullPolicy=Never
pullPolicy=Never确保使用 Kind 中预加载的本地镜像,而不是尝试从远程仓库拉取。
4.2 验证部署
# 检查 Pod 状态(默认 2 副本)
kubectl get pods -n podmutator-system
# NAME READY STATUS RESTARTS AGE
# pod-resource-mutator-xxxxx 1/1 Running 0 30s
# pod-resource-mutator-yyyyy 1/1 Running 0 30s
# 检查 webhook 配置
kubectl get mutatingwebhookconfigurations
# NAME WEBHOOKS AGE
# pod-resource-mutator 1 30s
# 检查 CRD 已注册
kubectl get crd podmutationrules.podmutator.io
# 检查 TLS 证书 Secret 已生成
kubectl get secret -n podmutator-system | grep tls
# pod-resource-mutator-tls kubernetes.io/tls 3 30s
如果 Pod 未就绪,排查:
kubectl logs -n podmutator-system -l app.kubernetes.io/name=pod-resource-mutator
kubectl get events -n podmutator-system --sort-by='.lastTimestamp'
五、创建测试 namespace 和规则
5.1 创建目标 namespace
kubectl create namespace test-ci
kubectl label namespace test-ci podmutator.io/enabled=true
验证:
kubectl get namespace test-ci --show-labels
# NAME STATUS AGE LABELS
# test-ci Active 5s podmutator.io/enabled=true,...
5.2 创建测试用 PodMutationRule
cat <<'EOF' | kubectl apply -f -
apiVersion: podmutator.io/v1alpha1
kind: PodMutationRule
metadata:
name: test-memory-fix
spec:
description: "测试规则:将匹配 Pod 的资源修改为指定值"
match:
namespaces: ["test-ci"]
labelSelector:
matchLabels:
app: builder
priority: 10
mode: Enforce
mutations:
containers:
resources:
requests:
memory: "4Gi"
cpu: "500m"
limits:
memory: "8Gi"
cpu: "2"
strategy: Replace
env:
- name: MUTATED_BY
value: "pod-resource-mutator"
action: Upsert
pod:
annotations:
podmutator.io/mutated: "true"
podmutator.io/rule: "test-memory-fix"
EOF
验证规则创建:
kubectl get podmutationrules
# NAME MODE PRIORITY MATCHES AGE
# test-memory-fix Enforce 10 0 5s
六、测试用例
6.1 测试一:匹配的 Pod 被正确 mutate
创建一个匹配规则的 Pod:
cat <<'EOF' | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
name: test-builder-match
namespace: test-ci
labels:
app: builder
spec:
containers:
- name: worker
image: busybox
command: ["sleep", "3600"]
resources:
requests:
memory: "1Gi"
cpu: "100m"
limits:
memory: "2Gi"
cpu: "1"
EOF
验证资源被修改:
# 检查 resource 被 Replace 为规则定义的值
kubectl get pod test-builder-match -n test-ci -o jsonpath='{.spec.containers[0].resources}' | jq .
# {
# "limits": { "cpu": "2", "memory": "8Gi" },
# "requests": { "cpu": "500m", "memory": "4Gi" }
# }
# 检查 env 被注入
kubectl get pod test-builder-match -n test-ci -o jsonpath='{.spec.containers[0].env}' | jq .
# [
# { "name": "MUTATED_BY", "value": "pod-resource-mutator" }
# ]
# 检查审计注解
kubectl get pod test-builder-match -n test-ci -o jsonpath='{.metadata.annotations}' | jq .
# {
# "podmutator.io/mutated": "true",
# "podmutator.io/rule": "test-memory-fix",
# "podmutator.io/rules": "test-memory-fix"
# }
预期结果:
- memory request: 1Gi → 4Gi
- memory limit: 2Gi → 8Gi
- cpu request: 100m → 500m
- cpu limit: 1 → 2
- 新增环境变量
MUTATED_BY=pod-resource-mutator - 新增审计注解
podmutator.io/mutated=true
6.2 测试二:不匹配的 Pod 不受影响
cat <<'EOF' | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
name: test-no-match
namespace: test-ci
labels:
app: web-server
spec:
containers:
- name: nginx
image: nginx
resources:
requests:
memory: "256Mi"
EOF
验证资源未被修改:
kubectl get pod test-no-match -n test-ci -o jsonpath='{.spec.containers[0].resources.requests.memory}'
# 256Mi ← 保持原值
kubectl get pod test-no-match -n test-ci -o jsonpath='{.metadata.annotations.podmutator\.io/mutated}'
# (空,无此注解)
6.3 测试三:Escape Hatch — skip label
cat <<'EOF' | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
name: test-skip
namespace: test-ci
labels:
app: builder
podmutator.io/skip: "true"
spec:
containers:
- name: worker
image: busybox
command: ["sleep", "3600"]
resources:
requests:
memory: "1Gi"
EOF
验证跳过了 mutation:
kubectl get pod test-skip -n test-ci -o jsonpath='{.spec.containers[0].resources.requests.memory}'
# 1Gi ← 保持原值,skip 生效
6.4 测试四:非目标 namespace 不受影响
kubectl create namespace test-other
cat <<'EOF' | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
name: test-other-ns
namespace: test-other
labels:
app: builder
spec:
containers:
- name: worker
image: busybox
command: ["sleep", "3600"]
resources:
requests:
memory: "1Gi"
EOF
kubectl get pod test-other-ns -n test-other -o jsonpath='{.spec.containers[0].resources.requests.memory}'
# 1Gi ← namespace 未启用 podmutator.io/enabled label,webhook 未拦截
6.5 测试五:DryRun 模式
# 将规则切换为 DryRun
kubectl patch podmutationrule test-memory-fix --type=merge -p '{"spec":{"mode":"DryRun"}}'
# 创建新 Pod
cat <<'EOF' | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
name: test-dryrun
namespace: test-ci
labels:
app: builder
spec:
containers:
- name: worker
image: busybox
command: ["sleep", "3600"]
resources:
requests:
memory: "1Gi"
EOF
# 资源不应被修改
kubectl get pod test-dryrun -n test-ci -o jsonpath='{.spec.containers[0].resources.requests.memory}'
# 1Gi ← DryRun 不实际 patch
# webhook 日志中可以看到 dry-run 记录
kubectl logs -n podmutator-system -l app.kubernetes.io/name=pod-resource-mutator | grep dry-run
测试完恢复为 Enforce:
kubectl patch podmutationrule test-memory-fix --type=merge -p '{"spec":{"mode":"Enforce"}}'
6.6 测试六:Resource 策略 — Max
cat <<'EOF' | kubectl apply -f -
apiVersion: podmutator.io/v1alpha1
kind: PodMutationRule
metadata:
name: test-max-strategy
spec:
match:
namespaces: ["test-ci"]
labelSelector:
matchLabels:
app: max-test
priority: 50
mode: Enforce
mutations:
containers:
resources:
requests:
memory: "4Gi"
strategy: Max
EOF
场景 A:Pod 原始 request 低于 4Gi → 被提升
cat <<'EOF' | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
name: test-max-low
namespace: test-ci
labels:
app: max-test
spec:
containers:
- name: app
image: busybox
command: ["sleep", "3600"]
resources:
requests:
memory: "1Gi"
EOF
kubectl get pod test-max-low -n test-ci -o jsonpath='{.spec.containers[0].resources.requests.memory}'
# 4Gi ← max(1Gi, 4Gi) = 4Gi
场景 B:Pod 原始 request 高于 4Gi → 保持不变
cat <<'EOF' | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
name: test-max-high
namespace: test-ci
labels:
app: max-test
spec:
containers:
- name: app
image: busybox
command: ["sleep", "3600"]
resources:
requests:
memory: "8Gi"
EOF
kubectl get pod test-max-high -n test-ci -o jsonpath='{.spec.containers[0].resources.requests.memory}'
# 8Gi ← max(8Gi, 4Gi) = 8Gi,保持原值
6.7 测试七:Rule Status 更新
# 等待 status updater 刷新(默认 10s 间隔)
sleep 15
kubectl get podmutationrule test-memory-fix -o jsonpath='{.status.matchCount}'
# 应该 > 0
kubectl get podmutationrule test-memory-fix -o jsonpath='{.status.lastMatchTime}'
# 应该有时间戳
6.8 测试八:Webhook 高可用
# 删掉一个副本,验证另一个继续工作
POD=$(kubectl get pod -n podmutator-system -l app.kubernetes.io/name=pod-resource-mutator -o jsonpath='{.items[0].metadata.name}')
kubectl delete pod $POD -n podmutator-system
# 立即创建新 Pod 测试
cat <<'EOF' | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
name: test-ha
namespace: test-ci
labels:
app: builder
spec:
containers:
- name: worker
image: busybox
command: ["sleep", "3600"]
resources:
requests:
memory: "1Gi"
EOF
kubectl get pod test-ha -n test-ci -o jsonpath='{.spec.containers[0].resources.requests.memory}'
# 4Gi ← 另一个副本正常工作
七、检查日志和指标
7.1 Webhook 日志
kubectl logs -n podmutator-system -l app.kubernetes.io/name=pod-resource-mutator
示例输出:
{
"level": "info",
"msg": "mutation applied",
"pod": "test-builder-match",
"namespace": "test-ci",
"matchedRules": ["test-memory-fix"],
"patchCount": 5,
"dryRun": false
}
7.2 Prometheus 指标
# 端口转发到 metrics 端口
kubectl port-forward -n podmutator-system svc/pod-resource-mutator 8080:8080 &
# 查看指标
curl -s localhost:8080/metrics | grep podmutator
# podmutator_requests_total{mode="enforce",namespace="test-ci",rule="test-memory-fix"} 3
# podmutator_request_duration_seconds_bucket{namespace="test-ci",le="0.005"} 3
# podmutator_rules_matched_total{namespace="test-ci",rule="test-memory-fix"} 3
# podmutator_patches_applied_total{namespace="test-ci",rule="test-memory-fix"} 2
# 结束端口转发
kill %1
八、清理
# 删除测试 Pod
kubectl delete pod --all -n test-ci
kubectl delete pod --all -n test-other
# 删除测试规则
kubectl delete podmutationrule --all
# 卸载 webhook
helm uninstall pod-resource-mutator -n podmutator-system
# 删除 namespace
kubectl delete namespace test-ci test-other podmutator-system
# 删除 Kind 集群
kind delete cluster --name podmutator-test
九、测试 Checklist
| # | 测试项 | 预期结果 | 通过 |
|---|---|---|---|
| 1 | 匹配的 Pod 资源被修改 | memory: 1Gi → 4Gi, cpu: 100m → 500m | ☐ |
| 2 | 匹配的 Pod env 被注入 | MUTATED_BY=pod-resource-mutator | ☐ |
| 3 | 匹配的 Pod 有审计注解 | podmutator.io/mutated=true | ☐ |
| 4 | 不匹配的 Pod 不受影响 | memory 保持 256Mi | ☐ |
| 5 | Skip label 生效 | memory 保持 1Gi | ☐ |
| 6 | 非目标 namespace 不受影响 | memory 保持 1Gi | ☐ |
| 7 | DryRun 模式不修改资源 | memory 保持 1Gi,日志有 dry-run 记录 | ☐ |
| 8 | Max 策略:低值被提升 | 1Gi → 4Gi | ☐ |
| 9 | Max 策略:高值保持 | 8Gi 不变 | ☐ |
| 10 | Rule status matchCount 更新 | > 0 | ☐ |
| 11 | 删掉一个副本仍正常 | mutation 继续生效 | ☐ |
| 12 | Prometheus 指标存在 | podmutator_* 有数据 | ☐ |