Kind集群使用Helm部署Mysql和Clickhouse

基于Kind搭建测试集群

可以参考基于Kind搭建测试集群搭建一个测试集群。

搭建集群

和基于Kind搭建测试集群不同的是,此处还要增加33006、30123、39000端口的映射,以实现mysql和clickhouse使用nodeport方式将服务暴露到集群外部。

将下面的配置写入进huari.yaml文件里

kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
networking:
  apiServerAddress: "10.10.151.201"
nodes:
  - role: control-plane
    extraPortMappings:
      - containerPort: 6443
        hostPort: 6443
        listenAddress: "10.10.151.201"
        protocol: tcp
  - role: control-plane
  - role: control-plane
  - role: worker
    extraPortMappings:
      - containerPort: 80
        hostPort: 7080
        listenAddress: "0.0.0.0"
        protocol: tcp
      - containerPort: 443
        hostPort: 7443
        listenAddress: "0.0.0.0"
        protocol: tcp
      - containerPort: 30036
        hostPort: 3306
        listenAddress: "0.0.0.0"
        protocol: tcp
      - containerPort: 30123
        hostPort: 8123
        listenAddress: "0.0.0.0"
        protocol: tcp
      - containerPort: 30046
        hostPort: 30046
        listenAddress: "0.0.0.0"
        protocol: tcp
  - role: worker
    extraPortMappings:
      - containerPort: 80
        hostPort: 8080
        listenAddress: "0.0.0.0"
        protocol: tcp
      - containerPort: 443
        hostPort: 8443
        listenAddress: "0.0.0.0"
        protocol: tcp
      - containerPort: 30036
        hostPort: 3307
        listenAddress: "0.0.0.0"
        protocol: tcp
      - containerPort: 30123
        hostPort: 8124
        listenAddress: "0.0.0.0"
        protocol: tcp
      - containerPort: 30046
        hostPort: 30047
        listenAddress: "0.0.0.0"
        protocol: tcp
  - role: worker
    extraPortMappings:
      - containerPort: 80
        hostPort: 9080
        listenAddress: "0.0.0.0"
        protocol: tcp
      - containerPort: 443
        hostPort: 9443
        listenAddress: "0.0.0.0"
        protocol: tcp
      - containerPort: 30036
        hostPort: 3308
        listenAddress: "0.0.0.0"
        protocol: tcp
      - containerPort: 30123
        hostPort: 8125
        listenAddress: "0.0.0.0"
        protocol: tcp
      - containerPort: 30046
        hostPort: 30048
        listenAddress: "0.0.0.0"
        protocol: tcp

创建高可用集群命令:

sudo kind create cluster --config=huari.yaml --name huari-test --image kindest/node:v1.34.0 --retain; sudo kind export logs --name huari-test

切换kubectl上下文:

sudo kubectl cluster-info --context kind-huari-test

查看信息:

# 查看集群节点
sudo kubectl get nodes

# 查看集群全部的pod
sudo kubectl get pods -A -owide

删除集群:

sudo kind delete cluster --name huari-test

安装ingress

需要安装官方定制版本的ingress。

安装ingress:

sudo kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/main/deploy/static/provider/kind/deploy.yaml

等待 Controller 就绪:

sudo kubectl -n ingress-nginx wait --for=condition=ready pod -l app.kubernetes.io/component=controller --timeout=90s

Helm安装mysql集群

添加Repository

helm repo add bitnami https://charts.bitnami.com/bitnami

拉取chart到本地

可以通过下面的命令将chart拉取到本地,然后看内部的定义:

helm pull bitnami/mysql --untar

这会在当前目录创建一个 mysql 文件夹,里面就是Chart的全部内容。

mysql集群配置

查看支持的storageclasses:

$ sudo kubectl get storageclasses.storage.k8s.io
NAME                 PROVISIONER             RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
standard (default)   rancher.io/local-path   Delete          WaitForFirstConsumer   false                  70m

将下面的配置写入进values.yaml文件里:

# 使用DaoCloud镜像代理
global:
  imageRegistry: "m.daocloud.io/docker.io"
  security:
    allowInsecureImages: true

# 集群架构 - 主从复制
architecture: replication

# 认证配置
auth:
  rootPassword: "mysql123"
  replicationPassword: "repl123"
  database: "myapp"

# 主节点配置
primary:
  persistence:
    enabled: false

  extraFlags: "--bind-address=0.0.0.0"

  # 通过环境变量覆盖配置
  extraEnvVars:
    - name: MYSQL_BIND_ADDRESS
      value: "0.0.0.0"
    - name: MYSQL_ROOT_PASSWORD
      value: "mysql123"

  # 完全禁用所有探针
  livenessProbe:
    enabled: false
  readinessProbe:
    enabled: false
  startupProbe:
    enabled: false

  # MySQL 配置 - 允许远程访问
  configuration: |-
    [mysqld]
    bind-address=0.0.0.0
    port=3306
    socket=/opt/bitnami/mysql/tmp/mysql.sock
    skip-name-resolve
    explicit_defaults_for_timestamp

  service:
    type: NodePort
    nodePorts:
      mysql: 30036

# 从节点配置
secondary:
  replicaCount: 1
  persistence:
    enabled: false

安装Mysql集群

# 安装MySQL集群
sudo helm install mysql-cluster bitnami/mysql \
  --namespace mysql \
  --create-namespace \
  -f values.yaml

# 等待安装完成,查看状态
sudo kubectl get pods -n mysql -w

修改root密码:

sudo kubectl exec -it mysql-cluster-primary-0 -n mysql -- mysql -S /opt/bitnami/mysql/tmp/mysql.sock -uroot -e "ALTER USER 'root'@'localhost' IDENTIFIED BY 'mysql123'; FLUSH PRIVILEGES;"

允许远程访问:

# 创建允许远程连接的用户
sudo kubectl exec -it mysql-cluster-primary-0 -n mysql -- mysql -S /opt/bitnami/mysql/tmp/mysql.sock -uroot -pmysql123 -e "
CREATE USER 'root'@'%' IDENTIFIED BY 'mysql123';
GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' WITH GRANT OPTION;
FLUSH PRIVILEGES;
"

验证远程用户

sudo kubectl exec -it mysql-cluster-primary-0 -n mysql -- mysql -h 127.0.0.1 -P 3306 -uroot -pmysql123 -e "SELECT 1;"

访问mysql集群

查看 mysql主节点落在了哪个node上

$ sudo kubectl get pods -n mysql -owide
NAME                        READY   STATUS    RESTARTS   AGE   IP            NODE                 NOMINATED NODE   READINESS GATES
mysql-cluster-primary-0     1/1     Running   0          28s   10.244.4.13   huari-test-worker    <none>           <none>
mysql-cluster-secondary-0   0/1     Running   0          28s   10.244.5.14   huari-test-worker3   <none>           <none>

现在mysql主节点落在huari-test-worker这个节点上,那么就可以通过宿主机ip加3306端口访问。

这里有两层端口映射:

测试mysql连接:

~ docker run -it --rm mysql:8.0 mysql -h 10.10.151.201 -P 3306 -uroot -pmysql123 -e "SELECT 1;"
mysql: [Warning] Using a password on the command line interface can be insecure.
+---+
| 1 |
+---+
| 1 |
+---+

Helm安装clickhouse

添加Repository

helm repo add bitnami https://charts.bitnami.com/bitnami

拉取chart到本地

可以通过下面的命令将chart拉取到本地,然后看内部的定义:

helm pull bitnami/clickhouse --untar

这会在当前目录创建一个 clickhouse-cluster 文件夹,里面就是Chart的全部内容。

clickhouse配置

ClickHouse主要使用两个端口来服务不同类型的客户端连接,它们对应的协议和用途可参考下表:

端口号 协议 主要用途 客户端/工具
8123 HTTP 用于HTTP接口。适合轻量级操作,方便通过REST API跨平台和编程语言使用。 您的可视化工具、curl、JDBC驱动
9000 TCP (原生协议) 用于ClickHouse原生二进制协议。由clickhouse-client、服务器间通信使用。 clickhouse-client
9004 MySQL 模拟协议 用于MySQL兼容接口。允许部分MySQL客户端与ClickHouse进行交互。 您的应用程序(使用3306端口时,需映射到此)

将下面的配置写入进values.yaml文件里:

# 使用DaoCloud镜像代理
global:
  imageRegistry: "m.daocloud.io/docker.io"
  security:
    allowInsecureImages: true

# 集群架构 - 主从复制
architecture: replication

# 认证配置
auth:
  username: "clickhouse"
  password: "clickhouse123"

# 服务配置 - NodePort访问
service:
  type: NodePort
  nodePorts:
    http: 30123
    mysql: 30046

# 暴露MySQL协议
exposeMysql: true

# 禁用持久化
persistence:
  enabled: false

# 简化探针配置
livenessProbe:
  enabled: false
readinessProbe:
  enabled: false

# 最小资源
resources:
  requests:
    memory: "512Mi"
    cpu: "250m"

# 使用内置Keeper
keeper:
  enabled: true
  replicaCount: 1
  persistence:
    enabled: false
  livenessProbe:
    enabled: false
  readinessProbe:
    enabled: false


安装clickhouse

# 安装clickhouse
sudo helm install clickhouse bitnami/clickhouse \
  --namespace clickhouse \
  --create-namespace \
  -f values.yaml

# 等待安装完成,查看状态
sudo kubectl get pods -n clickhouse -w

访问测试

# 宿主机测试:
curl http://localhost:8123/

# 其他机器测试:
curl http://10.10.151.201:8123/

期望的结果:

➜  ~ curl http://10.10.151.201:8123/
Ok.