在Kubernetes集群中设置3个副本的StatefulSet,MongoDB处于ReplicaSet模式,绑定到本地持久化

nzkunb0c  于 2023-11-17  发布在  Go
关注(0)|答案(2)|浏览(155)

我从K8开始,我还没有完全理解如何使用PV + PVC + SCvolumeClaimTemplates + HostPath场景来部署一个绑定到本地磁盘的多个副本的StatefulSet。我的目标是部署一个MongoDB StatefulSet,其中设置了3个副本。(mongo的副本集)副本集模式,并将每个副本绑定到本地ssd。我做了一些测试,并得到了一些概念,以获得直。
场景a)使用PV + PVC + SC:如果在我的StatefulSet的容器中(设置副本:1)我声明了一个volumeMounts和一个Volume,我可以将其指向一个PVC,该PVC使用PV使用的SC,该SC指向物理本地SSD文件夹。概念是直接的,它都Map得很漂亮。如果我增加副本到更多,那么从第二个pod开始,它们将找不到要绑定的Volume..
这让我意识到,从该PV上的PVC保留的存储容量不会复制为StatefulSet中的Pod,并Map到每个创建的POD。
场景B)volumeClaimTemplates + HostPath
我注解掉了Volume,而是使用了volumeClaimTemplates,它确实像我在场景a中所期望的那样工作,对于每个创建的Pod,都会创建一个关联的声明,并为该Pod保留一些存储容量。这里也是一个非常简单的概念,但它只工作,只要我使用volumeClaimTemplates中的storageClassName: hostpath。我尝试使用我的SC和结果是相同的1 node(s) didn't find available persistent volumes to bind错误。
此外,当使用volumeClaimTemplates创建时,PV名称是无用的,并且与PVC开始时一样令人困惑。

vincenzocalia@vincenzos-MacBook-Air server-node % kubectl get pv
NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS      CLAIM                        STORAGECLASS   REASON   AGE
mg-pv                                      3Gi        RWO            Delete           Available                                mg-sc                   64s
pvc-32589cce-f472-40c9-b6e4-dc5e26c2177a   50Mi       RWO            Delete           Bound       default/mg-pv-cont-mongo-3   hostpath                36m
pvc-3e2f4e50-30f8-4ce8-8a62-0b923fd6aa79   50Mi       RWO            Delete           Bound       default/mg-pv-cont-mongo-1   hostpath                37m
pvc-8f4ff966-c30a-469f-a68d-ed579ef2a96f   50Mi       RWO            Delete           Bound       default/mg-pv-cont-mongo-4   hostpath                36m
pvc-9f8c933b-85d6-4024-8bd0-6668feee8757   50Mi       RWO            Delete           Bound       default/mg-pv-cont-mongo-2   hostpath                37m
pvc-d6c212f3-2391-4137-97c3-07836c90b8f3   50Mi       RWO            Delete           Bound       default/mg-pv-cont-mongo-0   hostpath                37m
vincenzocalia@vincenzos-MacBook-Air server-node % kubectl get pvc
NAME                 STATUS    VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
mg-pv-cont-mongo-0   Bound     pvc-d6c212f3-2391-4137-97c3-07836c90b8f3   50Mi       RWO            hostpath       37m
mg-pv-cont-mongo-1   Bound     pvc-3e2f4e50-30f8-4ce8-8a62-0b923fd6aa79   50Mi       RWO            hostpath       37m
mg-pv-cont-mongo-2   Bound     pvc-9f8c933b-85d6-4024-8bd0-6668feee8757   50Mi       RWO            hostpath       37m
mg-pv-cont-mongo-3   Bound     pvc-32589cce-f472-40c9-b6e4-dc5e26c2177a   50Mi       RWO            hostpath       37m
mg-pv-cont-mongo-4   Bound     pvc-8f4ff966-c30a-469f-a68d-ed579ef2a96f   50Mi       RWO            hostpath       37m
mg-pvc               Pending                                                                        mg-sc          74s

字符串
有没有什么方法可以设置volumeClaimTemplates的PV名称,使其在声明PV时更有用?
如何将volumeClaimTemplates的PV指向SSD,就像我在场景a中所做的那样?
非常感谢

PV

apiVersion: v1
kind: PersistentVolume
metadata:
  name: mg-pv
  labels:
    type: local
spec:
  capacity:
    storage: 3Gi
  persistentVolumeReclaimPolicy: Delete
  accessModes:
    - ReadWriteOnce
  volumeMode: Filesystem 
  storageClassName: mg-sc
  local:
    path: /Volumes/ProjectsSSD/k8s_local_volumes/mongo/mnt/data/unt
  nodeAffinity:
    required:
      nodeSelectorTerms:
        - matchExpressions:
            - key: kubernetes.io/hostname
              operator: In
              values:
                - docker-desktop

SC

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
 name: mg-sc
provisioner: kubernetes.io/no-provisioner
volumeBindingMode: WaitForFirstConsumer
reclaimPolicy: Delete

PVC

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: mg-pvc
spec:
  storageClassName: mg-sc
  # volumeName: mg-pv 
  resources:
    requests:
      # storage: 1Gi
      storage: 50Mi
  accessModes:
    - ReadWriteOnce

StatefulSet

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: mongo
spec:
  selector:
    matchLabels:
      role: mongo
      environment: test
  serviceName: 'mongo'
  replicas: 5
  template:
    metadata:
      labels:
        role: mongo
        environment: test
    spec:
      terminationGracePeriodSeconds: 10
      containers:
        - name: mongo
          image: mongo
          command:
            - mongod
            - '--bind_ip'
            - all
            - '--replSet'
            - rs0
            # - "--smallfiles"
            # - "--noprealloc"
          ports:
            - containerPort: 27017
          volumeMounts:
            - name: mg-pv-cont
              mountPath: /data/db
        - name: mongo-sidecar
          image: cvallance/mongo-k8s-sidecar
          env:
            - name: MONGO_SIDECAR_POD_LABELS
              value: 'role=mongo,environment=test'
            - name: KUBERNETES_MONGO_SERVICE_NAME
              value: 'mongo'

      ### using volumes you have to have one persistent volume for each created pod..useful only for static set of pods
      # volumes:
      #   - name: mg-pv-cont
      #     persistentVolumeClaim:
      #       claimName: mg-pvc
  ## volume claim templates create a claim for each created pos, so if scaling up or down the number of pod they¡ll clame their own space in the persistent volume.
  volumeClaimTemplates:
    - metadata:
        name: mg-pv-cont # this binds
        # name: mg-pv-pvc-template     # same name as volumeMounts or it won't bind.
        ### Waiting for deployments to stabilize...
        ### - statefulset/mongo: Waiting for statefulset spec update to be observed...
      spec:
        # storageClassName: mg-sc
        storageClassName: hostpath
        accessModes: ['ReadWriteOnce']
        resources:
          requests:
            storage: 50Mi

bkhjykvo

bkhjykvo1#

好吧,在对它进行了更多的摆弄并测试了更多的配置之后,我发现PVCPV的绑定发生在1:1方式,所以一旦PV绑定到一个声明,(PVCvolumeClaimTemplates)没有其他声明可以绑定到它。所以解决方案就是创建许多PV,就像你希望创建的那样。还有一些额外的用于扩展和缩小StatefulSet的副本。现在在volumeClaimTemplates: spec: storageClassName:中,你可以使用你定义的SC,这样那些PV就可以使用了。如果使用volumeClassTemplatesPVC就没有用了。。它只会创建一个声明,没有人使用..
希望这将帮助其他人开始在Kubernetes世界。干杯。

tyg4sfes

tyg4sfes2#

我在SC自动创建PV PVC后使用SC + PVC<-->,在我的配置成员mongodb with replica set = 3中,我看到当我删除pod primary然后不resyc到SECONDARY现在当我删除pod然后status --> not reachable/healthy我认为PV不匹配PVC的问题。

相关问题