docker Bitnami气流调度程序无法连接数据库,而Web服务器可以连接,即使他们有相同的属性?

soat7uwm  于 2023-10-16  发布在  Docker
关注(0)|答案(1)|浏览(85)

我想在openshift上配置气流。
我在openshift上设置数据库如下:

kind: Service
apiVersion: v1
metadata:
  name: airflow-database
  namespace: ersin-poc
spec:
  ports:
    - name: 5432-tcp
      protocol: TCP
      port: 5432
      targetPort: 5432
  selector:
    deployment: airflow-database
  type: ClusterIP
  sessionAffinity: None
  ipFamilies:
    - IPv4
  ipFamilyPolicy: SingleStack

我的数据库部署如下:

kind: Deployment
apiVersion: apps/v1
metadata:
  name: airflow-database
  namespace: ersin-poc 
labels:
  deployment: airflow-database
spec:
  replicas: 1
  selector:
    matchLabels:
      deployment: airflow-database
  template:
    metadata:
      creationTimestamp: null
      labels:
        deployment: airflow-database
    spec:
      volumes:
        - name: generic
          persistentVolumeClaim:
            claimName: generic
        - name: empty1
          emptyDir: {}
      containers:
        - resources: {}
          name: airflow-database
          env:
            - name: POSTGRESQL_USERNAME
              value: 'bn_airflow'
            - name: POSTGRESQL_PASSWORD
              value: 'bitnami1'
            - name: POSTGRESQL_DATABASE
              value: 'bitnami_airflow'
          ports:
            - containerPort: 5432
              protocol: TCP
          volumeMounts:
            - name: generic
              mountPath: /bitnami/postgresql/
          image: >-
            bitnami/postgresql:latest
          hostname: airflow-database

我可以像下面这样从我的Web服务器连接这个DB:

kind: Deployment
apiVersion: apps/v1
metadata:
  name: airflow-webserver
  namespace: ersin-poc 
labels:
  deployment: airflow-webserver
spec:
  replicas: 1
  selector:
    matchLabels:
      deployment: airflow-webserver
  template:
    metadata:
      creationTimestamp: null
      labels:
        deployment: airflow-webserver
    spec:
      volumes:
        - name: generic
          persistentVolumeClaim:
            claimName: generic
        - name: empty1
          emptyDir: {}
      containers:
        - resources: {}
          name: airflow-webserver
          env:
            - name: AIRFLOW_HOME
              value: /home/appuser
            - name: USER
              value: appuser
            - name: AIRFLOW_FERNET_KEY
              value: '46BKJoQYlPPOexq0OhDZnIlNepKFf87WFwLbfzqDDho='
            - name: AIRFLOW_SECRET_KEY
              value: 'a25mQ1FHTUh3MnFRSk5KMEIyVVU2YmN0VGRyYTVXY08='
            - name: AIRFLOW_EXECUTOR
              value: 'CeleryExecutor'
            - name: AIRFLOW_DATABASE_NAME
              value: 'bitnami_airflow'
            - name: AIRFLOW_DATABASE_USERNAME
              value: 'bn_airflow'
            - name: AIRFLOW_DATABASE_PASSWORD
              value: 'bitnami1'
            - name: AIRFLOW_LOAD_EXAMPLES
              value: 'yes'
            - name: AIRFLOW_PASSWORD
              value: 'bitnami123'
            - name: AIRFLOW_USERNAME
              value: 'user'
            - name: AIRFLOW_EMAIL
              value: '[email protected]'
            - name: AIRFLOW_DATABASE_HOST
              value: 'airflow-database'
            - name: AIRFLOW_DATABASE_PORT_NUMBER
              value: '5432'
          ports:
            - containerPort: 8080
              protocol: TCP
          volumeMounts:
            - name: generic
              mountPath: /home/appuser
            - name: generic
              mountPath: /home/appuser/logs/
            - name: generic
              mountPath: /home/appuser/dags/
          image: >-
            bitnami/airflow:latest
          hostname: airflow-webserver

但当我尝试它与airflow-scheduler它给出错误:

airflow-scheduler 09:29:43.31 INFO ==> Trying to connect to the database server airflow-scheduler 09:30:47.42 ERROR ==> Could not connect to the database

我的调度yaml是:

kind: Deployment
apiVersion: apps/v1
metadata:
  name: airflow-scheduler
  namespace: ersin-poc 
labels:
  deployment: airflow-scheduler
spec:
  replicas: 1
  selector:
    matchLabels:
      deployment: airflow-scheduler
  template:
    metadata:
      labels:
        deployment: airflow-scheduler
    spec:
      volumes:
        - name: generic
          persistentVolumeClaim:
            claimName: generic
        - name: empty1
          emptyDir: {}
      containers:
        - resources: {}
          name: airflow-scheduler
          env:
            - name: AIRFLOW_HOME
              value: /home/appuser
            - name: USER
              value: appuser
            - name: AIRFLOW_FERNET_KEY
              value: '46BKJoQYlPPOexq0OhDZnIlNepKFf87WFwLbfzqDDho='
            - name: AIRFLOW_SECRET_KEY
              value: 'a25mQ1FHTUh3MnFRSk5KMEIyVVU2YmN0VGRyYTVXY08='
            - name: AIRFLOW_EXECUTOR
              value: 'CeleryExecutor'
            - name: AIRFLOW_DATABASE_NAME
              value: 'bitnami_airflow'
            - name: AIRFLOW_DATABASE_USERNAME
              value: 'bn_airflow'
            - name: AIRFLOW_DATABASE_PASSWORD
              value: 'bitnami1'
            - name: AIRFLOW_DATABASE_HOST
              value: 'airflow-database'
            - name: AIRFLOW_DATABASE_PORT_NUMBER
              value: '5432'
            - name: AIRFLOW_WEBSERVER_HOST
              value: 'airflow-webserver'
            - name: AIRFLOW_WEBSERVER_PORT_NUMBER
              value: '8080'
            - name: REDIS_HOST
              value: 'airflow-redis'
            - name: REDIS_PORT_NUMBER
              value: '6379'
          ports:
            - containerPort: 8080
              protocol: TCP
          volumeMounts:
            - name: generic
              mountPath: /home/appuser
            - name: generic
              mountPath: /home/appuser/logs/
            - name: generic
              mountPath: /home/appuser/dags/
          image: >-
            bitnami/airflow-scheduler:latest
          hostname: airflow-scheduler

所以我不能理解为什么我得到这个错误与相同的属性?
thanks in advance

编辑

我在scheduler pod中尝试这个命令,看看我是否可以连接到db:

psql -h airflow-database -p 5432 -U bn_airflow -d bitnami_airflow -W

通行证:bitnami 1

select * from public.ab_user;

是的,我可以。

a64a0gku

a64a0gku1#

经过大量的搜索,我决定使用apache/airflow图像。(posgresql和redis仍然是bitnami -这并不重要)
你可以在openshift上看到所有关于气流的ymal文件。
https://github.com/ersingulbahar/airflow_on_openshift
它现在按预期工作

相关问题