我想在openshift上配置气流。
我在openshift上设置数据库如下:
kind: Service
apiVersion: v1
metadata:
name: airflow-database
namespace: ersin-poc
spec:
ports:
- name: 5432-tcp
protocol: TCP
port: 5432
targetPort: 5432
selector:
deployment: airflow-database
type: ClusterIP
sessionAffinity: None
ipFamilies:
- IPv4
ipFamilyPolicy: SingleStack
我的数据库部署如下:
kind: Deployment
apiVersion: apps/v1
metadata:
name: airflow-database
namespace: ersin-poc
labels:
deployment: airflow-database
spec:
replicas: 1
selector:
matchLabels:
deployment: airflow-database
template:
metadata:
creationTimestamp: null
labels:
deployment: airflow-database
spec:
volumes:
- name: generic
persistentVolumeClaim:
claimName: generic
- name: empty1
emptyDir: {}
containers:
- resources: {}
name: airflow-database
env:
- name: POSTGRESQL_USERNAME
value: 'bn_airflow'
- name: POSTGRESQL_PASSWORD
value: 'bitnami1'
- name: POSTGRESQL_DATABASE
value: 'bitnami_airflow'
ports:
- containerPort: 5432
protocol: TCP
volumeMounts:
- name: generic
mountPath: /bitnami/postgresql/
image: >-
bitnami/postgresql:latest
hostname: airflow-database
我可以像下面这样从我的Web服务器连接这个DB:
kind: Deployment
apiVersion: apps/v1
metadata:
name: airflow-webserver
namespace: ersin-poc
labels:
deployment: airflow-webserver
spec:
replicas: 1
selector:
matchLabels:
deployment: airflow-webserver
template:
metadata:
creationTimestamp: null
labels:
deployment: airflow-webserver
spec:
volumes:
- name: generic
persistentVolumeClaim:
claimName: generic
- name: empty1
emptyDir: {}
containers:
- resources: {}
name: airflow-webserver
env:
- name: AIRFLOW_HOME
value: /home/appuser
- name: USER
value: appuser
- name: AIRFLOW_FERNET_KEY
value: '46BKJoQYlPPOexq0OhDZnIlNepKFf87WFwLbfzqDDho='
- name: AIRFLOW_SECRET_KEY
value: 'a25mQ1FHTUh3MnFRSk5KMEIyVVU2YmN0VGRyYTVXY08='
- name: AIRFLOW_EXECUTOR
value: 'CeleryExecutor'
- name: AIRFLOW_DATABASE_NAME
value: 'bitnami_airflow'
- name: AIRFLOW_DATABASE_USERNAME
value: 'bn_airflow'
- name: AIRFLOW_DATABASE_PASSWORD
value: 'bitnami1'
- name: AIRFLOW_LOAD_EXAMPLES
value: 'yes'
- name: AIRFLOW_PASSWORD
value: 'bitnami123'
- name: AIRFLOW_USERNAME
value: 'user'
- name: AIRFLOW_EMAIL
value: '[email protected]'
- name: AIRFLOW_DATABASE_HOST
value: 'airflow-database'
- name: AIRFLOW_DATABASE_PORT_NUMBER
value: '5432'
ports:
- containerPort: 8080
protocol: TCP
volumeMounts:
- name: generic
mountPath: /home/appuser
- name: generic
mountPath: /home/appuser/logs/
- name: generic
mountPath: /home/appuser/dags/
image: >-
bitnami/airflow:latest
hostname: airflow-webserver
但当我尝试它与airflow-scheduler它给出错误:
airflow-scheduler 09:29:43.31 INFO ==> Trying to connect to the database server airflow-scheduler 09:30:47.42 ERROR ==> Could not connect to the database
我的调度yaml是:
kind: Deployment
apiVersion: apps/v1
metadata:
name: airflow-scheduler
namespace: ersin-poc
labels:
deployment: airflow-scheduler
spec:
replicas: 1
selector:
matchLabels:
deployment: airflow-scheduler
template:
metadata:
labels:
deployment: airflow-scheduler
spec:
volumes:
- name: generic
persistentVolumeClaim:
claimName: generic
- name: empty1
emptyDir: {}
containers:
- resources: {}
name: airflow-scheduler
env:
- name: AIRFLOW_HOME
value: /home/appuser
- name: USER
value: appuser
- name: AIRFLOW_FERNET_KEY
value: '46BKJoQYlPPOexq0OhDZnIlNepKFf87WFwLbfzqDDho='
- name: AIRFLOW_SECRET_KEY
value: 'a25mQ1FHTUh3MnFRSk5KMEIyVVU2YmN0VGRyYTVXY08='
- name: AIRFLOW_EXECUTOR
value: 'CeleryExecutor'
- name: AIRFLOW_DATABASE_NAME
value: 'bitnami_airflow'
- name: AIRFLOW_DATABASE_USERNAME
value: 'bn_airflow'
- name: AIRFLOW_DATABASE_PASSWORD
value: 'bitnami1'
- name: AIRFLOW_DATABASE_HOST
value: 'airflow-database'
- name: AIRFLOW_DATABASE_PORT_NUMBER
value: '5432'
- name: AIRFLOW_WEBSERVER_HOST
value: 'airflow-webserver'
- name: AIRFLOW_WEBSERVER_PORT_NUMBER
value: '8080'
- name: REDIS_HOST
value: 'airflow-redis'
- name: REDIS_PORT_NUMBER
value: '6379'
ports:
- containerPort: 8080
protocol: TCP
volumeMounts:
- name: generic
mountPath: /home/appuser
- name: generic
mountPath: /home/appuser/logs/
- name: generic
mountPath: /home/appuser/dags/
image: >-
bitnami/airflow-scheduler:latest
hostname: airflow-scheduler
所以我不能理解为什么我得到这个错误与相同的属性?
thanks in advance
编辑
我在scheduler pod中尝试这个命令,看看我是否可以连接到db:
psql -h airflow-database -p 5432 -U bn_airflow -d bitnami_airflow -W
通行证:bitnami 1
select * from public.ab_user;
是的,我可以。
1条答案
按热度按时间a64a0gku1#
经过大量的搜索,我决定使用apache/airflow图像。(posgresql和redis仍然是bitnami -这并不重要)
你可以在openshift上看到所有关于气流的ymal文件。
https://github.com/ersingulbahar/airflow_on_openshift
它现在按预期工作