我按照offcial website of Airflow生成我的Airflow DAG连接到DB2。当我运行DAG插入数据或更新数据时,会引发jaydebeapi. Error。即使Airflow引发了jaydebeapi.Error,数据仍然在DB2中成功插入/更新。Airflow UI上的DAG将被标记为FAILED。我不知道我错过了哪些步骤。
我的DAG代码片段:
with DAG("my_dag1", default_args=default_args,
schedule_interval="@daily", catchup=False) as dag:
cerating_table = JdbcOperator(
task_id='creating_table',
jdbc_conn_id='db2',
sql=r"""
insert into DB2ECIF.T2(C1,C1_DATE) VALUES('TEST',CURRENT DATE);
""",
autocommit=True,
dag=dag
)
DAG日志:
[2022-06-20 02:16:03,743] {base.py:68} INFO - Using connection ID 'db2' for task execution.
[2022-06-20 02:16:04,785] {dbapi.py:213} INFO - Running statement:
insert into DB2ECIF.T2(C1,C1_DATE) VALUES('TEST',CURRENT DATE);
, parameters: None
[2022-06-20 02:16:04,842] {dbapi.py:221} INFO - Rows affected: 1
[2022-06-20 02:16:04,844] {taskinstance.py:1889} ERROR - Task failed with exception
Traceback (most recent call last):
File "/home/airflow/.local/lib/python3.7/site-packages/airflow/providers/jdbc/operators/jdbc.py", line 76, in execute
return hook.run(self.sql, self.autocommit, parameters=self.parameters, handler=fetch_all_handler)
File "/home/airflow/.local/lib/python3.7/site-packages/airflow/hooks/dbapi.py", line 195, in run
result = handler(cur)
File "/home/airflow/.local/lib/python3.7/site-packages/airflow/providers/jdbc/operators/jdbc.py", line 30, in fetch_all_handler
return cursor.fetchall()
File "/home/airflow/.local/lib/python3.7/site-packages/jaydebeapi/__init__.py", line 596, in fetchall
row = self.fetchone()
File "/home/airflow/.local/lib/python3.7/site-packages/jaydebeapi/__init__.py", line 561, in fetchone
raise Error()
jaydebeapi.Error
[2022-06-20 02:16:04,847] {taskinstance.py:1400} INFO - Marking task as FAILED. dag_id=my_dag1, task_id=creating_table, execution_date=20210101T000000, start_date=, end_date=20220620T021604
我已经安装了所需的python包的气流。列表如下:
程序包(系统)名称/版本
1.气流/2.3.2
- IBM db2 /11.5.7
1.开放式JDK/15.0.2 - JayDeBeApi/1.2.0版本的开发工具
- JPype 1/0.7.2版本
- Apache气流提供程序-jdbc/3.0.0
我已经尝试使用最新版本的项目4(1.2.3)和项目5(1.4.0)仍然不工作。我还将Airflow版本降级为2.2.3或2.2.5,得到了相同的结果。
如何解决这个问题呢?
1条答案
按热度按时间eh57zj3b1#
该错误不会在原始插入查询中发生,而是由于此PR -https://github.com/apache/airflow/pull/23817中引入的fetchall
使用
apache-airflow-providers-jdbc/2.1.3
可能是一种简单的解决方法。要获取根本原因,请在Airflow中设置DEBUG日志记录级别,并查看fetchall导致错误的原因。