python 气流配置的SMTP出错

new9mtju  于 2023-02-11  发布在  Python
关注(0)|答案(1)|浏览(114)

配置后出现错误

*** Reading local file: /opt/airflow/logs/dag_id=DocImageExec/run_id=manual__2023-02-09T16:06:23.630116+00:00/task_id=execute_docker_command/attempt=1.log
[2023-02-09, 16:06:26 UTC] {taskinstance.py:1083} INFO - Dependencies all met for <TaskInstance: DocImageExec.execute_docker_command manual__2023-02-09T16:06:23.630116+00:00 [queued]>
[2023-02-09, 16:06:26 UTC] {taskinstance.py:1083} INFO - Dependencies all met for <TaskInstance: DocImageExec.execute_docker_command manual__2023-02-09T16:06:23.630116+00:00 [queued]>
[2023-02-09, 16:06:26 UTC] {taskinstance.py:1279} INFO - 
--------------------------------------------------------------------------------
[2023-02-09, 16:06:26 UTC] {taskinstance.py:1280} INFO - Starting attempt 1 of 2
[2023-02-09, 16:06:26 UTC] {taskinstance.py:1281} INFO - 
--------------------------------------------------------------------------------
[2023-02-09, 16:06:26 UTC] {taskinstance.py:1300} INFO - Executing <Task(BashOperator): execute_docker_command> on 2023-02-09 16:06:23.630116+00:00
[2023-02-09, 16:06:26 UTC] {standard_task_runner.py:55} INFO - Started process 17492 to run task
[2023-02-09, 16:06:26 UTC] {standard_task_runner.py:82} INFO - Running: ['***', 'tasks', 'run', 'DocImageExec', 'execute_docker_command', 'manual__2023-02-09T16:06:23.630116+00:00', '--job-id', '109', '--raw', '--subdir', 'DAGS_FOLDER/docimage.py', '--cfg-path', '/tmp/tmptln30ewq']
[2023-02-09, 16:06:26 UTC] {standard_task_runner.py:83} INFO - Job 109: Subtask execute_docker_command
[2023-02-09, 16:06:26 UTC] {task_command.py:388} INFO - Running <TaskInstance: DocImageExec.execute_docker_command manual__2023-02-09T16:06:23.630116+00:00 [running]> on host 14b5d43a840e
[2023-02-09, 16:06:26 UTC] {taskinstance.py:1509} INFO - Exporting the following env vars:
AIRFLOW_CTX_DAG_EMAIL=austin.jackson@xxxxx.com
AIRFLOW_CTX_DAG_OWNER=***
AIRFLOW_CTX_DAG_ID=DocImageExec
AIRFLOW_CTX_TASK_ID=execute_docker_command
AIRFLOW_CTX_EXECUTION_DATE=2023-02-09T16:06:23.630116+00:00
AIRFLOW_CTX_TRY_NUMBER=1
AIRFLOW_CTX_DAG_RUN_ID=manual__2023-02-09T16:06:23.630116+00:00
[2023-02-09, 16:06:26 UTC] {subprocess.py:63} INFO - Tmp dir root location: 
 /tmp
[2023-02-09, 16:06:26 UTC] {subprocess.py:75} INFO - Running command: ['/bin/bash', '-c', 'docker run -d -p 5000:5000 image-docker']
[2023-02-09, 16:06:26 UTC] {subprocess.py:86} INFO - Output:
[2023-02-09, 16:06:26 UTC] {subprocess.py:93} INFO - docker: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?.
[2023-02-09, 16:06:26 UTC] {subprocess.py:93} INFO - See 'docker run --help'.
[2023-02-09, 16:06:26 UTC] {subprocess.py:97} INFO - Command exited with return code 125
[2023-02-09, 16:06:27 UTC] {taskinstance.py:1768} ERROR - Task failed with exception
Traceback (most recent call last):
  File "/home/airflow/.local/lib/python3.7/site-packages/airflow/operators/bash.py", line 197, in execute
    f"Bash command failed. The command returned a non-zero exit code {result.exit_code}."
airflow.exceptions.AirflowException: Bash command failed. The command returned a non-zero exit code 125.
[2023-02-09, 16:06:27 UTC] {taskinstance.py:1323} INFO - Marking task as UP_FOR_RETRY. dag_id=DocImageExec, task_id=execute_docker_command, execution_date=20230209T160623, start_date=20230209T160626, end_date=20230209T160627
[2023-02-09, 16:06:27 UTC] {warnings.py:110} WARNING - /home/***/.local/lib/python3.7/site-packages/***/utils/email.py:152: RemovedInAirflow3Warning: Fetching SMTP credentials from configuration variables will be deprecated in a future release. Please set credentials using a connection instead.
  send_mime_email(e_from=mail_from, e_to=recipients, mime_msg=msg, conn_id=conn_id, dryrun=dryrun)

[2023-02-09, 16:06:27 UTC] {configuration.py:663} WARNING - section/key [smtp/smtp_user] not found in config
[2023-02-09, 16:06:27 UTC] {email.py:268} INFO - Email alerting: attempt 1
[2023-02-09, 16:06:27 UTC] {configuration.py:663} WARNING - section/key [smtp/smtp_user] not found in config
[2023-02-09, 16:06:27 UTC] {email.py:268} INFO - Email alerting: attempt 1
[2023-02-09, 16:06:27 UTC] {taskinstance.py:1831} ERROR - Failed to send email to: ['austin.jackson@xxxx.com']
Traceback (most recent call last):
  File "/home/airflow/.local/lib/python3.7/site-packages/airflow/models/taskinstance.py", line 1374, in _run_raw_task
    self._execute_task_with_callbacks(context, test_mode)
  File "/home/airflow/.local/lib/python3.7/site-packages/airflow/models/taskinstance.py", line 1520, in _execute_task_with_callbacks
    result = self._execute_task(context, task_orig)
  File "/home/airflow/.local/lib/python3.7/site-packages/airflow/models/taskinstance.py", line 1581, in _execute_task
    result = execute_callable(context=context)
  File "/home/airflow/.local/lib/python3.7/site-packages/airflow/operators/bash.py", line 197, in execute
    f"Bash command failed. The command returned a non-zero exit code {result.exit_code}."
airflow.exceptions.AirflowException: Bash command failed. The command returned a non-zero exit code 125.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/airflow/.local/lib/python3.7/site-packages/airflow/models/taskinstance.py", line 2231, in email_alert
    send_email(task.email, subject, html_content)
  File "/home/airflow/.local/lib/python3.7/site-packages/airflow/utils/email.py", line 91, in send_email
    **kwargs,
  File "/home/airflow/.local/lib/python3.7/site-packages/airflow/utils/email.py", line 152, in send_email_smtp
    send_mime_email(e_from=mail_from, e_to=recipients, mime_msg=msg, conn_id=conn_id, dryrun=dryrun)
  File "/home/airflow/.local/lib/python3.7/site-packages/airflow/utils/email.py", line 270, in send_mime_email
    smtp_conn = _get_smtp_connection(smtp_host, smtp_port, smtp_timeout, smtp_ssl)
  File "/home/airflow/.local/lib/python3.7/site-packages/airflow/utils/email.py", line 317, in _get_smtp_connection
    else smtplib.SMTP(host=host, port=port, timeout=timeout)
  File "/usr/local/lib/python3.7/smtplib.py", line 251, in __init__
    (code, msg) = self.connect(host, port)
  File "/usr/local/lib/python3.7/smtplib.py", line 336, in connect
    self.sock = self._get_socket(host, port, self.timeout)
  File "/usr/local/lib/python3.7/smtplib.py", line 307, in _get_socket
    self.source_address)
  File "/usr/local/lib/python3.7/socket.py", line 728, in create_connection
    raise err
  File "/usr/local/lib/python3.7/socket.py", line 716, in create_connection
    sock.connect(sa)
OSError: [Errno 99] Cannot assign requested address

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/airflow/.local/lib/python3.7/site-packages/airflow/models/taskinstance.py", line 1829, in handle_failure
    self.email_alert(error, task)
  File "/home/airflow/.local/lib/python3.7/site-packages/airflow/models/taskinstance.py", line 2233, in email_alert
    send_email(task.email, subject, html_content_err)
  File "/home/airflow/.local/lib/python3.7/site-packages/airflow/utils/email.py", line 91, in send_email
    **kwargs,
  File "/home/airflow/.local/lib/python3.7/site-packages/airflow/utils/email.py", line 152, in send_email_smtp
    send_mime_email(e_from=mail_from, e_to=recipients, mime_msg=msg, conn_id=conn_id, dryrun=dryrun)
  File "/home/airflow/.local/lib/python3.7/site-packages/airflow/utils/email.py", line 270, in send_mime_email
    smtp_conn = _get_smtp_connection(smtp_host, smtp_port, smtp_timeout, smtp_ssl)
  File "/home/airflow/.local/lib/python3.7/site-packages/airflow/utils/email.py", line 317, in _get_smtp_connection
    else smtplib.SMTP(host=host, port=port, timeout=timeout)
  File "/usr/local/lib/python3.7/smtplib.py", line 251, in __init__
    (code, msg) = self.connect(host, port)
  File "/usr/local/lib/python3.7/smtplib.py", line 336, in connect
    self.sock = self._get_socket(host, port, self.timeout)
  File "/usr/local/lib/python3.7/smtplib.py", line 307, in _get_socket
    self.source_address)
  File "/usr/local/lib/python3.7/socket.py", line 728, in create_connection
    raise err
  File "/usr/local/lib/python3.7/socket.py", line 716, in create_connection
    sock.connect(sa)
OSError: [Errno 99] Cannot assign requested address
[2023-02-09, 16:06:27 UTC] {standard_task_runner.py:105} ERROR - Failed to execute job 109 for task execute_docker_command (Bash command failed. The command returned a non-zero exit code 125.; 17492)
[2023-02-09, 16:06:27 UTC] {local_task_job.py:208} INFO - Task exited with return code 1
[2023-02-09, 16:06:27 UTC] {taskinstance.py:2578} INFO - 0 downstream tasks scheduled from follow-on schedule check

脚本如下所示

"""
Code that goes along with the Airflow located at:
http://airflow.readthedocs.org/en/latest/tutorial.html
"""
import airflow 
from airflow import DAG
from airflow.operators.bash_operator import BashOperator
from datetime import datetime, timedelta
from airflow.operators.docker_operator import DockerOperator
from airflow.operators.python_operator import PythonOperator 
from airflow.operators.email_operator import EmailOperator

default_args = {
    "owner": "airflow",
    "depends_on_past": False,
    "start_date": datetime(2023, 2, 8),
    "email": ["austin.jackson@xxxx.com"],
    #"email_on_failure": False,
    "email_on_failure": True,
    "email_on_retry": True,
    "retries": 1,
    "retry_delay": timedelta(minutes=5),
    # 'queue': 'bash_queue',
    # 'pool': 'backfill',
    # 'priority_weight': 10,
    # 'end_date': datetime(2016, 1, 1),
}

dag = DAG("DocImageExec", default_args=default_args, schedule_interval=timedelta(1))

# t1, t2 and t3 are examples of tasks created by instantiating operators
t1 = BashOperator(task_id="execute_docker_command", bash_command="docker run -d -p 5000:5000 image-v1", dag=dag)

t1

请帮助与气流的邮件工作所需的正确配置，我需要为Apache气流警报Office 365邮件集成。
下面是代码：
"""代码与气流一起位于：http://airflow.readthedocs.org/en/latest/tutorial.html """

import airflow 
from airflow import DAG
from airflow.operators.bash_operator import BashOperator
from datetime import datetime, timedelta
from airflow.operators.docker_operator import DockerOperator
from airflow.operators.python_operator import PythonOperator 
from airflow.operators.email_operator import EmailOperator

default_args = {
    "owner": "airflow",
    "depends_on_past": False,
    "start_date": datetime(2023, 2, 8),
    "email": ["austin.jackson@xxxx.com"],
    #"email_on_failure": False,
    "email_on_failure": True,
    "email_on_retry": True,
    "retries": 1,
    "retry_delay": timedelta(minutes=5),
    # 'queue': 'bash_queue',
    # 'pool': 'backfill',
    # 'priority_weight': 10,
    # 'end_date': datetime(2016, 1, 1),
}

dag = DAG("DocImageExec", default_args=default_args, schedule_interval=timedelta(1))

# t1 example of tasks created by instantiating operators
t1 = BashOperator(task_id="execute_docker_command", bash_command="docker run -d -p 6000:6000 image-docker", dag=dag)

t1

正如请检查代码输出，所以请帮助，因为SMTP是给错误，无法发送后/电子邮件到所需的电子邮件ID

python

1条答案

按热度按时间
x4shl7ld

x4shl7ld1#

你提到的错误是一些不同的东西与SMTP无关，首先弄清楚。它说docker demon没有运行？你是如何运行airflow standalone/K8s（我猜是独立的）。通常设置SMTP发送警报的电子邮件最好使用EmailOperator
1.为SMTP启用IMAP
1.使用smtp_host、SSL、用户、密码和端口等smtp详细信息更新气流配置文件。
1.在对话框中使用EmailOperator
它应该能完成任务。

赞(0）分享  回复(0）举报  2023-02-11
我来回答

相关问题

    查看更多

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新问答

    更多
    • 蜀ICP备2022004421号-2 NULL123 https://www.null123.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com