django 启用延迟确认时,超出软时限后Celery任务重试

xvw2m8pv  于 2023-02-06  发布在  Go
关注(0)|答案(1)|浏览(112)

我有一个任务,运行了大约两分钟做一些数据库操作。我启用了延迟确认,但我发现任务从来没有重试后,软时间限制超过。
所以我添加了autoretry_for and retry_backoof,正如我正在使用的celery版本的文档中提到的那样。对延迟确认的任务进行重试是一个好的实践吗?
如果我没有设置autoretry_for,那么任务将不会从它终止的地方运行。

任务.py

@shared_task(soft_time_limit=20, autoretry_for=(SoftTimeLimitExceeded, ), retry_backoff=True)
def test_func_celery(current_time):
    fake = Faker()
    logger = logging.getLogger()
    time1 = time.perf_counter()
    main_time_end = current_time + 25
    logger.info('Task started here \n \n')

    while time.time() < main_time_end:
        logger.info('Before doing main operations')
        time.sleep(10)
        logger.info(f'Inside Main operation ===>>> {time.time()}')

        while True:
            name = fake.first_name()
            logger.info(f'Inside inner operation for {name}')
            logger.info(f'inner operation before starts for {name}')
            time.sleep(12)
            logger.info(f'Inside inner {name}===>>> {time.time()}')
            logger.info(f'inner operation after finishes for {name}')
            break
        logger.info('Completed Main operation')
        break

    if time.time() >= main_time_end:
        logger.info('Out from main operations')
    time2 = time.perf_counter()
    logger.info(f'Total time =======>>>>>>> {time2 - time1}')
    return None

查看次数.py

def home(request):
    current_time = time.time()
    test_func_celery.apply_async(args=(current_time, ), acks_late=True)
    return HttpResponse("Celery being tested here")

celery .原木

[2020-01-21 15:28:30,708: INFO/ForkPoolWorker-7] Task started here 
 

[2020-01-21 15:28:30,708: INFO/ForkPoolWorker-7] Before doing main operations
[2020-01-21 15:28:40,715: INFO/ForkPoolWorker-7] Inside Main operation ===>>> 1579599820.7153742
[2020-01-21 15:28:40,716: INFO/ForkPoolWorker-7] Inside inner operation for Joe
[2020-01-21 15:28:40,716: INFO/ForkPoolWorker-7] inner operation before starts for Joe
[2020-01-21 15:28:50,680: WARNING/MainProcess] Soft time limit (20s) exceeded for home.tasks.test_func_celery[4234d05e-672e-4565-bdb6-cd3d234c82c4]
[2020-01-21 15:28:50,726: INFO/MainProcess] Received task: home.tasks.test_func_celery[4234d05e-672e-4565-bdb6-cd3d234c82c4]  ETA:[2020-01-21 15:28:51.683861+05:45] 
[2020-01-21 15:28:50,728: INFO/ForkPoolWorker-7] Task home.tasks.test_func_celery[4234d05e-672e-4565-bdb6-cd3d234c82c4] retry: Retry in 1s: SoftTimeLimitExceeded()
[2020-01-21 15:28:51,938: INFO/ForkPoolWorker-7] Task started here 
 

[2020-01-21 15:28:51,938: INFO/ForkPoolWorker-7] Before doing main operations
[2020-01-21 15:29:01,948: INFO/ForkPoolWorker-7] Inside Main operation ===>>> 1579599841.948412
[2020-01-21 15:29:01,949: INFO/ForkPoolWorker-7] Inside inner operation for Lisa
[2020-01-21 15:29:01,949: INFO/ForkPoolWorker-7] inner operation before starts for Lisa
[2020-01-21 15:29:11,915: WARNING/MainProcess] Soft time limit (20s) exceeded for home.tasks.test_func_celery[4234d05e-672e-4565-bdb6-cd3d234c82c4]
[2020-01-21 15:29:11,965: INFO/MainProcess] Received task: home.tasks.test_func_celery[4234d05e-672e-4565-bdb6-cd3d234c82c4]  ETA:[2020-01-21 15:29:12.917995+05:45] 
[2020-01-21 15:29:11,967: INFO/ForkPoolWorker-7] Task home.tasks.test_func_celery[4234d05e-672e-4565-bdb6-cd3d234c82c4] retry: Retry in 1s: SoftTimeLimitExceeded()
[2020-01-21 15:29:13,136: INFO/ForkPoolWorker-7] Task started here 
 

[2020-01-21 15:29:13,136: INFO/ForkPoolWorker-7] Out from main operations
[2020-01-21 15:29:13,136: INFO/ForkPoolWorker-7] Total time =======>>>>>>> 0.00026752900157589465
[2020-01-21 15:29:13,138: INFO/ForkPoolWorker-7] Task home.tasks.test_func_celery[4234d05e-672e-4565-bdb6-cd3d234c82c4] succeeded in 0.05194566200225381s: None

有没有更好的解决方案来描述上面解释的场景?这将是一个真正的巨大帮助。谢谢。

s4chpxco

s4chpxco1#

将retry_backoff设置为False会导致任务在出现异常后立即运行。当任务依赖于外部服务器或网络时,建议将retry_backoff设置为True,以便在重试任务之前等待任何网络问题得到解决。如果任务没有外部依赖项,则不必将retry_backoff设置为True。

相关问题