我有一个任务,运行了大约两分钟做一些数据库操作。我启用了延迟确认,但我发现任务从来没有重试后,软时间限制超过。
所以我添加了autoretry_for and retry_backoof,正如我正在使用的celery版本的文档中提到的那样。对延迟确认的任务进行重试是一个好的实践吗?
如果我没有设置autoretry_for
,那么任务将不会从它终止的地方运行。
任务.py
@shared_task(soft_time_limit=20, autoretry_for=(SoftTimeLimitExceeded, ), retry_backoff=True)
def test_func_celery(current_time):
fake = Faker()
logger = logging.getLogger()
time1 = time.perf_counter()
main_time_end = current_time + 25
logger.info('Task started here \n \n')
while time.time() < main_time_end:
logger.info('Before doing main operations')
time.sleep(10)
logger.info(f'Inside Main operation ===>>> {time.time()}')
while True:
name = fake.first_name()
logger.info(f'Inside inner operation for {name}')
logger.info(f'inner operation before starts for {name}')
time.sleep(12)
logger.info(f'Inside inner {name}===>>> {time.time()}')
logger.info(f'inner operation after finishes for {name}')
break
logger.info('Completed Main operation')
break
if time.time() >= main_time_end:
logger.info('Out from main operations')
time2 = time.perf_counter()
logger.info(f'Total time =======>>>>>>> {time2 - time1}')
return None
查看次数.py
def home(request):
current_time = time.time()
test_func_celery.apply_async(args=(current_time, ), acks_late=True)
return HttpResponse("Celery being tested here")
celery .原木
[2020-01-21 15:28:30,708: INFO/ForkPoolWorker-7] Task started here
[2020-01-21 15:28:30,708: INFO/ForkPoolWorker-7] Before doing main operations
[2020-01-21 15:28:40,715: INFO/ForkPoolWorker-7] Inside Main operation ===>>> 1579599820.7153742
[2020-01-21 15:28:40,716: INFO/ForkPoolWorker-7] Inside inner operation for Joe
[2020-01-21 15:28:40,716: INFO/ForkPoolWorker-7] inner operation before starts for Joe
[2020-01-21 15:28:50,680: WARNING/MainProcess] Soft time limit (20s) exceeded for home.tasks.test_func_celery[4234d05e-672e-4565-bdb6-cd3d234c82c4]
[2020-01-21 15:28:50,726: INFO/MainProcess] Received task: home.tasks.test_func_celery[4234d05e-672e-4565-bdb6-cd3d234c82c4] ETA:[2020-01-21 15:28:51.683861+05:45]
[2020-01-21 15:28:50,728: INFO/ForkPoolWorker-7] Task home.tasks.test_func_celery[4234d05e-672e-4565-bdb6-cd3d234c82c4] retry: Retry in 1s: SoftTimeLimitExceeded()
[2020-01-21 15:28:51,938: INFO/ForkPoolWorker-7] Task started here
[2020-01-21 15:28:51,938: INFO/ForkPoolWorker-7] Before doing main operations
[2020-01-21 15:29:01,948: INFO/ForkPoolWorker-7] Inside Main operation ===>>> 1579599841.948412
[2020-01-21 15:29:01,949: INFO/ForkPoolWorker-7] Inside inner operation for Lisa
[2020-01-21 15:29:01,949: INFO/ForkPoolWorker-7] inner operation before starts for Lisa
[2020-01-21 15:29:11,915: WARNING/MainProcess] Soft time limit (20s) exceeded for home.tasks.test_func_celery[4234d05e-672e-4565-bdb6-cd3d234c82c4]
[2020-01-21 15:29:11,965: INFO/MainProcess] Received task: home.tasks.test_func_celery[4234d05e-672e-4565-bdb6-cd3d234c82c4] ETA:[2020-01-21 15:29:12.917995+05:45]
[2020-01-21 15:29:11,967: INFO/ForkPoolWorker-7] Task home.tasks.test_func_celery[4234d05e-672e-4565-bdb6-cd3d234c82c4] retry: Retry in 1s: SoftTimeLimitExceeded()
[2020-01-21 15:29:13,136: INFO/ForkPoolWorker-7] Task started here
[2020-01-21 15:29:13,136: INFO/ForkPoolWorker-7] Out from main operations
[2020-01-21 15:29:13,136: INFO/ForkPoolWorker-7] Total time =======>>>>>>> 0.00026752900157589465
[2020-01-21 15:29:13,138: INFO/ForkPoolWorker-7] Task home.tasks.test_func_celery[4234d05e-672e-4565-bdb6-cd3d234c82c4] succeeded in 0.05194566200225381s: None
有没有更好的解决方案来描述上面解释的场景?这将是一个真正的巨大帮助。谢谢。
1条答案
按热度按时间s4chpxco1#
将retry_backoff设置为False会导致任务在出现异常后立即运行。当任务依赖于外部服务器或网络时,建议将retry_backoff设置为True,以便在重试任务之前等待任何网络问题得到解决。如果任务没有外部依赖项,则不必将retry_backoff设置为True。