如何安排Azure数据工厂管道在每月的第n个工作日运行

ghhaqwfi  于 2023-06-24  发布在  其他
关注(0)|答案(2)|浏览(105)

我需要安排一个Azure数据工厂管道,每个月的第10个工作日运行一次。星期六和星期日不是工作日。例如,如果不包括星期六和星期日,则2023年6月的第10个工作日福尔斯6月14日。下一个第10个工作日福尔斯28日,依此类推。因此,ADF管道应在这些日期运行。
尝试翻转窗口触发器,但不知道如何实现它。

dkqlctbz

dkqlctbz1#

使用常规ADF触发器无法实现此目的。
您可以在几个月内触发管道,即10,11,12,13,14,然后运行一些代码来检查今天是否是工作日。您还需要使用一些配置表或文件来检查接下来的几天您的进程是否尚未运行。

hgc7kmma

hgc7kmma2#

您可以执行以下步骤来实现要求。触发器可能无法帮助您实现您的要求。我使用管道活动来确定一个月中10的倍数的工作日。以下是我使用过的步骤。

  • 首先获取特定月份的开始日期。使用一个until循环,它会迭代到每月的最后一天。
  • 在until活动中,我使用了可变活动的组合来获取该月的所有日期。
  • 现在使用过滤器活动过滤出星期日或星期六的日子。
  • 最后,选择是n的倍数的工作日(例如n=10)。
  • 以下是上面的完整管道JSON:
{
    "name": "pipeline2",
    "properties": {
        "activities": [
            {
                "name": "Until1",
                "type": "Until",
                "dependsOn": [
                    {
                        "activity": "month_start",
                        "dependencyConditions": [
                            "Succeeded"
                        ]
                    }
                ],
                "userProperties": [],
                "typeProperties": {
                    "expression": {
                        "value": "@equals(variables('flag'),startOfMonth(addDays(variables('month_start'),35)))",
                        "type": "Expression"
                    },
                    "activities": [
                        {
                            "name": "append date",
                            "type": "AppendVariable",
                            "dependsOn": [],
                            "userProperties": [],
                            "typeProperties": {
                                "variableName": "dates",
                                "value": {
                                    "value": "@variables('flag')",
                                    "type": "Expression"
                                }
                            }
                        },
                        {
                            "name": "update flag using temp",
                            "type": "SetVariable",
                            "dependsOn": [
                                {
                                    "activity": "append date",
                                    "dependencyConditions": [
                                        "Succeeded"
                                    ]
                                }
                            ],
                            "policy": {
                                "timeout": "0.12:00:00",
                                "retry": 0,
                                "retryIntervalInSeconds": 30,
                                "secureOutput": false,
                                "secureInput": false
                            },
                            "userProperties": [],
                            "typeProperties": {
                                "variableName": "temp",
                                "value": {
                                    "value": "@addDays(variables('flag'),1)",
                                    "type": "Expression"
                                }
                            }
                        },
                        {
                            "name": "update flag",
                            "type": "SetVariable",
                            "dependsOn": [
                                {
                                    "activity": "update flag using temp",
                                    "dependencyConditions": [
                                        "Succeeded"
                                    ]
                                }
                            ],
                            "policy": {
                                "timeout": "0.12:00:00",
                                "retry": 0,
                                "retryIntervalInSeconds": 30,
                                "secureOutput": false,
                                "secureInput": false
                            },
                            "userProperties": [],
                            "typeProperties": {
                                "variableName": "flag",
                                "value": {
                                    "value": "@variables('temp')",
                                    "type": "Expression"
                                }
                            }
                        }
                    ],
                    "timeout": "0.12:00:00"
                }
            },
            {
                "name": "initialize flag",
                "type": "SetVariable",
                "dependsOn": [],
                "policy": {
                    "timeout": "0.12:00:00",
                    "retry": 0,
                    "retryIntervalInSeconds": 30,
                    "secureOutput": false,
                    "secureInput": false
                },
                "userProperties": [],
                "typeProperties": {
                    "variableName": "flag",
                    "value": {
                        "value": "@formatDateTime(startOfMonth(utcNow()))",
                        "type": "Expression"
                    }
                }
            },
            {
                "name": "month_start",
                "type": "SetVariable",
                "dependsOn": [
                    {
                        "activity": "initialize flag",
                        "dependencyConditions": [
                            "Succeeded"
                        ]
                    }
                ],
                "policy": {
                    "timeout": "0.12:00:00",
                    "retry": 0,
                    "retryIntervalInSeconds": 30,
                    "secureOutput": false,
                    "secureInput": false
                },
                "userProperties": [],
                "typeProperties": {
                    "variableName": "month_start",
                    "value": {
                        "value": "@variables('flag')",
                        "type": "Expression"
                    }
                }
            },
            {
                "name": "Filter1",
                "type": "Filter",
                "dependsOn": [
                    {
                        "activity": "Until1",
                        "dependencyConditions": [
                            "Succeeded"
                        ]
                    }
                ],
                "userProperties": [],
                "typeProperties": {
                    "items": {
                        "value": "@variables('dates')",
                        "type": "Expression"
                    },
                    "condition": {
                        "value": "@not(or(equals(dayOfWeek(item()),0),equals(dayOfWeek(item()),6)))",
                        "type": "Expression"
                    }
                }
            },
            {
                "name": "ForEach1",
                "type": "ForEach",
                "dependsOn": [
                    {
                        "activity": "Filter1",
                        "dependencyConditions": [
                            "Succeeded"
                        ]
                    }
                ],
                "userProperties": [],
                "typeProperties": {
                    "items": {
                        "value": "@range(0,length(activity('Filter1').output.value))",
                        "type": "Expression"
                    },
                    "isSequential": true,
                    "activities": [
                        {
                            "name": "If Condition1",
                            "type": "IfCondition",
                            "dependsOn": [],
                            "userProperties": [],
                            "typeProperties": {
                                "expression": {
                                    "value": "@equals(mod(add(item(),1),pipeline().parameters.nth_day),0)",
                                    "type": "Expression"
                                },
                                "ifTrueActivities": [
                                    {
                                        "name": "Set variable1",
                                        "type": "SetVariable",
                                        "dependsOn": [],
                                        "policy": {
                                            "timeout": "0.12:00:00",
                                            "retry": 0,
                                            "retryIntervalInSeconds": 30,
                                            "secureOutput": false,
                                            "secureInput": false
                                        },
                                        "userProperties": [],
                                        "typeProperties": {
                                            "variableName": "tp",
                                            "value": {
                                                "value": "@activity('Filter1').output.value[item()]",
                                                "type": "Expression"
                                            }
                                        }
                                    }
                                ]
                            }
                        }
                    ]
                }
            }
        ],
        "parameters": {
            "nth_day": {
                "type": "int",
                "defaultValue": 10
            }
        },
        "variables": {
            "flag": {
                "type": "String"
            },
            "temp": {
                "type": "String"
            },
            "dates": {
                "type": "Array"
            },
            "month_start": {
                "type": "String"
            },
            "final_dates_to_trigger": {
                "type": "Array"
            },
            "tp": {
                "type": "String"
            }
        },
        "annotations": []
    }
}
  • 6月第10个工作日

  • 6月第20个工作日

  • 您可以将if条件中的set变量活动替换为execute pipeline活动,并计划每天运行此管道。您还必须更改条件,以检查今天的日期是否等于过滤器活动输出日期的第n个(n=10)个元素中的任何一个。

相关问题