AWS DMS - PostgreSQL枚举和数据范围的输入值无效

aiazj4mn  于 2023-01-02  发布在  PostgreSQL
关注(0)|答案(1)|浏览(174)

我正在使用数据库迁移工具(DMS)执行从PostgreSQL服务器到AWS RDS的逻辑复制。大多数表都迁移成功了,但有几个表出现了问题。
当我运行任务来加载表时,我在RDS的日志中得到了这个错误:

ERROR:  invalid input value for enum property_type: ""
CONTEXT:  unnamed portal parameter $17 = ''
STATEMENT:  INSERT INTO "public"."loans"("id","account_id","loan_number","created_at","updated_at","folio","mers_min","mers_status","mers_status_date","application_number","servicer","servicer_loan_number","status","primary_borrower_last_name","primary_borrower_first_name","property_number_of_units","property_type","property_address_line1","property_address_line2","property_city","property_state","property_zip","property_county_code","property_country_code","property_census_tract_code","property_parcel_id","mortgage_type","qm_loan","purpose","ltv","amortization_type","amount","interest_rate","term","lien_priority","application_date","approval_date","rejected_date","closing_date","funding_date","purchase_date","source","officer","processor","underwriter","appraiser","property_usage","fha_case_number","var_payload","approval_type","approval_message","housing_expense_ratio","total_debt_expense_ratio","subordinate_financing_amount","combined_ltv","property_appraised_value","property_purchase_price","property_appraised_date","property_year_built","credit_score","au_type","au_recommendation","lender_product","heloc_indicator","reverse_indicator","property_pud_indicator","closer","additional_financing_amount","rejected_reason","mortgage_insurance_certificate_number","mortgage_insurance_coverage_amount","mortgage_insurance_premium","day_one_certainty","first_payment_date","maturity_date","principal_and_interest_payment_amount","is_portfolio","financing_concessions_amount","sales_concessions_amount","transaction_costs_amount","is_investment_quality","application_received_date","lender_program","refinance_cash_out_type") values ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,$14,$15,$16,$17,$18,$19,$20,$21,$22,$23,$24,$25,$26,$27,$28,$29,$30,$31,$32,$33,$34,$35,$36,$37,$38,$39,$40,$41,$42,$43,$44,$45,$46,$47,$48,$49,$50,$51,$52,$53,$54,$55,$56,$57,$58,$59,$60,$61,$62,$63,$64,$65,$66,$67,$68,$69,$70,$71,$72,$73,$74,$75,$76,$77,$78,$79,$80,$81,$82,$83,$84)

property_type的类型是可空的enum,其取值为CondominiumCooperativeManufacturedHomeSingleFamilyTownhouseTwoToFourFamily
我在同一个数据库的另一个表中遇到了类似的问题,我得到:

ERROR:  malformed range literal: ""
DETAIL:  Missing left parenthesis or bracket.
CONTEXT:  unnamed portal parameter $3 = ''
STATEMENT:  INSERT INTO "public"."selections_sampling_data"("id","sow_id","period","loan_number","field_data","selected_on","substituted_on","substitution_for_id","received_for_review_on","created_at","updated_at","selected_for","selection_reason","selections_sampling_strategy_id") values ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,$14)

这个参数没有明确告诉我是哪一列产生了错误,但是参数$3应该与period列相对应,period列是daterange类型。
在这两种情况下,DMS任务似乎都在查询空字符串作为值,这是这些数据类型所不允许的。我不知道这是我的架构错误,还是我如何配置DMS任务。

[更新]

我的任务配置如下:

{
    "StreamBufferSettings": {
        "StreamBufferCount": 3,
        "CtrlStreamBufferSizeInMB": 5,
        "StreamBufferSizeInMB": 8
    },
    "ErrorBehavior": {
        "FailOnNoTablesCaptured": true,
        "ApplyErrorUpdatePolicy": "LOG_ERROR",
        "FailOnTransactionConsistencyBreached": false,
        "RecoverableErrorThrottlingMax": 1800,
        "DataErrorEscalationPolicy": "SUSPEND_TABLE",
        "ApplyErrorEscalationCount": 0,
        "RecoverableErrorStopRetryAfterThrottlingMax": true,
        "RecoverableErrorThrottling": true,
        "ApplyErrorFailOnTruncationDdl": false,
        "DataTruncationErrorPolicy": "LOG_ERROR",
        "ApplyErrorInsertPolicy": "LOG_ERROR",
        "EventErrorPolicy": "IGNORE",
        "ApplyErrorEscalationPolicy": "LOG_ERROR",
        "RecoverableErrorCount": -1,
        "DataErrorEscalationCount": 0,
        "TableErrorEscalationPolicy": "STOP_TASK",
        "RecoverableErrorInterval": 5,
        "ApplyErrorDeletePolicy": "IGNORE_RECORD",
        "TableErrorEscalationCount": 0,
        "FullLoadIgnoreConflicts": true,
        "DataErrorPolicy": "LOG_ERROR",
        "TableErrorPolicy": "SUSPEND_TABLE"
    },
    "ValidationSettings": {
        "ValidationPartialLobSize": 0,
        "PartitionSize": 10000,
        "RecordFailureDelayLimitInMinutes": 0,
        "SkipLobColumns": false,
        "FailureMaxCount": 10000,
        "HandleCollationDiff": false,
        "ValidationQueryCdcDelaySeconds": 0,
        "ValidationMode": "ROW_LEVEL",
        "TableFailureMaxCount": 1000,
        "RecordFailureDelayInMinutes": 5,
        "MaxKeyColumnSize": 8096,
        "EnableValidation": true,
        "ThreadCount": 5,
        "RecordSuspendDelayInMinutes": 30,
        "ValidationOnly": false
    },
    "TTSettings": {
        "TTS3Settings": null,
        "TTRecordSettings": null,
        "EnableTT": false
    },
    "FullLoadSettings": {
        "CommitRate": 1000,
        "StopTaskCachedChangesApplied": false,
        "StopTaskCachedChangesNotApplied": false,
        "MaxFullLoadSubTasks": 2,
        "TransactionConsistencyTimeout": 600,
        "CreatePkAfterFullLoad": false,
        "TargetTablePrepMode": "DO_NOTHING"
    },
    "TargetMetadata": {
        "ParallelApplyBufferSize": 0,
        "ParallelApplyQueuesPerThread": 0,
        "ParallelApplyThreads": 0,
        "TargetSchema": "",
        "InlineLobMaxSize": 0,
        "ParallelLoadQueuesPerThread": 0,
        "SupportLobs": true,
        "LobChunkSize": 64,
        "TaskRecoveryTableEnabled": false,
        "ParallelLoadThreads": 0,
        "LobMaxSize": 0,
        "BatchApplyEnabled": true,
        "FullLobMode": true,
        "LimitedSizeLobMode": false,
        "LoadMaxFileSize": 0,
        "ParallelLoadBufferSize": 0
    },
    "BeforeImageSettings": null,
    "ControlTablesSettings": {
        "historyTimeslotInMinutes": 5,
        "HistoryTimeslotInMinutes": 5,
        "StatusTableEnabled": false,
        "SuspendedTablesTableEnabled": false,
        "HistoryTableEnabled": false,
        "ControlSchema": "",
        "FullLoadExceptionTableEnabled": false
    },
    "LoopbackPreventionSettings": null,
    "CharacterSetSettings": null,
    "FailTaskWhenCleanTaskResourceFailed": false,
    "ChangeProcessingTuning": {
        "StatementCacheSize": 50,
        "CommitTimeout": 1,
        "BatchApplyPreserveTransaction": true,
        "BatchApplyTimeoutMin": 1,
        "BatchSplitSize": 0,
        "BatchApplyTimeoutMax": 30,
        "MinTransactionSize": 1000,
        "MemoryKeepTime": 60,
        "BatchApplyMemoryLimit": 500,
        "MemoryLimitTotal": 1024
    },
    "ChangeProcessingDdlHandlingPolicy": {
        "HandleSourceTableDropped": true,
        "HandleSourceTableTruncated": true,
        "HandleSourceTableAltered": true
    },
    "PostProcessingRules": null
}
yv5phkfx

yv5phkfx1#

因此,在深入研究AWS文档并联系支持部门之后,我了解到enum数据类型不能使用DMS迁移,无论如何,任务都会失败。
如果您的数据库有enum列,您需要找到另一个解决方案,即barman,并在EC2示例上运行它。

相关问题