aws-glue-data没有插入到所需的目标中

ukqbszuj  于 2021-05-27  发布在  Spark
关注(0)|答案(0)|浏览(245)

我有一个胶水的工作插入数据到s3的位置。我想将数据插入到我文件夹中的一个特定csv文件中,但粘合作业是在每次运行中创建新的目标文件。你能告诉我这段代码有什么问题吗?

applymapping1 = ApplyMapping.apply(frame = datasource_lk, mappings = [("row_id", "bigint", "row_id", "bigint"), ("Quantity", "long", "Quantity", "long"),("Category", "string", "Category", "string") ], transformation_ctx = "applymapping1")

selectfields2 = SelectFields.apply(frame = applymapping1, paths = ["row_id", "Quantity", "Category"], transformation_ctx = "selectfields2")

resolvechoice3 = ResolveChoice.apply(frame = selectfields2, choice = "MATCH_CATALOG", database = "mydb", table_name = "order_summary_csv", transformation_ctx = "resolvechoice3")

datasink4 = glueContext.write_dynamic_frame.from_catalog(frame = resolvechoice3, database = "mydb", table_name = "order_summary_csv", transformation_ctx = "datasink4")
job.commit()

我在一个特定的s3文件夹中有一个csv文件(order\u summary.csv),但是glue没有在其中插入记录,而是创建了一个名为order\u summary.csv的新文件夹,并在这个文件夹中插入一些随机名称的文件。

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题