kafka连接数据库源连接器:如何从外键复制数据

dgsult0t  于 2021-06-04  发布在  Kafka
关注(0)|答案(1)|浏览(405)

我正在使用数据库源连接器将数据从postgres数据库表移动到kafka主题。我有一个orders表,它有一个使用customernumber字段的带有customers表的外键。
下面是连接器,它正在将订单复制到kafka,但没有将客户数据复制到json中。我想知道如何将客户订单的完整对象构造成json。

连接器为:

{
"name": "SOURCE_CONNECTOR",
"config": {
    "connector.class": "io.confluent.connect.jdbc.JdbcSourceConnector",
    "transforms.createKey.type": "org.apache.kafka.connect.transforms.ValueToKey",
    "connection.password": "postgres_pwd",
    "transforms.cast.type": "org.apache.kafka.connect.transforms.Cast$Value",
    "transforms.cast.spec": "amount:float64",
    "tasks.max": "1",
    "transforms": "cast,createKey,extractInt",
    "transforms.extractInt.type": "org.apache.kafka.connect.transforms.ExtractField$Key",
    "batch.max.rows": "25",
    "table.whitelist": "orders",
    "mode": "bulk",
    "topic.prefix": "data_",
    "transforms.extractInt.field": "uuid",
    "connection.user": "postgres_user",
    "transforms.createKey.fields": "uuid",
    "poll.interval.ms": "3600000",
    "sql.quote.identifiers": "false",
    "name": "SOURCE_CONNECTOR",
    "numeric.mapping": "best_fit",
    "connection.url": "url"
    }
  }
5tmbdcev

5tmbdcev1#

您可以使用基于查询的摄取。只需指定 query 配置选项。

相关问题