使用debezium链接postgresql 11无法获取数据库测试的编码

ycggw6v2 于 2021-06-04 发布在 Kafka

关注(0)|答案(2)|浏览(592)

我使用debeziumcdc连接pg，我构建了由docker使用的pg11，pg运行良好。当我在kafka connector中使用debezium时，它报告：
无法获取数据库测试的编码
curl 是：

curl -H "Accept: application/json" -H "Content-type: application/json" -X POST http://localhost:8083/connectors/ -d '{
    "name": "debezium",
    "config": {
        "name": "debezium",
        "connector.class": "io.debezium.connector.postgresql.PostgresConnector",
        "tasks.max": "1",
        "database.hostname": "localhost",
        "database.port": "5432",
        "database.dbname": "test",
        "database.user": "pg",
        "database.password": "135790",
        "database.server.name": "ls",
        "table.whitelist": "public.test",
        "plugin.name": "pgoutput"
    }
}'

Kafka的例外是：

[2020-07-08 09:24:35,076] ERROR Uncaught exception in REST call to /connectors/ (org.apache.kafka.connect.runtime.rest.errors.ConnectExceptionMapper:61)
java.lang.RuntimeException: Couldn't obtain encoding for database test
    at io.debezium.connector.postgresql.connection.PostgresConnection.determineDatabaseCharset(PostgresConnection.java:434)
    at io.debezium.connector.postgresql.connection.PostgresConnection.<init>(PostgresConnection.java:77)
    at io.debezium.connector.postgresql.connection.PostgresConnection.<init>(PostgresConnection.java:87)
    at io.debezium.connector.postgresql.PostgresConnector.validate(PostgresConnector.java:102)
    at org.apache.kafka.connect.runtime.AbstractHerder.validateConnectorConfig(AbstractHerder.java:277)
    at org.apache.kafka.connect.runtime.distributed.DistributedHerder$6.call(DistributedHerder.java:534)
    at org.apache.kafka.connect.runtime.distributed.DistributedHerder$6.call(DistributedHerder.java:531)
    at org.apache.kafka.connect.runtime.distributed.DistributedHerder.tick(DistributedHerder.java:267)
    at org.apache.kafka.connect.runtime.distributed.DistributedHerder.run(DistributedHerder.java:216)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
Caused by: org.postgresql.util.PSQLException: FATAL: database "test" does not exist
    at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2532)
    at org.postgresql.core.v3.QueryExecutorImpl.readStartupMessages(QueryExecutorImpl.java:2644)
    at org.postgresql.core.v3.QueryExecutorImpl.<init>(QueryExecutorImpl.java:137)
    at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:255)
    at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49)
    at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:217)
    at org.postgresql.Driver.makeConnection(Driver.java:458)
    at org.postgresql.Driver.connect(Driver.java:260)
    at io.debezium.jdbc.JdbcConnection.lambda$patternBasedFactory$1(JdbcConnection.java:190)
    at io.debezium.jdbc.JdbcConnection.connection(JdbcConnection.java:788)
    at io.debezium.jdbc.JdbcConnection.connection(JdbcConnection.java:783)
    at io.debezium.connector.postgresql.connection.PostgresConnection.determineDatabaseCharset(PostgresConnection.java:431)
    ... 13 more
[2020-07-08 09:24:35,128] INFO 127.0.0.1 - - [08/Jul/2020:01:24:34 +0000] "POST /connectors/ HTTP/1.1" 500 73  330 (org.apache.kafka.connect.runtime.rest.RestServer:60)

postgresql apache-kafka apache-kafka-connect debezium

来源：https://stackoverflow.com/questions/62786240/use-debezium-link-postgresql-11-couldnt-obtain-encoding-for-database-test

2条答案

按热度按时间

7cjasjjr1#

有两件事在你的有效载荷中是不准确的。保留可解析名称而不是 localhost . 其次，放置正确的db名称空间。

"database.hostname": "FQDN",
        "database.server.name": "test_table_name",

在这种情况下，postgresql主机可能没有验证身份验证 pg 用户。它需要进入pg_hba.conf（在postgresql服务器上）并从客户机（即kafka连接器）建立信任/身份验证。


# host       DATABASE  USER  ADDRESS                       METHOD  [OPTIONS]

# hostssl    DATABASE  USER  ADDRESS                       METHOD  [OPTIONS]

host         test      pg    Kafka.connector.server.ip/32  md5
hostssl      test      pg    Kafka.connector.server.ip/32  md5

然后，重新启动postgresql服务器以获取的身份验证 pg 实际上，在我的情况下 pg_ctl reload .
正如，curl发生在restapi调用时，获取postgresql服务器防火墙设置中添加的kafka rest（8082）和kafka connect rest（8083）端口。

赞(0）回复(0）举报 2021-06-05

vxbzzdmp2#

在我看来 test 不存在或用户不可见 pg .

赞(0）回复(0）举报 2021-06-04