使用debezium链接postgresql 11无法获取数据库测试的编码

ycggw6v2  于 2021-06-04  发布在  Kafka
关注(0)|答案(2)|浏览(610)

我使用debeziumcdc连接pg,我构建了由docker使用的pg11,pg运行良好。当我在kafka connector中使用debezium时,它报告:
无法获取数据库测试的编码
curl 是:

curl -H "Accept: application/json" -H "Content-type: application/json" -X POST http://localhost:8083/connectors/ -d '{
    "name": "debezium",
    "config": {
        "name": "debezium",
        "connector.class": "io.debezium.connector.postgresql.PostgresConnector",
        "tasks.max": "1",
        "database.hostname": "localhost",
        "database.port": "5432",
        "database.dbname": "test",
        "database.user": "pg",
        "database.password": "135790",
        "database.server.name": "ls",
        "table.whitelist": "public.test",
        "plugin.name": "pgoutput"
    }
}'

Kafka的例外是:

[2020-07-08 09:24:35,076] ERROR Uncaught exception in REST call to /connectors/ (org.apache.kafka.connect.runtime.rest.errors.ConnectExceptionMapper:61)
java.lang.RuntimeException: Couldn't obtain encoding for database test
    at io.debezium.connector.postgresql.connection.PostgresConnection.determineDatabaseCharset(PostgresConnection.java:434)
    at io.debezium.connector.postgresql.connection.PostgresConnection.<init>(PostgresConnection.java:77)
    at io.debezium.connector.postgresql.connection.PostgresConnection.<init>(PostgresConnection.java:87)
    at io.debezium.connector.postgresql.PostgresConnector.validate(PostgresConnector.java:102)
    at org.apache.kafka.connect.runtime.AbstractHerder.validateConnectorConfig(AbstractHerder.java:277)
    at org.apache.kafka.connect.runtime.distributed.DistributedHerder$6.call(DistributedHerder.java:534)
    at org.apache.kafka.connect.runtime.distributed.DistributedHerder$6.call(DistributedHerder.java:531)
    at org.apache.kafka.connect.runtime.distributed.DistributedHerder.tick(DistributedHerder.java:267)
    at org.apache.kafka.connect.runtime.distributed.DistributedHerder.run(DistributedHerder.java:216)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
Caused by: org.postgresql.util.PSQLException: FATAL: database "test" does not exist
    at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2532)
    at org.postgresql.core.v3.QueryExecutorImpl.readStartupMessages(QueryExecutorImpl.java:2644)
    at org.postgresql.core.v3.QueryExecutorImpl.<init>(QueryExecutorImpl.java:137)
    at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:255)
    at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49)
    at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:217)
    at org.postgresql.Driver.makeConnection(Driver.java:458)
    at org.postgresql.Driver.connect(Driver.java:260)
    at io.debezium.jdbc.JdbcConnection.lambda$patternBasedFactory$1(JdbcConnection.java:190)
    at io.debezium.jdbc.JdbcConnection.connection(JdbcConnection.java:788)
    at io.debezium.jdbc.JdbcConnection.connection(JdbcConnection.java:783)
    at io.debezium.connector.postgresql.connection.PostgresConnection.determineDatabaseCharset(PostgresConnection.java:431)
    ... 13 more
[2020-07-08 09:24:35,128] INFO 127.0.0.1 - - [08/Jul/2020:01:24:34 +0000] "POST /connectors/ HTTP/1.1" 500 73  330 (org.apache.kafka.connect.runtime.rest.RestServer:60)
7cjasjjr

7cjasjjr1#

有两件事在你的有效载荷中是不准确的。保留可解析名称而不是 localhost . 其次,放置正确的db名称空间。

"database.hostname": "FQDN",
        "database.server.name": "test_table_name",

在这种情况下,postgresql主机可能没有验证身份验证 pg 用户。它需要进入pg_hba.conf(在postgresql服务器上)并从客户机(即kafka连接器)建立信任/身份验证。


# host       DATABASE  USER  ADDRESS                       METHOD  [OPTIONS]

# hostssl    DATABASE  USER  ADDRESS                       METHOD  [OPTIONS]

host         test      pg    Kafka.connector.server.ip/32  md5
hostssl      test      pg    Kafka.connector.server.ip/32  md5

然后,重新启动postgresql服务器以获取的身份验证 pg 实际上,在我的情况下 pg_ctl reload .
正如,curl发生在restapi调用时,获取postgresql服务器防火墙设置中添加的kafka rest(8082)和kafka connect rest(8083)端口。

vxbzzdmp

vxbzzdmp2#

在我看来 test 不存在或用户不可见 pg .

相关问题