sqoop:如何关闭准备好的语句?

798qvoo8  于 2021-06-03  发布在  Sqoop
关注(0)|答案(1)|浏览(571)

我使用sqoop1.4.5-cdh5.4.2和postgresql。
如果sqoop直接连接到数据库-好的。但是我需要使用sqoop而不是pgbouncer,我对此有问题。
在pgbouncer中,您不能在事务模式下执行准备语句。
... connect命令:

sqoop import \
    --connect "$db_name" \
    --username "$db_user" \
    --password "$db_pass" \
    --direct \
    --hive-import \
    --hive-table "$hive_schema.$t" \
    --hive-overwrite \
    --num-mappers 10 \
    --fetch-size 100000 \
    --split-by "object_id" \
    --target-dir "/user/$hive_schema/$t" \
    --table "$t"

... 和错误:

org.postgresql.util.PSQLException: ERROR: prepared statement "S_3" already exists
    at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2270)
    at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1998)
    at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255)
    at org.postgresql.jdbc2.AbstractJdbc2Connection.executeTransactionCommand(AbstractJdbc2Connection.java:791)
    at org.postgresql.jdbc2.AbstractJdbc2Connection.commit(AbstractJdbc2Connection.java:815)
    at org.apache.sqoop.manager.SqlManager.getColumnInfoForRawQuery(SqlManager.java:315)
    at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:241)
    at org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:227)
    at org.apache.sqoop.hive.TableDefWriter.getCreateTableStmt(TableDefWriter.java:126)
    at org.apache.sqoop.hive.HiveImport.importTable(HiveImport.java:188)
    at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:514)
    at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
    at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
    at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
    at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
    at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
    at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
bqjvbblv

bqjvbblv1#

sqoop不适用于pgbouncer和事务池:(

相关问题