DataX 写mysql遇到的问题

qni6mghb 于 4个月前发布在 Mysql

关注(0)|答案(5)|浏览(148)

mysql同步mysql
当配置文件配置如下时报错，读端配置没报错写端配置报错，尽管都是mysql 并且密码没错
"writer":{
"name":"mysqlwriter",
2023-12-26 17:24:13.904 [job-0] INFO OriginalConfPretreatmentUtil - Available jdbcUrl:jdbc:mysql://rm....
...
2023-12-26 17:24:13.990 [job-0] ERROR RetryUtil - Exception when calling callable, 异常Msg:Code:[DBUtilErrorCode-10], Description:[连接数据库失败. 请检查您的账号、密码、数据库名称、IP、Port或者向 DBA 寻求帮助(注意网络环境).]. - 具体错误信息为：java.sql.SQLException: No suitable driver found for ["jdbc:mysql://rm-uf.......

但只要将mysqlwriter改为streamwriter，就没有连接错误了，可以运行，但运行完成后写端查不到数据，仿佛没有commit一样
2023-12-26 17:14:00.698 [job-0] INFO AbstractScheduler - Scheduler accomplished all tasks.
2023-12-26 17:14:00.699 [job-0] INFO JobContainer - DataX Writer.Job [streamwriter] do post work.
2023-12-26 17:14:00.699 [job-0] INFO JobContainer - DataX Reader.Job [mysqlreader] do post work.
2023-12-26 17:14:00.699 [job-0] INFO JobContainer - DataX jobId [0] completed successfully.
.....
2023-12-26 17:14:00.701 [job-0] INFO JobContainer -
任务启动时刻 : 2023-12-26 17:07:39
任务结束时刻 : 2023-12-26 17:14:00
任务总计耗时 : 381s
任务平均流量 : 1.21MB/s
记录写入速度 : 8242rec/s
读出记录总数 : 3132088
读写失败总数 : 0

太离谱的问题

DataX

来源：https://github.com/alibaba/DataX/issues/2029

5条答案

按热度按时间

unhi4e5o1#

我也是写mysql 的时候报No suitable driver，换了几个版本的驱动还是报错。求解决

赞(0）回复(0）举报 4个月前

e1xvtsh32#

我的问题解决了，mysqlwriter 配置的jdbcUrl 要把中括号[] 去掉

赞(0）回复(0）举报 4个月前

332nm8kg3#

Xuejicha0
我这怎么还是出错
2024-01-02 16:05:30.670 [job-0] INFO JobContainer - jobContainer starts to do prepare ...
2024-01-02 16:05:30.670 [job-0] INFO JobContainer - DataX Reader.Job [mysqlreader] do prepare work .
2024-01-02 16:05:30.671 [job-0] INFO JobContainer - DataX Writer.Job [mysqlwriter] do prepare work .
2024-01-02 16:05:30.675 [job-0] ERROR JobContainer - Exception when job run
java.lang.ClassCastException: class java.lang.String cannot be cast to class java.util.List (java.lang.String and java.util.List are in module java.base of loader 'bootstrap')
.....
经DataX智能分析,该任务最可能的错误原因是:
com.alibaba.datax.common.exception.DataXException: Code:[Framework-02], Description:[DataX引擎运行过程出错，具体原因请参看DataX运行结束时的错误诊断信息 .]. - java.lang.ClassCastException: class java.lang.String cannot be cast to class java.util.List (java.lang.String and java.util.List are in module java.base of loader 'bootstrap')
at com.alibaba.datax.common.util.Configuration.getList(Configuration.java:426)

赞(0）回复(0）举报 4个月前

dldeef674#

如果你的Mysql是8.0以上，需要手动将MySQL的mysql-connector-java-xxx的jar包放到datax/lib目录下。

赞(0）回复(0）举报 4个月前

mrphzbgm5#

如果你的Mysql是8.0以上，需要手动将MySQL的mysql-connector-java-xxx的jar包放到datax/lib目录下。
多了一行这。别的没变化
Loading class com.mysql.jdbc.Driver'. This is deprecated. The new driver class is com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary.
看我这配置文件也没什么错吧
{
"content":[
{
"reader":{
"name":"mysqlreader",
"parameter":{
"column":[
"brandId",
"areaRange",
"areaId",
"pid",
"storeName",
"storeLogo",
"provinceName",
"cityName",
"districtName",
"address",
"location",
"createTime",
"gaodeLocation"
],
"connection":[
{
"jdbcUrl":[
"jdbc:mysql://rm-xxxxxxx:3306/dw_rs"
],
"table":[
"brand_store"
]
}
],
"password":"**",
"username":"wang",
"where":"1=1"
}
},
"writer":{
"name":"mysqlwriter",
"parameter":{
"column":[
"brandId",
"areaRange",
"areaId",
"pid",
"storeName",
"storeLogo",
"provinceName",
"cityName",
"districtName",
"address",
"location",
"createTime",
"gaodeLocation"
],
"connection":[
{
"jdbcUrl":"jdbc:mysql://rm-yyyyyy:3306/dw_rs",
"table":[
"brand_store_102"
]
}
],
"password":"**",
"preSql":"",
"session":[],
"username":"wang",
"writeMode":"insert"
}
}
}
],
"setting":{
"speed":{
"channel":"10"
}
}
}

赞(0）回复(0）举报 4个月前

我来回答

DataX 写mysql遇到的问题

5条答案

相关问题

热门标签

最新问答