我有3个节点,一个名为node1,datanode1和datanode2。scoop和mysql安装在namenode1上。
什么时候可以用c作为测试数据库的列表。
hadoop@namenode1:/usr/local/sqoop/lib$ sqoop list-databases --connect jdbc:mysql://localhost/ --username root --password hadoop
18/08/25 19:49:58 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7
18/08/25 19:49:58 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
18/08/25 19:49:58 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
information_schema
metastore
mysql
performance_schema
test
但当我运行sqoop import时--连接jdbc:mysql://localhost:3306/test --用户名root——密码hadoop——表student;
hadoop@namenode1:/usr/local/sqoop/lib$ sqoop import --connect jdbc:mysql://localhost:3306/test --username root --password hadoop --table student;
18/08/25 19:43:37 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7
18/08/25 19:43:37 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
18/08/25 19:43:37 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
18/08/25 19:43:37 INFO tool.CodeGenTool: Beginning code generation
18/08/25 19:43:38 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `student` AS t LIMIT 1
18/08/25 19:43:38 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `student` AS t LIMIT 1
18/08/25 19:43:38 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/local/hadoop
Note: /tmp/sqoop-hadoop/compile/2a56cd695f49348fad38af086755acd8/student.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
18/08/25 19:43:41 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-hadoop/compile/2a56cd695f49348fad38af086755acd8/student.jar
18/08/25 19:43:41 WARN manager.MySQLManager: It looks like you are importing from mysql.
18/08/25 19:43:42 WARN manager.MySQLManager: This transfer can be faster! Use the --direct
18/08/25 19:43:42 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
18/08/25 19:43:42 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
18/08/25 19:43:42 INFO mapreduce.ImportJobBase: Beginning import of student
18/08/25 19:43:42 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
18/08/25 19:43:43 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
18/08/25 19:43:43 INFO client.RMProxy: Connecting to ResourceManager at namenode1/192.168.1.2:8050
18/08/25 19:43:46 INFO db.DBInputFormat: Using read commited transaction isolation
18/08/25 19:43:46 INFO db.DataDrivenDBInputFormat: BoundingValsQuery: SELECT MIN(`id`), MAX(`id`) FROM `student`
18/08/25 19:43:46 INFO db.IntegerSplitter: Split size: 0; Num splits: 4 from: 1 to: 2
18/08/25 19:43:46 INFO mapreduce.JobSubmitter: number of splits:2
18/08/25 19:43:47 INFO Configuration.deprecation: yarn.resourcemanager.system-metrics-publisher.enabled is deprecated. Instead, use yarn.system-metrics-publisher.enabled
18/08/25 19:43:47 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1535200795373_0015
18/08/25 19:43:48 INFO impl.YarnClientImpl: Submitted application application_1535200795373_0015
18/08/25 19:43:48 INFO mapreduce.Job: The url to track the job: http://namenode1:8088/proxy/application_1535200795373_0015/
18/08/25 19:43:48 INFO mapreduce.Job: Running job: job_1535200795373_0015
18/08/25 19:43:57 INFO mapreduce.Job: Job job_1535200795373_0015 running in uber mode : false
18/08/25 19:43:57 INFO mapreduce.Job: map 0% reduce 0%
18/08/25 19:44:03 INFO mapreduce.Job: Task Id : attempt_1535200795373_0015_m_000000_0, Status : FAILED
**Error: java.lang.RuntimeException: java.lang.RuntimeException: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Unknown database 'test'**
at org.apache.sqoop.mapreduce.db.DBInputFormat.setDbConf(DBInputFormat.java:170)
at org.apache.sqoop.mapreduce.db.DBInputFormat.setConf(DBInputFormat.java:161)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:77)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:137)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:755)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:177)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1886)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:171)
**Caused by: java.lang.RuntimeException: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Unknown database 'test'**
at org.apache.sqoop.mapreduce.db.DBInputFormat.getConnection(DBInputFormat.java:223)
at org.apache.sqoop.mapreduce.db.DBInputFormat.setDbConf(DBInputFormat.java:168)
... 10 more
Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Unknown database 'test'
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
at com.mysql.jdbc.Util.getInstance(Util.java:386)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1054)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4237)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4169)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:928)
at com.mysql.jdbc.MysqlIO.proceedHandshakeWithPluggableAuthentication(MysqlIO.java:1750)
at com.mysql.jdbc.MysqlIO.doHandshake(MysqlIO.java:1290)
at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2493)
at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2526)
at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2311)
at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:834)
at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:47)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
我已经给了格兰特
GRANT ALL PRIVILEGES ON *.* TO 'root'@'%';
GRANT ALL PRIVILEGES ON *.* TO 'root'@'localhost';
GRANT ALL PRIVILEGES ON *.* TO 'root'@'127.0.0.1';
FLUSH PRIVILEGES;
但我不知道为什么会出现这样的错误。当数据库存在时。
1条答案
按热度按时间niknxzdl1#
mysql需要安装在所有节点上
当我们在分布式平台上运行mysql命令时,sqoop希望mysql命令应该在所有节点上解释,这就是为什么我们需要在所有节点上安装它的原因。希望这能解释答案