我是新到这个网站,实际上我与下面的问题得到“不能从一个空字符串创建路径”错误卡住。我的hadoop版本是2.5.2,hive版本是0.13.1。下面是我使用的文件详细信息和我的查询,谁能帮我解决这个问题谢谢。
文件名:tweet\ u info.json文件中的数据如下:
{
"user": {
"lang": "en",
"profile_background_image_url": "http:\/\/a0.twimg.com\/images\/themes\/theme1\/bg.png",
"id_str": "568286862",
"default_profile_image": false,
"statuses_count": 595,
"profile_link_color": "0084B4",
"favourites_count": 5,
"profile_image_url_https": "https:\/\/si0.twimg.com\/profile_images\/2206373396\/image_normal.jpg",
"following": null,
"profile_background_color": "C0DEED",
"description": "Nobody can ever affect you unless you allow yourself to be affected.",
"notifications": null,
"profile_background_tile": false,
"time_zone": null,
"profile_sidebar_fill_color": "DDEEF6",
"listed_count": 0,
"contributors_enabled": false,
"geo_enabled": false,
"created_at": "Tue May 01 13:29:22 +0000 2012",
"screen_name": "Chin_Hean",
"follow_request_sent": null,
"profile_sidebar_border_color": "C0DEED",
"protected": false,
"url": null,
"default_profile": true,
"name": "\u0106hr\u00ed\u0161",
"is_translator": false,
"show_all_inline_media": false,
"verified": false,
"profile_use_background_image": true,
"followers_count": 37,
"profile_image_url": "http:\/\/a0.twimg.com\/profile_images\/2206373396\/image_normal.jpg",
"id": 568286862,
"profile_background_image_url_https": "https:\/\/si0.twimg.com\/images\/themes\/theme1\/bg.png",
"utc_offset": null,
"friends_count": 65,
"profile_text_color": "333333",
"location": "Stark Industries"
}
}
在配置单元中创建表查询:
CREATE TABLE tweet_info(
user struct<lang: string, id: int, statuses_count: int, favourites_count: int, description: string,
screen_name: string, url: string, friends_count: int, followers_count: int, location: string>)
ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.JsonSerde';
我的选择查询:
SELECT user.id UserId, user.screen_name UserName, user.location UserLocation, user.description UserDescription, user.lang UserLanguage, user.friends_count UserFrndsCount from tweet_info;
错误日志:
total jobs=1启动job 1(共1个reduce任务)由于没有reduce运算符,因此将reduce任务数设置为0
java.lang.IllegalArgumentException: Can not create a Path from an empty string
at org.apache.hadoop.fs.Path.checkPathArg(Path.java:127)
at org.apache.hadoop.fs.Path.<init>(Path.java:135)
at org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:212)
at org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:301)
at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:389)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1285)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1282)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1282)
at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:562)
at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:557)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:557)
at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:548)
at org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(ExecDriver.java:420)
at org.apache.hadoop.hive.ql.exec.mr.MapRedTask.execute(MapRedTask.java:136)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153)
at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1503)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1270)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1088)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:911)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:901)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:268)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:423)
at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:792)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:686)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
Job Submission failed with exception 'java.lang.IllegalArgumentException(Can not create a Path from an empty string)'
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
1条答案
按热度按时间cgfeq70w1#
我终于想出了解决我问题的办法。下面是我所做的修改,解决了我所有的错误。
我用hive-serdes-1.0-snapshot.jar替换了serde
将上面的jar放入我的hive/lib文件夹,如下所示:
'/usr/lib/hive/apache-hive-0.13.1-bin/lib/hive-serdes-1.0-snapshot.jar'
将jar添加到我的类路径:add jar/usr/lib/hive/apache-hive-0.13.1-bin/lib/hive-serdes-1.0-snapshot.jar
我在create table查询中将行格式serde更改为“com.cloudera.hive.serde.jsonserde”。
这解决了我所有的错误,最终得到了我的输出。
谢谢