在配置单元中发出select查询时获取异常“java.lang.illegalargumentexception(无法从空字符串创建路径)”

vkc1a9a2  于 2021-05-29  发布在  Hadoop
关注(0)|答案(1)|浏览(390)

我是新到这个网站,实际上我与下面的问题得到“不能从一个空字符串创建路径”错误卡住。我的hadoop版本是2.5.2,hive版本是0.13.1。下面是我使用的文件详细信息和我的查询,谁能帮我解决这个问题谢谢。
文件名:tweet\ u info.json文件中的数据如下:

{
    "user": {
        "lang": "en",
        "profile_background_image_url": "http:\/\/a0.twimg.com\/images\/themes\/theme1\/bg.png",
        "id_str": "568286862",
        "default_profile_image": false,
        "statuses_count": 595,
        "profile_link_color": "0084B4",
        "favourites_count": 5,
        "profile_image_url_https": "https:\/\/si0.twimg.com\/profile_images\/2206373396\/image_normal.jpg",
        "following": null,
        "profile_background_color": "C0DEED",
        "description": "Nobody can ever affect you unless you allow yourself to be affected.",
        "notifications": null,
        "profile_background_tile": false,
        "time_zone": null,
        "profile_sidebar_fill_color": "DDEEF6",
        "listed_count": 0,
        "contributors_enabled": false,
        "geo_enabled": false,
        "created_at": "Tue May 01 13:29:22 +0000 2012",
        "screen_name": "Chin_Hean",
        "follow_request_sent": null,
        "profile_sidebar_border_color": "C0DEED",
        "protected": false,
        "url": null,
        "default_profile": true,
        "name": "\u0106hr\u00ed\u0161",
        "is_translator": false,
        "show_all_inline_media": false,
        "verified": false,
        "profile_use_background_image": true,
        "followers_count": 37,
        "profile_image_url": "http:\/\/a0.twimg.com\/profile_images\/2206373396\/image_normal.jpg",
        "id": 568286862,
        "profile_background_image_url_https": "https:\/\/si0.twimg.com\/images\/themes\/theme1\/bg.png",
        "utc_offset": null,
        "friends_count": 65,
        "profile_text_color": "333333",
        "location": "Stark Industries"
    }
}

在配置单元中创建表查询:

CREATE TABLE tweet_info(
user struct<lang: string, id: int, statuses_count: int, favourites_count: int, description: string, 
screen_name: string, url: string, friends_count: int, followers_count: int, location: string>)

ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.JsonSerde';

我的选择查询:

SELECT user.id UserId, user.screen_name UserName, user.location UserLocation, user.description UserDescription, user.lang UserLanguage, user.friends_count UserFrndsCount from tweet_info;

错误日志:

total jobs=1启动job 1(共1个reduce任务)由于没有reduce运算符,因此将reduce任务数设置为0

java.lang.IllegalArgumentException: Can not create a Path from an empty string
    at org.apache.hadoop.fs.Path.checkPathArg(Path.java:127)
    at org.apache.hadoop.fs.Path.<init>(Path.java:135)
    at org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:212)
    at org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:301)
    at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:389)
    at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1285)
    at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1282)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
    at org.apache.hadoop.mapreduce.Job.submit(Job.java:1282)
    at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:562)
    at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:557)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
    at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:557)
    at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:548)
    at org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(ExecDriver.java:420)
    at org.apache.hadoop.hive.ql.exec.mr.MapRedTask.execute(MapRedTask.java:136)
    at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153)
    at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85)
    at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1503)
    at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1270)
    at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1088)
    at org.apache.hadoop.hive.ql.Driver.run(Driver.java:911)
    at org.apache.hadoop.hive.ql.Driver.run(Driver.java:901)
    at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:268)
    at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220)
    at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:423)
    at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:792)
    at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:686)
    at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
Job Submission failed with exception 'java.lang.IllegalArgumentException(Can not create a Path from an empty string)'
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
cgfeq70w

cgfeq70w1#

我终于想出了解决我问题的办法。下面是我所做的修改,解决了我所有的错误。
我用hive-serdes-1.0-snapshot.jar替换了serde
将上面的jar放入我的hive/lib文件夹,如下所示:
'/usr/lib/hive/apache-hive-0.13.1-bin/lib/hive-serdes-1.0-snapshot.jar'
将jar添加到我的类路径:add jar/usr/lib/hive/apache-hive-0.13.1-bin/lib/hive-serdes-1.0-snapshot.jar
我在create table查询中将行格式serde更改为“com.cloudera.hive.serde.jsonserde”。
这解决了我所有的错误,最终得到了我的输出。
谢谢

相关问题