importtsv命令在hbase中不起作用

aiazj4mn  于 2021-06-03  发布在  Hadoop
关注(0)|答案(2)|浏览(425)

我正在使用 HBase 0.98.1-cdh5.1.3 . 我正在尝试接收一个csv文件,该文件存在于我的hdfs中的位置 /user/hdfs/exp 到hbase。我的文件包含以下格式的数据:

1,abc,xyz

2,def,uvw

3,ghi,rst

我正在使用下面的命令:

bin/hbase org.apache.hadoop.hbase.mapreduce.ImportTsv '-Dimporttsv.separator=,' -Dimporttsv.columns=HBASE_ROW_KEY,CF:firstname,CF:lastname tablename /user/hdfs/exp

我也使用了不同的组合,比如

bin/hbase org.apache.hadoop.hbase.mapreduce.ImportTsv -Dimporttsv.columns=HBASE_ROW_KEY,CF:firstname,CF:lastname tablename /user/hdfs/exp '-Dimporttsv.separator=,'

bin/hbase org.apache.hadoop.hbase.mapreduce.ImportTsv -Dimporttsv.columns=HBASE_ROW_KEY,CF:firstname,CF:lastname '-Dimporttsv.separator=,' tablename /user/hdfs/exp

但什么都不管用。它无法检测分隔符,也就是说,在我的例子中,它没有被正确地解析。谁能帮我找出哪里出了问题。
这只是一行数据集:
10000064202896309897,1000006420,2896309897,10180,hdfs://btc5x015:8020/user/mr\u test/logsjan/log\u jan20\u 29/10180\u log201501260000.log,3.2.3.1,92015-01-26,15:46:12.12,参考肩部4,n,n,肩部60,17.0,m,487093458,[研究\u 16004\u 16004],检查\u 16004,[患者\u 16004\u 1],schulter std,schultergelenk rechts,8.10,不存在,-8.1,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,y,n,hf,hf,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,y,y,n,hf,hf,y,n,hf,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在时间:12.12,9.5,9.4,1.002,不存在,真,不,不存在,0.0,不存在,不存在,不存在,不存在,不存在,0.0,不存在,不存在,0.0,0.09,0.3,3.3,0,假,假,假,n,n,26.1,lt,0.33,0.03,不存在,null,1,dres.grafkernhausmann,e:\export\datamonitoring\p\u i\u 20150126\u 154530.frame,hdfs://btc5x015.code1.emi.philips.com:8020/user/mr\u test/logsjan/log\u jan20\u 29/10180\u log201501260000.log,317774,883,0,0,0,0,0,1,1,8,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,00:00:00,00,00,00,00:00,00,00:00,00:00,00:00,00:00,00:00,00:00:00,00:00,00:00:00,00,00:00:00,00:00:00,00:00:00:00,00:00,00:夸德,190190,临床,0,094166709,放射学家gemeinschaftspraxis,医学博士。迈克尔·格拉夫,医学博士。安德烈亚斯·克恩,医学博士。韦茨拉尔豪斯曼,邮编:35578。47,6,不存在,1,不存在,否,不存在,不存在,不存在,1,不存在,不存在,不存在,不存在,不存在,不存在,不存在,最短,不存在,最短,1,不,是,默认,ffe,不存在,不存在,不存在,不存在,不存在,不存在,不存在,3,不存在,笛卡尔,是,否,不存在,不存在,不存在,低,满,不存在,不存在,不存在,不存在,3d,不存在,不存在,不存在,不存在,不存在,不存在,不存在,是,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,用户定义,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,450405,是,仰卧,hf,不存在,不存在,不存在,不存在,不存在,不存在,不存在,默认,不存在,2,sense-shoulder8 body quad,f,400,,,100,5.625,7.23214293,4,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,80,平行,不,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不,不存在,0,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不,不,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,不存在,0,5.625,不存在,不存在,不存在,t30,3.0t,否,f2000,水冷2274-d,主,无,,,0,16,0,1,s26\U 128,无,8,空,cdas,logfolder\U sysfolder\U matched\U release\U not\U checked,false,空,空,空,空,12.15,sense-shoulder8/(19)q-body,0,scan\U parse\U success,shoulder,-4.64757729 5.37445641,

xurqigkl

xurqigkl1#

在指定分隔符时,单个引号似乎放错了位置。尝试使用以下选项: -Dimporttsv.separator=',' 而不是 '-Dimporttsv.separator=,' 如果准备输入文件时,任何列值都由字段分隔符(在本例中为逗号)组成,则输入文件将失败。在准备csv文件时最好保留不同的分隔符(如|)

falq053o

falq053o2#

我刚刚用importtsv命令将一行give加载到hbase表中,它提供了414列,对我来说非常有效。

hbase org.apache.hadoop.hbase.mapreduce.ImportTsv -Dimporttsv.columns=HBASE_ROW_KEY,CF:c1,CF:c2,CF:c3,CF:c4,CF:c5,CF:c6,CF:c7,CF:c8,CF:c9,CF:c10,CF:c11,CF:c12,CF:c13,CF:c14,CF:c15,CF:c16,CF:c17,CF:c18,CF:c19,CF:c20,CF:c21,CF:c22,CF:c23,CF:c24,CF:c25,CF:c26,CF:c27,CF:c28,CF:c29,CF:c30,CF:c31,CF:c32,CF:c33,CF:c34,CF:c35,CF:c36,CF:c37,CF:c38,CF:c39,CF:c40,CF:c41,CF:c42,CF:c43,CF:c44,CF:c45,CF:c46,CF:c47,CF:c48,CF:c49,CF:c50,CF:c51,CF:c52,CF:c53,CF:c54,CF:c55,CF:c56,CF:c57,CF:c58,CF:c59,CF:c60,CF:c61,CF:c62,CF:c63,CF:c64,CF:c65,CF:c66,CF:c67,CF:c68,CF:c69,CF:c70,CF:c71,CF:c72,CF:c73,CF:c74,CF:c75,CF:c76,CF:c77,CF:c78,CF:c79,CF:c80,CF:c81,CF:c82,CF:c83,CF:c84,CF:c85,CF:c86,CF:c87,CF:c88,CF:c89,CF:c90,CF:c91,CF:c92,CF:c93,CF:c94,CF:c95,CF:c96,CF:c97,CF:c98,CF:c99,CF:c100,CF:c101,CF:c102,CF:c103,CF:c104,CF:c105,CF:c106,CF:c107,CF:c108,CF:c109,CF:c110,CF:c111,CF:c112,CF:c113,CF:c114,CF:c115,CF:c116,CF:c117,CF:c118,CF:c119,CF:c120,CF:c121,CF:c122,CF:c123,CF:c124,CF:c125,CF:c126,CF:c127,CF:c128,CF:c129,CF:c130,CF:c131,CF:c132,CF:c133,CF:c134,CF:c135,CF:c136,CF:c137,CF:c138,CF:c139,CF:c140,CF:c141,CF:c142,CF:c143,CF:c144,CF:c145,CF:c146,CF:c147,CF:c148,CF:c149,CF:c150,CF:c151,CF:c152,CF:c153,CF:c154,CF:c155,CF:c156,CF:c157,CF:c158,CF:c159,CF:c160,CF:c161,CF:c162,CF:c163,CF:c164,CF:c165,CF:c166,CF:c167,CF:c168,CF:c169,CF:c170,CF:c171,CF:c172,CF:c173,CF:c174,CF:c175,CF:c176,CF:c177,CF:c178,CF:c179,CF:c180,CF:c181,CF:c182,CF:c183,CF:c184,CF:c185,CF:c186,CF:c187,CF:c188,CF:c189,CF:c190,CF:c191,CF:c192,CF:c193,CF:c194,CF:c195,CF:c196,CF:c197,CF:c198,CF:c199,CF:c200,CF:c201,CF:c202,CF:c203,CF:c204,CF:c205,CF:c206,CF:c207,CF:c208,CF:c209,CF:c210,CF:c211,CF:c212,CF:c213,CF:c214,CF:c215,CF:c216,CF:c217,CF:c218,CF:c219,CF:c220,CF:c221,CF:c222,CF:c223,CF:c224,CF:c225,CF:c226,CF:c227,CF:c228,CF:c229,CF:c230,CF:c231,CF:c232,CF:c233,CF:c234,CF:c235,CF:c236,CF:c237,CF:c238,CF:c239,CF:c240,CF:c241,CF:c242,CF:c243,CF:c244,CF:c245,CF:c246,CF:c247,CF:c248,CF:c249,CF:c250,CF:c251,CF:c252,CF:c253,CF:c254,CF:c255,CF:c256,CF:c257,CF:c258,CF:c259,CF:c260,CF:c261,CF:c262,CF:c263,CF:c264,CF:c265,CF:c266,CF:c267,CF:c268,CF:c269,CF:c270,CF:c271,CF:c272,CF:c273,CF:c274,CF:c275,CF:c276,CF:c277,CF:c278,CF:c279,CF:c280,CF:c281,CF:c282,CF:c283,CF:c284,CF:c285,CF:c286,CF:c287,CF:c288,CF:c289,CF:c290,CF:c291,CF:c292,CF:c293,CF:c294,CF:c295,CF:c296,CF:c297,CF:c298,CF:c299,CF:c300,CF:c301,CF:c302,CF:c303,CF:c304,CF:c305,CF:c306,CF:c307,CF:c308,CF:c309,CF:c310,CF:c311,CF:c312,CF:c313,CF:c314,CF:c315,CF:c316,CF:c317,CF:c318,CF:c319,CF:c320,CF:c321,CF:c322,CF:c323,CF:c324,CF:c325,CF:c326,CF:c327,CF:c328,CF:c329,CF:c330,CF:c331,CF:c332,CF:c333,CF:c334,CF:c335,CF:c336,CF:c337,CF:c338,CF:c339,CF:c340,CF:c341,CF:c342,CF:c343,CF:c344,CF:c345,CF:c346,CF:c347,CF:c348,CF:c349,CF:c350,CF:c351,CF:c352,CF:c353,CF:c354,CF:c355,CF:c356,CF:c357,CF:c358,CF:c359,CF:c360,CF:c361,CF:c362,CF:c363,CF:c364,CF:c365,CF:c366,CF:c367,CF:c368,CF:c369,CF:c370,CF:c371,CF:c372,CF:c373,CF:c374,CF:c375,CF:c376,CF:c377,CF:c378,CF:c379,CF:c380,CF:c381,CF:c382,CF:c383,CF:c384,CF:c385,CF:c386,CF:c387,CF:c388,CF:c389,CF:c390,CF:c391,CF:c392,CF:c393,CF:c394,CF:c395,CF:c396,CF:c397,CF:c398,CF:c399,CF:c400,CF:c401,CF:c402,CF:c403,CF:c404,CF:c405,CF:c406,CF:c407,CF:c408,CF:c409,CF:c410,CF:c411,CF:c412,CF:c413,CF:c414 '-Dimporttsv.separator=,' tablename /user/hdfs/exp

我已经给了随机列名,你可以根据需要更新。
注意:确保通过命令传递的列数与输入数据源匹配。甚至当我通过412列而不是414列时,我也遇到了糟糕的行问题。
希望这会有帮助。:)

相关问题