关于如何在shell脚本中运行impala shell

am46iovg  于 2021-06-26  发布在  Impala
关注(0)|答案(2)|浏览(586)

我在尝试执行此bash代码时遇到问题:

function createImpalaPartition() {

period_id=$1;
database=$2
node=$3

actual_full=$(date -d@"$period_id" +%Y/%m/%d/%H/%M/)
template="use c2d;create EXTERNAL TABLE exptopology_$period_id (child_id bigint,parent_id bigint,level INT) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' WITH SERDEPROPERTIES ('serialization.format'=',', 'field.delim'=',') STORED AS TEXTFILE LOCATION '/hfc/sip/service/topology/$actual_full'"

echo "template is $template";

# impala-shell -V -i $node -d $database -q $template

impala-shell -V -i $node -q $template
}

我是这样调用它的:

createImpalaPartition $actual $impalaDatabase $impalaNode

哪里

actual=$(date +'%s')
impalaDatabase="dbName"
impalaNode="name_of_the_node"

脚本的执行返回:

[core@dub-vcd-vms170 ~]$ createImpalaPartition $actual $impalaDatabase $impalaNode
template is use c2d;create EXTERNAL TABLE exptopology_1428326587 (child_id bigint,parent_id bigint,level INT) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' WITH SERDEPROPERTIES ('serialization.format'=',', 'field.delim'=',') STORED AS TEXTFILE LOCATION '/hfc/sip/service/topology/2015/04/06/14/23/'
Error, could not parse arguments "c2d;create EXTERNAL TABLE exptopology_1428326587 (child_id bigint,parent_id bigint,level INT) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' WITH SERDEPROPERTIES ('serialization.format'=',', 'field.delim'=',') STORED AS TEXTFILE LOCATION '/hfc/sip/service/topology/2015/04/06/14/23/'"
 Usage: impala_shell.py [options]

如您所见,我必须使用shell脚本创建表。
更新:
通过这个链接,我可以看到 Impala 壳可以这样使用,但我没有使用正确的参数。
我使用了-f而不是-q,同样的错误仍然存在。有人能帮我吗?

3zwtqj6y

3zwtqj6y1#

你需要引用 $template 将整个字符串视为 impala-shell :

impala-shell -V -i $node "$template"

哪里 -V 启用详细输出。对于非详细(安静)输出,替换 -V-q .

zsohkypk

zsohkypk2#

最后我发现如何解决我的问题。

function createImpalaPartition() {

period_id=$1;
database=$2
node=$3

actual_full=$(date -d@"$period_id" +%Y/%m/%d/%H/%M/)

# UC=$(impala-shell -r -q "select count(1) from table where condition=1" -d $DB -i $HOST -B)

# attention, i have to use this way impala-shell

impala-shell -V -i $node -d $database -q "create EXTERNAL TABLE exptopology_$period_id (child_id bigint,parent_id bigint,level INT) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' WITH SERDEPROPERTIES ('serialization.format'=',', 'field.delim'=',') STORED AS TEXTFILE LOCATION '/hfc/sip/service/topology/$actual_full'"
}

我不能在create命令中使用模板var,我必须这样传递命令。

相关问题