scala 如何在spark sql中执行多行sql

knpiaxh1  于 2023-01-09  发布在  Scala
关注(0)|答案(6)|浏览(147)

如何在Spark SQL中执行冗长的多行配置单元查询?

val sqlContext = new HiveContext (sc)
val result = sqlContext.sql ("
 select ...
 from ...
");
mwecs4sa

mwecs4sa1#

请使用“"”,例如

val results = sqlContext.sql ("""
     select ....
     from ....
""");

或者,如果要设置代码格式,请用途:

val results = sqlContext.sql ("""
     |select ....
     |from ....
""".stripMargin);
2lpgd968

2lpgd9682#

您可以在SQL代码的开始/结束处使用三引号,或在每行的结尾处使用反斜杠。

val results = sqlContext.sql ("""
    create table enta.scd_fullfilled_entitlement as
    select *
    from my_table
    """);

results = sqlContext.sql (" \
    create table enta.scd_fullfilled_entitlement as \
    select * \
    from my_table \
    ")
cxfofazt

cxfofazt3#

val query = """(SELECT
      a.AcctBranchName,
    c.CustomerNum,
    c.SourceCustomerId,
    a.SourceAccountId,
    a.AccountNum,
    c.FullName,
    c.LastName,
    c.BirthDate,
    a.Balance,
    case when [RollOverStatus] = 'Y' then 'Yes' Else 'No' end as RollOverStatus
    FROM
    v_Account AS a left join v_Customer AS c
      ON c.CustomerID = a.CustomerID AND c.Businessdate = a.Businessdate
    WHERE
    a.Category = 'Deposit' AND
    c.Businessdate= '2018-11-28'  AND
    isnull(a.Classification,'N/A') IN ('Contractual Account','Non-Term Deposit','Term Deposit')
    AND IsActive = 'Yes' ) tmp """
irtuqstp

irtuqstp4#

值得注意的是,长度不是问题,只是书写的问题。为此,您可以像Gaweda建议的那样使用“"”,或者简单地使用一个字符串变量,例如,通过使用字符串构建器构建它。例如:

val selectElements = Seq("a","b","c")
val builder = StringBuilder.newBuilder
builder.append("select ")
builder.append(selectElements.mkString(","))
builder.append(" where d<10")
val results = sqlContext.sql(builder.toString())
mo49yndu

mo49yndu5#

除了上述方式外,还可以采用以下方式:

val results = sqlContext.sql("select .... " +
" from .... " +
" where .... " +
" group by ....
");
gmxoilav

gmxoilav6#

在三重引号内编写sql,如“sql code“
df = spark.sql(”从表1中选择 *“)
Scala spark和Pyspark也是如此。

相关问题