simple tableapi sql查询在flink 1.10和blink上不起作用

guicsvcw 于 2021-06-26 发布在 Flink

关注(0)|答案(2)|浏览(828)

我想使用tableapi定义kafka连接器，并在这样描述的表（由kafka支持）上运行sql。不幸的是，似乎 Rowtime 定义没有按预期工作。
下面是一个可复制的示例：

object DefineSource extends App {

  import org.apache.flink.streaming.api.scala._
  import org.apache.flink.table.api.scala._

  val env = StreamExecutionEnvironment.getExecutionEnvironment
  env.setStreamTimeCharacteristic(TimeCharacteristic.EventTime)

  val config = EnvironmentSettings.newInstance().inStreamingMode().useBlinkPlanner().build()
  val tEnv = StreamTableEnvironment.create(env, config)

  val rowtime = new Rowtime().watermarksPeriodicBounded(5000)
  val schema = new Schema()
    .field("k", "string")
    .field("ts", "timestamp(3)").rowtime(rowtime)

  tEnv.connect(new Kafka()
    .topic("test")
    .version("universal"))
    .withSchema(schema)
    .withFormat(new Csv())
    .createTemporaryTable("InputTable")

  val output = tEnv.sqlQuery(
    """SELECT k, COUNT(*)
      |  FROM InputTable
      | GROUP BY k, TUMBLE(ts, INTERVAL '15' MINUTE)
      |""".stripMargin
  )

  tEnv.toAppendStream[(String, Long)](output).print()

  env.execute()
}

这就产生了

org.apache.flink.table.api.TableException: Window aggregate can only be defined over a time attribute column, but TIMESTAMP(3) encountered.
    at org.apache.flink.table.planner.plan.rules.logical.StreamLogicalWindowAggregateRule.getInAggregateGroupExpression(StreamLogicalWindowAggregateRule.scala:51)
    at org.apache.flink.table.planner.plan.rules.logical.LogicalWindowAggregateRuleBase.onMatch(LogicalWindowAggregateRuleBase.scala:79)
    at org.apache.calcite.plan.AbstractRelOptPlanner.fireRule(AbstractRelOptPlanner.java:319)
    at org.apache.calcite.plan.hep.HepPlanner.applyRule(HepPlanner.java:560)
    at org.apache.calcite.plan.hep.HepPlanner.applyRules(HepPlanner.java:419)
    at org.apache.calcite.plan.hep.HepPlanner.executeInstruction(HepPlanner.java:256)
    at org.apache.calcite.plan.hep.HepInstruction$RuleInstance.execute(HepInstruction.java:127)
    at org.apache.calcite.plan.hep.HepPlanner.executeProgram(HepPlanner.java:215)
    at org.apache.calcite.plan.hep.HepPlanner.findBestExp(HepPlanner.java:202)
    at org.apache.flink.table.planner.plan.optimize.program.FlinkHepProgram.optimize(FlinkHepProgram.scala:69)
    at org.apache.flink.table.planner.plan.optimize.program.FlinkHepRuleSetProgram.optimize(FlinkHepRuleSetProgram.scala:87)
    at org.apache.flink.table.planner.plan.optimize.program.FlinkChainedProgram.$anonfun$optimize$1(FlinkChainedProgram.scala:62)
    at scala.collection.TraversableOnce.$anonfun$foldLeft$1(TraversableOnce.scala:160)
    at scala.collection.TraversableOnce.$anonfun$foldLeft$1$adapted(TraversableOnce.scala:160)
    at scala.collection.Iterator.foreach(Iterator.scala:941)
    at scala.collection.Iterator.foreach$(Iterator.scala:941)
    at scala.collection.AbstractIterator.foreach(Iterator.scala:1429)
    at scala.collection.IterableLike.foreach(IterableLike.scala:74)
    at scala.collection.IterableLike.foreach$(IterableLike.scala:73)
    at scala.collection.AbstractIterable.foreach(Iterable.scala:56)
    at scala.collection.TraversableOnce.foldLeft(TraversableOnce.scala:160)
    at scala.collection.TraversableOnce.foldLeft$(TraversableOnce.scala:158)
    at scala.collection.AbstractTraversable.foldLeft(Traversable.scala:108)
    at org.apache.flink.table.planner.plan.optimize.program.FlinkChainedProgram.optimize(FlinkChainedProgram.scala:58)
    at org.apache.flink.table.planner.plan.optimize.StreamCommonSubGraphBasedOptimizer.optimizeTree(StreamCommonSubGraphBasedOptimizer.scala:170)
    at org.apache.flink.table.planner.plan.optimize.StreamCommonSubGraphBasedOptimizer.doOptimize(StreamCommonSubGraphBasedOptimizer.scala:94)
    at org.apache.flink.table.planner.plan.optimize.CommonSubGraphBasedOptimizer.optimize(CommonSubGraphBasedOptimizer.scala:77)
    at org.apache.flink.table.planner.delegation.PlannerBase.optimize(PlannerBase.scala:248)
    at org.apache.flink.table.planner.delegation.PlannerBase.translate(PlannerBase.scala:151)
    at org.apache.flink.table.api.scala.internal.StreamTableEnvironmentImpl.toDataStream(StreamTableEnvironmentImpl.scala:210)
    at org.apache.flink.table.api.scala.internal.StreamTableEnvironmentImpl.toAppendStream(StreamTableEnvironmentImpl.scala:107)

我在Flink 1.10.0 .

scala apache-flink flink-sql flink-table-api

来源：https://stackoverflow.com/questions/62135758/simple-tableapi-sql-query-doesnt-work-on-flink-1-10-and-blink

2条答案

按热度按时间

dwthyt8l1#

不幸的是，它是1.10中的一个bug，正如@lijiayan所说的，应该在1.11中修复+
作为1.10中的解决方法，您可以改用ddl：

tEnv.sqlUpdate(
"CREATE TABLE InputTable (\n" +
"    k STRING,\n" +
"    ts TIMESTAMP(3),\n" +
"    WATERMARK FOR ts AS ts - INTERVAL '5' SECOND\n" +
") WITH (\n" + 
" 'connector.type' = 'kafka',\n" + 
" 'connector.version' = 'universal',\n" +
" 'connector.topic' = 'test',\n" +
" 'connector.properties.zookeeper.connect' = 'localhost:2181',\n" +
" 'connector.properties.bootstrap.servers' = 'localhost:9092',\n" +
" 'format.type' = 'csv'\n" +
")"
);

赞(0）回复(0）举报 2021-06-26

nnt7mjpx2#

这是一个bug，修复了1.10.0+https://issues.apache.org/jira/browse/flink-16160

赞(0）回复(0）举报 2021-06-26