这是我将spark streaming与flume集成的代码:
val conf = new SparkConf()
.setAppName("File Count")
.setMaster("local[2]")
val sc = new SparkContext(conf)
val ssc = new StreamingContext(sc, Seconds(10))
val flumeStream = FlumeUtils.createPollingStream(ssc,192.168.1.31,8020)
但我有一个错误:找不到:value flumeutils
这是我的pom.xml:
<dependency>
<groupId>org.scala-lang</groupId>
<artifactId>scala-library</artifactId>
<version>2.10.4</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.10</artifactId>
<version>1.6.1</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-streaming-flume-sink_2.10</artifactId>
<version>1.5.0</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-streaming_2.10</artifactId>
<version>1.5.0</version>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-lang3</artifactId>
<version>3.3.2</version>
</dependency>
我的spark版本是1.5.0
任何帮助!!!先谢谢你。
1条答案
按热度按时间lstz6jyr1#
FlumeUtils
一个班在下面吗org.apache.spark.streaming.flume
. 在pom.xml中,您尝试导入的工件是spark-streaming-flume-sink_2.10
它没有FlumeUtils
班级。而是导入以下工件:
希望这有帮助。