当我运行spark示例时,我得到这个错误
20/09/04 17:06:35警告taskschedulerimpl:初始作业未接受任何资源;检查集群ui以确保worker已注册并且具有足够的资源
我在本地运行这个应用程序,我的spark服务器部署在 47.111.185.105
```
public class WordCountApp {
private static final Logger logger = LoggerFactory.getLogger(WordCountApp.class);
public static void main(String[] args) {
// Should be some file on your system
String logFile = "src/main/resources/people.json";
SparkConf conf = new SparkConf().set("spark.shuffle.service.enabled", "false")
.set("spark.dynamicAllocation.enabled", "false")
.set("spark.cores.max", "1")
.set("spark.executor.instances","2")
.set("spark.executor.memory","500m")
.set("spark.executor.cores","1")
.setMaster("spark://47.111.185.105:7077");
//.set("deploy-mode", "client");
SparkSession spark = SparkSession.builder()
.appName("Word Count Application")
.config(conf)
.getOrCreate();
Dataset<String> logData = spark.read().textFile(logFile).cache();
System.out.println("Spark version = " + spark.version());
logger.info("Spark version = " + spark.version());
long numAs = logData.filter((FilterFunction<String>) s -> s.contains("a")).count();
long numBs = 3;//logData.filter((FilterFunction<String>) s -> s.contains("b")).count();
System.out.println("Lines with a: " + numAs + ", lines with b: " + numBs);
logger.info("Lines with a: " + numAs + ", lines with b: " + numBs);
spark.stop();
}
}
打包并上传jar文件,使用sparkshell效果很好
检查剩余资源是否足够
尝试重新启动spark
![](https://i.stack.imgur.com/MwnZC.png)
暂无答案!
目前还没有任何答案,快来回答吧!