如何在java实现的并行算法中指定核心数?

8yoxcaq7  于 2021-06-27  发布在  Java
关注(0)|答案(0)|浏览(198)

下面的并行算法是用 Spark 以及 Java . 但我想知道如何指定可以从机器中使用的内核数来查看性能。一开始我以为我可以在 setMaster("local[2]") 提供它可以使用的核心数。但我认为这不管用。

import org.apache.spark.SparkConf;
import org.apache.spark.api.java.JavaRDD;
import org.apache.spark.api.java.JavaSparkContext;
import org.apache.commons.lang.time.StopWatch;
import java.util.ArrayList;
import java.util.List;

    public class Prime {

        List<Integer> primes = new ArrayList<>();

        //Method to calculate and count the prime numbers
        public void countPrime(int n){
            for (int i = 2; i < n; i++){
                boolean  isPrime = true;

                //check if the number is prime or not
                for (int j = 2; j < i; j++){
                    if (i % j == 0){
                        isPrime = false;
                        break;  // exit the inner for loop
                    }
                }

                //add the primes into the List
                if (isPrime){
                    primes.add(i);
                }
            }
        }

        //Main method to run the program
        public static void main(String[]args){
//
           StopWatch watch = new StopWatch();
           watch.start();

            //creating javaSparkContext object
            SparkConf conf = new SparkConf().setAppName("haha").setMaster("local");
            JavaSparkContext sc = new JavaSparkContext(conf);

            //new prime object
            Prime prime = new Prime();
            prime.countPrime(1000000);

            //parallelize the collection
            JavaRDD<Integer> rdd = sc.parallelize(prime.primes,4);
            long count = rdd.filter(e  -> e == 2|| e % 2 != 0).count();

            watch.stop();
            System.out.println("Time took to run the process is " + watch);
            System.out.println("The number of prime 0 to 100000 after parallelization  is " + count);
            sc.stop();

        }
    }

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题