我正在用下面的代码运行pycharm中的python文件。我面临着同样的问题,当我给Pypark在cmd以及pycharm,有人能帮我这个。提前谢谢。
代码:
from pyspark.sql import SparkSession
from pyspark.sql.types import StructType, StructField, StringType, DateType, DecimalType, IntegerType
spark = SparkSession.builder.master("local[*]").appName("ETL").getOrCreate()
spark.sparkContext.setLogLevel("WARN")
source_data_file = "C:\Python_pgms\apache-spark-etl-pipeline-example-master\apache-spark-etl-pipeline-example-master\data\20160104\*"
print("Fetching")
面对以下问题:
Traceback (most recent call last):
File "C:/Python_pgms/apache-spark-etl-pipeline-example-master/apache-spark-etl-pipeline-example-master/src/etl.py", line 5, in <module>
spark = SparkSession.builder.master("local[*]").appName("ETL").getOrCreate()
File "C:\Spark\spark-3.0.1-bin-hadoop2.7\python\lib\pyspark.zip\pyspark\sql\session.py", line 186, in getOrCreate
File "C:\Spark\spark-3.0.1-bin-hadoop2.7\python\lib\pyspark.zip\pyspark\context.py", line 376, in getOrCreate
File "C:\Spark\spark-3.0.1-bin-hadoop2.7\python\lib\pyspark.zip\pyspark\context.py", line 133, in __init__
File "C:\Spark\spark-3.0.1-bin-hadoop2.7\python\lib\pyspark.zip\pyspark\context.py", line 325, in _ensure_initialized
File "C:\Spark\spark-3.0.1-bin-hadoop2.7\python\lib\pyspark.zip\pyspark\java_gateway.py", line 98, in launch_gateway
File "C:\Users\comp\AppData\Local\Programs\Python\Python36\lib\subprocess.py", line 709, in __init__
restore_signals, start_new_session)
File "C:\Users\comp\AppData\Local\Programs\Python\Python36\lib\subprocess.py", line 997, in _execute_child
startupinfo)
FileNotFoundError: [WinError 2] The system cannot find the file specified
Process finished with exit code 1
1条答案
按热度按时间vdzxcuhz1#
pycharm->运行->编辑配置->环境变量
根据你的路径添加Python和星火之家