我尝试将DataFrame写入任何类型的文件格式。我尝试以不同的方式和不同的版本重新安装Spark几次,但每次都收到相同的错误,即使是在另一台机器上。目前在Windows 11本地Hadoop 2.7上使用Spark 3.3.1:
data = [[1, 43, 41], [2, 43, 41], [3, 43, 4]]
x = spark.createDataFrame(data)
x.write.csv('qqq')
收到这个:
File "D:\venvs\spark2\spark_hw.py", line 77, in <module>
x.write.csv('qqq')
File "D:\venvs\spark2\lib\site-packages\pyspark\sql\readwriter.py", line 1240, in csv
self._jwrite.csv(path)
File "D:\venvs\spark2\lib\site-packages\py4j\java_gateway.py", line 1321, in __call__
return_value = get_return_value(
File "D:\venvs\spark2\lib\site-packages\pyspark\sql\utils.py", line 190, in deco
return f(*a, **kw)
File "D:\venvs\spark2\lib\site-packages\py4j\protocol.py", line 326, in get_return_value
raise Py4JJavaError(
py4j.protocol.Py4JJavaError: An error occurred while calling o44.csv.
: org.apache.spark.SparkException: Job aborted.
1条答案
按热度按时间mrzz3bfm1#
将把 Dataframe 写入保存方法中指定路径的csv文件