我正在尝试用python在windows上使用mllib。所以我需要spark,而spark又需要hadoop。我已经安装了anaconda2,它包含python2.7、numpy等。
我一直在遵循这个食谱,这在我看来主要是让我到我需要去的地方,但我想我被困在最后一个错误:
Python 2.7.13 |Anaconda 4.3.1 (64-bit)| (default, Dec 19 2016, 13:29:36) [MSC v.1500 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
Traceback (most recent call last):
File "C:\spark\bin\..\python\pyspark\shell.py", line 43, in <module>
spark = SparkSession.builder\
File "C:\spark\python\pyspark\sql\session.py", line 179, in getOrCreate
session._jsparkSession.sessionState().conf().setConfString(key, value)
File "C:\spark\python\lib\py4j-0.10.4-src.zip\py4j\java_gateway.py", line 1133, in __call__
File "C:\spark\python\pyspark\sql\utils.py", line 79, in deco
raise IllegalArgumentException(s.split(': ', 1)[1], stackTrace)
pyspark.sql.utils.IllegalArgumentException: u"Error while instantiating 'org.apache.spark.sql.hive.HiveSessionState':"
从这个输出可以清楚地看到,没有关于找不到winutils.exe的错误。
此外,异常源于py4j的java域,但是由于illegalargumentexception,我们已经丢失了回溯。
感谢所有指导!
干杯
暂无答案!
目前还没有任何答案,快来回答吧!