这是个老问题，但没有一个答案能解决我的问题。我将azurehdinsight用于我的spark集群和kafka消息代理的事件中心。
例如，我使用了和这个答案完全相同的方法，但仍然有错误。

'JavaPackage' object is not callable
Traceback (most recent call last):
  File "/usr/hdp/current/spark2-client/python/pyspark/streaming/kafka.py", line 78, in createStream
    helper = KafkaUtils._get_helper(ssc._sc)
  File "/usr/hdp/current/spark2-client/python/pyspark/streaming/kafka.py", line 217, in _get_helper
    return sc._jvm.org.apache.spark.streaming.kafka.KafkaUtilsPythonHelper()
TypeError: 'JavaPackage' object is not callable

我的代码是：

import os
sparkVersion = '2.3.2'  # update this accordingly 
os.environ['PYSPARK_SUBMIT_ARGS'] = '--packages org.apache.spark:spark-streaming-kafka-0-8:{} pyspark-shell'.format(sparkVersion) 

# import Spark core

from pyspark.sql import SparkSession 
from pyspark.streaming import StreamingContext

# import extra packages

from pyspark.streaming.kafka import KafkaUtils

ssc = StreamingContext(sc, 60)
kafkaStream = KafkaUtils.createStream(ssc, 'hellos.servicebus.windows.net:9093', 'test-group', {'twitter':1})

这是我的spark版本：

sc.version
'2.3.2.2.6.5.3026-7'