我正在EMR上使用Pyspark，并尝试按照此处的指南连接到Kinesis：https://spark.apache.org/docs/latest/streaming-kinesis-integration.html
我一直在跑：

spark-submit --jars '/usr/lib/spark/jars/spark-streaming-kinesis-asl_2.12-3.2.1.jar' kinesis_wordcount_asl.py sparkEnrichedDev abc_decoded https://kinesis.eu-west-2.amazonaws.com eu-west-2

但得到了误差

Traceback (most recent call last):
  File "/home/hadoop/kinesis_wordcount_asl.py", line 76, in <module>
    ssc, appName, streamName, endpointUrl, regionName, InitialPositionInStream.LATEST, 2)
  File "/usr/lib/spark/python/lib/pyspark.zip/pyspark/streaming/kinesis.py", line 116, in createStream
  File "/usr/lib/spark/python/lib/py4j-0.10.9.3-src.zip/py4j/java_gateway.py", line 1322, in __call__
  File "/usr/lib/spark/python/lib/py4j-0.10.9.3-src.zip/py4j/protocol.py", line 328, in get_return_value
py4j.protocol.Py4JJavaError: An error occurred while calling o65.createStream.
: java.lang.NoClassDefFoundError: com/amazonaws/services/kinesis/clientlibrary/lib/worker/InitialPositionInStream
        at org.apache.spark.streaming.kinesis.KinesisUtilsPythonHelper.createStream(KinesisUtilsPythonHelper.scala:60)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
        at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
        at py4j.Gateway.invoke(Gateway.java:282)
        at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
        at py4j.commands.CallCommand.execute(CallCommand.java:79)
        at py4j.ClientServerConnection.waitForCommands(ClientServerConnection.java:182)
        at py4j.ClientServerConnection.run(ClientServerConnection.java:106)
        at java.lang.Thread.run(Thread.java:750)
Caused by: java.lang.ClassNotFoundException: com.amazonaws.services.kinesis.clientlibrary.lib.worker.InitialPositionInStream
        at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
        ... 13 more

其他有同样问题的用户似乎说这是版本的问题，但从maven下载的jar是Spark 3.2.1和Scala 2.12的，这与我的版本相同。
我在尝试运行带有packages标志spark-submit --packages org.apache.spark:spark-streaming-kinesis-asl_2.12:3.2.1的spark-submit时也遇到了同样的错误
我也尝试过在Jupyter Notebooks中运行类似的代码，并尝试了许多方法来引用jar/包，但错误是相同的。
有什么好主意吗？