Note:
Spark 2.3 or above no longer supports Kafka 0.8.2.
EMR Version | Spark Version | Kafka Version | Maven Example |
---|---|---|---|
v1.3.1 | 2.0.2 | 0.8.2.1 or higher | spark-streaming-kafka-0-8_2.11 |
v2.0.1 | 2.2.1 | 0.8.2.1 or higher | spark-streaming-kafka-0-8_2.11 |
v2.1.0 | 2.3.2 | 0.10.0 or higher | spark-streaming-kafka-0-10_2.11 |
v3.0.0 | 2.4.3 | 0.10.0 or higher | spark-streaming-kafka-0-10_2.12 |
https://spark.apache.org/docs/{spark.version}/streaming-kafka-integration.html
Replace {spark.version}
with the corresponding Spark version. For example, if you want to query the dependencies of Spark 2.0.2, the access link will be as follows:https://spark.apache.org/docs/2.0.2/streaming-kafka-integration.html
Was this page helpful?