Kafka and Spark Versions in Each EMR Version

Last updated: 2020-02-28 20:02:30

    Dependencies

    EMR Version Spark Version Kafka Version Maven Example
    v1.3.1 2.0.2 0.8.2.1 or higher spark-streaming-kafka-0-8_2.11
    v2.0.1 2.2.1 0.8.2.1 or higher spark-streaming-kafka-0-8_2.11
    v2.1.0 2.3.2 0.10.0 or higher spark-streaming-kafka-0-10_2.11
    v3.0.0 2.4.3 0.10.0 or higher spark-streaming-kafka-0-10_2.12

    Spark 2.3 or above no longer supports Kafka 0.8.2.

    Query Method

    1. Access the official website URL and enter the version number link template:
      https://spark.apache.org/docs/{spark.version}/streaming-kafka-integration.html
      Replace {spark.version} with the corresponding Spark version. For example, if you want to query the dependencies of Spark 2.0.2, the access link will be as follows:
      https://spark.apache.org/docs/2.0.2/streaming-kafka-integration.html
    2. View the dependencies and click the corresponding link to view detailed descriptions.

    Was this page helpful?

    Was this page helpful?

    • Not at all
    • Not very helpful
    • Somewhat helpful
    • Very helpful
    • Extremely helpful
    Send Feedback
    Help