Spark Environment Info
Last updated: 2020-03-25 10:53:56PDF
Currently, the Spark component in EMR supports Spark 2.0.2, 2.2.1, and 2.3.2. The software environment information is as follows:
- By default, Spark is installed on a master node.
- After the log-in, run the
su hadoopcommand then you can switch to the Hadoop user.
- The Spark software path is
- The relevant logs are stored in
For more information, please see the community documentation. Here is mainly about how to access COS through Spark.