Currently, EMR supports Spark version 2.1.0. The software environment information is as follows:
- By default, Spark is installed on a master node.
- After the log-in, run su hadoop command then you can switch to the Hadoop user.
- The Spark software path is /usr/local/service/spark
- The relevant logs are stored in /data/emr
For more information, see the community documentation. Here is mainly about how to access COS using Spark.
