Setting Bootstrap Actions
Last updated: 2020-05-28 17:44:44PDF
A bootstrap action is a custom script executed when a cluster is created to help you modify the cluster environment, install third-party software, and use your own data.
Create a small pay-as-you-go cluster first to test whether the bootstrap action works properly, and if yes, create a production cluster.
An EMR instance can be created in the following steps:
A bootstrap action can be executed at the following three time slots:
- a (after server initialization): after server resource initialization before EMR cluster software installation.
- b (before cluster startup): before cluster service startup.
- c (after cluster startup): after cluster service startup.
A bootstrap action will run bootstrap scripts during cluster creation and scaling. Bootstrap scripts will be executed in sequence in the order of addition, and there can be up to 16 bootstrap actions.
Adding Bootstrap Action
- Add a bootstrap action when creating a cluster on the purchase page.
Select Basic Configuration > Advanced Settings > Add Bootstrap Action to add a bootstrap action.
- Add a bootstrap action on the cluster details page.
In the Cluster List in the EMR Console, select the target cluster and click Details > Bootstrap Action > Add Bootstrap Action to add a bootstrap action.
You can edit or delete the added bootstrap action.
When you add, edit, or delete a bootstrap action in an existing cluster, differences may exist between it and the new environment, which will lead to execution failure. Therefore, please do so with caution.
- Select the running time slot and enter relevant parameters.
- Name: you are recommended to keep it the same as your "object name".
- Script Location: you are recommended to copy the location information on the COS details page. Enter the COS Console, click Bucket List, select the desired script, and select Operation > Details.
On the Details page, you can view the "Object Name" and "Object Address".
SecretKEY: they can be obtained from the COS API key management page. Enter the COS Console and select Key Management > TencentCloud API Key.
Enter the API Key Management page. If there are existing
SecretKEY, they can be used directly; otherwise, you can click Create Key to generate a new pair of them.
- Parameter: this refers to the parameters for running the script. Separate multiple parameters by spaces, and do not add spaces in individual parameters. The total length of "Parameter" and "Name" cannot exceed 240 characters.
Viewing Bootstrap Result
Currently, you cannot specify a bootstrap action during scaling in the console. The bootstrap action specified during cluster creation will be executed during scaling.
If you want to specify a bootstrap action for scaling, use APIs to scale. If a bootstrap action is specified for scaling, it will be executed during scaling; otherwise, the one specified during cluster creation will be executed.
Logs and script files to be executed are stored in the
/usr/local/service/scripts/ directory. The script system log is
- Naming convention: "execution sequence" + "_" + "running time slot" + "script name" + "_" + stderr.
- Naming convention: "execution sequence" + "_" + "running time slot" + "script name" + "_" + stdout.
- The scripts will be executed on all types of nodes, and the script files and output log files of script execution will be stored on each node.
- Bootstrap script content needs to be encoded in UTF-8.