How do I set up an edge node for my Amazon EMR cluster?

Last updated: 2019-04-18

How do I create an edge node (also known as a gateway node) for my Amazon EMR cluster?

Short Description

To create an edge node:

  1. Create an Amazon Machine Image (AMI) for the master node's Amazon Elastic Compute Cloud (Amazon EC2) instance.
  2. Launch a new EC2 instance from the AMI. Be sure that the new instance's security group allows access to the Amazon EMR cluster's nodes and allows inbound SSH traffic.
  3. Connect to the EC2 instance using SSH and then submit jobs to the Amazon EMR cluster.

Resolution

  1. Open the Amazon EMR console.
  2. Choose the name of the cluster that you want to create an edge node for.
  3. Choose the Hardware tab, and then choose the ID of the master node. Note the EC2 instance ID—you'll need it in step 5.
  4. Open the Amazon Elastic Compute Cloud (Amazon EC2) console.
  5. Choose the master node's instance ID.
  6. Choose Actions, Image, and then Create Image.
  7. Enter an Image name and Image description.
  8. Choose the No reboot option, and then choose Create Image.
  9. From the navigation pane, choose AMIs.
  10. When the image's status changes to available, select the image, and then choose Launch.
  11. Choose an instance type, and then choose Next: Configure Instance Details.
  12. In Step 3: Configure Instance Details, for IAM role, either choose EMR_EC2_DefaultRole, or choose another role that allows Amazon EMR to call other AWS services.
  13. Choose Next: Add Storage, Next: Add Tags, Next: Configure Security Group. You can use the default configurations or change them if needed.
  14. In Step 6: Configure Security Group, choose Create a new security group or Select an existing security group:
    For Create a new security group: Be sure that the security group allows access to the Amazon EMR cluster's nodes and allows inbound SSH traffic.
    For Select an existing security group: Select the master node's security group (the default name is "ElasticMapReduce-master"), and then select a second security group that allows inbound SSH traffic.
  15. Launch the instance.
  16. Connect to the EC2 instance using SSH and then submit jobs to the Amazon EMR cluster.

Did this article help you?

Anything we could improve?


Need more help?