Launch an EMR Job Flow with the MapR Distribution for Hadoop

Introduction

MapR offers an open, enterprise-grade distribution that makes Hadoop easier-to-use and more dependable. Combined with Amazon Elastic MapReduce's managed Hadoop environment, seamless integration with other AWS services, and hourly pricing with no upfront fees or long-term commitments, Amazon EMR with MapR offers customers a powerful tool for generating insights from their data. For more details on EMR with MapR, visit the EMR with the MapR Distribution for Hadoop detail page.

Starting an EMR Job Flow with the MapR Distribution for Hadoop

Follow these steps to start an Amazon EMR with MapR cluster:

Log in to your Amazon Web Services Account

Use your normal Amazon Web Services (AWS) credentials to log in to your AWS account.

Start a New Job Flow and Select a MapR Product

From the AWS Management Console

1. Select Elastic MapReduce, then Create New Job Flow.
2. Select MapR M3 Edition or MapR M5 Edition from the Hadoop Version drop-down selector.




• MapR M3 Edition is a complete Hadoop distribution that provides many unique capabilities such as industry-standard NFS and ODBC interfaces, end-to-end management, high reliability and automatic compression. You can manage a MapR cluster via the AWS Management Console, the command line, or a REST API. Amazon EMR's standard rates include the full functionality of MapR M3 at no additional cost.
• MapR M5 Edition expands the capabilities of M3 with enterprise-grade capabilities such as high availability, snapshots and mirroring.


3. Continue to specify your job flow as described in Creating a Job Flow.

From the Command Line Interface

Use the --with-supported-products parameter with the elastic-mapreduce command to specify a MapR distribution:

• Use mapr-m3 for MapR M3.
• Use mapr-m5 for MapR M5.

This example launches a job flow that uses the MapR M3 Edition distribution:
./elastic-mapreduce --create --alive \
--instance-type m1.xlarge\
--num-instances 5 \
--with-supported-products mapr-m3

From the REST API

Make a call to RunJobFlow that specifies a MapR distribution as a member of the SupportedProducts list:

• Use mapr-m3 for MapR M3.
• Use mapr-m5 for MapR M5.

This example launches a job flow that uses the MapR M3 distribution:
https://elasticmapreduce.amazonaws.com?Action=RunJobFlow
&Name=MyJobFlowName&LogUri=s3n%3A%2F%2Fmybucket%2Fsubdir
&Instances.MasterInstanceType=m1.xlarge&Instances.SlaveInstanceType=m1.xlarge&Instances.InstanceCount=4
&Instances.Ec2KeyName=myec2keyname&Instances.Placement.AvailabilityZone=us-east-1a
&Instances.KeepJobFlowAliveWhenNoSteps=true
&Instances.TerminationProtected=true
&Steps.member.1.Name=MyStepName
&Steps.member.1.ActionOnFailure=CONTINUE
&Steps.member.1.HadoopJarStep.Jar=MyJarFile
&Steps.member.1.HadoopJarStep.MainClass=MyMainClass
&Steps.member.1.HadoopJarStep.Args.member.1=arg1
&Steps.member.1.HadoopJarStep.Args.member.2=arg2
&SupportedProducts.member.1=mapr-m3
&AuthParams

Configuring your MapR Job Flow

After your MapR job flow is running, you need to open a port to enable access to the MapR Control System (MCS). Follow these steps to open a port.

1. Select your job from the list of jobs displayed in Your Elastic MapReduce Job Flows in the Elastic MapReduce tab of the AWS Management Console, then select the Description tab in the lower pane. Make a note of the Master Public DNS Name value. Click the Amazon EC2 tab in the AWS Management Console to open the Amazon EC2 Console Dashboard.

2. Select Security Groups from the Network & Security group in the Navigation pane at the left of the EC2 Console Dashboard.

3. Select Elastic MapReduce-master from the list displayed in Security Groups.

4. In the lower pane, click the Inbound tab.

5. In Port Range:, type 8453. Leave the default value in the Source: field.

6. Click Add Rule, then click Apply Rule Changes.

You can now navigate to the master node's DNS address. Connect to port 8453 to log in to the MapR Control System. Use the string hadoop for both login and password at the MCS login screen.

Testing Your Cluster

Amazon EMR with MapR provides a Debian environment with MapR software running on each node. MapR's NFS interface mounts the cluster is mounted on localhost at the /mapr directory. Follow these steps to create a file and run your first MapReduce job: