Getting Started with CUDA and Parallel Programming in the AWS Cloud

Articles & Tutorials>Getting Started with CUDA and Parallel Programming in the AWS Cloud
This guide takes you through the steps necessary to set up the CUDA development environment on Amazon Web Services (AWS). You will launch the NVIDIA programming environment on Amazon EC2 virtual servers configured as Cluster GPU instances.


Submitted By: Jinesh Varia
AWS Products Used: Amazon EC2
Created On: March 20, 2013 12:13 AM GMT
Last Updated: March 26, 2013 9:22 PM GMT

This guide takes you through the steps necessary to set up the CUDA development environment on Amazon Web Services (AWS). You will launch the NVIDIA programming environment on Amazon EC2 virtual servers configured as Cluster GPU instances. To do so, you will deploy an AWS CloudFormation template. This configuration can be used for the Udacity CS344: Intro to Parallel Programming online course, or for your own development projects.

When you are done using the CUDA cloud, you should stop or terminate your EC2 instances to avoid incurring additonal charges.

Step 1: Sign Up for the Service

If you don't already have an AWS account, you'll need to get one.Your AWS account gives you access to all services, but you will be charged only for the resources that you use.

To sign up for AWS
  1. Go to and click Sign Up.
  2. Follow the on-screen instructions.

AWS notifies you by email when your account is active and available for you to use.

Step 2: Create a Key Pair

After you launch the virtual server, you will use SSH to connect to it and configure it further. The credentials you'll use to make the SSH connection are an EC2 key pair. For security reasons, EC2 instances use a public/private key pair, rather than a user name and password, to authenticate connection requests. The public key half of this pair is embedded in the instance, so you can use the private key to log in securely without a password. If you already have an existing key pair, you can use that set of credentials. Otherwise, you will need to generate a new EC2 key pair.

To generate a key pair
  1. Open the Amazon EC2 console at
  2. In the Navigation pane, in the Region box, click US East (Virginia).
  3. In the Navigation pane, under Network and Security, click Key Pairs.
  4. Click Create Key Pair.
  5. In the Key Pair dialog box, in the Key Pair Name box, type a name (for example: mykeypair) for the new key pair and then click Create.
  6. Download the private key PEM file (for example: mykeypair.pem) and keep it in a safe place. You will need it to access the EC2 instances that you launch with this key pair. If you lose the key pair, you cannot connect to your EC2 instances.

Step 3: Launch the Virtual Server

You will use an AWS Cloudformation template to launch the virtual servers as EC2 instances, and to install the necessary software. Using a template ensures that you are launching the instances with the proper configuration.

To create an AWS CloudFormation stack using the Udacity template
  1. Open the AWS CloudFormation console at
  2. Click Create Stack.
  3. In the SELECT TEMPLATE pane of the Create Stack dialog:
    • Enter UdacityGPUDevelopment for Stack Name. This name must be unique in your AWS account. You cannot create two stacks with the same name.
    • Select Provide a Template URL.
    • In the text box that appears, enter
    • Click Continue.
  4. In the SPECIFY PARAMETERS pane of the Create Stack dialog: SPECIFY PARAMETERS
  5. In the ADD TAGS pane of the Create Stack dialog:
    • Click Continue.
  6. In the REVIEW pane of the Create Stack dialog:
    • Review Stack Name, Stack Template, and KeyPair to ensure the values are correct.
    • Click Continue.

The AWS CloudFormation now displays your new stack. When the value of Status of your stack is CREATE_COMPLETE, your virtual server is launched, configured, and ready to use.

Step 4: Configure Remote Access to the Desktop of the Virtual Server

The virtual server has been launched with the operating system and development environment you need installed. The next step is to connect to the server to set a password and start the NX Connection Wizard so you access its desktop remotely.

To locate the public DNS name of your virtual server
  1. Open the Amazon EC2 console at
  2. Under Resources, click Running Instances.
  3. Click on the server instance you launched with the AWS CloudFormation stack. It will be of type cg1.4xlarge in a security group that begins with "UdacityGPUDevelopment-".
  4. In the details pane that appears, locate the value of Public DNS. It should be in the format:
To configure the permissions of the keypair file
  1. Open a terminal window. This is found at Applications/Utilities/Terminal on Mac OS X and at Applications/Accessories/Terminal on many Linux distributions.
  2. Before you can use the keypair file to create an SSH connection, you must set permissions on the PEM file for your Amazon EC2 key pair so that only the key owner has permissions to access the key. For example, if you saved the file as mykeypair.pem in the user's home directory, the command is:
    chmod og-rwx ~/mykeypair.pem 
To connect to your virtual server and create a user and password
  1. From the terminal window, enter the following command line, where ssh is the command, gpudev1 is the user name you are using to connect, the at symbol (@) joins the username and the DNS of the machine you are connecting to, and the -i parameter indicates the location of the private key file you saved in Step 2: Create a Key Pair. In this example, we�re assuming it�s been saved to your home directory.
    ssh -i ~/mykeypair.pem
  2. Create a new user and set a password for the user. The command for this is the following.
    sudo passwd gpudev1
  3. When prompted, enter a password value. Make a note of the password you set, you will need this in the next procedure.
To open a remote desktop of the virtual server
  1. On your local machine, install NX NoMachine Client. You can download the latest version for your operating system from If your local operating system is Mac 10.7 or later, you will need to install NoMachine Player for Mac OS X.
  2. Open the NX Connection Wizard.
    • Enter a name for the connection in Name, such as Udacity GPU Development
    • Enter the DNS name of your virtual server as Host
    • Click Next.
  3. On the second page of the wizard:
    • Select Unix/GNOME.
    • Set the size you want for your remote desktop.
    • Click Next.
  4. Click Finish.
  5. Using the NoMachine Client connect to the connection you just created. Log in as user gpudev1 with the password you supplied when you created the user account on the virtual server.
  6. NoMachine Client warns you that the host authenticity cannot be established. Check that the DNS name is correct. If it is, click Yes.
  7. After the connection completes, NoMachine Client displays a remote desktop window for the virtual server. There should be icons on the desktop for Nsight and other developer tools. Your development environment is now configured.

Step 5: Clean Up

When you are done with your development work, you should stop or terminate the EC2 instance hosting your virtual server to avoid accrruing additional charges. You can choose either to stop your EC2 instances or to terminate the CloudFormation stack.

Stopping the instances gives you the option of restarting them at a later time. While the instances are stopped, you will not be charged for compute time, but you may be charged for storing the instance configuration and state.

To stop the virtual server instance
  1. Open the Amazon EC2 console at
  2. Under Resources, click Running Instances.
  3. From the list, select the EC2 instance you want to stop.
  4. Click Actions and select Stop.

Terminating the AWS CloudFormation stack also terminates the EC2 instance and deletes all resources associated with the stack. You will not be able to restart the same instance later. If you need this development environment again, you must launch a new stack.

To terminate the AWS CloudFormation stack
  1. Open the AWS CloudFormation console at
  2. In the list, select your CloudFormation stack.
  3. Click Delete Stack.
©2017, Amazon Web Services, Inc. or its affiliates. All rights reserved.