AWS Developer Tools Blog
Getting started with the AWS Cloud Development Kit and Python
This post introduces you to the new Python bindings for the AWS Cloud Development Kit (AWS CDK).
What’s the AWS CDK, you might ask? Good question! You are probably familiar with the concept of infrastructure as code (IaC). When you think of IaC, you might think of things like AWS CloudFormation.
AWS CloudFormation allows you to define your AWS infrastructure in JSON or YAML files that can be managed within your source code repository, just like any other code. You can do pull requests and code reviews. When everything looks good, you can use these files as input into an automated process (CI/CD) that deploys your infrastructure changes.
The CDK actually builds on AWS CloudFormation and uses it as the engine for provisioning AWS resources. Rather than using a declarative language like JSON or YAML to define your infrastructure, the CDK lets you do that in your favorite imperative programming language. This includes languages such as TypeScript, Java, C#, and now Python.
About this post | |
Time to read | 19 minutes |
Time to complete (estimated) | 30 minutes |
Cost to complete | $0 free tier (tiny fraction of a penny if you aren’t free tier) |
Learning level | Intermediate (200) |
Services used |
Why would an imperative language be better than a declarative language? Well, it may not always be but there are some real advantages: IDE integration and composition.
IDE integration
You probably have your favorite IDE for your favorite programming language. It provides all kinds of useful features that make you a more productive developer (for example, code completion, integrated documentation, or refactoring tools).
With CDK, you automatically get all of those same advantages when defining your AWS infrastructure. That’s because you’re doing it in the same language that you use for your application code.
Composition
One of the things that modern programming languages do well is composition. By that, I mean the creation of new, higher-level abstractions that hide the details of what is happening underneath and expose a much simpler API. This is one of the main things that we do as developers, creating higher levels of abstraction to simplify code.
It turns out that this is also useful when defining your infrastructure. The existing APIs to AWS services are, by design, fairly low level because they are trying to expose as much functionality as possible to a broad audience of developers. IaC tools like AWS CloudFormation expose a declarative interface, but that interface is at the same level of the API, so it’s equally complex.
In contrast, CDK allows you to compose new abstractions that hide details and simplify common use cases. Then, it packages that code up as a library in your language of choice so that others can easily take advantage.
One of the other neat things about the CDK is that it is designed to support multiple programming languages. The core of the system is written in TypeScript, but bindings for other languages can be added.
That brings me back to the topic of this post, the Python bindings for CDK.
Sample Python application
First, there is some installation that must happen. Rather than describe all of that here, see Getting Started with the AWS CDK.
Create the application
Now, create a sample application.
The first thing you do is create a directory that contains your Python CDK sample. The CDK provides a CLI tool to make it easy to perform many CDK-related operations. You can see that you are running the init
command with no parameters.
The CLI is responding with information about all the things that the init
command can do. There are different types of apps that you can initialize and there are a number of different programming languages available. Choose sample-app
and python
, of course.
So, what just happened? Quite a bit, actually. The CDK CLI created some Python source code for your sample application. It also created other support files and infrastructure to make it easy to get started with CDK in Python. Here’s what your directory contains now:
Take a closer look at the contents of your directory:
README.md
—The introductory README for this project.app.py
—The “main” for this sample application.cdk.json
—A configuration file for CDK that defines what executable CDK should run to generate the CDK construct tree.hello
—A Python module directory.hello_construct.py
—A custom CDK construct defined for use in your CDK application.hello_stack.py
—A custom CDK stack construct for use in your CDK application.
requirements.txt
—This file is used by pip to install all of the dependencies for your application. In this case, it contains only-e
. This tells pip to install the requirements specified insetup.py
. It also tells pip to runpython setup.py develop
to install the code in thehello
module so that it can be edited in place.setup.py
—Defines how this Python package would be constructed and what the dependencies are.tests
—Contains all tests.unit
—Contains unit tests.test_hello_construct.py—
A trivial test of the custom CDK construct created in thehello
package. This is mainly to demonstrate how tests can be hooked up to the project.
You may have also noticed that as the init
command was running, it mentioned that it had created a virtualenv
for the project as well. I don’t have time to go into virtualenvs
in detail for this post. They are basically a great tool in the Python world for isolating your development environments from your system Python environment and from other development environments.
All dependencies are installed within this virtual environment and have no effect on anything else on your machine. When you are done with this example, you can just delete the entire directory and everything goes away.
You don’t have to use the virtualenv
created here but I highly recommend that you do. Here’s how you would initialize your virtualenv
and then install all of your dependencies.
As you can see, you even have tests included, although they are admittedly simple at this point. It does give you a way to make sure your sample application and all of its dependencies are installed correctly.
Generate an AWS CloudFormation template
Okay, now that you know what’s here, try to generate an AWS CloudFormation template for the constructs that you are defining in your CDK app. You use the CDK Toolkit (the CLI) to do this.
Hmm, that was unexpected. What does this mean? Well, as you will see in a minute, your CDK app actually defines two stacks, hello-cdk-1 and hello-cdk-2. The synth command can only synthesize one stack at a time. It is telling you about the two that it has found and asking you to choose one of them.
That’s a lot of YAML. 147 lines to be exact. If you take some time to study this, you can probably understand all of the AWS resources that are being created. You could probably even understand why they are being created. Rather than go through that in detail right now, instead focus on the Python code that makes up your CDK app. It’s a lot shorter and a lot easier to understand.
First, look at your “main,” app.py
.
Well, that’s short and sweet. You are creating an App
, adding two instances of some class called MyStack
to the app, and then calling the run
method of the App
object.
Now find out what’s going on in the MyStack
class.
This is a bit more interesting. This code is importing some CDK packages and then using those to create a few AWS resources.
First, you create an SQS queue called MyFirstQueue
and set the visibility_timeout
value for the queue. Then you create an SNS topic called MyFirstTopic
.
The next line of code is interesting. You subscribe the SNS topic to the SQS queue and it’s all happening in one simple and easy to understand line of code.
If you have ever done this with the SDKs or with the CLI, you know that there are several steps to this process. You have to create an IAM policy that grants the topic permission to send messages to the queue, you have to create a topic subscription, etc. You can see the details in the AWS CloudFormation stack generated earlier.
All of that gets simplified into a single, readable line of code. That’s an example of what CDK constructs can do to hide complexity in your infrastructure.
The final thing happening here is that you are creating an instance of a HelloConstruct
class. Look at the code behind this.
This code shows an example of creating your own custom constructs in CDK that define arbitrary AWS resources under the hood while exposing a simple API.
Here, your construct accepts an integer parameter num_buckets
in the constructor and then creates that number of buckets inside the scope passed in. It also exposes a grant_read
method that automatically grants the IAM principal passed in read permissions to all buckets associated with your construct.
Deploy the AWS CloudFormation templates
The whole point of CDK is to create AWS infrastructure and so far you haven’t done any of that. So now use your CDK program to generate the AWS CloudFormation templates. Then, deploy those templates to your AWS account and validate that the right resources got created.
Here, the CDK is telling you about the security-related changes that this deployment includes. It shows you the resources or ARN patterns involved, the actions being granted, and the IAM principals to which the grants apply. You can review these and press y when ready. You then see status reported about the resources being created.
At this point, the CLI presents you with another summary of IAM changes and asks you to confirm. This is because your CDK sample application creates two stacks in two different AWS Regions. Approve the changes for the second stack and you see similar status output.
Clean up
Now you can use the AWS Management Console to look at the resources that were created and validate that it all makes sense. After you are finished, you can easily destroy all of these resources with a single command.
Conclusion
In this post, I introduced you to the AWS Cloud Development Kit. You saw how it enables you to define your AWS infrastructure in modern programming languages like TypeScript, Java, C#, and now Python. I showed you how to use the CDK CLI to initialize a new sample application in Python, and walked you though the project structure. I taught you how to use the CDK to synthesize your Python code into AWS CloudFormation templates and deploy them through AWS CloudFormation to provision AWS infrastructure. Finally, I showed you how to clean up these resources when you’re done.
Now it’s your turn. Go build something amazing with the AWS CDK for Python! To help get you started, see the following resources:
- AWS CDK Developer Guide
- API reference for the AWS Construct Library
- Python CDK code examples for fully functional applications
The CDK and the Python language binding are currently in developer preview, so I’d love to get feedback on what you like, and where AWS can do better. The team lives on GitHub at https://github.com/awslabs/aws-cdk where it’s easy to get directly in touch with the engineers building the CDK. Raise an issue if you discover a bug or want to make a feature request. Join the conversation on the aws-cdk Gitter channel to ask questions.