AWS Partner Network (APN) Blog
Designing High-Performance Applications Using Serverless TiDB Cloud and AWS Lambda
By Ayan Ray, Sr. Partner Solutions Architect, Data and Analytics – AWS
By Zhen Chen, Software Engineer of Developer Ecosystem – PingCAP
By Arun Vijayraghavan, Principal Product Manager Developer Ecosystem – PingCAP
 |
| PingCAP |
 |
Traditional monolithic applications often struggle to meet the increasing demands of digital businesses.
To achieve better performance, scalability, and agility, many organizations have turned to microservices, which break down large applications into smaller, independent, and more manageable services. Databases are also a major component of microservices.
In this post, you will learn how to build scalable, cost-effective, and serverless microservices using TiDB Cloud and AWS Lambda. TiDB Cloud is based on TiDB which is cloud-native, open-source distributed SQL database with built-in hybrid transactional and analytical processing (HTAP) that can handle multiple workloads seamlessly and effectively.
You will learn how pairing TiDB Cloud with AWS Lambda enables you to build serverless, event-driven microservices, further enhancing the scalability and cost-effectiveness of your architecture.
TiDB Cloud is offered by PingCAP, an AWS Partner and AWS Marketplace Seller that provides enterprise-grade software service and is committed to delivering open-source, cloud-native solutions for growth-oriented clients.
Benefits of AWS Lambda
AWS Lambda is a serverless, event-driven compute service that lets you run code for virtually any type of application or backend service without provisioning or managing servers.
You can trigger AWS Lambda from other Amazon Web Services (AWS) products and software as a service (SaaS) applications, and only pay for what you use.
Lambda runs your code on highly available, fault-tolerant infrastructure spread across multiple AWS Availability Zones (AZs) in a single AWS region. It seamlessly deploys code and provides all of the administration, maintenance, and patches of the infrastructure.
Lambda also offers built-in logging and monitoring, including integration with Amazon CloudWatch, CloudWatch Logs, and AWS CloudTrail. It frees up your programming resources by taking over infrastructure management, allowing you to focus more on innovation and development of business logic.
AWS Lambda precisely manages scaling of your functions (or application) by running event-initiated code in parallel and processing each event individually.
Benefits and Strengths of TiDB Cloud
TiDB Cloud enables organizations to unlock the full potential of their data by providing scalability, high availability, and strong consistency. This open source, distributed SQL database is an ideal choice for performing real-time analytics and global data access.
TiDB Cloud has a growing community and has been widely adopted by various industries including financial technology (FinTech), Web3, and SaaS. For these industries, TiDB Cloud serves as the primary database for workloads that require high availability and strong consistency with large data.
TiDB Cloud has numerous strengths and capabilities, with its most notable including:
- Horizontal scalability: Makes it simple to enhance storage and processing power as data and user volumes grow.
- High availability: Ensures data is always accessible and consistent, even in case of node failures, through features such as automatic failover, load balancing, and data replication.
- MySQL compatibility: Smoothly integrates with existing MySQL applications and tools.
- Cloud-native: Makes deployment and management on cloud platforms more manageable while enabling better utilization of cloud resources.
- Real-time analytics: Offers advanced features like HTAP, which enables both transactional and analytical queries to be executed on the same dataset in real time. With HTAP, there’s no need for data duplication or movement, resulting in faster and more accurate insights, as well as reduced complexity and lower costs.
Solution Overview
In this section, we will describe how to build secure, scalable, and cost-effective RESTful microservices using AWS Lambda, Amazon API Gateway, AWS Secrets Manager, and TiDB Cloud.
Our goal is to guide users through the process of creating a backend service API app for bookstore management. This application will provide a set of APIs for managing book data, such as listing, creating, updating, and deleting books.
Throughout this walkthrough, we’ll provide step-by-step instructions for building a demo app that showcases the backend APIs in action. This demo is not intended to be a fully functional bookstore application, but rather a simplified example that demonstrates the core functionality of the API.
By following along with our instructions, you’ll be able to build a similar demo app to use as a starting point for your own projects.
Figure 1 – AWS Lambda secure interaction with TiDB Cloud using Amazon API Gateway.
Prerequisites and Assumptions
Before getting started, please complete the following prerequisites:
- Create a TiDB Cloud cluster, as we’ll be using TiDB Cloud to store book information.
- Create an AWS account, as we’ll be using various services such as Lambda, Amazon API Gateway, and AWS Secrets Manager.
For the purposes of this post, we’ll create the solution in the us-east-1 region. There are also detailed guidelines in appropriate sections for creating the solution in different regions.
Integration Walkthrough
This integration utilizes Amazon API Gateway to create, publish, and manage APIs, as well as handle incoming requests to the Lambda functions.
We will also use AWS Secrets Manager to securely store and manage access credentials for the TiDB Cloud instance, allowing easy rotation and management without the need to update the application code.
The project’s README file explains in detail the process and code. During this walkthrough, we’ll reference the relevant sections as necessary.
Step 1: Create Secret
Create a secret called aws/lambda/bookstore using AWS Secrets Manager to store TiDB Cloud’s authentication credentials in the default us-east-1 region.
Step 2: Create IAM Role
For a Lambda function to interact with AWS Secret Manager and Amazon CloudWatch, we need to create an AWS Identity and Access Management (IAM) role named LambdaSecretsManagerRole that has both SecretsManagerReadWrite and AWSLambdaBasicExecutionRole permissions. Learn more about creating an IAM role for this project.
Step 3: Create Lambda Function
Follow these instructions to create a Lambda function called bookstoreLambda and associate it to the IAM role created earlier.
When configuring the Lambda function, it’s important to increase the timeout value. This is because our API will also be responsible for initializing and creating various database workloads, such as tables and sample data, which may take longer to complete.
Now that we have created an empty function, it’s time to deploy our code to AWS Lambda and make it fully operational. The simplest method to do this is by uploading a pre-packaged index.zip from the Amazon Simple Storage Service (Amazon S3) location.
Figure 2 – Deploy AWS Lambda code from Amazon S3.
Alternatively, you can package, build, and deploy the Lambda code if you want to configure it in an AWS region different from us-east-1.
Project Walkthrough
The code for this project can be found in PingCAP’s GitHub repository. Below is a quick walkthrough of the project.
The file secretManager.ts securely manages the database’s secrets using the AWS Software Developer Kit (SDK). This file contains key information such as the secret name and region, which must be specified accurately to ensure proper functionality.
Use the web framework Fastify to define routes in the app.ts file, making it a convenient reference point for any changes to the routing structure.
The model.ts file defines the book model and utilizes Sequelize ORM to interact with the TiDB Cloud database, initializing the book table. This is a place to look if you wish to extend the book model.
The index.ts file initializes the database connection using Sequelize and accesses secrets stored in AWS Secrets Manager.
The core functionality of handling HTTP requests can be found within the files located in the routers folder. If you wish to add a new model and its corresponding handlers, this is the appropriate location to do so.
Step 4: Configure Amazon API Gateway
Next, we will configure Amazon API Gateway by defining routes and mapping them to the appropriate Lambda function.
Keep the “Invoke URL” handy as it allows you to perform CRUD operations on the books via the properly configured routes on the API Gateway.
Step 5: Testing the Books API and Confirming the Setup
We’ll use CURL on a terminal to verify the setup of Books API built with Lambda and Amazon API Gateway. But first, let’s initialize the books by running the following command:
curl -X POST -H "Content-Type: application/json" -d '{"count":10}' <invoke-url>/book/init
This command creates the table and insert 10 random books into it.
To create a single book, execute the following command:
curl -X POST -H "Content-Type: application/json" -d '{ "title": "Book Title", "type": "Test", "publishAt": "2022-12-15T21:01:49.000Z", "stock": 123, "price": 12.34, "authors": "Test Test" }' <invoke-url>/book
A new book with the specified title, type, publish date, stock, price, authors, and auto-generated ID will be created.
To delete a book, you can execute the following command:
curl -X DELETE <invoke-url>/book/3
Finally, to retrieve the books you can execute the following command:
curl <invoke-url>/book
The output should retrieve all of the books created in the previous steps, excluding the ones that were deleted.
Congratulations! Your AWS Lambda function can now interact with your TiDB Cloud database.
For more examples of how to interact with the bookstore API using CURL, refer to this section of the README file.
Step 6: Clean Up Your Resources (Optional)
To avoid any unnecessary charges to your AWS account, you can now delete the resources that were created for this demo, unless you wish to retain them.
- Delete the Lambda function by navigating to the Functions page of the Lambda console, selecting the function that was created, and choosing Actions and Delete.
- Next, delete the execution role by navigating to the Roles page of the IAM console, selecting the role that was created, and choosing Delete.
- You can delete the API Gateway by navigating to the API page of the Amazon API Gateway console, selecting the API that was created, and choosing Actions and Delete.
- Finally, navigate to the Secret List page, select the secret that was created earlier, and delete it by choosing Actions and Delete.
Conclusion
Throughout this post, you learned about TiDB Cloud benefits, saw how to use TiDB Cloud to improve application performance, and walked through how to set up an AWs Lambda function for TiDB Cloud access.
Try out TiDB Cloud for free and visit the TiDB Forum for any questions or feedback. You can also learn more about TiDB Cloud on AWS Marketplace.
.
.
PingCAP – AWS Partner Spotlight
PingCAP is an AWS Partner that provides enterprise-grade software service and is committed to delivering open-source, cloud-native solutions for growth-oriented clients.