Tag: serverless-application


AWS Toolkit for Eclipse: Improved Support for Serverless Applications (Part 3 of 3)

In the first part of the blog series, we created a new application named rekognition-service from the rekognition blueprint. In the second part, we deployed this serverless application to AWS CloudFormation. In this last part of the blog series, we describe how to test and check the result of the newly deployed rekognition-service application.

Test the rekognition-service application by using the Amazon S3 bucket editor

You can drag and drop a group of files, including folders, to the Amazon S3 bucket editor so you can upload them to an Amazon S3 bucket. The .jpg files trigger the underlying Lambda function to be tagged with the name and confidence value returned by Amazon Rekognition. You can also manually update and delete these tags by using the tag dialog box.

Check the Lambda logs by using the AWS Lambda function editor

You can also check the Lambda function logs by using the Lambda function editor. All the Amazon CloudWatch streams for the Lambda function are listed on the Logs tab in the editor. You can double-click one item to open the underlying one stream in Eclipse. You can also select multiple items, right-click, and then select Show Log Events to open the underlying streams in one batch.

This concludes our three-part series. What do you think of the rekognition serverless blueprint and the working flow in the AWS Toolkit for Eclipse? If you have any requests for new blueprints and features in the AWS Toolkit for Eclipse, please let us know. We appreciate your feedback.

AWS Toolkit for Eclipse: Improved Support for Serverless Applications (Part 2 of 3)

In the first part of this blog post, we talked about how to create a new AWS SAM application from the rekognition serverless blueprint. In this second part, we describe how to deploy the application to AWS CloudFormation.

Deploy the rekognition-service application to a new CloudFormation stack

This .gif animation shows the steps to deploy an AWS SAM application to AWS CloudFormation.

What does the AWS Toolkit for Eclipse do for you during deployment

  • Creates a .zip file that contains the project and all its dependencies. Then uploads the file to the specified Amazon S3 bucket.
  • Updates the serverless.template (as shown in the following snippet) to fill in the complete properties for the AWS::Serverless::Function resource type:
    • Replaces the Handler with the FQCN of the AWS Lambda function handler.
    • Generates the actual code URI for CodeUri so that AWS CloudFormation can reference the Lambda function artifact in the S3 bucket.
    • Adds the missing configurations (Runtime, Description, MemorySize, Timeout, Role) and use the default values.
  • Creates a new AWS CloudFormation stack using the updated serverless.template file.

Here is the updated snippet for TagImage in the CloudFormation template.

"TagImage" : {
      "Type" : "AWS::Serverless::Function",
      "Properties" : {
        "Handler" : "com.serverless.demo.function.TagImage",
        "Runtime" : "java8", "CodeUri" : "s3://zhaoxiz-us-west-1/rekognition-service-stack-1497642692569-1497643074359.zip", "Description" : null, "MemorySize" : 512, "Timeout" : 300, "Role" : null,
        "Policies" : [ "AmazonS3FullAccess", "AmazonRekognitionFullAccess" ],
        "Events" : {
          "ProcessNewImage" : {
            "Type" : "S3",
            "Properties" : {
              "Bucket" : {"Ref" : "ImageBucket"},
              "Events" : "s3:ObjectCreated:*",
              "Filter" : {
                "S3Key" : {
                  "Rules" : [{"Name": "suffix", "Value": ".jpg"}]
                }
              }
            }
          }
        }
      }
    }

Deploy the rekognition-service application to an existing CloudFormation stack

We want to update the recognition confidence value to 80 in the Lambda function code and redeploy it to the CloudFormation stack. The following .gif animation shows how you can achieve that. When doing a second deployment for the same project, the AWS toolkit for Eclipse remembers the parameters used in the last deployment, so if you want to keep them the same, you don’t have to retype them.

Notice that we need to change the parameter value of ImageBucketExists to true in the parameter page (Fill in stack template parameters) because the bucket was already created during the first deployment. The underlying CloudFormation stack is updated with the new version of the Lambda function whether or not you update the parameters.

Update the Lambda event source by using the parameters page

Now, we want to configure the trigger event for the Lambda function to another new S3 bucket. This removes the bucket we created in the first deployment and creates a new bucket for this deployment. We only need to redeploy the application and update the ImageBucketExists parameter to false, and the ImageBucketName parameter to the new bucket name. After deployment, you see that the name of the ImageStack in the stack outputs is updated to the new name.

In the third part of this blog post, we’ll talk about how to use the AWS Toolkit for Eclipse to check the result of the rekognition-service application.

AWS Toolkit for Eclipse: Improved Support for Serverless Applications (Part 1 of 3)

I am happy to announce that the latest release of the AWS Toolkit for Eclipse includes a couple new enhancements for developing AWS Serverless Application Model (AWS SAM) applications. In this release, we added a new blueprint: rekognition.

In part 1 of this blog post, we describe and show with an animation what this blueprint does, and how to use the AWS Toolkit for Eclipse to create an application from it. In part 2, we’ll deploy the AWS SAM application to AWS CloudFormation. In part 3, we’ll check the result of the application and test the AWS Lambda function by using the AWS Explorer in the AWS Toolkit for Eclipse.

About the rekognition blueprint

The rekognition blueprint includes a Lambda function TagImage. This Lambda function automatically tags .jpg files uploaded to a specified Amazon S3 bucket by using the Amazon Rekognition service. It applies the top five confident tags recognized by the Amazon Rekognition service as the keys to the Amazon S3 object. It then applies the confident values as tag values, accordingly.

Create an application named rekognition-service from the rekognition blueprint.

This .gif animation shows the steps to create an application from the rekognition blueprint.

About the AWS SAM template

Here is the template snippet from the serverless.template in the project we just created for defining the Lambda function TagImage. Notice that this is a simplified configuration for this Lambda function. This is because during the deployment phase, the AWS Toolkit for Eclipse will fill in all other properties we needed. For a complete configuration set, see Deploying Lambda-based Applications in the AWS Lambda Developer Guide.

In this snippet, we grant the Lambda function permissions to access Amazon S3 and Amazon Rekognition. We also define a triggering event for the Lambda function when uploading .jpg files to the specified Amazon S3 bucket.

"TagImage": {
  "Type": "AWS::Serverless::Function",
  "Properties": {
    "Handler": "TagImage",
    "Policies": [
      "AmazonS3FullAccess",
      "AmazonRekognitionFullAccess"
    ],
    "Events": {
      "ProcessNewImage": {
        "Type": "S3",
        "Properties": {
          "Bucket": {"Ref" : "ImageBucket"},
          "Events": "s3:ObjectCreated:*",
          "Filter": {
            "S3Key": {
              "Rules": [{"Name": "suffix", "Value": ".jpg"}]
            }
          }
        }
      }
    }
  }
}

How the Lambda function works

Here is a snippet from the Lambda function ImageTag. This Lambda function retrieves the Amazon S3 object from the S3Event. Then it calls the Amazon Rekognition service to detect labels with a confidence value of at least 77.

Image imageToTag = new Image().withS3Object(new S3Object().withName(objectKey).withBucket(bucketName));
// Call Rekognition to identify image labels
DetectLabelsRequest request = new DetectLabelsRequest()
                    .withImage(imageToTag)
                    .withMaxLabels(5)
                    .withMinConfidence(77F);
List<Label> labels = rekognitionClient.detectLabels(request).getLabels();

In part 2 of this blog post, we’ll deploy the newly created AWS SAM application to AWS. Then we’ll configure the parameters in the template during the deployment phase. Stay tuned!

Creating and Deploying a Serverless Web Application with CloudFormation and Ember.js

Serverless computing enables you to build scalable and cost-effective applications that scale up or down automatically without provisioning, scaling, and managing servers. You can use AWS Lambda to execute your back-end application code, Amazon API Gateway for a fully managed service to create, publish, maintain, monitor, and secure your REST API, and Amazon S3 to host and serve your static web files.

Ember.js is a popular, long-standing front-end web application framework for developing rich HTML5 web applications. Ember.js has an intuitive command line interface. You can write dramatically less code by using its integrated Handlebars templates. Ember.js also contains an abstracted data access pattern that enables you to write application adapters that communicate with REST APIs.

In this tutorial, we build a simple Ember.js application that initializes the AWS SDK for JavaScript using native Ember.js initializers. The application communicates with API Gateway, which runs backend Lambda code that reads and writes to Amazon DynamoDB. In addition, we use the Ember.js command line interface to package and deploy the web application to Amazon S3 for hosting and for delivery through Amazon CloudFront. We also use AWS CloudFormation with the AWS Serverless Application Model (AWS SAM) to automate the creation and management of our serverless cloud architecture.

Configuring your local development environment

Requirements:

Clone or fork the aws-serverless-ember repository and install the dependencies. Based on your setup, you might need to use the sudo argument when installing these. If you get an EACCESS error, run these commands with sudo npm install:

npm install -g ember-cli
npm install -g bower
git clone https://github.com/awslabs/aws-serverless-ember

You should now have two folders in the project’s root directory, client and cloud. The cloud folder contains our CloudFormation templates and Lambda function code. The client folder contains the client-side Ember.js web application that deploys to S3. Next, install the dependencies:

cd aws-serverless-ember
cd client
npm install && bower install

When this is complete, navigate to the cloud directory in your project’s root directory:

cd ../cloud

Create and deploy the AWS Cloud environment using CloudFormation

The infrastructure for this application uses API Gateway, Lambda, S3, and DynamoDB.

Note: You will incur charges after running these templates. See the appropriate pricing page for details.

The cloud folder of the repository contains the following files:

  • api.yaml – An AWS SAM CloudFormation template for your serverless backend
  • deploy.sh – A bash helper script for deploying with CloudFormation
  • hosting.yaml – A CloudFormation template for your S3 bucket and website
  • swagger.yaml – The API definition for API Gateway
  • index.js – The Lambda function code
  • swagger.yaml – The OpenAPI definition for your serverless REST API

First, create a website bucket with static website hosting. You also create a bucket for deploying your serverless code.

Note: For this section be sure you’ve installed and configured the AWS CLI with appropriate permissions (your CLI should at least have access to read/write to CloudFormation and read/write to S3) for this tutorial.

Run the following command to create the S3 hosting portion of our back-end, wait for completion, and finally describe stacks once complete:

aws cloudformation create-stack --stack-name ember-serverless-hosting --template-body file://$PWD/hosting.yaml && \
aws cloudformation wait stack-create-complete --stack-name ember-serverless-hosting && \ 
aws cloudformation describe-stacks --stack-name ember-serverless-hosting

This creates a CloudFormation stack named ember-serverless-hosting, waits for the stack to complete, and then display the output results. While you wait, you can monitor the events or view the resource creation in the CloudFormation console, or you run the following command in another terminal:

aws cloudformation describe-stack-events –-stack-name ember-serverless-hosting

When the stack is created, the console returns JSON that includes the stack outputs specified in your template. If the JSON does not output for some reason, re-run the last portion of the command: aws cloudformation describe-stacks --stack-name ember-serverless-hosting. This output includes the bucket name you’ll use to deploy your serverless code. It also includes the bucket URL you’ll use to deploy your ember application. For example:

{
    "Stacks": [
        {
            "StackId": "arn:aws:cloudformation:<aws-region>:<account-id>:stack/ember-serverless-hosting/<unique-id>",
            "Description": "Ember Serverless Hosting",
            "Tags": [],
            "Outputs": [
                {
                    "Description": "The bucket used to deploy Ember serverless code",
                    "OutputKey": "CodeBucketName",
                    "OutputValue": "ember-serverless-codebucket-<unique-id>"
                },
                {
                    "Description": "The bucket name of our website bucket",
                    "OutputKey": "WebsiteBucketName",
                    "OutputValue": "ember-serverless-websitebucket-<unique-id>"
                },
                {
                    "Description": "Name of S3 bucket to hold website content",
                    "OutputKey": "S3BucketSecureURL",
                    "OutputValue": "https://ember-serverless-websitebucket-<unique-id>.s3.amazonaws.com"
                },
                {
                    "Description": "URL for website hosted on S3",
                    "OutputKey": "WebsiteURL",
                    "OutputValue": "http://ember-serverless-websitebucket-<unique-id>.s3-website-<aws-region>.amazonaws.com"
                }
            ],
            "CreationTime": "2017-02-22T14:40:46.797Z",
            "StackName": "ember-serverless",
            "NotificationARNs": [],
            "StackStatus": "CREATE_COMPLETE",
            "DisableRollback": false
        }
    ]
}

Next, you create your REST API with CloudFormation and AWS SAM. For this, you need the CodeBucketName output parameter from the previous stack JSON output when running the following command to package the template:

aws cloudformation package --template-file api.yaml --output-template-file api-deploy.yaml --s3-bucket <<CodeBucketName>>

This command creates a packaged template file named api-deploy.yaml. This file contains the S3 URI to your Lambda code, which was uploaded by the previous command. To deploy your serverless REST API, run the following:

aws cloudformation deploy --template-file api-deploy.yaml --stack-name ember-serverless-api --capabilities CAPABILITY_IAM

Next, run the following command to retrieve the outputs:

aws cloudformation describe-stacks --stack-name ember-serverless-api

Note the API Gateway ID OutputValue:

...
"Outputs": [
                {
                    "Description": "URL of your API endpoint", 
                    "OutputKey": "ApiUrl", 
                    "OutputValue": "https://xxxxxxxx.execute-api.us-east-1.amazonaws.com/Prod"
                }, 
                {
                    "Description": "API Gateway ID", 
                    "OutputKey": "Api", 
                    "OutputValue": "xxxxxxxx" //<- This Value is needed in the below command
                }
            ],
...

Now, run the following command to download the JavaScript SDK for your API using the ID noted above:

aws apigateway get-sdk --rest-api-id <<api-id>> --stage-name Prod --sdk-type javascript ./apiGateway-js-sdk.zip

This downloads a .zip file that contains the JavaScript interface generated by API Gateway. You use this interface to interact with API Gateway from your Ember.js application. Extract the contents of the .zip file, which produces a folder named apiGateway-js-sdk , directly into your client/vendor/ folder. You should now have the following client/vendor folder structure:

  • client
    • vendor
      • apiGateway-js-sdk
      • amazon-cognito

The additional files are libraries that the SDK generated by API Gateway uses to properly sign your API Gateway request. For details about how the libraries are used, see the Amazon API Gateway Developer Guide.

To use the SDK in your Ember.js application, you need to ensure Ember loads them properly. We do this by using app.import() statements within the ember-cli-build.js file.

Initialize the AWS SDK for JavaScript with Ember

You’ll set up and initialize the AWS SDK for JavaScript within your application by using ember initializers. Ember initializers allow you to initialize code before your application loads. By using them, you can ensure the AWS SDK and your API SDK are properly initialized before your application is presented to the user.

Open your client/config/environment.js file and add your AWS configuration to the development section. Add the region in which you are running, the Amazon Cognito identity pool ID created in the previous section, the Amazon Cognito user pool ID created in the previous section, and the user pool app client ID created in the previous section.

You can retrieve these values by running:

aws cloudformation describe-stacks --stack-name ember-serverless-api

Use the following values returned in the Output within the client/config/environment.js file:

  • ENV.AWS_POOL_ID -> CognitoIdentityPoolId
  • ENV.AWS_USER_POOL_ID -> CognitoUserPoolsId
  • ENV.AWS_CLIENT_ID -> CognitoUserPoolsClientId
// client/config/environment.js L26
if (environment === 'development') {
    ENV.AWS_REGION = ‘aws-region-1’
    ENV.AWS_POOL_ID = ‘aws-region:unique-hash-id’
    ENV.AWS_USER_POOL_ID = ‘aws-region_unique-id’
    ENV.AWS_CLIENT_ID = ‘unique-user-pool-app-id’
}

Now, run your client and ensure the SDK loads properly. From the client directory, run:

ember s

Then visit http://localhost:4200/ in your web browser, and open the developer tools. (If you’re using Chrome on a macOS machine, press cmd+option+i. Otherwise, press Ctrl+Shift+i.), you should see the following messages:

AWS SDK Initialized, Registering API Gateway Client: 
API Gateway client initialized: 
Object {docsGet: function, docsPost: function, docsDelete: function, docsOptions: function}

This confirms your generated SDK and the AWS SDK are properly loaded and ready for use.

Now try the following:

1. Log in with any user/credentials, and observe the error. (onFailure: Error: User does not exist.)
2. Register with a weak password, and observe the error. (Password did not conform with policy. Password not long enough.)
3. Register a new user, and use an email address you have access to so you receive a confirmation code.
4. Try re-sending the confirmation code.
5. Enter the confirmation code.
6. Log in with your newly created user.

The JWT access token is retrieved from Amazon Cognito user pools, it is decoded and displayed for reference after logging into the client application. The section entitled “Dynamo Items” within the application creates and deletes items within your DynamoDB table (that was previously created with CloudFormation) via API Gateway and Lambda, and authenticated with Cognito User Pools.

Deploying the web application to Amazon S3

In this command, you use the S3 WebsiteBucketName output from the ember-serverless-hosting CloudFormation stack as the sync target.

Note: you can retrieve the output values again by running:
aws cloudformation describe-stacks --stack-name ember-serverless-hosting

cd client
ember build
aws s3 sync dist/ s3://WebsiteBucketName/ --acl public-read

Your website should now be available at the WebsiteURL output value from the ember-serverless-hostingCloudFormation stack’s outputs.

Questions or comments? Reach out to us on the AWS Development Forum.

Source code and issues on GitHub: https://github.com/awslabs/aws-serverless-ember.

AWS Toolkit for Eclipse: Serverless Applications

I’m glad to announce that the AWS Lambda plugin in the AWS Toolkit for Eclipse now supports serverless application development for Java. The serverless application (also called a Lambda-based application) is composed of functions triggered by events. In this blog, I provide two examples to show you how to leverage the Eclipse IDE to create and deploy a serverless application quickly.

Install the AWS Toolkit for Eclipse

To install the latest AWS Toolkit for Eclipse, go to this page and follow the instructions at the top right of the page. You should install the AWS Toolkit for Eclipse Core, AWS CloudFormation Tool, and AWS Lambda Plugin to use this feature. The following figure shows where you can choose these three components in the installation wizard. To complete the installation, you need to review and accept the license and restart Eclipse.

InstallServerless

Create a Serverless Project

To create a serverless project, click the Toolbar AWS and choose New AWS Serverless Project…, The following wizard opens. You can also create a new serverless project using the AWS Toolkit for Eclipse in the usual way: Choose File, New, Other, AWS and then choose AWS Serverless Java Project. As you can see in the following figure, the Toolkit provides two blueprints for you to start with: article and hello-world.

  • article – This is a simple serverless application that helps manage articles. It consists of two Lambda functions triggered by API events. The two functions are GetArticle and PutArticle, which manage storing articles to the backend service and retrieving articles to the front end. This blueprint also leverages an Amazon S3 bucket for storing article content and an Amazon DynamoDB table for storing article metadata.
  • hello-world – This blueprint project only includes a simple stand alone Lambda function, HelloWorld, which is not triggered by any event and not bound to any resource. It simply takes in a String and outputs it with the prefix “Hello”. If an empty String is provided, it outputs “Hello World”.

NewServerlessProject

You can also use a serverless template to create a serverless project by choosing Select a Serverless template file and then importing the template file. This template file is a simplified version of the SAM (AWS Serverless Application Model) file that is used in a serverless application to define the application resources stack. The following snippet is from the blueprint articles template for defining the Lambda function GetArticle. Different from the real SAM file, you don’t need to provide the CodeUri and Runtime properties, and you only need to provide the class name for the Handler property instead of the Fully Qualified Class Name. By importing a template file, the AWS Toolkit for Eclipse will generate all the Lambda function hooks and the Lambda Proxy Integration models used as the API event Input and Output for the Lambda functions.

{
  "Type": "AWS::Serverless::Function",
  "Properties": {
    "Handler": "com.serverless.demo.GetArticle",
    "Runtime": "Java",
    "CodeUri": "s3://serverless-bucket/get-article.zip",
    "Policies": [
      "AmazonDynamoDBReadOnlyAccess",
      "AmazonS3ReadOnlyAccess"
    ],
    ...
}

The following figure shows the startup view after you create the article blueprint project. As you can see from the project structure, the AWS Toolkit for Eclipse puts all the Lambda functions defined in the template into a function package, and all the required models into a model package. You can check the serverless.template file for a closer look at this project. As we mentioned earlier, this is a simplified version of a SAM file, which is derived from AWS CloudFormation template. See the README.html page for the next.

articleStartup

Deploy a Serverless Project

If the serverless project is created from a blueprint, you can deploy it directly to AWS. Notice that the article blueprint will create an S3 bucket and a DynamoDB table for use of the Lambda functions. You can open the serverless.template file and customize the resource names in the Parameters property section, as shown in the following snippet.

"Parameters" : {
    "ArticleBucketName" : {
        "Type" : "String",
        "Default" : "serverless-blueprint-article-bucket",
        "Description" : "Name of S3 bucket used to store the article content.",
        "MinLength" : "0"
    },
    "ArticleTableName" : {
        "Type" : "String",
        "Default" : "serverless-blueprint-article-table",
        "Description" : "Name of DynamoDB table used to store the article metadata.",
        "MinLength" : "0"
      },
      ...
}

To deploy this project to AWS, click the project name in the explorer view, choose Amazon Web Services, and then choose Deploy Serverless Project. Or right click the workspace of any Lambda function file, choose AWS Lambda, and then choose Deploy Serverless Project. You will see the following wizard. Choose the S3 bucket, type the CloudFormation stack name, and then choose Finish. The AWS Toolkit for Eclipse generates the fat JAR file for the underlying Lambda functions, and uploads it to the S3 bucket you chose. It’ll also update the serverless.template file in memory to be a real SAM file and upload it to the S3 bucket. AWS CloudFormation reads this file to create the stack.

DeployArticle

While the AWS CloudFormation stack is being created, a Stack Editor view is shown to indicate the current status of the stack. This page is automatically refreshed every five seconds, but you can also manually refresh it by clicking the refresh icon at the top right of the view. Upon CREATE_COMPLETE, you will see a link to the right  of the Output label in the top section. This link is the Prod stage endpoint of the API Gateway API created by this serverless project.

DeploymentStackEditor

Test a Serverless Project

After successfully deploying the article project, you can test the two APIs by hitting the API Prod endpoint through browser tools or command line tools.

  • Using the Curl command line tool.
    $ curl --data "This is an article!" https://s5cvlouqwe.execute-api.us-west-2.amazonaws.com/Prod?id=1
    Successfully inserted article 1
    $ curl -X GET https://s5cvlouqwe.execute-api.us-west-2.amazonaws.com/Prod?id=1
    This is an article!
  • Using the Simple rest client plugin in Chrome. You can also use this plugin to send a POST request to the endpoint.

We’d like to know what you think of the workflow for developing serverless applications with the AWS Toolkit for Eclipse. Please let us know if there are other features you want to see in this toolkit. We appreciate your comments.