Category: .NET


Creating .NET Core AWS Lambda Projects without Visual Studio

by Norm Johanson | on | in .NET | Permalink | Comments |  Share

In the last post, we talked about AWS Lambda deployment integration with the dotnet CLI, using the Amazon.Lambda.Tools NuGet package to deploy Lambda functions and serverless applications. But what if you want to create an AWS Lambda project outside of Visual Studio? This is especially important if you’re working on platforms other than Windows.

The “dotnet new” Command

The dotnet CLI has a command named new that you can use to create .NET Core projects from the command line. For example, by default there are options for creating many of the common project types.

C:\BlogContent> dotnet new -all                                                                                                             
Template Instantiation Commands for .NET Core CLI.                                                                            
                                                   
Templates                 Short Name       Language      Tags                                                                 
------------------------------------------------------------------------------------------------------                                                      
Console Application       console          [C#], F#      Common/Console                                                       
Class library             classlib         [C#], F#      Common/Library                                                       
Unit Test Project         mstest           [C#], F#      Test/MSTest                                                          
xUnit Test Project        xunit            [C#], F#      Test/xUnit                                                           
ASP.NET Core Empty        web              [C#]          Web/Empty                                                            
ASP.NET Core Web App      mvc              [C#], F#      Web/MVC                                                              
ASP.NET Core Web API      webapi           [C#]          Web/WebAPI                                                           
Nuget Config              nugetconfig                    Config                                                               
Web Config                webconfig                      Config                                                               
Solution File             sln                            Solution                                                             
                                                                                                                             
Examples:                                                                                                                     
    dotnet new mvc --auth None --framework netcoreapp1.1                                                                      
    dotnet new mvc --framework netcoreapp1.1                                                                                  
    dotnet new --help   

The new command also has the ability to add more project types via NuGet. We recently released a new NuGet package named Amazon.Lambda.Templates that wraps up all the templates we expose in Visual Studio as project types you can create from the dotnet CLI. To install this NuGet package, run the following command.

dotnet new -i Amazon.Lambda.Templates::*

The trailing ::* in the command specifies to install the latest version. Once the install is complete, the Lambda templates show up as part of dotnet new.

C:\BlogContent> dotnet new -all                                                                                                             
Template Instantiation Commands for .NET Core CLI.                                                                            
                                                                                                                             
Templates                            Short Name                    Language      Tags                                         
------------------------------------------------------------------------------------------------------                                                      
Lambda Detect Image Labels           lambda.DetectImageLabels      [C#]          AWS/Lambda/Function                          
Lambda Empty Function                lambda.EmptyFunction          [C#]          AWS/Lambda/Function                          
Lambda Simple DynamoDB Function      lambda.DynamoDB               [C#]          AWS/Lambda/Function                          
Lambda Simple Kinesis Function       lambda.Kinesis                [C#]          AWS/Lambda/Function                          
Lambda Simple S3 Function            lambda.S3                     [C#]          AWS/Lambda/Function                          
Lambda ASP.NET Core Web API          lambda.AspNetCoreWebAPI       [C#]          AWS/Lambda/Serverless                        
Lambda DynamoDB Blog API             lambda.DynamoDBBlogAPI        [C#]          AWS/Lambda/Serverless                        
Lambda Empty Serverless              lambda.EmptyServerless        [C#]          AWS/Lambda/Serverless                        
Console Application                  console                       [C#], F#      Common/Console                               
Class library                        classlib                      [C#], F#      Common/Library                               
Unit Test Project                    mstest                        [C#], F#      Test/MSTest                                  
xUnit Test Project                   xunit                         [C#], F#      Test/xUnit                                   
ASP.NET Core Empty                   web                           [C#]          Web/Empty                                    
ASP.NET Core Web App                 mvc                           [C#], F#      Web/MVC                                      
ASP.NET Core Web API                 webapi                        [C#]          Web/WebAPI                                   
Nuget Config                         nugetconfig                                 Config                                       
Web Config                           webconfig                                   Config                                       
Solution File                        sln                                         Solution                                     
                                                                                                                             
Examples:                                                                                                                     
    dotnet new mvc --auth None --framework netcoreapp1.1                                                                      
    dotnet new classlib                                                                                                       
    dotnet new --help                                                                                                         
C:\BlogContent>  

To get details about a template, you can use the help command.


dotnet new lambda.EmptyFunction –help

C:\BlogContent> dotnet new lambda.EmptyFunction --help                                                                                                    
Template Instantiation Commands for .NET Core CLI.                                                                                          
                                                                                                                                           
Lambda Empty Function (C#)                                                                                                                  
Author: AWS                                                                                                                                 
Options:                                                                                                                                    
  -p|--profile  The AWS credentials profile set in aws-lambda-tools-defaults.json and used as the default profile when interacting with AWS.
                string - Optional                                                                                                           
                                                                                                                                           
  -r|--region   The AWS region set in aws-lambda-tools-defaults.json and used as the default region when interacting with AWS.              
                string - Optional       

You can see here that the template takes two optional parameters to set the profile and region. These values are written to the aws-lambda-tools-default.json so you can get started deploying with the Lambda tooling right away.

To create a function, run the following command.

dotnet new lambda.EmptyFunction --name BlogFunction --profile default --region us-east-2

This creates a project for the Lambda function and a test project. We can now use any editor we want to build and test our .NET Core Lambda function. Once we’re ready to deploy the function, we run the following commands.

cd ./BlogFunction/src/BlogFunction
dotnet restore
dotnet lambda deploy-function BlogFunction –function-role TestRole

After deployment,we can even test the function from the command line by using the following command.

dotnet lambda invoke-function BlogFunction --payload "Hello World"
C:\BlogContent> dotnet lambda invoke-function BlogFunction --payload "Hello World"
Payload:
"HELLO WORLD"

Log Tail:
START RequestId: a54b750b-0dca-11e7-9099-27598ea7c35d Version: $LATEST
END RequestId: a54b750b-0dca-11e7-9099-27598ea7c35d
REPORT RequestId: a54b750b-0dca-11e7-9099-27598ea7c35d  Duration: 0.99 ms       Billed Duration: 100 ms         Memory Size: 256 MB     Max Memory Used: 42 MB

Summary

With our Lambda tooling provided by Amazon.Lambda.Tools and our project templates provided by Amazon.Lambda.Templates, you can develop .NET Core Lambda functions on any platform. As always, let us know what you think on our GitHub repository.

Deploying .NET Core AWS Lambda Functions from the Command Line

by Norm Johanson | on | in .NET | Permalink | Comments |  Share

In previous posts about our .NET Core support with AWS Lambda, we’ve shown how you can create Lambda functions and serverless applications with Visual Studio. But one of the most exciting things about .NET Core is its cross-platform support with the new command line interface (CLI) named dotnet. To help you develop Lambda functions outside of Visual Studio, we’ve released the Amazon.Lambda.Tools NuGet package that integrates with the dotnet CLI.

We released Amazon.Lambda.Tools as a preview with our initial release of .NET Core on Lambda. We kept it in preview while .NET Core tooling, including the dotnet CLI, was in preview. With the recent release of Visual Studio 2017, the dotnet CLI and our integration with it is now marked as generally available (GA). If you’re still using preview versions of the dotnet CLI and the pre-Visual Studio 2017 project structure, the GA release of Amazon.Lambda.Tools will still work for those projects.

.NET Core Project Structure

When .NET Core was originally released last summer, you would define a project in a JSON file named project.json. At that time, it was announced that this was temporary, that .NET Core was moving to be in line with other .NET projects and would be based on the msbuild XML format, and that each project would contain a .csproj file. As part of the GA release of the dotnet CLI tooling, the release includes the switch to the msbuild format.

Amazon.Lambda.Tools Registration

If you create an AWS Lambda project in Visual Studio, the command line integration is set up automatically so that you can easily transition from Visual Studio to the command line. If you inspect a project created in Visual Studio 2017, you’ll notice a DotNetCliToolReference for the Amazon.Lambda.Tools NuGet package.


<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>netcoreapp1.0</TargetFramework>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Amazon.Lambda.Core" Version="1.0.0" />
    <PackageReference Include="Amazon.Lambda.Serialization.Json" Version="1.0.1" />
  </ItemGroup>

  <ItemGroup>
    <DotNetCliToolReference Include="Amazon.Lambda.Tools" Version="1.4.0" />
  </ItemGroup>

</Project>

In Visual Studio 2015, which uses the older project.json format, the Amazon.Lambda.Tools package is declared as a build dependency and is also registered in the tools section.


{
  "version": "1.0.0-*",
  "buildOptions": {
  },

  "dependencies": {
    "Microsoft.NETCore.App": {
      "type": "platform",
      "version": "1.0.0"
    },

    "Amazon.Lambda.Core": "1.0.0*",
    "Amazon.Lambda.Serialization.Json": "1.0.1",

    "Amazon.Lambda.Tools" : {
      "type" :"build",
      "version":"1.4.0 "
    }
  },

  "tools": {
    "Amazon.Lambda.Tools" : "1.4.0 "
  },

  "frameworks": {
    "netcoreapp1.0": {
      "imports": "dnxcore50"
    }
  }
}

Adding to an Existing Project

The Amazon.Lambda.Tools NuGet package is marked as DotNetCliTool package type. Right now Visual Studio 2017 doesn’t understand the new package type. If you attempt to add the NuGet package through Visual Studio’s Manage NuGet Packages dialog it won’t be able to add the reference. Till Visual Studio 2017 is updated you will need to manually add the DotNetCliToolReference in the csproj file.

Deploying Lambda Functions

All the tooling we developed for Visual Studio to deploy Lambda functions originated in the Amazon.Lambda.Tools package. That means all the deployment features you use inside Visual Studio you can also do from the command line.

To get started, in a command window navigate to a project you created in Visual Studio. To see the available commands, enter dotnet lambda help.

C:\BlogContent\BlogExample\BlogExample> dotnet lambda help                                                                                     
AWS Lambda Tools for .NET Core functions                                                                 
Project Home: https://github.com/aws/aws-lambda-dotnet                                                   
                                                                                                        
                                                                                                        
Commands to deploy and manage AWS Lambda functions:                                                      
                                                                                                        
        deploy-function         Command to deploy the project to AWS Lambda                              
        invoke-function         Command to invoke a function in Lambda with an optional input            
        list-functions          Command to list all your Lambda functions                                
        delete-function         Command to delete an AWS Lambda function                                 
        get-function-config     Command to get the current runtime configuration for a Lambda function   
        update-function-config  Command to update the runtime configuration for a Lambda function        
                                                                                                        
                                                                                                        
Commands to deploy and manage AWS Serverless applications using AWS CloudFormation:                      
                                                                                                        
        deploy-serverless       Command to deploy an AWS Serverless application                          
        list-serverless         Command to list all your AWS Serverless applications                     
        delete-serverless       Command to delete an AWS Serverless application                          
                                                                                                        
                                                                                                        
Other Commands:                                                                                          
                                                                                                        
        package                 Command to package a Lambda project into a zip file ready for deployment
                                                                                                        
                                                                                                        
To get help on individual commands execute:                                                              
        dotnet lambda help <command>  

By using the dotnet lambda command you have access to a collection of commands to manage Lambda functions and serverless applications. There is also a package command that packages your project into a .zip file, ready for deployment. This can be useful for CI systems.

To see help for an individual command, type dotnet lambda help followed by the command name; for example, dotnet lambda help deploy-function.

C:\BlogContent\BlogExample\BlogExample> dotnet lambda help deploy-function
AWS Lambda Tools for .NET Core functions
Project Home: https://github.com/aws/aws-lambda-dotnet

deploy-function:
   Command to deploy the project to AWS Lambda

   dotnet lambda deploy-function [arguments] [options]
   Arguments:
      <FUNCTION-NAME> The name of the function to deploy
   Options:
     --region                                The region to connect to AWS services, if not set region will be detected from the environment (Default Value: us-east-2)
      --profile                               Profile to use to look up AWS credentials, if not set environment credentials will be used (Default Value: normj+vpc)
     --profile-location                      Optional override to the search location for Profiles, points at a shared credentials file
      -pl    | --project-location             The location of the project, if not set the current directory will be assumed
      -cfg   | --config-file                  Configuration file storing default values for command line arguments. Default is aws-lambda-tools-defaults.json
      -c     | --configuration                Configuration to build with, for example Release or Debug (Default Value: Release)
      -f     | --framework                    Target framework to compile, for example netcoreapp1.0 (Default Value: netcoreapp1.0)
      -pac   | --package                      Application package to use for deployment, skips building the project
      -fn    | --function-name                AWS Lambda function name
      -fd    | --function-description         AWS Lambda function description
      -fp    | --function-publish             Publish a new version as an atomic operation
      -fh    | --function-handler             Handler for the function <assembly>::<type>::<method> (Default Value: BlogExample::BlogExample.Function::FunctionHandler)
      -fms   | --function-memory-size         The amount of memory, in MB, your Lambda function is given (Default Value: 256)
      -frole | --function-role                The IAM role that Lambda assumes when it executes your function
      -ft    | --function-timeout             The function execution timeout in seconds (Default Value: 30)
      -frun  | --function-runtime             The runtime environment for the Lambda function (Default Value: dotnetcore1.0)
      -fsub  | --function-subnets             Comma delimited list of subnet ids if your function references resources in a VPC
      -fsec  | --function-security-groups     Comma delimited list of security group ids if your function references resources in a VPC
      -dlta  | --dead-letter-target-arn       Target ARN of an SNS topic or SQS Queue for the Dead Letter Queue
      -ev    | --environment-variables        Environment variables set for the function. Format is <key1>=<value1>;<key2>=<value2>
      -kk    | --kms-key                      KMS Key ARN of a customer key used to encrypt the function's environment variables
      -sb    | --s3-bucket                    S3 bucket to upload the build output
      -sp    | --s3-prefix                    S3 prefix for for the build output
      -pcfg  | --persist-config-file          If true the arguments used for a successful deployment are persisted to a config file. Default config file is aws-lambda-tools-defaults.json
C:\BlogContent\BlogExample\BlogExample>

As you can see, you can set many options with this command. This is where the aws-lambda-tools-defaults.json file, which is created as part of your project, comes in handy. You can set the options in this file, which is read by the Lambda tooling by default. The project templates created in Visual Studio set many of these fields with default values.


{                                                                                   
  "profile":"default",                                                            
  "region" : "us-east-2",                                                           
  "configuration" : "Release",                                                      
  "framework" : "netcoreapp1.0",                                                    
  "function-runtime":"dotnetcore1.0",                                               
  "function-memory-size" : 256,                                                     
  "function-timeout" : 30,                                                          
  "function-handler" : "BlogExample::BlogExample.Function::FunctionHandler"         
}

When you use this aws-lambda-tools-default.json file, the only things left that the Lambda tooling needs to deploy the function are the name of the Lambda function and the IAM role. You do this by using the following command:

dotnet lambda deploy-function TheFunction --function-role TestRole
C:\BlogContent\BlogExample\BlogExample> dotnet lambda deploy-function TheFunction --function-role TestRole                                                                                                  
Executing publish command                                                                                                                                                           
Deleted previous publish folder                                                                                                                                                     
... invoking 'dotnet publish', working folder 'C:\BlogContent\BlogExample\BlogExample\bin\Release\netcoreapp1.0\publish'                                                            
... publish: Microsoft (R) Build Engine version 15.1.548.43366                                                                                                                      
... publish: Copyright (C) Microsoft Corporation. All rights reserved.                                                                                                              
... publish:   BlogExample -> C:\BlogContent\BlogExample\BlogExample\bin\Release\netcoreapp1.0\BlogExample.dll                                                                      
Zipping publish folder C:\BlogContent\BlogExample\BlogExample\bin\Release\netcoreapp1.0\publish to C:\BlogContent\BlogExample\BlogExample\bin\Release\netcoreapp1.0\BlogExample.zip
... zipping: Amazon.Lambda.Core.dll                                                                                                                                                 
... zipping: Amazon.Lambda.Serialization.Json.dll                                                                                                                                   
... zipping: BlogExample.deps.json                                                                                                                                                  
... zipping: BlogExample.dll                                                                                                                                                        
... zipping: BlogExample.pdb                                                                                                                                                        
... zipping: Newtonsoft.Json.dll                                                                                                                                                    
... zipping: System.Runtime.Serialization.Primitives.dll                                                                                                                            
Creating new Lambda function TheFunction                                                                                                                                            
New Lambda function created                                                                                                                                                         
C:\BlogContent\BlogExample\BlogExample>             

You can also pass an alternative file that contains option defaults by using the –config-file option. This enables you to reuse multiple Lambda configurations.

“dotnet publish” vs “dotnet lambda” Commands

Using the Amazon.Lambda.Tools package is the preferred way to deploy functions to Lambda from the command line versus using the dotnet publish command, zipping that output folder, and sending the zip file to Lambda. The Lambda tooling looks at the publish folder and removes any duplicate native dependency in it, which reduces the size of your Lambda function. For example, if you reference the SQL Server client NuGet package, System.Data.SqlClient, the Lambda tooling produces a package file that is about 1 MB smaller than the zipped publish folder from dotnet publish. It also reworks the layout of native dependencies to ensure that the Lambda service finds the native dependencies.

Summary

We hope the Amazon.Lambda.Tools package helps you with the transition from working in Visual Studio to working in the command line to script and automate your deployments. Let us know what you think on our GitHub repository, and what you’d like to see us add to the tooling.

AWS SDK for .NET Supports Assume Role Profiles and the Shared Credentials File

by John Vellozzi | on | in .NET | Permalink | Comments |  Share

The AWS SDK for .NET, AWS Tools for PowerShell, and the AWS Toolkit for Visual Studio now support the use of the AWS CLI credentials file. Some of the AWS SDKs have supported shared use of the AWS CLI credentials file for some time, and we’re happy to add the SDK for .NET to that list.

For a long time, the SDK for .NET has supported reading and writing of its own credentials file. We’ve added support for new credential profile types to facilitate feature parity with the shared credentials file. The SDK for .NET and Tools for PowerShell now support reading and writing of basic, session, and assume role credential profiles to both the .NET credentials file and the shared credentials file. The .NET credentials file maintains its support for federated credential profiles.

With the new Amazon.Runtime.CredentialManagement namespace, you now have programmatic access to read and write credential profiles to the .NET credentials file and the shared credentials file. This is a new namespace, and some older classes have been deprecated. Please see the developer guide topic Configuring AWS Credentials and the API Reference for details.

AWS Tools for PowerShell now enable you to read and write credential profiles to both credentials files as well. We’ve added parameters to the credentials-related cmdlets to support the new profile types and the shared credentials file. You can reference the new profiles with the -ProfileName argument in the service cmdlets. You can find more details about the changes to Tools for PowerShell in Shared Credentials in AWS Tools for PowerShell and the AWS Tools for PowerShell Cmdlet Reference.

In Visual Studio you’ll now see profiles stored in (user’s home directory)\.aws\credentials listed in the AWS Explorer. Reading is supported for all profile types and you can edit basic profiles.

What You Need to Know

In addition to the new Amazon.Runtime.CredentialManagement classes, the SDK has some internal changes. The SDK’s region resolution logic now looks for the region in the default credential profile. This is especially important for SDK for .NET applications running in Amazon EC2. The SDK for .NET determines the region for a request from:

  1. The client configuration, or what is explicitly set on the AWS service client.
  2. The AWSConfigs.RegionEndpoint property (set explicitly or in AppConfig).
  3. The AWS_REGION environment variable, if it’s non-empty.
  4. The “default” credential profile. (See “Credential Profile Resolution” below for details.).
  5. EC2 instance metadata.

Checking the “default” credential profile is a new step in the process. If your application relies on EC2 instance metadata for the region, ensure that the SDK doesn’t pick up the wrong region from one of the credentials files.

Although there aren’t any changes to the credentials resolution logic, it’s important to understand how credential profiles fit into that as well. The SDK for .NET will (continue to) determine the credentials to use for service requests from:

  1. The client configuration, or what is explicitly set on the AWS service client.
  2. BasicAWSCredentials that are created from the AWSAccessKey and AWSSecretKey AppConfig values, if they’re available.
  3. A search for a credentials profile with a name specified by a value in AWSConfigs.AWSProfileName (set explicitly or in AppConfig). (See “Credential Profile Resolution” below for details.)
  4. The “default” credentials profile. (See “Credential Profile Resolution” below for details.)
  5. SessionAWSCredentials that are created from the AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_SESSION_TOKEN environment variables, if they’re all non-empty.
  6. BasicAWSCredentials that are created from the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables, if they’re both non-empty.
  7. EC2 instance metadata.

Credential Profile Resolution

With two different credentials file types, it’s important to understand how to configure the SDK and Tools for PowerShell to use them. The AWSConfigs.AWSProfilesLocation (set explicitly or in AppConfig) controls how the SDK finds credential profiles. The -ProfileLocation command line argument controls how Tools for PowerShell find a profile. Here’s how the configuration works in both cases:

Profile Location Value Profile Resolution Behavior
null (not set) or empty *First search the .NET credentials file for a profile with the specified name. If the profile isn’t there, search (user’s home directory)\.aws\credentials. If the profile isn’t there, search (user’s home directory)\.aws\config.
The path to a file in the shared credentials file format Search only the specified file for a profile with the specified name.

*The .NET credentials file is not supported on Mac and Linux platforms, and is skipped when resolving credential profiles.

Time-to-Live Support in Amazon DynamoDB

by Pavel Safronov | on | in .NET | Permalink | Comments |  Share

Amazon DynamoDB recently added Time-to-Live (TTL) support, a way to automatically delete expired items from your DynamoDB table. This blog post discusses this feature, how it’s exposed in the AWS SDK for .NET, and how you can take advantage of it.

Using Time-to-Live

At a high-level, you configure TTL by choosing a particular attribute on a table that will be treated as a timestamp. Then you simply store an expiration time into this attribute on every item that you need to expire. A periodic process in DynamoDB identifies whether an item’s TTL timestamp attribute is now in the past, and then schedules the removal of that item from the table. The timestamps must be stored as epoch seconds (number of seconds since 12:00:00 AM January 1st, 1970 UTC), which you must calculate or have the SDK calculate for you.

The AWS SDK for .NET has three different DynamoDB APIs, so you have three different ways to use TTL. In the following sections, we discuss these APIs and how you use the TTL feature from each of them.

Low-Level Model – Control Plane

First, the low-level model. This is a thin wrapper around the DynamoDB service operations that you use by instantiating AmazonDynamoDBClient and calling its various operations. This model provides you with the most control, but it also doesn’t have the helpful abstractions of the higher-level APIs. Using the low-level model, you can enable and disable the TTL feature and configure Time-to-Live for your data.

Here’s an example of checking the status of TTL for a table.


using (var client = new AmazonDynamoDBClient())
{
    // Retrieve TTL status
    var ttl = client.DescribeTimeToLive(new DescribeTimeToLiveRequest
    {
        TableName = "SessionData"
    }).TimeToLiveDescription;
    Console.WriteLine($"TTL status = {ttl.TimeToLiveStatus}");
    Console.WriteLine($"TTL attribute {(ttl.AttributeName == null ? "has not been set" : $"= {ttl.AttributeName}")}");

    // Enable TTL
    client.UpdateTimeToLive(new UpdateTimeToLiveRequest
    {
        TableName = "SessionData",
        TimeToLiveSpecification = new TimeToLiveSpecification
        {
            Enabled = true,
            AttributeName = "ExpirationTime"
        }
    });

    // Disable TTL
    client.UpdateTimeToLive(new UpdateTimeToLiveRequest
    {
        TableName = "SessionData",
        TimeToLiveSpecification = new TimeToLiveSpecification
        {
            Enabled = false,
            AttributeName = "ExpirationTime"
        }
    });
}

Note: There is a limit to how often you can enable or disable TTL in a given period of time. Running this sample multiple times will likely result in a ValidationException being thrown.

Low Level – Data Plane

Actually writing and reading TTL data in an item is fairly straightforward, but you are required to write epoch seconds into an AttributeValue. You can calculate the epoch seconds manually or use helper methods in AWSSDKUtils, as shown below.

Here’s an example of using the low-level API to work with TTL data.


using (var client = new AmazonDynamoDBClient())
{
    // Writing TTL attribute
    DateTime expirationTime = DateTime.Now.AddDays(7);
    Console.WriteLine($"Storing expiration time = {expirationTime}");
    int epochSeconds = AWSSDKUtils.ConvertToUnixEpochSeconds(expirationTime);
    client.PutItem("SessionData", new Dictionary<string, AttributeValue>
    {
        { "UserName", new AttributeValue { S = "user1" } },
        { "ExpirationTime", new AttributeValue { N = epochSeconds.ToString() } }
    });

    // Reading TTL attribute
    var item = client.GetItem("SessionData", new Dictionary<string, AttributeValue>
    {
        { "UserName", new AttributeValue { S = "user1" } },
    }).Item;
    string epochSecondsString = item["ExpirationTime"].N;
    epochSeconds = int.Parse(epochSecondsString);
    expirationTime = AWSSDKUtils.ConvertFromUnixEpochSeconds(epochSeconds);
    Console.WriteLine($"Stored expiration time = {expirationTime}");
}

Document Model

The Document Model provides you with Table objects that represent a DynamoDB table, and Document objects that represent a single row of data in a table. You can store primitive .NET types directly in a Document, with the required conversion to DynamoDB types happening in the background. This makes the Document Model API easier to use than the low-level model.

Using the Document Model API, you can easily configure which attributes you’d like to store as epoch seconds by setting the TableConfig.AttributesToStoreAsEpoch collection. Then you can use DateTime objects without needing to convert the data to epoch seconds manually. If you don’t specify which attributes to store as epoch seconds, then instead of writing epoch seconds in that attribute you would end up storing the DateTime as an ISO-8601 string, such as “2017-03-09T05:49:38.631Z”. In that case, DynamoDB Time-to-Live would NOT automatically delete the item. So you need to be sure to specify AttributesToStoreAsEpoch correctly when you’re creating the Table object.

Here’s an example of configuring the Table object, then writing and reading TTL items.


// Set up the Table object
var tableConfig = new TableConfig("SessionData")
{
    AttributesToStoreAsEpoch = new List { "ExpirationTime" }
};
var table = Table.LoadTable(client, tableConfig);

// Write TTL data
var doc = new Document();
doc["UserName"] = "user2";

DateTime expirationTime = DateTime.Now.AddDays(7);
Console.WriteLine($"Storing expiration time = {expirationTime}");
doc["ExpirationTime"] = expirationTime;

table.PutItem(doc);

// Read TTL data
doc = table.GetItem("user2");
expirationTime = doc["ExpirationTime"].AsDateTime();
Console.WriteLine($"Stored expiration time = {expirationTime}");

Object Persistence Model

The Object Persistence Model simplifies interaction with DynamoDB even more, by enabling you to use .NET classes with DynamoDB. This interaction is done by passing objects to the DynamoDBContext, which handles all the conversion logic. Using TTL with the Object Persistence Model is just as straightforward as using it with the Document model: you simply identify the attributes to store as epoch seconds and the SDK performs the required conversions for you.

Consider the following class definition.


[DynamoDBTable("SessionData")]
public class User
{
    [DynamoDBHashKey]
    public string UserName { get; set; }

    [DynamoDBProperty(StoreAsEpoch = true)]
    public DateTime ExpirationTime { get; set; }
}

Once we’ve added the [DynamoDBProperty(StoreAsEpoch = true)] attribute, we can use DateTime objects with the class just like we normally would. However, this time we store epoch seconds, and the items we create are eligible for TTL automatic deletion. And just as with the Document Model, if you omit the StoreAsEpoch attribution, the objects you write will contain ISO-8601 dates and are not eligible for TTL deletion.

Here’s an example of creating the DynamoDBContext object, writing a User object, and reading it out again.


using (var context = new DynamoDBContext(client))
{
    // Writing TTL data
    DateTime expirationTime = DateTime.Now.AddDays(7);
    Console.WriteLine($"Storing expiration time = {expirationTime}");

    var user = new User
    {
        UserName = "user3",
        ExpirationTime = expirationTime
    };
    context.Save(user);

    // Reading TTL data
    user = context.Load("user3");
    expirationTime = user.ExpirationTime;
    Console.WriteLine($"Stored expiration time = {expirationTime}");
}

Conclusion

In this blog post we showed how you can toggle the new Time-to-Live feature for a table. We’ve also showed multiple ways to work with this data. The approach you choose is up to you and, hopefully with these examples, you’ll find it quite easy to schedule your data for automatic deletion. Happy coding!

Preview of the AWS Toolkit for Visual Studio 2017

by Norm Johanson | on | in .NET | Permalink | Comments |  Share

Today we released a preview of our AWS Toolkit for Visual Studio that includes support for the release candidate (RC) version of Visual Studio 2017. Because this Visual Studio release contains some significant changes for extension developers, we’re making this preview available in advance of the formal release. We highly encourage you to pass along feedback about any issues you find or whether you were successful using the preview by adding to the GitHub issue we describe below.

AWS Lambda .NET Core Support

Visual Studio 2017 also contains support for the new MSBuild project system for .NET Core projects. With this preview of the toolkit, we’ve updated the .NET Core Lambda support to use the new build system. For existing Lambda projects based on the Visual Studio 2015 project.json-based build system, Visual Studio 2017 offers to migrate them to the new build system when you open the projects.

Downloading and Installing the Preview

The AWS Toolkit for Visual Studio contains MSBuild target files for AWS CloudFormation projects. In previous releases of Visual Studio, you had to install these files and dependencies for an extension outside of the Visual Studio folder hierarchy. This required us to use a Windows Installer to install the toolkit. Starting with Visual Studio 2017, these MSBuild extensions exist within the Visual Studio folder hierarchy and can be installed from the VSIX extension package without an installer. As the installer technology we use doesn’t yet support Visual Studio 2017, we’ve decided to distribute the preview as a VSIX file only.

To install the preview

  1. Download the VSIX file from the preview link
  2. Double-click the VSIX file. This launches the VSIX Installer process, as shown.

After the installer finishes, the toolkit functions as it has in previous versions of Visual Studio.

Feedback

To track issues with installing and using the toolkit in Visual Studio 2017, we opened the following GitHub issue in our AWS SDK for .NET repository. For any issues or comments about our Visual Studio 2017 support, please add to the issue. Also, by letting us know if everything is working as you expected, you help us evaluate the readiness of our support for Visual Studio 2017.

Deploy an Existing ASP.NET Core Web API to AWS Lambda

by Norm Johanson | on | in .NET | Permalink | Comments |  Share

In the previous post, we talked about the new ASP.NET Core Web API blueprint for AWS Lambda, and the Amazon.Lambda.AspNetCoreServer NuGet package that made it possible to run the ASP.NET Core Web API through Lambda. But what if you already have an existing ASP.NET Core Web API that you want to try as a serverless application? You can do this by following these steps:

  • Add the Amazon.Lambda.AspNetCoreServer NuGet package.
  • Add a Lambda function and bootstrap the ASP.NET Core framework.
  • Add the Amazon.Lambda.Tools NuGet package to enable the toolkit’s deployment features.
  • Add a serverless.template file to define Amazon API Gateway.
  • Deploy the project.

Let’s take a deeper look at each step.

Setting Up the Lambda Function

The first step is to add the Amazon.Lambda.AspNetCoreServer NuGet package that bridges the communication between Amazon API Gateway and the ASP.NET Core framework.

After you add the package, add a new class named LambdaFunction and have it extend from Amazon.Lambda.AspNetCoreServer.APIGatewayProxyFunction. You have to implement the abstract method Init to bootstrap the ASP.NET Core framework.


public class LambdaFunction : Amazon.Lambda.AspNetCoreServer.APIGatewayProxyFunction
{
    protected override void Init(IWebHostBuilder builder)
    {
        builder
            .UseContentRoot(Directory.GetCurrentDirectory())
            .UseStartup()
            .UseApiGateway();
    }
}

Enable Tool Support in the AWS Toolkit for Visual Studio

In order for the AWS Toolkit for Visual Studio to recognize the project as a Lambda project, you have to add the Amazon.Lambda.Tools NuGet package. This package isn’t used as part of the runtime of the function and is added as a build tool.


{  
  "dependencies": {
    ...

    "Amazon.Lambda.AspNetCoreServer": "0.8.4-preview1",
    "Amazon.Lambda.Tools": {
      "type": "build",
      "version": "1.1.0-preview1"
    }
  },

  ...
}

To also enable the integration with the .NET Core CLI, list the Amazon.Lambda.Tools NuGet package in the tools section in the project.json file.


{
  ...

  "tools": {
    "Microsoft.AspNetCore.Server.IISIntegration.Tools": "1.0.0-preview2-final",
    "Amazon.Lambda.Tools": "1.1.0-preview1"
  },

  ...
}

Configuring Amazon API Gateway

At this point, you could right-click the project and deploy it to Lambda, but it wouldn’t be fronted by API Gateway exposing the function as an HTTP REST API. The easiest way to do that is to add a serverless.template file to the project and deploy the project as an AWS Serverless project.

Add a serverless.template file to the project by right-clicking the project and choosing Add, AWS Serverless Template.

add-serverless

The default serverless.template file contains one function definition configured to be exposed by API Gateway using proxy integration, so all requests will go to that function. This is exactly what you need for an ASP.NET Core Web API project. The only thing that needs to be updated is the handler field. The format for the handler field is <assembly-name>::<namespace>.LambdaFunction::FunctionHandlerAsync. The FunctionHandlerAsync method is inherited from the base class of our LambdaFunction class.


{
  "AWSTemplateFormatVersion" : "2010-09-09",
  "Transform" : "AWS::Serverless-2016-10-31",
  "Description" : "Starting template for an AWS Serverless Application.",
  "Parameters" : {
  },
  "Resources" : {
    "DefaultFunction" : {
      "Type" : "AWS::Serverless::Function",
      "Properties": {
        "Handler": "ExistingWebAPI::ExistingWebAPI.LambdaFunction::FunctionHandlerAsync",
        "Runtime": "dotnetcore1.0",
        "CodeUri": "",
        "Description": "Default function",
        "MemorySize": 256,
        "Timeout": 30,
        "Role": null,
        "Policies": [ "AWSLambdaFullAccess" ],
        "Events": {
          "PutResource": {
            "Type": "Api",
            "Properties": {
              "Path": "/{proxy+}",
              "Method": "ANY"
            }
          }
        }
      }
    }
  },
  "Outputs" : {
  }
}

Deploy

Now you can deploy the ASP.NET Core Web API to either AWS Elastic Beanstalk or Lambda. The deployment process works in the same way that we’ve shown in previous blog posts about AWS Serverless projects.

deploy-selector

And that’s all you have to do to deploy an existing ASP.NET Core Web API project to Lambda.

Visit our .NET Core Lambda GitHub repository to let us know what you think of running ASP.NET Core applications as an AWS Serverless functions and issues you might have. This will help us take the Amazon.Lambda.AspNetCoreServer NuGet package out of preview status.

Running Serverless ASP.NET Core Web APIs with Amazon Lambda

by Norm Johanson | on | in .NET | Permalink | Comments |  Share

One of the coolest things we demoed at our recent AWS re:Invent talk about .NET Core support for AWS Lambda was how to run an ASP.NET Core Web API with Lambda. We did this with the NuGet package Amazon.Lambda.AspNetCoreServer (which is currently in preview) and Amazon API Gateway. Today we’ve released a new AWS Serverless blueprint that you’ll see in Visual Studio or with our Yeoman generator that makes it easy to set up an ASP.NET Core Web API project as a Lambda project.

Blueprint Picker

How Does It Work?

Depending on your platform, a typically deployed ASP.NET Core application is fronted by either IIS or NGINX, which forwards requests to the ASP.NET Core web server named Kestrel. Kestrel marshals the request into the ASP.NET Core hosting framework.

Normal Flow

When running an ASP.NET Core application as an AWS Serverless application, IIS is replaced with API Gateway and Kestrel is replaced with a Lambda function contained in the Amazon.Lambda.AspNetCoreServer package which marshals the request into the ASP.NET Core hosting framework.

Serverless Flow

The Blueprint

The blueprint creates a project that’s very similar to the one you would get if you selected the .NET Core ASP.NET Core Web Application and chose the Web API template. The key difference is instead of having a Program.cs file that contains a Main function bootstrapping the ASP.NET Core framework, the blueprint has LambdaEntryPoint.cs that bootstraps the ASP.NET Core framework.


public class LambdaEntryPoint : Amazon.Lambda.AspNetCoreServer.APIGatewayProxyFunction
{
    protected override void Init(IWebHostBuilder builder)
    {
        builder
            .UseContentRoot(Directory.GetCurrentDirectory())
            .UseStartup()
            .UseApiGateway();
    }
}

The actual Lambda function comes from the base class. The function handler for the Lambda function is set in the AWS CloudFormation template named serverless.template, which will be in the format <assembly-name>::<namespace>.LambdaEntryPoint::FunctionHandlerAsync.

The blueprint also has LocalEntryPoint.cs that works in the same way as the original Program.cs file, enabling you to run and develop your application locally and then deploy it to Lambda.

The remainder of the project’s files are the usual ones you would find in an ASP.NET Core application. The blueprint contains two Web API controllers. The first is the example ValuesController, which is found in the starter ASP.NET Core Web API project. The other controller is S3ProxyController, which demonstrates how to use HTTP GET, PUT, and DELETE requests to a controller and uses the AWS SDK for .NET to make the calls to an Amazon S3 bucket. The name of the S3 bucket to use is obtained from the Configuration object, which means you can set the bucket in the appsettings.json file for local development.


{
  ...

  "AppS3Bucket": "ExampleBucketName"
}

The Configuration object is built by using environment variables.


public Startup(IHostingEnvironment env)
{
    var builder = new ConfigurationBuilder()
        .SetBasePath(env.ContentRootPath)
        .AddJsonFile("appsettings.json", optional: false, reloadOnChange: true)
        .AddJsonFile($"appsettings.{env.EnvironmentName}.json", optional: true);

    builder.AddEnvironmentVariables();
    Configuration = builder.Build();
}

When the application is deployed, serverless.template is used to create the bucket and then pass the bucket’s name to the Lambda function as an environment variable.


...

"Get" : {
  "Type" : "AWS::Serverless::Function",
  "Properties": {
    "Handler": "AspNetCoreWithLambda::AspNetCoreWithLambda.LambdaEntryPoint::FunctionHandlerAsync",
    "Runtime": "dotnetcore1.0",
    "CodeUri": "",
    "MemorySize": 256,
    "Timeout": 30,
    "Role": null,
    "Policies": [ "AWSLambdaFullAccess" ],
    "Environment" : {
      "Variables" : {
        "AppS3Bucket" : { "Fn::If" : ["CreateS3Bucket", {"Ref":"Bucket"}, { "Ref" : "BucketName" } ] }
      }
    },
    "Events": {
      "PutResource": {
        "Type": "Api",
        "Properties": {
          "Path": "/{proxy+}",
          "Method": "ANY"
        }
      }
    }
  }
},

...

Logging

ASP.NET Core introduced a new logging framework. To help integrate with the logging framework, we’ve also released the NuGet package Amazon.Lambda.Logging.AspNetCore. This logging provider allows any code that uses the ILogger interface to record log messages to the associated Amazon CloudWatch log group for the Lambda function. When used outside of a Lambda function, the log messages are written to the console.

The blueprint enables the provider in Startup.cs, where other services are configured.


public void Configure(IApplicationBuilder app, IHostingEnvironment env, ILoggerFactory loggerFactory)
{
    loggerFactory.AddLambdaLogger(Configuration.GetLambdaLoggerOptions());
    app.UseMvc();
}

This following snippet shows the call GetLambdaLoggerOptions from the Configuration object, which grabs the configuration of what messages to write to CloudWatch Logs. The appsettings.json file in the blueprint configures logging so that messages coming from classes under the Microsoft namespace are written if they’re informational level and above. For all other log messages, write debug level messages and above.


{
  "Lambda.Logging": {
    "LogLevel": {
      "Default": "Debug",
      "Microsoft": "Information"
    }
  },

  ...
}

For more information about this package, see the GitHub repository.

Deployment

Deploying the ASP.NET Core Web API works exactly as we showed you in the previous post about the AWS Serverless projects.

Deploy from Solution Explorer

Once deployed, a single Lambda function and an API Gateway REST API are configured to send all requests to the Lambda function. Then the Lambda function uses the ASP.NET Core framework to route to the correct Web API controller. You can test the deployment by accessing the two controllers using the AWS Serverless URL found in the CloudFormation stack view.

  • <aws-serverless-url>/api/values – Example controller
  • <aws-serverless-url>/api/s3proxy – S3 Proxy controller.

Feedback

We’re very excited about running ASP.NET Core applications on AWS Lambda. As you can imagine, the option of running the ASP.NET Core framework on top of Lambda opens lots of possibilities. The Amazon.Lambda.AspNetCoreServer package is in preview while we explore those possibilities. I highly encourage .NET developers to check out this blueprint and the Amazon.Lambda.AspNetCoreServer package and let us know on our GitHub repository or our new Gitter channel what you think and how we can continue to improve the library.

Amazon CloudWatch Logs and .NET Logging Frameworks

by Norm Johanson | on | in .NET | Permalink | Comments |  Share

You can use Amazon CloudWatch Logs to monitor, store, and access your application’s logs. To get log data into CloudWatch Logs, you can use an AWS SDK or install the CloudWatch Log agent to monitor certain log folders. Today, we’ve made it even easier to use CloudWatch Logs with .NET applications by integrating CloudWatch Logs with several popular .NET logging frameworks.

The supported .NET logging frameworks are NLog, Log4net, and the new built-in ASP.NET Core logging framework. For each framework, all you need to do is add the appropriate NuGet package, add CloudWatch Logs as an output source, and then use your logging library as you normally would.

For example to use CloudWatch Logs with a .NET application using NLog, add the AWS.Logger.NLog NuGet package, and then add the AWS target into your NLog.config file. Here is an example of an NLog.config file that enables both CloudWatch Logs and the console as output for the log messages.


<?xml version="1.0" encoding="utf-8" ?>
<nlog xmlns="http://www.nlog-project.org/schemas/NLog.xsd"
      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      throwExceptions="true">
  <targets>
    <target name="aws" type="AWSTarget" logGroup="NLog.ConfigExample" region="us-east-1"/>
    <target name="logfile" xsi:type="Console" layout="${callsite} ${message}" />
  </targets>
  <rules>
    <logger name="*" minlevel="Info" writeTo="logfile,aws" />
  </rules>
</nlog>

After performing these steps, when you run your application the log messages written with NLog will be sent to CloudWatch Logs. Then you can view your application’s log messages in near real time from the CloudWatch Logs console. You can also set up metrics and alarms from the CloudWatch Logs console, based on your application’s log messages.

These logging plugins are all built on top of the AWS SDK for .NET, and use the same behavior used by the SDK to find AWS credentials. The credentials used by the logging plugins must have the following permissions to access CloudWatch Logs.


{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents",
        "logs:DescribeLogGroups"
      ],
      "Resource": [
        "arn:aws:logs:*:*:*"
      ]
    }
  ]
}

The AWS .NET logging plugins are a new open source project on GitHub. All of the plugins are there, including samples and instructions on how to configure CloudWatch Logs for each of the supported .NET logging frameworks.

For any comments or issues for the new libraries, open an issue in the GitHub repository.

AWS Serverless Applications in Visual Studio

by Norm Johanson | on | in .NET | Permalink | Comments |  Share

In the last post, I talked about the AWS Lambda Project template. The other new project template we added to Visual Studio is the AWS Serverless Application. This is our AWS Toolkit for Visual Studio implementation of the new AWS Serverless Application Model. Using this project type, you can develop a collection of AWS Lambda functions and deploy them with any necessary AWS resources as a whole application, using AWS CloudFormation to orchestrate the deployment.

To demonstrate this, let’s create a new AWS Serverless Application and name it Blogger.

serverless-new-project

As in the AWS Lambda Project, we can then choose a blueprint to help get started. For this post, we’re going to use the Blog API using DynamoDB blueprint.

serverless-blueprints

The Project Files

Now let’s take a look at the files in our serverless application.

Blog.cs

This is a simple class used to represent the blog items that are stored in Amazon DynamoDB.

Functions.cs

This file defines all the C# functions we want to expose as Lambda functions. There are four functions defined to manage a blog platform:

  • GetBlogsAsync – gets a list of all the blogs.
  • GetBlogAsync – gets a single blog identified by either the query parameter Id or the ID added to the URL resource path.
  • AddBlogAsync – adds a blog to DynamoDB table.
  • RemoveBlogAsync – removes a blog from the DynamoDB table.

Each of these functions accepts an APIGatewayProxyRequest object and returns an APIGatewayProxyResponse. That is because these Lambda functions will be exposed as an HTTP API using Amazon API Gateway. The APIGatewayProxyRequest contains all the information representing the HTTP request. In the GetBlogAsync operation, you can see how we can find the ID of the blog from the resource path or query string.


public async Task GetBlogAsync(APIGatewayProxyRequest request, ILambdaContext context)
{
    string blogId = null;
    if (request.PathParameters != null && request.PathParameters.ContainsKey(ID_QUERY_STRING_NAME))
        blogId = request.PathParameters[ID_QUERY_STRING_NAME];
    else if (request.QueryStringParameters != null && request.QueryStringParameters.ContainsKey(ID_QUERY_STRING_NAME))
        blogId = request?.QueryStringParameters[ID_QUERY_STRING_NAME];

    ...
}

In the default constructor for this class, we can also see how the name of the DynamoDB table storing our blogs is passed in as an environment variable. This environment variable is set when Lambda deploys our function.


public Functions()
{
    // Check to see if a table name was passed in through environment variables and, if so,
    // add the table mapping
    var tableName = System.Environment.GetEnvironmentVariable(TABLENAME_ENVIRONMENT_VARIABLE_LOOKUP);
    if(!string.IsNullOrEmpty(tableName))
    {
        AWSConfigsDynamoDB.Context.TypeMappings[typeof(Blog)] = new Amazon.Util.TypeMapping(typeof(Blog), tableName);
    }

    var config = new DynamoDBContextConfig { Conversion = DynamoDBEntryConversion.V2 };
    this.DDBContext = new DynamoDBContext(new AmazonDynamoDBClient(), config);
}

serverless.template

This file is the AWS CloudFormation template used to deploy the four functions. The parameters for the template enable us to set the name of the DynamoDB table, and choose whether we want CloudFormation to create the table or to assume the table is already created.

The template defines four resources of type AWS::Serverless::Function. This is a special meta resource defined as part of the AWS Serverless Application Model specification. The specification is a transform that is applied to the template as part of the CloudFormation deployment. The transform expands the meta resource type into the more concrete resources, like AWS::Lambda::Function and AWS::IAM::Role. The transform is declared at the top of the template file, as follows.


{
  "AWSTemplateFormatVersion" : "2010-09-09",
  "Transform" : "AWS::Serverless-2016-10-31",

  ...
  
 }

Now let’s take a look at the GetBlogs declaration in the template, which is very similar to the other function declarations.


"GetBlogs" : {
  "Type" : "AWS::Serverless::Function",
  "Properties": {
    "Handler": "Blogger::Blogger.Functions::GetBlogsAsync",
    "Runtime": "dotnetcore1.0",
    "CodeUri": "",
    "Description": "Function to get a list of blogs",
    "MemorySize": 256,
    "Timeout": 30,
    "Role": null,
    "Policies": [ "AWSLambdaFullAccess" ],
    "Environment" : {
      "Variables" : {
        "BlogTable" : { "Fn::If" : ["CreateBlogTable", {"Ref":"BlogTable"}, { "Ref" : "BlogTableName" } ] }
      }
    },
    "Events": {
      "PutResource": {
        "Type": "Api",
        "Properties": {
          "Path": "/",
          "Method": "GET"
        }
      }
    }
  }
}

You can see a lot of the fields here are very similar to what we saw when we did a Lambda project deployment. In the Environment property, notice how the name of the DynamoDB table is being passed in as an environment variable. The CodeUri property tells CloudFormation where in Amazon S3 your application bundle is stored. Leave this property blank because the toolkit will fill it in during deployment, after it uploads the application bundle to S3 (it won’t change the template file on disk when it does so).

The Events section is where we can define the HTTP bindings for our Lambda function. This takes care of all the API Gateway setup we need to do for our function. You can also set up other types of event sources in this section.

template-addeventsource

One of the great benefits of using CloudFormation to manage the deployment is we can also add and configure any other AWS resources necessary for our application in the template, and let CloudFormation take care of creating and deleting the resources.

template-addresources

Deploying

We deploy our serverless application in the same way we deployed the Lamba project previously: right-click the project and choose Publish to AWS Lambda.

serverless-publishmenu

This launches the deployment wizard, but this time it’s quite a bit simpler. Because all the Lambda configuration was done in the serverless.template file, all we need to supply are the following:

  • The name of our CloudFormation stack, which will be the container for all the resources declared in the template.
  • The S3 bucket to upload our application bundle to.

These should exist in the same AWS Region.

serverless-first-page

Because the serverless template has parameters, an additional page is displayed in the wizard where we specify the values for the parameters. We can leave the BlogTableName property blank and let CloudFormation generate a unique name for the table. We do need to set ShouldCreateTable to true so that CloudFormation will create the table. To use an existing table, enter the table name and set the ShouldCreateTable parameter to false. We can leave the other fields at their default values and choose Publish.

serverless-second-page

Once the publish step is complete, the CloudFormation stack view is displayed in AWS Explorer. This view shows the progress of the creation of all the resources declared in our serverless template.

serverless-stack-view

When the stack creation is complete, the root URL for the API Gateway is displayed on the page. If we click that link, it returns an empty JSON array because we haven’t added any blogs to our table. To get blogs in our table, we need to make an HTTP PUT method to this URL, passing in a JSON document that represents the blog. We can do that in code or in any number of tools. I’ll use the Postman tool, which is a Chrome browser extension, but you can use any tool you like. In this tool, I’ll set the URL and change the method to be PUT. In the Body tab, I’ll put in some sample content. When we make the HTTP call, you can see that we get back the blog ID.

procman-addpost

Now if we go back to the browser with the link to our AWS Serverless URL, you can see we are getting back the blog we just posted.

get-post

Conclusion

Using the AWS Serverless Application template, you can manage a collection of Lambda functions and the application’s other AWS resources. Also, with the new AWS Serverless Application Model specification, we can use a simplified syntax to declare our serverless application in the CloudFormation template. If you have any questions or suggestions for blueprints, feel free to reach out to us on our .NET Lambda GitHub repository.

Using the AWS Lambda Project in Visual Studio

by Norm Johanson | on | in .NET | Permalink | Comments |  Share

Last week we launched C# and .NET Core support for AWS Lambda. That release provided updated tooling for Visual Studio to help you get started writing your AWS Lambda functions and deploy them right from Visual Studio. In this post, we describe how to create, deploy, and test an AWS Lambda project.

Creating a Lambda Project

To get started writing Lambda functions in Visual Studio, you first need to create an AWS Lambda project. You can do this by using the Visual Studio 2015 New Project wizard. Under the Visual C# templates, there is a new category called AWS Lambda. You can choose between two types of project, AWS Lambda Project and AWS Serverless Application, and you also have the option to add a test project. In this post, we’ll focus on the AWS Lambda project and save AWS Serverless Application for a separate post. To begin, choose AWS Lambda Project with Tests (.NET Core), name the project ImageRekognition, and then choose OK.

lambda-new-project

On the next page, you choose the blueprint you want to get started with. Blueprints provide starting code to help you write your Lambda functions. For this example, choose the Detect Image Labels blueprint. This blueprint provides code for listening to Amazon S3 events and uses the newly released Amazon Rekognition service to detect labels and then add them to the S3 object as tags.

lambda-blueprints

When the project is complete, you will have a solution with two projects, as shown: the source project that contains your Lambda function code that will be deployed to AWS Lambda, and a test project using xUnit for testing your function locally.

lambda-solution-explorer

You might notice when you first create your projects that Visual Studio does not find all the NuGet references. This happens because these blueprints require dependencies that must be retrieved from NuGet. When new projects are created, Visual Studio only pulls in local references and not remote references from NuGet. You can fix this easily by right-clicking your references and choosing Restore Packages.

Lambda Function Source

Now let’s open the Function.cs file and look at the code that came with the blueprint. The first bit of code is the assembly attribute that is added to the top of the file.

// Assembly attribute to enable the Lambda function's JSON input to be converted into a .NET class.
[assembly: LambdaSerializerAttribute(typeof(Amazon.Lambda.Serialization.Json.JsonSerializer))]

By default, Lambda accepts only input parameters and return types of type System.IO.Stream. To use typed classes for input parameters and return types, we have to register a serializer. This assembly attribute is registering the Lambda JSON serializer, which uses Newtonsoft.Json to convert the streams to typed classes. The serializer can be set at the assembly or method level.

The class has two constructors. The first is a default constructor that is used when Lambda invokes your function. This constructor creates the S3 and Rekognition service clients, and will get the AWS credentials for these clients from the IAM role we’ll assign to the function when we deploy it. The AWS Region for the clients will be set to the region your Lambda function is running in. In this blueprint, we only want to add tags to our S3 object if the Rekognition service has a minimum level of confidence about the label. This constructor will check the environment variable MinConfidence to determine the acceptable confidence level. We can set this environment variable when we deploy the Lambda function.

public Function()
{
    this.S3Client = new AmazonS3Client();
    this.RekognitionClient = new AmazonRekognitionClient();

    var environmentMinConfidence = System.Environment.GetEnvironmentVariable(MIN_CONFIDENCE_ENVIRONMENT_VARIABLE_NAME);
    if(!string.IsNullOrWhiteSpace(environmentMinConfidence))
    {
        float value;
        if(float.TryParse(environmentMinConfidence, out value))
        {
            this.MinConfidence = value;
            Console.WriteLine($"Setting minimum confidence to {this.MinConfidence}");
        }
        else
        {
            Console.WriteLine($"Failed to parse value {environmentMinConfidence} for minimum confidence. Reverting back to default of {this.MinConfidence}");
        }
    }
    else
    {
        Console.WriteLine($"Using default minimum confidence of {this.MinConfidence}");
    }
}

We can use the second constructor for testing. Our test project configures its own S3 and Rekognition clients and passes them in.

public Function(IAmazonS3 s3Client, IAmazonRekognition rekognitionClient, float minConfidence)
{
    this.S3Client = s3Client;
    this.RekognitionClient = rekognitionClient;
    this.MinConfidence = minConfidence;
}

FunctionHandler is the method Lambda will call after it constructs the instance. Notice that the input parameter is of type S3Event and not a Stream. We can do this because of our registered serializer. The S3Event contains all the information about the event triggered in S3. The function loops through all the S3 objects that were part of the event and tells Rekognition to detect labels. After the labels are detected, they are added as tags to the S3 object.

public async Task FunctionHandler(S3Event input, ILambdaContext context)
{
    foreach(var record in input.Records)
    {
        if(!SupportedImageTypes.Contains(Path.GetExtension(record.S3.Object.Key)))
        {
            Console.WriteLine($"Object {record.S3.Bucket.Name}:{record.S3.Object.Key} is not a supported image type");
            continue;
        }

        Console.WriteLine($"Looking for labels in image {record.S3.Bucket.Name}:{record.S3.Object.Key}");
        var detectResponses = await this.RekognitionClient.DetectLabelsAsync(new DetectLabelsRequest
        {
            MinConfidence = MinConfidence,
            Image = new Image
            {
                S3Object = new Amazon.Rekognition.Model.S3Object
                {
                    Bucket = record.S3.Bucket.Name,
                    Name = record.S3.Object.Key
                }
            }
        });

        var tags = new List();
        foreach(var label in detectResponses.Labels)
        {
            if(tags.Count < 10)
            {
                Console.WriteLine($"\tFound Label {label.Name} with confidence {label.Confidence}");
                tags.Add(new Tag { Key = label.Name, Value = label.Confidence.ToString() });
            }
            else
            {
                Console.WriteLine($"\tSkipped label {label.Name} with confidence {label.Confidence} because maximum number of tags reached");
            }
        }

        await this.S3Client.PutObjectTaggingAsync(new PutObjectTaggingRequest
        {
            BucketName = record.S3.Bucket.Name,
            Key = record.S3.Object.Key,
            Tagging = new Tagging
            {
                TagSet = tags
            }
        });
    }
    return;
}

Notice that the code contains calls to Console.WriteLine(). When the function is being run in AWS Lambda, all calls to Console.WriteLine() will redirect to Amazon CloudWatch Logs.

Default Settings File

Another file that was created with the blueprint is aws-lambda-tools-defaults.json. This file contains default values that the blueprint has set to help prepopulate some of the fields in the deployment wizard. It is also helpful in setting command line options with our integration with the new .NET Core CLI. We’ll dive deeper into the CLI integration in a later post, but to get started using it, navigate to the function’s project directory and type dotnet lambda help.

{
  "Information" : [
    "This file provides default values for the deployment wizard inside Visual Studio and the AWS Lambda commands added to the .NET Core CLI.",
    "To learn more about the Lambda commands with the .NET Core CLI execute the following command at the command line in the project root directory.",

    "dotnet lambda help",

    "All the command line options for the Lambda command can be specified in this file."
  ],

  "profile":"",
  "region" : "",
  "configuration" : "Release",
  "framework" : "netcoreapp1.0",
  "function-runtime":"dotnetcore1.0",
  "function-memory-size" : 256,
  "function-timeout" : 30,
  "function-handler" : "ImageRekognition::ImageRekognition.Function::FunctionHandler"
}

An important field to understand is the function-handler. This indicates to Lambda the method to call in our code in response to our function being invoked. The format of this field is <assembly-name>::<full-type-name>::<method-name>. Be sure to include the namespace with the type name.

Deploying the Function

To get started deploying the function, right-click the Lambda project and then choose

Publish to AWS Lambda. This starts the deployment wizard. Notice that many of the fields are already set. These values came from the aws-lambda-tools-defaults.json file described earlier. We do need to enter a function name. For this example, let’s name it ImageRekognition, and then choose Next.

lambda-deployment-wizard-page1

On the next page, we need to select an IAM role that gives permission for our code to access S3 and Rekognition. To keep this post short, let’s select the Power User managed policy; the tools create a role for us based on this policy. Note that the Power User managed policy was added to use to create a role in version 1.11.1.0 of the toolkit.

Finally, we set the environment variable MinConfidence to 60, and then choose Publish.

lambda-deployment-wizard-page2

This launches the deployment process, which builds and packages the Lambda project and then creates the Lambda function. Once publishing is complete, the Function view in the AWS Explorer window is displayed. From here, we can invoke a test function, view CloudWatch Logs for the function, and configure event sources.

lambda-function-view

With our function deployed, we need to configure S3 to send its events to our new function. We do this by going to the event source tab and choosing Add. Then, we choose Amazon S3 and choose the bucket we want to connect to our Lambda function. The bucket must be in the same region as the region where the Lambda function was deployed.

Testing the Function

Now that the function is deployed and an S3 bucket is configured as an event source for it, open the S3 bucket browser from the AWS Explorer for the bucket we selected and upload some images.

When the upload is complete, we can confirm that our function ran by looking at the logs from our function view. Or, we can right-click the images in the bucket browser and select Properties. In the Properties dialog box on the Tags tab, we can view the tags that were applied to our object.

lambda-object-properties

Conclusion

We hope this post gives you a good understanding of how our tooling inside Visual Studio works for developing and creating Lambda functions. We’ll be adding more blueprints over time to help you get started using other AWS services with Lambda. The blueprints are hosted in our new Lambda .NET GitHub repository. If you have any suggestions for new blueprints, open an issue and let us know.