AWS Developer Tools Blog

ASP.NET Core and AWS CodeStar Deep Dive

The AWS CodeStar team recently announced the addition of two ASP.NET Core project templates. As you might know, AWS CodeStar creates a code-integration and code-deployment(CI/CD) pipeline on behalf of developers, so they can spend their valuable time building applications instead of building infrastructure. With the new ASP.NET Core project templates, .NET developers can build and deploy their AWS applications on day one. Tara Walker’s excellent blog post covers how to create ASP.NET Core applications on AWS CodeStar. In this blog post, we take a deeper look into what goes on behind the scenes as we learn how to add tests to your ASP.NET Core project for AWS CodeStar.

Adding a unit test project

Our goal is to add a simple test case that exercises HelloController’s functionality. I’m assuming that you have a brand new ASP.Net Core web service project. If you don’t, you can follow Tara’s blog post (mentioned above) to create one. Be sure to choose the ASP.NET Core Web service template. After you create the ASP.NET Core for AWS CodeStar project, clone the project repository through Team Explorer, and load the AspNetCoreWebService solution, you should be able to follow along with the rest of the blog post. If you need some guidance setting up your repo through Team Explorer, check out Steve Robert’s Visual Studio and AWS CodeCommit integration announcement in May.

First, add a new xUnit project named AspNetCoreWebServiceTest to the AspNetCoreWebService solution. Our new test project will reference the HelloController class and JsonResult, so we should add AspNetCoreWebService as a project reference and Microsoft.AspNetCore.Mvc as a NuGet reference. Once you add them to the test project, you should see the following addition in AspNetCoreWebServiceTest.csproj.

<ItemGroup>
    <PackageReference Include="Microsoft.AspNetCore.Mvc" Version="1.1.3" />
    ...
</ItemGroup>
...
<ItemGroup>
    <ProjectReference Include="..\AspNetCoreWebService\AspNetCoreWebService.csproj" />
</ItemGroup>

This should allow you to make direct references to the HelloController class and unpack JsonResult. Let’s add a simple test case, as follows.

using System;
using Xunit;
using Microsoft.AspNetCore.Mvc;
using AspNetCoreWebService.Controllers;

namespace AspNetCoreWebServiceTest
{
    public class HelloControllerTest
    {
        [Fact]
        public void SimpleTest()
        {
            HelloController controller = new HelloController();
            var response = controller.Get("AWS").Value as Response;
            Assert.Equal(response.output, "Hello AWS!");
        }
    }
}

Notice that we have renamed the file name, namespace, class name, and the method name. Run the test and verify that it passes. You should see the following in Solution Explorer.

Now that we have a working test project, we should update our pipeline to build and run the test before deploying the application.

Updating the AWS CodeBuild job

Let’s first look at how the project is built. When you or your team member pushes a change to the repo, your pipeline automatically begins the build process against the latest change. During this step, AWS CodeBuild uses the buildspec.yml file in the root of the repository to drive the build process.

version: 0.2
phases:
  pre_build:
    commands:
      - echo Restore started on `date`
      - dotnet restore AspNetCoreWebService/AspNetCoreWebService.csproj
  build:
    commands:
      - echo Build started on `date`
      - dotnet publish -c release -o ./build_output AspNetCoreWebService/AspNetCoreWebService.csproj
artifacts:
  files:
    - AspNetCoreWebService/build_output/**/*
    - scripts/**/*
    - appspec.yml

The AWS CodeBuild job uses the .NET Core image for AWS CodeBuild, which contains the .NET Core SDK and CLI you will invoke in buildspec.yml.  Since this project consists of one web service, a single buildspec.yml file should be sufficient. As your project grows and the complexity of the build process increases, you may want to drive the build process externally via a shell script or an MSBuild .proj file and simply invoke the script/build file in buildspec.yml.

I would like to bring your attention to the dotnet publish command. This publishing step is crucial here, because it packages all dependencies together so that they are immediately available on the host machine. As defined in the artifacts section of the buildspec.yml file shown above, the list of files will be stored in an Amazon S3 bucket for AWS CodeDeploy to use to deploy your application onto the host.  scripts/**/* contains all scripts that appsec.yml depends on. If you’re not familiar with appsec.yml or want to know more about it, we’ll go over it in the next section.

In the previous section, we added a test project to our AWS CodeCommit repository. Now we should update buildspec.yml to build our new test project. We could simply run dotnet vstest as part of the build stage. However, in this exercise, let’s follow best practices by building separate stages for build and test. Let’s modify the buildspec.yml to build the test binaries and publish the bits into the AspNetCoreWebServiceTest/test_output directory.

pre_build:
    commands:
        ...
        - dotnet restore AspNetCoreWebServiceTest/AspNetCoreWebServiceTest.csproj
post_build:
    commands:
        ...
        - dotnet publish -c release -o ./test_output AspNetCoreWebServiceTest/AspNetCoreWebServiceTest.csproj  
artifacts:
    files:
        ...
        - AspNetCoreWebServiceTest/test_output/**/*

Notice that we added AspNetCoreWebServiceTest/test_output/**/* as an artifact. In effect, this directs the AWS CodeBuild service to upload the published test binaries to Amazon S3, so that we can reference them in the test job we will create next.

Updating AWS CodePipeline

In the previous sections, we added a new test project and modified buildspec.yml to build and save the binaries we need to run the tests. Now we’ll go over how to add a test stage in our pipeline. Let’s begin by adding a Test stage and a UnitTest action to the pipeline in the console.

Follow the rest of the UI and fill in these parameters:

  • Action category: Test
  • Action name: UnitTest
  • Test provider: AWS CodeBuild
  • Select Create a new build project
  • Project name: <your project name>-test
  • Operating system: Ubuntu
  • Runtime: .NET Core
  • Version: aws/codebuild/dot-net:core-1
  • For Build specification, select Insert build Commands
  • Build command: dotnet vstest AspNetCoreWebServiceTest/test_output/AspNetCoreWebServiceTest.dll
  • For Role name, select CodeStarWorker-<your project name>-CodeBuild from the list
  • For Input artifacts #1, select <your project name>-BuildArtifact from the list

The key piece of information here is the build command you provide. Our test job will run dotnet vstest against the test .dll built in the previous stage. Your pipeline should now look like this.

We’re almost done! If you run this pipeline by pressing Release change, the pipeline will fail on the Test stage with the message Error Code: AccessDeniedException. This is because the AWS CodeStar service doesn’t have permission to run our new Test stage. Let’s figure out how to grant appropriate access to our AWS CodeStar project.

Updating the role policy

Your AWS CodeStar project created policies for minimum permission for various services and workers to sync, build, and deploy your application. Because we added a new AWS CodeBuild job, we need to grant access to our new resource in CodeStarWorkerCodePipelinePolicy. Let’s navigate to the IAM console to make this change. On the Roles tab, search using the “codebuild” keyword. The role should be in the format CodeStarWorker-<project name>-CodePipeline. Then, edit the policy attached to the role. This is shown below.

The change we want to make is to add our new codebuild resource arn:aws:codebuild:us-east-1:532345249509:project/<your project name>-test that is associated with AWS CodeBuild actions in the policy.

{
    "Action": [
        "codebuild:StartBuild",
        "codebuild:BatchGetBuilds",
        "codebuild:StopBuild"
    ],
    "Resource": [
        "arn:aws:codebuild:us-east-1:532345249509:project/<your project name>"
        "arn:aws:codebuild:us-east-1:532345249509:project/<your project name>-test"
    ],
    "Effect": "Allow"
}

That’s it. Your AWS CodeStar project should now have appropriate permission to build the new job. Give it a try by pressing Release change.

ASP.NET Core application deployment

So far we’ve seen how AWS CodeStar builds and tests your project. In this section, we look closer at the deployment process. As part of the AWS CodeStar project creation, the AWS CodeStar service creates an Amazon EC2 instance to host your application. It also installs code-deploy-agent, which runs the deployment process on that instance following the instructions in appspec.yml. Let’s take a look at appspec.yml.

version: 0.0
os: linux
files:
  - source: AspNetCoreWebService/build_output
    destination: /home/ubuntu/aspnetcoreservice
  - source: scripts/virtualhost.conf
    destination: /home/ubuntu/aspnetcoreservice 
hooks:
  ApplicationStop:
    - location: scripts/stop_service
      timeout: 300
      runas: root

  BeforeInstall:
    - location: scripts/remove_application
      timeout: 300
      runas: root

  AfterInstall:
    - location: scripts/install_dotnetcore
      timeout: 500
      runas: root

    - location: scripts/install_httpd
      timeout: 300
      runas: root

  ApplicationStart:
    - location: scripts/start_service
      timeout: 300
      runas: root

Each script is run at various stages of the deployment process:

  • install_dotnetcore – Installs dotnet core if it isn’t already installed, and updates the package cache on the first run. This is Microsoft’s recommended way of installing .NET Core on Ubuntu.
  • install_httpd – Installs HTTPD daemon and mods, and overwrites the HTTPD configuration file to enable reverse-proxy.
  • start_service – Restarts the HTTPD service and restarts the existing ASP.NET application/service process.
  • scripts/stop_service  – Stops the HTTPD service and stops the ASP.NET application/service if it is already running.
  • remove_application – Removes the deployed application from the instance.

The code-deploy-agent on the instance runs these hooks during the application deployment to install and start the service. You can monitor the event activities on the AWS CodeDeploy console and we can grab a detailed log from the EC2 instance. After opening an SSH connection to the instance, navigate to /var/log/aws/codedeploy-agent to find the deployment logs.

Conclusion

In this blog post, you learned how your ASP.NET Core project for AWS CodeStar is built and deployed through the example of adding a test stage to your application’s pipeline. I hope this post helped you understand how various components and AWS services interact to provide you with a complete CI/CD system under AWS CodeStar. To learn more, visit the AWS CodeStar user guide. If you run into issues that are specific to AWS CodeStar, see the AWS CodeStar troubleshooting guide.