Category: AWS SDK for Go*


Mocking Out the AWS SDK for Go for Unit Testing

In our previous post, we showed how you could use the request handler stack in the AWS SDK for Go to extend or modify how requests are sent and received. Now, we’d like to expand the idea of extending the SDK and discuss how you can unit test code that uses the SDK. The SDK’s service clients are a common component to short circuit with custom unit test functionality.

You can easily mock out the SDK service clients by taking advantage of Go’s interfaces. By using the methods of an interface, your code can use that interface instead of using the concrete service client directly. This enables you to mock out the implementation of the service client for your unit tests. You can define these interfaces yourself or use the interfaces that the SDK already defines for each service client’s API. The service client’s API interfaces are very simple to use, and give you the flexibility to test your code. This pattern of using the API interfaces for mocking out is simple to use for new or existing code.

You can find the API interface package nested under each service client package, named “iface“. For example, the Amazon SQS API interface package is “sqsiface“, with the import path of github.com/aws/aws-sdk-go/service/sqs/sqsiface.

The following example shows one pattern how your code can use the SDK’s SQS API interface instead of the concrete client. We’ll take a look at how you could mock out the SDK’s SQS client next.

func main() {
	sess := session.Must(session.NewSession())

	q := Queue{
		Client: sqs.New(sess),
		URL:    os.Args[1],
	}

	msgs, err := q.GetMessages(20)
	// ...
}

type Queue struct {
	Client sqsiface.SQSAPI
	URL    string
}

func (q *Queue) GetMessages(waitTimeout int64) ([]Message, error) {
	params := sqs.ReceiveMessageInput{
		QueueUrl: aws.String(q.URL),
	}
	if waitTimeout > 0 {
		params.WaitTimeSeconds = aws.Int64(waitTimeout)
	}
	resp, err := q.Client.ReceiveMessage(&params)
	// ...
}

Because the previous code uses the SQS API interface, our tests can mock out the client with the responses we want so that we can verify GetMessages returns the parsed messages correctly.

type mockedReceiveMsgs struct {
	sqsiface.SQSAPI
	Resp sqs.ReceiveMessageOutput
}

func (m mockedReceiveMsgs) ReceiveMessage(in *sqs.ReceiveMessageInput) (*sqs.ReceiveMessageOutput, error) {
	// Only need to return mocked response output
	return &m.Resp, nil
}

func TestQueueGetMessage(t *testing.T) {
	cases := []struct {
		Resp     sqs.ReceiveMessageOutput
		Expected []Message
	}{
		{
			Resp: sqs.ReceiveMessageOutput{
				Messages: []*sqs.Message{
					{Body: aws.String(`{"from":"user_1","to":"room_1","msg":"Hello!"}`)},
					{Body: aws.String(`{"from":"user_2","to":"room_1","msg":"Hi user_1 :)"}`)},
				},
			},
			Expected: []Message{
				{From: "user_1", To: "room_1", Msg: "Hello!"},
				{From: "user_2", To: "room_1", Msg: "Hi user_1 :)"},
			},
		},
	}

	for i, c := range cases {
		q := Queue{
			Client: mockedReceiveMsgs{Resp: c.Resp},
			URL:    fmt.Sprintf("mockURL_%d", i),
		}
		msgs, err := q.GetMessages(20)
		if err != nil {
			t.Fatalf("%d, unexpected error", err)
		}
		if a, e := len(msgs), len(c.Expected); a != e {
			t.Fatalf("%d, expected %d messages, got %d", i, e, a)
		}
		for j, msg := range msgs {
			if a, e := msg, c.Expected[j]; a != e {
				t.Errorf("%d, expected %v message, got %v", i, e, a)
			}
		}
	}
}

Using this pattern will help ensure your code that uses the SDK is easy to test and maintain. You can find the full code example used in this post on GitHub at https://github.com/aws/aws-sdk-go/tree/master/example/service/sqs/mockingClientsForTests.

Using Custom Request Handlers

Requests are the core of the Go SDK. They handle the network request to a specific service. When you call a service method in the AWS SDK for Go, the SDK creates a request object that contains the input parameters and request lifecycle handlers. The logic involved in each step of a request’s lifecycle, such as Build, Validate, or Send, is handled by its corresponding list of handlers. For example, when the request’s Send method is called, it will iterate through and call every function in the request’s Send handler list. For the full list of handlers, see this API reference page. These handlers  shape and mold  what your request does on the way to AWS and what the response does on the way out.

Customizability is an important aspect of request handlers. By customizing its handlers, you can modify a request’s lifecycle for various use cases. You can do this for an individual request object returned by DoFooRequest client methods or, more interestingly, for an entire service client object, which will affect every request processed by the client.

An example use case would be gathering additional information on retried requests. In this case, a simple addition to the Retry handler would provide information required for further debugging. Logging is another use case. For example, a logger function can be added to log every request, or a benchmark function can be added to profile requests. Let’s see how you can add a custom logger to different handler lists.


func customLogger(req *request.Request) {
	 log.Printf(Op: %s\nParams: %v\nTime:      %s\nRetryCount: %d\n\n",
	 req.Operation.Name, req.Params, req.Time, req.RetryCount
	)
}


svc.Handlers.Send.PushBack(customLogger)
svc.Handlers.Unmarshal.PushBack(func(req *request.Request) {
log.Println(“Unmarshal handler”)
customLogger(req)
})

We added a customLogger request handler to the Sign and Send handlers and a closure function in the Unmarshal handler. The only requirement for this function is that it takes a request.Request pointer. The customLogger, which was added to the Send handler, will log the operation’s name, the parameters passed, time executed, and the retry count. The anonymous function added to Unmarshal will log when the Unmarshal handler was executed and then call the customLogger.

These functions are being added to the end of the handler list. During the request’s Send step, it will iterate through the list and execute the functions. Having custom loggers can be useful when developing and debugging applications because they provide more visibility into each API operation’s request/response layers.

In addition to logging, sometimes benchmarking or profiling requests are important. This will give you an idea of what to expect when hitting your production environments.


dt := time.Time{}
svc.Handlers.Send.PushFront(func(req *request.Request) {
	 dt = time.Now()
})
svc.Handlers.Send.PushBack(func(req *request.Request) {
	 fmt.Println("SENDING", time.Now().Sub(dt))
})
// Output:
// SENDING 201.007364ms

Here we push one function to the beginning of the Send’s list and then compute the delta time in the handler added to the end of the Send’s list to measure how long the API request took to send and receive a response.

As we have shown in these code samples, request handlers allow you to easily customize the SDK’s behavior and add new functionality. If you have used handlers to augment the SDK’s request/response functionality, please let us know in the comments!

Welcome to the AWS Developer Blog for Go

Hi, everyone! Welcome to the AWS Developer Blog for Go. I’m Jason Del Ponte, and I’m a developer on the AWS SDK for Go team. This blog will be the place to go for information about:

  • Tips and tricks for using the AWS SDK for Go
  • New feature announcements
  • Deep dives into the AWS SDK for Go
  • Guest posts from AWS service teams

In the meantime, we’ve created several resources to introduce you to the SDK:

We’re excited to see how you will use the SDK. Please share your experience, feedback, and questions. Stay tuned!