Retry delivering failed SMS using Amazon Pinpoint

Organizations in many sectors and verticals have user bases to whom they send transactional SMS messages such as OTPs (one-time passwords), Notices, or transaction/purchase confirmations, among other things. Amazon Pinpoint enables customers to send transactional SMS messages to a global audience through a single API endpoint, and the messages are routed to recipients by the service. Amazon Pinpoint relies on downstream SMS providers and telecom operators to deliver the messages to end user’s device. While most of the times the SMS messages gets delivered to recipients but sometimes these messages could not get delivered due to  carrier/telecom related issues which are transient in nature. This impacts customer’s brand name. As a result, customers need to implement a solution that allows them to retry the transmission of SMS messages that fail due to transitory problems caused by downstream SMS providers or telecom operators.

In this blog post, you will discover how to retry sending unsuccessfully delivered SMS messages caused by transitory problems at the downstream SMS provider or telecom operator side.

Prerequisites

For this post, you should be familiar with the following:

Managing an AWS account
Amazon Pinpoint
Amazon Pinpoint SMS events
AWS Lambda
AWS CloudFormation
Amazon Kinesis Firehose
Kinesis Streams
Amazon DynamoDB WCU and RCU accordingly

Architecture Overview

The architecture depicted below is a potential architecture for re-sending unsuccessful SMS messages at real time. The application sends the SMS message to Amazon Pinpoint for delivery using sendMessge API. Pinpoint receives the message and returns a receipt notification with the Message ID; the application records the message content and ID to a Datastore or DynamoDB. Amazon Pinpoint delivers messages to users and then receives SMS engagement events. The same SMS engagement events are provided to Amazon Kinesis Data Streams which as an event source for Lambda function that validates the event type, If the event type indicates that the SMS message was unable to be sent and it make sense to retry, the Lambda function logic retrieves respective “message id” from the SMS events and then retrieves the message body from the database. Then it sends the SMS message to Amazon  Pinpoint for redelivery, you can choose same or an alternative origination number as origination identity while resending the SMS to end users. We recommend configuring the number of retries and adding a retry message tag within Pinpoint to analyse retries and also to avoid infinite loops. All events are also sent to Amazon Kinesis Firehose which then saved to your S3 data lake for later audit and analytics purpose.

Note: The Lambda concurrency and DynamoDB WCU/RCUs need to be provisioned accordingly. The AWS CloudFormation template provided in this post automatically sets up the different architecture components required to retry unsuccessful SMS messages

At the same time, if you use Amazon Kinesis Firehose delivery stream instead of Kinesis data stream to stream data to a storage location, you might consider utilising Transformation lambda as part of the kinesis Firehose delivery stream to retry unsuccessful messages. The architecture is as follows; application sends the SMS payload to Amazon Pinpoint using SendMessage API/SDK while also writing the message body to a persistent data store, in this case a DynamoDB database. The SMS related events are then sent to Amazon Kinesis Firehose, where a   transformation lambda is setup. In essence, if SMS event type returns no errors, the event is returned to Firehose as-is. However, if an event type fails and it makes sense to retry, lambda logic sends another SendMessage until the retry count (specified to 5 within the code) is reached. If just one retry attempt is made, S3 storage is not loaded with an event (thus the result=Dropped). Since Pinpoint event do not contain actual SMS content, a call to DynamoDB is made to get the message body for a new SendMessage.

Amazon Pinpoint provides event response for each transactional SMS communications for retrying unsuccessful SMS connections, there are primarily two factors to consider in this architecture. 1/ Type of event (event_type) 2/ Record Status (record_status). So whenever the event_type is “_SMS.FAILURE” and record_status is any of “UNREACHABLE”, “UNKNOWN”, “CARRIER_UNREACHABLE”, “EXPIRED”. Then surely customer application need to retry the SMS message delivery. Following pseudo code snippet explains the conditional flow for failed SMS sending logic within the lambda function.

Code Sample:

If event.event_type = '_SMS.FAILURE': and event.record_status == 'UNREACHABLE' 
	'| UNKNOWN | CARRIER_UNREACHABLE | TTL_EXPIRED'
	sendMessage(message content, Destination) # resend the SMS message then 
	output_record = { "recordId": record["recordId"], 'result': 'Dropped', 'data': 
		base64.b64encode(payload.encode('utf-8')) } 
else 
	output_record = { "recordId": record["recordId"], 'result': 'Ok', 
						'data': base64.b64encode(payload.encode('utf-8')) }

Getting started with solution deployment

Prerequisite tasks to be completed before deploying the logging solution

1. Go to CloudFormation Console and Click Create Stack.
2. Select Amazon S3 Url radio button and provide the cloud formation link. 
3. Click Next on Create Stack screen.
4. Specify Stack Name, for example “SMS-retry-stack”
5. Specify event stream configuration option, this will trigger the respective child cloud formation stack . There are three Event stream configuration you can choose from.
   • No Existing event stream setup – Select this option if you don’t have any event stream setup for Amazon Pinpoint.
   • Event stream setup with Amazon Kinesis Stream – Select this option if your Amazon Pinpoint project already have Amazon Kinesis as event stream destination.
   • Event stream setup with Amazon Kinesis Firehose – Select this option if you have configured Kinesis Firehose delivery stream as event stream destination.
6. Specify the Amazon Pinpoint project app ID (Pinpoint project ID), and click Next.
7. Click Next on Configure stack options screen.
8. Select “I acknowledge that AWS CloudFormation might create IAM resources” and click Create Stack.
9. Wait for the CloudFormation template to complete and then verify resources in the CloudFormation stack has been created. Click on individual resources and verify.
   • Parent stack-
   • Child Stack –
10. As described in the architectural overview session, the maxRetries configuration inside “RetryLambdaFunction” ensures that unsuccessful SMS messages are tried resending repeatedly. This number is set to 3 by default.” If you want to adjust the maxRetry count, go to the settings “RetryLambdaFunction” and change it to the desired number.

Notes :- The Cloudformation link in the blog specifically points to the parent cloudformation template, which has links to the child Cloudformation stack, these child stacks will be deployed automatically as you go through the patent stack.

Testing the solution

You can test the solution using “PinpointDDBProducerLambdaFunction” and SMS simulator numbers . PinpointDDBProducerLambdaFunction has sample code that shall trigger the SMS using Amazon Pinpoint.

Now follow the steps below to test the solution.

1. Go to environment variables for PinpointDDBProducerLambdaFunction­­
2. Update “destinationNumber” and “pinpointApplicationID,” where destination number is the recipient number for whom you wish to send the SMS as a failed attempt and Amazon Pinpoint application id is the Pinpoint Project ID for which the Pinpoint SMS channel has already been configured.
3. Deploy and test the Lambda function.
4. Check the “Pinpoint Message state” DyanamoDB table and open the Latest table ITEM.
5. If you observe the table Items, it states the retry_count=2 (SMS send retry has been attempted 2 times and all_retries_failed=true ( for both of the times the SMS could not get delivered.)

Notes :

• If existing Kinesis firehose has pre-existing transformation lambda then current stack does not add a transformation Lambda function.
• If existing Kinesis firehose has a destination other than Amazon S3 then current stack does not add a transformation Lambda function.

Remarks

This SMS retry solution is based on best effort. This means that the solution is dependent on event response data from SMS aggregators. If the SMS aggregator data is incorrect, this slotion may not produce the desired effec

Cost

Considering that the retry mechanism is applicable for 1000000 unsuccessful SMS messages per month, this solution will approximately cost around $20 per month. Here is AWS calculator link for reference

Clean up

When you’re done with this exercise, complete the following steps to delete your resources and stop incurring costs:

• On the CloudFormation console, select your stack and choose Delete.
• This cleans up all the resources created by the stack.

Conclusion

In this blog post, we have demonstrated how customers can retry sending the undelivered/failed SMS messages via Amazon Pinpoint. We explained how to leverage the Amazon kinesis data streams and AWS Lambda functions to assess the status of unsuccessful SMS messages and retry delivering them in an automatic manner.

Extending the solution

This blog provides a rightful frame work to Implement a solution to retry sending failed SMS messages. You can download the AWS Cloudformation templates, code, and scripts for this solution from our GitHub repository and modify it to fit your needs.

About the Authors
Satyasovan Tripathy works as a Senior Specialist Solution Architect at AWS. He is situated in Bengaluru, India, and focuses on the AWS Digital User Engagement product portfolio. He enjoys reading and travelling outside of work.

Nikhil Khokhar is a Solutions Architect at AWS. He specializes in building and supporting data streaming solutions that help customers analyze and get value out of their data. In his free time, he makes use of his 3D printing skills to solve everyday problems.