Monitor performance with AWS Elemental MediaStore HTTP access logs

AWS Elemental MediaStore is an AWS storage service optimized for media. It gives you the performance, consistency, and low latency required to deliver live streaming video content. To monitor ingest and egress for MediaStore, pipeline access logs provide detailed records for requests that are made for any object stored in a MediaStore container. This information can include the request type, the resources that are specified in the request, and the time and date that the request was processed. This blog will explain how to setup MediaStore access logs, how to view them, and why you need them.

Enable access logs in MediaStore

By default access logs are disabled in MediaStore. You can enable access logs using the AWS CLI or using the AWS Management Console. The command for the AWS CLI is:

aws mediastore start-access-logging --container-name LiveEvent

More details can be found in the MediaStore documentation.

In the AWS Management Console, click “Enable access logging” after selecting the MediaStore container.

Next, give MediaStore permission with IAM to send Access Logs to AWS CloudWatch

Follow the steps using this link in your AWS account to give MediaStore access to write logs to AWS CloudWatch.

Tip: For step 8 in the instruction, the account ID is the same as your account number without hyphens, e.g. 123123123123

Now upload a file to test the configuration

Once you have created a IAM role giving MediaStore access to write to AWS CloudWatch, upload a couple of files to see what the logs in CloudWatch look like. Again, you can either use the AWS CLI or the AWS management console to upload an object. Both methods do a PUT request to the MediaStore API which will show up in the CloudWatch logs. I’ve uploaded two .m3u8 files for this test.

Head to AWS CloudWatch in the AWS Management Console

Click on the Logs button on the left hand side of the screen, and you will now see an /aws/mediastore/[Container_Name] log group that has been created.

If you are not seeing logs look at this guide: Troubleshooting Access Logs

By clicking into one of the events you get a nice JSON structure. It captures the HTTP request, which in this case is HTTP PutObject. What you want to look at are Turn Around Time and Total Time.

For PUT requests, you’ll find TurnAroundTime in the MediaStore access logs. This value is a measurement of the time in milliseconds it takes for MediaStore to process a request. Specifically, the time is measured from the last byte of the request being received until the time of the first byte of the response being sent. For PUT requests, you’ll find TotalTime. This value is measuring the total time from when the request is received by MediaStore to time the last byte of the response is sent back to the client.

This information helps confirm your customers are having an optimal viewing experience. The TotalTime metric on PUT Requests gives you information about the internet performance from your workflow sending video (objects) into MediaStore.

Monitoring these access log files gives you valuable insight about your MediaStore container. More information on each of these fields in the JSON log can be found here.

{
    "Path": "/playlist-2.m3u8",
    "AWSAccountId": xxxxxxxxxxx,
    "RequestID": "F5HN5DOBPVAQEV4PNXLEOQC23WNFEVDJCO4UYXNIDQAXDO2TBJSYWBPLIE52Q67IX735SGJVBOY5GQZ7NBM7API",
    "ReceivedTime": "2019-05-24T22:47:02.494Z",
    "Operation": "PutObject",
    "Source": "0.0.0.0",
    "Requester": xxxxxxxxxxxxxxxxxxxxxxxx,
    "ContainerName": "LiveEvent",
    "TurnAroundTime": 36,
    "TotalTime": 37,
    "BytesSent": 184,
    "ErrorCode": null,
    "HTTPStatus": 200
}

In the example above, there is a TurnAroundTime of 36. This shows is took MediaStore 36 milliseconds from the time the last byte of the PUT request was received to the time the first byte of the response was sent back. The TotalTime is 37 which indicates MediaStore took 37 milliseconds from the time the last byte of the PUT request was received to the time the last byte of the response was sent back. This means MediaStore took 1 millisecond to complete sending the response.

CloudWatch Insights for creating Dashboards

In order to monitor PUT requests to MediaStore, you will need to create a CloudWatch Dashboard using CloudWatch insights. For more information about CloudWatch Insights please read this blog post.

Step 1: Under the CloudWatch console click the insights tab on the left under Logs.
Step 2: Search for /aws/mediastore/[Container_Name] which will allow for you to query a specific log group.

With the log group for your container selected, we can now paste the CloudWatch Insights query into the console. Here is the query:

fields @timestamp, @message
| stats percentile(TurnAroundTime, 50), percentile(TurnAroundTime, 99), max(TurnAroundTime) by Bin(1m)

The query will show a table of the p50 and p99 of the TurnAroundTime metrics. Click the Visualization tab to see a graph showing results over the time range selected.

In this example, over the last hour the Max TurnAroundTime peaked at around 220 milliseconds. The data queried with CloudWatch insights metrics can be inserted into a CloudWatch Dashboard, allowing you to keep track of metrics such as MediaStore PUT performance from your encoder.

Conclusion

AWS Elemental MediaStore Access Logs give valuable insight into the performance of your MediaStore container. These logs can be used with a CloudWatch dashboard or alarms to monitor performance and ensure that the internet pipeline between your data center and MediaStore is healthy, and ultimately ensure a great experience for your customers.

AWS for M&E Blog