Amazon CloudWatch aggregates logs and creates observability metrics and dashboards, providing visualizations to help you identify performance bottlenecks and troubleshoot requests. You can also set up CloudWatch alarms to identify trends that could be problematic and alert you before they impact your application or business. Additionally, you can use AWS CloudTrail , which keeps track of account activity, to enable governance and risk auditing, as well as facilitate the compliance of your AWS account.

The principle of least privilege is the industry best practice for reducing the surface area of security risks. AWS Identity and Access Management (IAM) policies use least-privilege access so that every policy is restrictive to the specific resource and operation. Additionally, to implement security in layers, this Guidance encrypts data in transit and transfers it over HTTPS, and AWS Key Management Service (AWS KMS) keys encrypt data at rest in Amazon S3 buckets. Finally, real-time bidding (RTB) applications access SageMaker endpoints and Amazon S3 only through PrivateLink , enhancing your security posture.

All the services used in this Guidance are serverless and can automatically scale horizontally based on workload demand. In the SageMaker inference endpoints, Amazon EC2 Auto Scaling groups launch instances across AZs to provide high availability. Additionally, Amazon S3 supports features like S3 Versioning , which helps you maintain data version control, prevent accidental deletions, and replicate data to the same or a different AWS Region. With the ability to preserve, retrieve, and restore every version of an object stored in Amazon S3 , you can recover from unintended user actions and application failures.

AWS managed services offload infrastructure management and scaling from you so that you can focus on solving your business needs. In this Guidance, SageMaker manages the hosting of your model inference endpoints. It retrieves the models from Amazon S3 buckets at deployment time, then hosts the most optimal implementation runtime containers. By using its inference recommender and a load-testing tool, SageMaker can choose the optimal instance size based on throughput capacity and incurred latencies. This enables it to manage the scaling of the inference compute through load balancers and Amazon EC2 Auto Scaling groups.

This Guidance uses serverless technologies and managed services so that you only pay for the resources you consume. You can also select options to further reduce costs. For example, Amazon SageMaker Savings Plans offer a flexible, usage-based pricing model in exchange for a commitment to a consistent amount of usage. You can also store data cost-effectively by choosing from a range of Amazon S3 storage classes built for specific use cases and access patterns. For example, if you use Amazon S3 Intelligent-Tiering for data with changing, unknown, or unpredictable access patterns—such as data lakes, analytics, or new applications—it will automatically optimize costs by moving your data between tiers for frequent, infrequent, and rare access. Additionally, by securing traffic over a private network using PrivateLink , you can reduce data transfer fees.

This Guidance uses serverless technologies that scale up and down to meet demand so that resources don’t consume energy while idle. Additionally, SageMaker endpoints use custom infrastructure that is optimal to the workload demands of model training and inference, helping you achieve more with fewer resources and a lower carbon footprint.