AWS Architecture Blog

Category: Amazon SageMaker

Figure 1 - Architecture Showing how to build an automated Image Processing and Model Training pipeline

Field Notes: Building an Automated Image Processing and Model Training Pipeline for Autonomous Driving

In this blog post, we demonstrate how to build an automated and scalable data pipeline for autonomous driving. This solution was built with the goal of accelerating the process of analyzing recorded footage and training a model to improve the experience of autonomous driving. We will demonstrate the extraction of images from ROS bag file […]

Figure 1. Notional architecture for improving forecasting accuracy solution and SAP integration

Improving Retail Forecast Accuracy with Machine Learning

The global retail market continues to grow larger and the influx of consumer data increases daily. The rise in volume, variety, and velocity of data poses challenges with demand forecasting and inventory planning. Outdated systems generate inaccurate demand forecasts. This results in multiple challenges for retailers. They are faced with over-stocking and lost sales, and […]

How to redact confidential information in your ML pipeline

Integrating Redaction of FinServ Data into a Machine Learning Pipeline

Financial companies process hundreds of thousands of documents every day. These include loan and mortgage statements that contain large amounts of confidential customer information. Data privacy requires that sensitive data be redacted to protect the customer and the institution. Redacting digital and physical documents is time-consuming and labor-intensive. The accidental or inadvertent release of personal information […]

Figure 1 - Architecture overview of the solution to launch a fully configured AWS Deep Learning Desktop with NICE DCV

Field Notes: Launch a Fully Configured AWS Deep Learning Desktop with NICE DCV

You want to start quickly when doing deep learning using GPU-activated Elastic Compute Cloud (Amazon EC2) instances in the AWS Cloud. Although AWS provides end-to-end machine learning (ML) in Amazon SageMaker, working at the deep learning frameworks level, the quickest way to start is with AWS Deep Learning AMIs (DLAMIs), which provide preconfigured Conda environments for […]

Figure 2: AI Factory high-level architecture

ERGO Breaks New Frontiers for Insurance with AI Factory on AWS

This post is co-authored with Piotr Klesta, Robert Meisner and Lukasz Luszczynski of ERGO Artificial intelligence (AI) and related technologies are already finding applications in our homes, cars, industries, and offices. The insurance business is no exception to this. When AI is implemented correctly, it adds a major competitive advantage. It enhances the decision-making process, […]

Figure 1. Data pipeline that cleans, processes, and segments data

How Financial Institutions can use AWS to Address Regulatory Reporting

Since the 2008 financial crisis, banking supervisory institutions such as the Basel Committee on Banking Supervision (BCBS) have strengthened regulations. There is now increased oversight over the financial services industry. For banks, making the necessary changes to comply with these rules is a challenging, multi-year effort. Basel IV, a massive update to existing rules, is […]

Figure 1 - Architecture for Automating Data Ingestion and Labeling for Autonomous Vehicle Development

Field Notes: Automating Data Ingestion and Labeling for Autonomous Vehicle Development

This post was co-written by Amr Ragab, AWS Sr. Solutions Architect, EC2 Engineering and Anant Nawalgaria, former AWS Professional Services EMEA. One of the most common needs we have heard from customers in Autonomous Vehicle (AV) development, is to launch a hybrid deployment environment at scale. As vehicle fleets are deployed across the globe, they […]

Figure 2. Lake House architecture on AWS

Architecting Persona-centric Data Platform with On-premises Data Sources

Many organizations are moving their data from silos and aggregating it in one location. Collecting this data in a data lake enables you to perform analytics and machine learning on that data. You can store your data in purpose-built data stores, like a data warehouse, to get quick results for complex queries on structured data. […]

The following diagram shows the components that are used in this solution. We use an AWS CloudFormation template to set up the required ntworking components (for example, VPC, subnets).

Field Notes: Develop Data Pre-processing Scripts Using Amazon SageMaker Studio and an AWS Glue Development Endpoint

This post was co-written with Marcus Rosen, a Principal  – Machine Learning Operations with Rio Tinto, a global mining company.  Data pre-processing is an important step in setting up Machine Learning (ML) projects for success. Many AWS customers use Apache Spark on AWS Glue or Amazon EMR to run data pre-processing scripts while using Amazon SageMaker […]

Figure 1. Example architecture using AWS Managed Services

Building a Cloud-based OLAP Cube and ETL Architecture with AWS Managed Services

For decades, enterprises used online analytical processing (OLAP) workloads to answer complex questions about their business by filtering and aggregating their data. These complex queries were compute and memory-intensive. This required teams to build and maintain complex extract, transform, and load (ETL) pipelines to model and organize data, oftentimes with commercial-grade analytics tools. In this […]