AWS Architecture Blog

Category: Analytics

Figure 5. Amazon Redshift federated query with Amazon Redshift ML

Address Modernization Tradeoffs with Lake House Architecture

Many organizations are modernizing their applications to reduce costs and become more efficient. They must adapt to modern application requirements that provide 24×7 global access. The ability to scale up or down quickly to meet demand and process a large volume of data is critical. This is challenging while maintaining strict performance and availability. For […]

reference architecture - build automated scene detection pipeline - Autonomous Driving

Field Notes: Building an automated scene detection pipeline for Autonomous Driving – ADAS Workflow

This Field Notes blog post in 2020 explains how to build an Autonomous Driving Data Lake using this Reference Architecture. Many organizations face the challenge of ingesting, transforming, labeling, and cataloging massive amounts of data to develop automated driving systems. In this re:Invent session, we explored an architecture to solve this problem using Amazon EMR, Amazon […]

High-level design for an AWS lake house implementation

Benefits of Modernizing On-premises Analytics with an AWS Lake House

Organizational analytics systems have shifted from running in the background of IT systems to being critical to an organization’s health. Analytics systems help businesses make better decisions, but they tend to be complex and are often not agile enough to scale quickly. To help with this, customers upgrade their traditional on-premises online analytic processing (OLAP) […]

Figure 1 - Architecture representing an IoT ingestion pipeline

Field Notes: Deliver Messages Using an IoT Rule Action to Amazon Managed Streaming for Apache Kafka

With IoT devices scaling up rapidly, real-time data integration and data processing has become a major challenge. This is why customers often choose Message Queuing Telemetry Transport (MQTT) for message ingestion, and Apache Kafka to build a real-time streaming data pipeline. AWS IoT Core now supports a new IoT rule action to deliver messages from […]

Figure 1. Notional architecture for improving forecasting accuracy solution and SAP integration

Improving Retail Forecast Accuracy with Machine Learning

The global retail market continues to grow larger and the influx of consumer data increases daily. The rise in volume, variety, and velocity of data poses challenges with demand forecasting and inventory planning. Outdated systems generate inaccurate demand forecasts. This results in multiple challenges for retailers. They are faced with over-stocking and lost sales, and […]

How to redact confidential information in your ML pipeline

Integrating Redaction of FinServ Data into a Machine Learning Pipeline

Financial companies process hundreds of thousands of documents every day. These include loan and mortgage statements that contain large amounts of confidential customer information. Data privacy requires that sensitive data be redacted to protect the customer and the institution. Redacting digital and physical documents is time-consuming and labor-intensive. The accidental or inadvertent release of personal information […]

Figure 2. Fraud detection using machine learning architecture on AWS

Analyze Fraud Transactions using Amazon Fraud Detector and Amazon Athena

Organizations with online businesses have to be on guard constantly for fraudulent activity, such as fake accounts or payments made with stolen credit cards. One way they try to identify fraudsters is by using fraud detection applications. Some of these applications use machine learning (ML). A common challenge with ML is the need for a […]

Fitness functions provide feedback to engineers via metrics

Using Cloud Fitness Functions to Drive Evolutionary Architecture

“It is not the strongest of the species that survives, nor the most intelligent. It is the one that is most adaptable to change.” – often attributed to Charles Darwin One common strategy for businesses that operate in dynamic market conditions (and thus need to continuously correct their course) is to aim for smaller, independent […]

Figure 2. Solution overview diagram

Intelligently Search Media Assets with Amazon Rekognition and Amazon ES

Media assets have become increasingly important to industries like media and entertainment, manufacturing, education, social media applications, and retail. This is largely due to innovations in digital marketing, mobile, and ecommerce. Successfully locating a digital asset like a video, graphic, or image reduces costs related to reproducing or re-shooting. An efficient search engine is critical […]

Figure 2: AI Factory high-level architecture

ERGO Breaks New Frontiers for Insurance with AI Factory on AWS

This post is co-authored with Piotr Klesta, Robert Meisner and Lukasz Luszczynski of ERGO Artificial intelligence (AI) and related technologies are already finding applications in our homes, cars, industries, and offices. The insurance business is no exception to this. When AI is implemented correctly, it adds a major competitive advantage. It enhances the decision-making process, […]