AWS Architecture Blog

Category: Analytics

Figure 1 - Architecture representing an IoT ingestion pipeline

Field Notes: Deliver Messages Using an IoT Rule Action to Amazon Managed Streaming for Apache Kafka

With IoT devices scaling up rapidly, real-time data integration and data processing has become a major challenge. This is why customers often choose Message Queuing Telemetry Transport (MQTT) for message ingestion, and Apache Kafka to build a real-time streaming data pipeline. AWS IoT Core now supports a new IoT rule action to deliver messages from […]

Figure 1. Notional architecture for improving forecasting accuracy solution and SAP integration

Improving Retail Forecast Accuracy with Machine Learning

The global retail market continues to grow larger and the influx of consumer data increases daily. The rise in volume, variety, and velocity of data poses challenges with demand forecasting and inventory planning. Outdated systems generate inaccurate demand forecasts. This results in multiple challenges for retailers. They are faced with over-stocking and lost sales, and […]

How to redact confidential information in your ML pipeline

Integrating Redaction of FinServ Data into a Machine Learning Pipeline

Financial companies process hundreds of thousands of documents every day. These include loan and mortgage statements that contain large amounts of confidential customer information. Data privacy requires that sensitive data be redacted to protect the customer and the institution. Redacting digital and physical documents is time-consuming and labor-intensive. The accidental or inadvertent release of personal information […]

Figure 2. Fraud detection using machine learning architecture on AWS

Analyze Fraud Transactions using Amazon Fraud Detector and Amazon Athena

Organizations with online businesses have to be on guard constantly for fraudulent activity, such as fake accounts or payments made with stolen credit cards. One way they try to identify fraudsters is by using fraud detection applications. Some of these applications use machine learning (ML). A common challenge with ML is the need for a […]

Fitness functions provide feedback to engineers via metrics

Using Cloud Fitness Functions to Drive Evolutionary Architecture

“It is not the strongest of the species that survives, nor the most intelligent. It is the one that is most adaptable to change.” – often attributed to Charles Darwin One common strategy for businesses that operate in dynamic market conditions (and thus need to continuously correct their course) is to aim for smaller, independent […]

Figure 2. Solution overview diagram

Intelligently Search Media Assets with Amazon Rekognition and Amazon ES

Media assets have become increasingly important to industries like media and entertainment, manufacturing, education, social media applications, and retail. This is largely due to innovations in digital marketing, mobile, and ecommerce. Successfully locating a digital asset like a video, graphic, or image reduces costs related to reproducing or re-shooting. An efficient search engine is critical […]

Figure 2: AI Factory high-level architecture

ERGO Breaks New Frontiers for Insurance with AI Factory on AWS

This post is co-authored with Piotr Klesta, Robert Meisner and Lukasz Luszczynski of ERGO Artificial intelligence (AI) and related technologies are already finding applications in our homes, cars, industries, and offices. The insurance business is no exception to this. When AI is implemented correctly, it adds a major competitive advantage. It enhances the decision-making process, […]

Figure 1. Architecture diagram depicting enterprise vertical integration with Amazon EventBridge

Vertical Integration Strategy Powered by Amazon EventBridge

Over the past few years, midsize and large enterprises have adopted vertical integration as part of their strategy to optimize operations and profitability. Vertical integration consists of separating different stages of the production line from other related departments, such as marketing and logistics. Enterprises implement such strategy to gain full control of their value chain: from the […]

Figure 2. A serverless architecture representing data workflow

Building a Showback Dashboard for Cost Visibility with Serverless Architectures

Enterprises with centralized IT organizations and multiple lines of businesses frequently use showback or chargeback mechanisms to hold their departments accountable for their technology usage and costs. Chargeback involves actually billing a department for the cost of their division’s usage. Showback focuses on visibility to make the department more cost conscientious and encourage operational efficiency. […]

Figure 1. RStudio/Shiny Open Source Deployment Pipeline on AWS Serverless Infrastructure

Field Notes: Accelerating Data Science with RStudio and Shiny Server on AWS Fargate

This post was updated November 18, 2021.  Data scientists continuously look for ways to accelerate time to value for analytics projects.  RStudio Server is a popular Integrated Development Environment (IDE) for R, which is used to render analytics visualizations for faster decision making. These visualizations are traditionally hosted on legacy unix servers along with Shiny […]