AWS Architecture Blog

Category: AWS Glue

Mainframe data integration conceptual flow

Mainframe data integration: Using mainframe data to build cloud native services with AWS

For International Women’s Day and Women’s History Month, we’re featuring more than a week’s worth of posts that highlight female builders and leaders. We’re showcasing women in the industry who are building, creating, and, above all, inspiring, empowering, and encouraging everyone—especially women and girls—in tech. Many companies in the financial services and insurance industries rely […]

Architecture Diagram showing How to Audit and Report S3 prefix level access using S3 Access Analyzer

How to Audit and Report S3 Prefix Level Access Using S3 Access Analyzer

Data Services teams in all industries are developing centralized data platforms that provide shared access to datasets across multiple business units and teams within the organization. This makes data governance easier, minimizes data redundancy thus reducing cost, and improves data integrity. The central data platform is often built with Amazon Simple Storage Service (Amazon S3). […]

Figure 2. Control plane architecture

How Ribbon Built a Scalable, Resilient Robocall Mitigation Platform

Ribbon provides communications software, and IP and optical networking end-to-end solutions that deliver innovation, unparalleled scale, performance, and agility to service providers and enterprise. Ribbon is helping customers modernize their networks. In today’s data-hungry, 24/7 world, this equates to improved competitive positioning and business outcomes. Companies are migrating from on-premises equipment for telephony services and looking […]

Figure 2. Credit application – technical solution using Amazon SageMaker and Experian CaaS ML models

How Experian uses Amazon SageMaker to Deliver Affordability Verification 

Financial Service (FS) providers must identify patterns and signals in a customer’s financial behavior to provide deeper, up-to-the-minute, insight into their affordability and credit risk. FS providers use these insights to improve decision making and customer management capabilities. Machine learning (ML) models and algorithms play a significant role in automating, categorising, and deriving insights from […]

Figure 1. Audit Surveillance data lake architecture diagram

How Parametric Built Audit Surveillance using AWS Data Lake Architecture

Parametric Portfolio Associates (Parametric), a wholly owned subsidiary of Morgan Stanley, is a registered investment adviser. Parametric provides investment advisory services to individual and institutional investors around the world. Parametric manages over 100,000 client portfolios with assets under management exceeding $400B (as of 9/30/21). As a registered investment adviser, Parametric is subject to numerous regulatory […]

Serverless S3 metadata search

Swiftly Search Metadata with an Amazon S3 Serverless Architecture

As you increase the number of objects in Amazon Simple Storage Service (Amazon S3), you’ll need the ability to search through them and quickly find the information you need. In this blog post, we offer you a cost-effective solution that uses a serverless architecture to search through your metadata. Using a serverless architecture helps you […]

Figure 2. Containerized application for ingestion and Amazon Kinesis for format conversion

Designing a High-volume Streaming Data Ingestion Platform Natively on AWS

The total global data storage is projected to exceed 200 zettabytes by 2025. This exponential growth of data demands increased vigilance against cybercrimes. Emerging cybersecurity trends include increasing service attacks, ransomware, and critical infrastructure threats. Businesses are changing how they approach cybersecurity and are looking for new ways to tackle these threats. In the past, […]

Field Notes: How to Build an AWS Glue Workflow using the AWS Cloud Development Kit

Many customers use AWS Glue workflows to build and orchestrate their ETL (extract-transform-load) pipelines directly in the AWS Glue console using the visual tool to author workflows. This can be time consuming, harder to version control, and error prone due to manual configurations, when compared to managing your workflows as code. To improve your operational […]

Figure 1. Data flow - Source to data lake target

Hybrid Cloud Architectures Using Self-hosted Apache Kafka and AWS Glue

Using analytics to gain insights from a variety of datasets is key to successful transformation. There are many options to consider to realize the full value and potential of our data in a hybrid cloud infrastructure. Common practice is to route data produced from on-premises to a central repository or data lake. Here it can […]

Figure 1. OR optimization options

Emerging Solutions for Operations Research on AWS

September 8, 2021: Amazon Elasticsearch Service has been renamed to Amazon OpenSearch Service. See details. Operations research (OR) uses mathematical and analytical tools to arrive at optimal solutions for complex business problems like workforce scheduling. The mathematical techniques used to solve these problems, such as linear programming and mixed-integer programming, require the use of optimization […]