AWS Architecture Blog

Category: AWS Glue

Zendesk data pipelines

Insights for CTOs: Part 3 – Growing your business with modern data capabilities

This post was co-wrtiten with Jonathan Hwang, head of Foundation Data Analytics at Zendesk. In my role as a Senior Solutions Architect, I have spoken to chief technology officers (CTOs) and executive leadership of large enterprises like big banks, software as a service (SaaS) businesses, mid-sized enterprises, and startups. In this 6-part series, I share […]

Read More
Mainframe data integration conceptual flow

Mainframe data integration: Using mainframe data to build cloud native services with AWS

For International Women’s Day and Women’s History Month, we’re featuring more than a week’s worth of posts that highlight female builders and leaders. We’re showcasing women in the industry who are building, creating, and, above all, inspiring, empowering, and encouraging everyone—especially women and girls—in tech. Many companies in the financial services and insurance industries rely […]

Read More
Architecture Diagram showing How to Audit and Report S3 prefix level access using S3 Access Analyzer

How to Audit and Report S3 Prefix Level Access Using S3 Access Analyzer

Data Services teams in all industries are developing centralized data platforms that provide shared access to datasets across multiple business units and teams within the organization. This makes data governance easier, minimizes data redundancy thus reducing cost, and improves data integrity. The central data platform is often built with Amazon Simple Storage Service (Amazon S3). […]

Read More
Figure 2. Control plane architecture

How Ribbon Built a Scalable, Resilient Robocall Mitigation Platform

Ribbon provides communications software, and IP and optical networking end-to-end solutions that deliver innovation, unparalleled scale, performance, and agility to service providers and enterprise. Ribbon is helping customers modernize their networks. In today’s data-hungry, 24/7 world, this equates to improved competitive positioning and business outcomes. Companies are migrating from on-premises equipment for telephony services and looking […]

Read More
Figure 2. Credit application – technical solution using Amazon SageMaker and Experian CaaS ML models

How Experian uses Amazon SageMaker to Deliver Affordability Verification 

Financial Service (FS) providers must identify patterns and signals in a customer’s financial behavior to provide deeper, up-to-the-minute, insight into their affordability and credit risk. FS providers use these insights to improve decision making and customer management capabilities. Machine learning (ML) models and algorithms play a significant role in automating, categorising, and deriving insights from […]

Read More
Figure 1. Audit Surveillance data lake architecture diagram

How Parametric Built Audit Surveillance using AWS Data Lake Architecture

Parametric Portfolio Associates (Parametric), a wholly owned subsidiary of Morgan Stanley, is a registered investment adviser. Parametric provides investment advisory services to individual and institutional investors around the world. Parametric manages over 100,000 client portfolios with assets under management exceeding $400B (as of 9/30/21). As a registered investment adviser, Parametric is subject to numerous regulatory […]

Read More
Serverless S3 metadata search

Swiftly Search Metadata with an Amazon S3 Serverless Architecture

As you increase the number of objects in Amazon Simple Storage Service (Amazon S3), you’ll need the ability to search through them and quickly find the information you need. In this blog post, we offer you a cost-effective solution that uses a serverless architecture to search through your metadata. Using a serverless architecture helps you […]

Read More
Figure 2. Containerized application for ingestion and Amazon Kinesis for format conversion

Designing a High-volume Streaming Data Ingestion Platform Natively on AWS

The total global data storage is projected to exceed 200 zettabytes by 2025. This exponential growth of data demands increased vigilance against cybercrimes. Emerging cybersecurity trends include increasing service attacks, ransomware, and critical infrastructure threats. Businesses are changing how they approach cybersecurity and are looking for new ways to tackle these threats. In the past, […]

Read More

Field Notes: How to Build an AWS Glue Workflow using the AWS Cloud Development Kit

Many customers use AWS Glue workflows to build and orchestrate their ETL (extract-transform-load) pipelines directly in the AWS Glue console using the visual tool to author workflows. This can be time consuming, harder to version control, and error prone due to manual configurations, when compared to managing your workflows as code. To improve your operational […]

Read More
Figure 1. Data flow - Source to data lake target

Hybrid Cloud Architectures Using Self-hosted Apache Kafka and AWS Glue

Using analytics to gain insights from a variety of datasets is key to successful transformation. There are many options to consider to realize the full value and potential of our data in a hybrid cloud infrastructure. Common practice is to route data produced from on-premises to a central repository or data lake. Here it can […]

Read More