AWS Architecture Blog

The following diagram shows the components that are used in this solution. We use an AWS CloudFormation template to set up the required ntworking components (for example, VPC, subnets).

Field Notes: Develop Data Pre-processing Scripts Using Amazon SageMaker Studio and an AWS Glue Development Endpoint

This post was co-written with Marcus Rosen, a Principal  – Machine Learning Operations with Rio Tinto, a global mining company.  Data pre-processing is an important step in setting up Machine Learning (ML) projects for success. Many AWS customers use Apache Spark on AWS Glue or Amazon EMR to run data pre-processing scripts while using Amazon SageMaker […]

Data ingestion options from on-premises devices to AWS

Enhancing Existing Building Systems with AWS IoT Services

With the introduction of cloud technology and by extension the rapid emergence of Internet of Things (IoT), the barrier to entry for creating smart building solutions has never been lower. These solutions offer commercial real estate customers potential cost savings and the ability to enhance their tenants’ experience. You can differentiate your business from competitors […]

Figure 1 – Visualizing read I/O operations on d3.2xlarge instances across three Availability Zones.

Field Notes: Benchmarking Performance of the New M5zn, D3, and R5b Instance Types with Datadog

This post was co-written with Danton Rodriguez, Product Manager at Datadog.  At re:Invent 2020, AWS announced the new Amazon Elastic Compute Cloud (Amazon EC2) M5zn, D3, and R5b instance types. These instances are built on top of the AWS Nitro System, a collection of AWS-designed hardware and software innovations that enable the delivery of private networking, […]

Architecture Monthly Magazine: 5G

In our 5G issue, we discuss the evolution, implementation and trends for 5G. We’ve included some case studies from Verizon and Ericsson, and take a look at Ribbon SBC SWe and network-slicing with Cloudify. There are detailed Reference Architectures for you to check out, and videos on DISH, Verizon, AWS Wavelength, VR, Automotive, and Telefónica. […]

Figure 1. Example architecture using AWS Managed Services

Building a Cloud-based OLAP Cube and ETL Architecture with AWS Managed Services

For decades, enterprises used online analytical processing (OLAP) workloads to answer complex questions about their business by filtering and aggregating their data. These complex queries were compute and memory-intensive. This required teams to build and maintain complex extract, transform, and load (ETL) pipelines to model and organize data, oftentimes with commercial-grade analytics tools. In this […]

Figure 2: A diagram visualizing the steps involved in data processing across two layers, the data layer and the knowledge layer and their implementations with AWS services.

Field Notes: Data-Driven Risk Analysis with Amazon Neptune and Amazon Elasticsearch Service

September 8, 2021: Amazon Elasticsearch Service has been renamed to Amazon OpenSearch Service. See details. This blog post is co-authored with Charles Crouspeyre and Angad Srivastava. Charles is Director at Accenture Applied Intelligence and ASEAN AI SME (Subject Matter Expert) and Angad is Data and Analytics Consultant at AWS and NLP (Natural Language Processing) expert. […]

Serverless compute with Fargate

Rate Limiting Strategies for Serverless Applications

Serverless technologies reduce the work needed to set up and maintain computing resources, provide built-in scalability, and optimize agility, performance, cost, and security. The pay-as-you-go model is particularly liberating for developers. You can fail fast, experiment more, and do it fairly cheaply. However, serverless brings its own challenges. In this blog, we’ll examine how to […]

Pilot consideration process

Designing a Successful Pilot Phase for Your Cloud Migration

Pilot phases, or pilots, as we will call them from now on, should be conducted to test and find the positive and negative aspects of a particular use case, design pattern, or application migration approach. They allow you to validate the foundation of your architecture (for example, with a landing zone governed by AWS Control […]

Figure 1. Managed File Transfer Process Flow

Managed File Transfer using AWS Transfer Family and Amazon S3

Financial, healthcare, retail and other companies exchange many different types of data. This can include stock information, healthcare claims, and sharing product data files with their partners. These companies need a managed file transfer solution that supports data transformation, and exchange of data over File Transfer Protocol over SSL (FTPS), and Secure File Transfer Protocol (SFTP). […]

Example of a basic organization

New Whitepaper Provides Best Practices for Optimizing AWS Accounts

Moving fast on the cloud while staying secure requires mechanisms to give the right people access to the right features at the right time. We recommend using multiple accounts, which helps you isolate applications, data, and business processes across your environment. The recently released whitepaper Organizing Your AWS Environment Using Multiple Accounts will show you […]