AWS Architecture Blog

Category: Analytics

Figure 5. The Full Architectural Diagram

Reduce Operational Load using AWS Managed Services for your Data Solutions

As the volume of customers’ data grows, companies are realizing the benefits that data has for their business. Amazon Web Services (AWS) offers many database and analytics services, which give companies the ability to build complex data management workloads. At the same time, these services can reduce the operational overhead compared to traditional operations. Using […]

Read More
Corrosion Monitoring Architecture

Field Notes: Building Automated Pipeline Corrosion Monitoring with AWS IoT Core

This post was authored by Venkatesh Muthusami, Principle Consultant, Infosys Technologies, Sudharsan Chinnappan, Analyst, System Development, Infosys Technologies, Kenneth Francis Dias, Technology Architect, Infosys Technologies, and Ashutosh Pateriya, Partner Solutions Architect, AWS Pipelines are crucial to the oil and gas industry across upstream, midstream, and downstream sectors. For industries like oil and gas, the pipeline […]

Read More
EMR solution diagram

Field Notes: Launch Amazon EMR with a Static Private IP in a Private Subnet

Organizations across every industry and sector are looking to easily and cost-effectively process vast amounts of data. Amazon EMR offers a way to instantly provision as much or as little capacity as needed to perform data- intensive tasks. When launching Amazon EMR, the IPs of the primary (master) and core node are automatically assigned at […]

Read More
Figure 1. RStudio/Shiny Open Source Deployment on AWS Serverless Infrastructure

Scaling RStudio/Shiny using Serverless Architecture and AWS Fargate

Data scientists use RStudio server as an Integrated Development Environment (IDE) to develop, publish, and share interactive web dashboards built on Shiny Server. Although it is possible to use virtual server infrastructure in the cloud to run R workloads, containerization offers significant operational benefits. Migrating R workloads into a serverless model in AWS, customers can […]

Read More
serverless repo screesnhot

Field Notes: Launch Amazon Elasticsearch Service with Fine-Grained Access Control and Amazon Cognito

Originally published March 2020 to the Field Notes blog, and reviewed for technical accuracy in March 2021. To get started with Amazon Elasticsearch Service (Amazon ES), you must have a concept for authentication and authorization for your search cluster. In addition to an IP-based access policy or a proxy server to protect your search cluster, you can […]

Read More
Figure 3. Replay Architecture

Amazon MSK Backup for Archival, Replay, or Analytics

Amazon MSK is a fully managed service that helps you build and run applications that use Apache Kafka to process streaming data. Apache Kafka is an open-source platform for building real-time streaming data pipelines and applications. With Amazon MSK, you can use native Apache Kafka APIs to populate data lakes. You can also stream changes to […]

Read More
Amazon Personalize: from datasets to a recommendation API

Automating Recommendation Engine Training with Amazon Personalize and AWS Glue

Customers from startups to enterprises observe increased revenue when personalizing customer interactions. Still, many companies are not yet leveraging the power of personalization, or, are relying solely on rule-based strategies. Those strategies are effort-intensive to maintain and not effective. Common reasons for not launching machine learning (ML) based personalization projects include: the complexity of aggregating […]

Read More
Olympus Tower - Grov Technologies

Building a Controlled Environment Agriculture Platform

This post was co-written by Michael Wirig, Software Engineering Manager at Grōv Technologies. A substantial percentage of the world’s habitable land is used for livestock farming for dairy and meat production. The dairy industry has leveraged technology to gain insights that have led to drastic improvements and are continuing to accelerate. A gallon of milk […]

Read More
Samba file share solution diagram

Field Notes: Ingest and Visualize Your Flat-file IoT Data with AWS IoT Services

Customers who maintain manufacturing facilities often find it challenging to ingest, centralize, and visualize IoT data that is emitted in flat-file format from their factory equipment. While modern IoT-enabled industrial devices can communicate over standard protocols like MQTT, there are still some legacy devices that generate useful data but are only capable of writing it […]

Read More
architecture for the solution

Real-Time In-Stream Inference with AWS Kinesis, SageMaker, & Apache Flink

As businesses race to digitally transform, the challenge is to cope with the amount of data, and the value of that data diminishes over time. The challenge is to analyze, learn, and infer from real-time data to predict future states, as well as to detect anomalies and get accurate results. In this blog post, we’ll […]

Read More