AWS Compute Blog

Category: Amazon EC2*

Building High-Throughput Genomics Batch Workflows on AWS: Workflow Layer (Part 4 of 4)

Aaron Friedman is a Healthcare and Life Sciences Partner Solutions Architect at AWS Angel Pizarro is a Scientific Computing Technical Business Development Manager at AWS This post is the fourth in a series on how to build a genomics workflow on AWS. In Part 1, we introduced a general architecture, shown below, and highlighted the […]

Read More

Building Scalable Applications and Microservices: Adding Messaging to Your Toolbox

Jakub Wojciak, Senior Software Development Engineer Throughout our careers, we developers keep adding new tools to our development toolboxes. These range from the programming languages we learn, use, and become experts in, to architectural components such as HTTP servers, load balancers, and databases (both relational and NoSQL). I’d like to kick off a series of […]

Read More

Disabling Intel Hyper-Threading Technology on Amazon Linux

Brian Beach, Solutions Architect Customers running high performance computing (HPC) workloads on Amazon Linux occasionally ask to disable the Intel Hyper-Threading Technology (HT Technology) that is enabled by default. In the pre-cloud world, this was usually performed by modifying the BIOS. That turned off HT Technology for all users, regardless of any possible benefits obtainable, […]

Read More

Automating the Creation of Consistent Amazon EBS Snapshots with Amazon EC2 Systems Manager (Part 2)

Nicolas Malaval, AWS Professional Consultant In my previous blog post, I discussed the challenge of creating Amazon EBS snapshots when you cannot turn off the instance during backup because this might exclude any data that has been cached by any applications or the operating system. I showed how you can use EC2 Systems Manager to […]

Read More

Automating the Creation of Consistent Amazon EBS Snapshots with Amazon EC2 Systems Manager (Part 1)

Nicolas Malaval, AWS Professional Consultant If an EC2 instance is up and running, there may be applications working, like databases, with data in memory or pending I/O operations that cannot be retrieved from an Amazon EBS snapshot. If your application is unable to recover from such a state, you might lose vital data for your […]

Read More

Building a Backup System for Scaled Instances using AWS Lambda and Amazon EC2 Run Command

Diego Natali, AWS Cloud Support Engineer When an Auto Scaling group needs to scale in, replace an unhealthy instance, or re-balance Availability Zones, the instance is terminated, data on the instance is lost and any on-going tasks are interrupted. This is normal behavior but sometimes there are use cases when you might need to run […]

Read More

Distributed Deep Learning Made Easy

This is a guest post from my colleagues Naveen Swamy and Joseph Spisak. ——————————— Machine learning is a field of computer science that enables computers to learn without being explicitly programmed. It focuses on algorithms that can learn from and make predictions on data. Most recently, one branch of machine learning, called deep learning, has […]

Read More

Real World AWS Scalability

This is a guest post from Linda Hedges, Principal SA, High Performance Computing. —– One question we often hear is, “How well will my application scale on AWS?” For high performance computing (HPC) workloads that cross multiple nodes, the cluster network is at the heart of scalability concerns. AWS uses advanced Ethernet networking technology, which, […]

Read More

Optimizing Joining Windows Server Instances to a Domain with PowerShell in AWS CloudFormation

Scott Zimmerman AWS Solutions Architect Deploying applications with Active Directory, including Microsoft SharePoint Server and custom .NET applications, can take several minutes and possibly even hours. My colleague Julien Lépine wrote an excellent post, Optimize AWS CloudFormation Templates, about parallelizing Amazon EC2 instance creation in AWS CloudFormation. His post shows a general technique for using […]

Read More

Powering your Amazon ECS Clusters with Spot Fleet

My colleague Drew Dennis sent a nice guest post that shows how to use Amazon ECS with Spot fleet. — There are advantages to using on-demand EC2 instances. However, for many workloads, such as stateless or task-based scenarios that simply run as long as they need to run and are easily replaced with subsequent identical […]

Read More