AWS Open Source Blog

User uploads data in BIDS format to S3 and starts the Lambda function → Lambda parses the uploaded data and launches a cluster of EC2 instances → EC2 instances run fMRIprep which preprocesses the data → preprocessed data are saved to S3.

fMRI data preprocessing on AWS using fMRIprep

A typical fMRI study often produces imaging data of terabytes or more. Storing and preprocessing this data can be challenging on a single computer because it often has neither enough disk space to store the data nor enough computing power to preprocess it. Traditionally, researchers use a combination of cloud-based storage and on-premises high-performance clusters […]

Read More
VMD over NICE DCV.

Deploying an HPC cluster and remote visualization in a single step using AWS ParallelCluster

Since its initial release in November 2018, AWS ParallelCluster (an AWS-supported open source tool) has made it easier and more cost effective for users to manage and deploy HPC clusters in the cloud. Since then, the team has continued to enhance the product with more configuration flexibility and enhancements like built-in support for the Elastic […]

Read More
Open Distro for Elasticsearch logo.

Introducing real-time anomaly detection in Open Distro for Elasticsearch

September 8, 2021: Amazon Elasticsearch Service has been renamed to Amazon OpenSearch Service. Visit the website to learn more. There is an enormous increase in real-time streaming applications across a wide range of industries such as finance, health, information technology, retail, and the Internet of Things (IoT). Organizations depend on log analytics solutions to detect aberrations […]

Read More

AWS Fargate container logs collection and analysis with AWS FireLens and Sumo Logic

AWS Fargate is a compute engine for Amazon ECS that allows you to run containers without having to manage servers or clusters. Fargate manages provisioning, configuration, and scaling of the clusters. With Fargate, you can focus on your application design and implementation instead of worrying about the infrastructure. In this post, we’ll provide an overview […]

Read More
Figure 1: Sumo Logic EKS Integration

The Sumo Logic integration with Amazon EKS

Amazon Elastic Kubernetes Service (Amazon EKS) makes it easy to deploy, manage, and scale containerized applications using Kubernetes on AWS. In this post, we’ll provide an overview on how Sumo Logic’s integration with Amazon EKS works using the open source tools Helm, Fluent Bit, Fluentd, Prometheus, and Falco, and how to use it to: Monitor the […]

Read More

Splitting an application’s logs into multiple streams: a Fluent tutorial

Not all logs are of equal importance. Some require real-time analytics, others simply need to be stored long term so that they can be analyzed if needed. In this tutorial, I will show three different methods by which you can “fork” a single application’s stream of logs into multiple streams which can be parsed, filtered, […]

Read More

Gearing up for re:Invent 2019 with Open Distro for Elasticsearch sessions

re:Invent 2019 has a new track this year and it’s all about Open Source! There are lots of great sessions coming up on Open Distro for Elasticsearch and its components such as Alerting, Security, and Performance Analyzer. Join in to learn more and participate in hands-on workshops! Keep a lookout for our sessions on machine […]

Read More
Robotics at re:Invent 2019.

Robotics at re:Invent 2019

  For the first time, at re:Invent 2019, we will have dedicated Robotics and Open Source tracks! We’re excited to be working closely with the open source community to make robotics easier than ever for everyone. Throughout the week, we are offering a diverse variety of content and learning opportunities about the open source Robot Operating System […]

Read More