AWS Open Source Blog

Category: Analytics

pickup – stock.adobe.com

Leverage deep learning in Scala with GPU on Spark 3.0

This post was contributed by Qing Lan, Carol McDonald, and Kong Zhao. With the growing interest in deep learning (DL), more users are using DL in their production environments. Because DL requires intensive computational power, developers are leveraging GPUs to do their training and inference jobs. As part of a major Apache Spark initiative to […]

Read More
AWS Open Source logo.

Stepping up for a truly open source Elasticsearch

Last week, Elastic announced they will change their software licensing strategy, and will not release new versions of Elasticsearch and Kibana under the Apache License, Version 2.0 (ALv2). Instead, new versions of the software will be offered under the Elastic License (which limits how it can be used) or the Server Side Public License (which […]

Read More
lead image: a diagram that explains the interaction between the individual components

Using open source FHIR APIs with FHIR Works on AWS

In September 2019, we published a blog post, Building a Serverless FHIR Interface on AWS, which explained why customers might want to use FHIR (Fast Healthcare interoperability Resources) as a healthcare interface, and why serverless technology is a cost-efficient and flexible approach to these interfaces. The post explained the basic concepts underpinning FHIR, and how […]

Read More

The Instaclustr sign of open source success

In a 2001 interview, Brian Behlendorf, then president of the Apache Software Foundation, was asked to identify the most foundational right in open source—the thing that, if removed, makes open source not open source. His response? “The right to fork.” Of course Behlendorf wasn’t riffing on flatware, but rather “the right for a user or […]

Read More
Open Distro for Elasticsearch logo.

Launching Open Distro for Elasticsearch security features on Amazon Elasticsearch Service

We are excited to announce that we are making new Open Distro for Elasticsearch security features available on Amazon Elasticsearch Service. Amazon Elasticsearch Service is frequently used for sensitive enterprise workloads, and today’s launch adds multiple capabilities to give you even tighter control over your data. New features include the ability to use roles to […]

Read More
Andy Jassy giving the 2019 re:Invent keynote.

re:Cap part one – open source at re:Invent 2019

As the dust settles after another re:Invent closes, I wanted to put together a quick summary of all the open source-related announcements that happened in the run up to this year’s re:Invent and the week itself. If you are interested in open source in mobile web development, devops, containers, security, big data and data analytics, […]

Read More
Open Distro for Elasticsearch logo.

Introducing real-time anomaly detection in Open Distro for Elasticsearch

  There is an enormous increase in real-time streaming applications across a wide range of industries such as finance, health, information technology, retail, and the Internet of Things (IoT). Organizations depend on log analytics solutions to detect aberrations in the data and identify critical situations. Examples include finding fraudulent behavior in financial transactions, discovering suspicious IP […]

Read More

Gearing up for re:Invent 2019 with Open Distro for Elasticsearch sessions

re:Invent 2019 has a new track this year and it’s all about Open Source! There are lots of great sessions coming up on Open Distro for Elasticsearch and its components such as Alerting, Security, and Performance Analyzer. Join in to learn more and participate in hands-on workshops! Keep a lookout for our sessions on machine […]

Read More

Deploying Spark jobs on Amazon EKS

UPDATE, March 2021: This blog post describes how to deploy self-managed Apache Spark jobs on Amazon EKS. AWS now provides a fully managed service with Amazon EMR on Amazon EKS. This new deployment option allows customers to automate the provisioning and management of Spark on Amazon EKS, and benefit from advanced features such as Amazon […]

Read More
Lucene in action Amazon search screenshot.

What Amazon gets by giving back to Apache Lucene

  At pretty much any scale, search is hard. It becomes dramatically harder, however, when searching at Amazon scale: think billions of products, complicated by millions of sellers constantly changing those products on a daily basis, with hundreds of millions of customers searching through that inventory at all hours. Although Amazon has powered its product […]

Read More