Sign in
Categories
Your Saved List Partners Sell in AWS Marketplace Amazon Web Services Home Help

SparkMLlib + Support

By: ATH Infosystems Latest Version: 3.2.1
Linux/Unix
Linux/Unix

Product Overview

An AWS product Spark Mllib Hadoop Scala powered by ATH Infosystems. MLlib is Spark's machine learning library, focusing on learning algorithms and utilities, including classification, regression, clustering, collaborative filtering, dimensional reduction, as well as underlying optimization primitives.

We are launching a product which will configure and publish Spark MLlib, an open source software solution which is embedded per-configured tool with Ubuntu OS and ready-to-launch AMI on Amazon EC2 that contains Spark MBlib, Hadoop 2.7, Scala, Linux, PHP (LAMP).

MLlib fits into Spark's APIs and interoperates with Scala. You can use any Hadoop data source (e.g. HDFS, HBase, or local files), making it easy to plug into Hadoop workflows.

Why MLlib? It is built on Apache Spark, which is a fast and general engine for large scale processing. Supposedly, running times or up to 100x faster than Hadoop Map Reduce, or 10x faster on disk. Supports writing applications in Java, Scala, or Python.
MLlib contains many algorithms and utilities

Classification: logistic regression, naive Bayes
Regression: generalized linear regression, survival regression
Decision trees, random forests, and gradient-boosted trees
Recommendation: alternating least squares (ALS)
Clustering: K-means, Gaussian mixtures (GMMs)
Topic modeling: latent Dirichlet allocation (LDA)
Frequent item sets, association rules, and sequential pattern mining
MLlib will still support the RDD-based API in spark.mllib with bug fixes.
MLlib will not add new features to the RDD-based API.
In the Spark 2.x releases, MLlib will add features to the Data Frames-based API to reach feature parity with the RDD-based API.
After reaching feature parity (roughly estimated for Spark 2.2), the RDD-based API will be deprecated.
The RDD-based API is expected to be removed in Spark 3.0.
Data Frames provide a more user-friendly API than RDDs. The many benefits of Data Frames include Spark Data sources, SQL/DataFrame queries, Tungsten and Catalyst optimizations, and uniform APIs across languages.
The Data Frame-based API for MLlib provides a uniform API across ML algorithms and across multiple languages.
Data Frames facilitate practical ML Pipelines, particularly feature transformations. See the Pipelines guide for details.
Data types
Classification and regression
Collaborative filtering
Clustering
Dimensional reduction
Feature extraction and transformation

Version

3.2.1

Operating System

Linux/Unix, Ubuntu 20.04LTS

Delivery Methods

  • Amazon Machine Image

Pricing Information

Usage Information

Support Information

Customer Reviews