Data-Based Global Market Prediction Using Brightics AI with AWS

By Woosik Kong, Solution Engineer – Samsung SDS
By Kyunghwan Kim, Solution Marketer – Samsung SDS
By Heungsik Kim, Solution Consultant – Samsung SDS
By Lalit Mangla, Sr. Partner Solution Architect – AWS

Samsung SDS

Strategic sales and marketing actions through market opportunities and risk forecasting can be a big challenge for enterprises.

This is often because of issues forecasting market size changes and short-term market share due to rapid environmental changes, including those that could not be expected such as the impact of the pandemic, conflicts between countries, and intensifying competition among manufacturers.

It has become increasingly important to build an artificial intelligence (AI)-based objective market forecasting process by forming a single analysis platform using flexible resources and the latest AI algorithms.

These are strengths of public cloud technologies, and enablement via intuitive forecasting models is the solution using proven pre-built models.

Enterprises are looking to achieve the following key business objectives:

Improved market share prediction accuracy.
Flexible resource acquisition and cost reduction.
Better platform usability with administrator features.
Stable service operations.

Samsung SDS is an information and communications technologies (ICT) arm of the Samsung Group. Samsung SDS solutions have been leading the digital transformation and innovation of clients for over 35 years across a wide range of industries.

With operations in 40 countries, Samsung SDS’s solutions utilize advanced analytics platforms, AI, and blockchain technologies to serve a diverse range of industries including financial services, smart manufacturing, global logistics, and retail.

Samsung SDS’s vision is to become a data-driven digital transformation leader by leveraging the most advanced ICT technologies and solutions to discover actionable insights. Samsung SDS is an AWS Advanced Tier Services Partner and Managed Cloud Services Provider (MSP) with the AWS Security Competency.

In this post, we will present the architecture for how a multi-national electronic manufacturing company established a model development and analysis platform using Brightics AI on Amazon Web Services (AWS). The customer used Brightics AI from Samsung SDS to predict markets and establish sales strategies through AI models.

Proposed Solution and Technical Architecture

The multi-national electronics manufacturing company reviewed various public cloud service offerings that could provide a market share prediction and sales marketing decision making solution for their customers around the world.

In particular, the customer was considering a public cloud that provides infrastructure stability, flexible resource use structure, security, and technical support for professionals. They selected AWS without hesitation because of the platform advantages it provides.

The customer established a model development and analysis platform using Brightics AI to use machine learning (ML) models to predict markets and establish sales strategies. The predictive model can be advanced further based on business requirements.

Brightics AI provides services from the perspective of an integrated analysis environment by containerizing open sources such as analytic functions, user interfaces (UI), and Apache Hadoop, Spark, Python, and R developed in a Kubernetes cluster.

In Brightics AI , users analyze data through self-developed function by Samsung SDS or open source. Metadata is saved to PostgreSQL, and resulting data is automatically saved to Hadoop and Redis. Brightics AI on AWS leverages services provided by AWS to provision solutions, including:

Amazon Elastic Kubernetes Services: Amazon EKS provides the container orchestration environment.
Amazon Elastic Container Registry: Amazon ECR provides the Python virtual environment image repository.
Amazon Elastic Compute Cloud: Amazon EC2 instances are created with AWS auto scaling group and used by EKS nodes.
Elastic Load Balancer: ELB is used as a load balancing gateway to efficiently forward user requests to Kubernetes ingress. Efficient load balancing is possible with a multi-Availability Zone (AZ) auto scaling group for incoming traffic.
Amazon Elastic File Service: Amazon EFS provides a persistent volume disk where Brightics AI analytics data (including Hadoop) are stored.
Amazon Elastic Block Store: Amazon EBS provides the operating system (OS) area used by EC2, EKS node, and local storage space for the Brightics AI app image, including the Python virtual environment image and various metadata, as well as Brightics analysis data. Basic OS and temp image are also stored.
Amazon Route 53: Provides global domain access to Brightics AI .

Additionally, Brightics AI can be driven and analyzed by using Amazon Simple Storage Service (Amazon S3), which leverages data analysis and storage through S3-related functions, and Amazon Relational Database Service (Amazon RDS), which utilizes data analysis and storage from an analysis point of view.

The following diagram is the Brightics AI solution architecture on AWS.

Figure 1 – High-level view of end-to-end architecture.

Here’s the flow of AWS architecture components from a user’s point of view:

Amazon Route 53 access when accessing the user.define.domain URL.
Call Elastic Load Balancer with mapping information registered in Route 53.
Ingress call in Amazon EKS cluster created as private subnet with mapping information registered in ELB.
EKS ingress calls the Brightics AI service pod launched in EKS according to each API.
Invoke your own or external storage/analysis pods or services (Hadoop, Spark, PostgreSQL) from Brightics AI pod depending on installation option.
After saving the analysis result in Hadoop, the result is stored in Redis cache and user result feedback.
When data outside the virtual private cloud (VPC) or external analytics package is required, additionally configure the NAT Gateway of the public subnet to configure communication between the private subnet of Brightics AI and the customer/outside network.

The configuration example of the components of the Brightics AI solution is as follows.

Figure 2 – Brightics AI cluster.

Brightics AI on AWS Capabilities

Brightics AI on AWS extends the following capabilities:

By using Amazon EKS as the Brightics AI running environment, version control is possible and Kubernetes control plane availability and scalability can be achieved.
Easy to expand Hadoop and file shared storage by using Amazon EFS as Brightics AI solution storage. By using an auto scaling group, it’s possible to secure availability by specifying CPU threshold and multi-AZ flexibly in preparation for sudden increase in analysis work.
Brightics AI solution database can be used with Amazon RDS, and an external database can be used through separate DB connection settings such as JDBC driver.
Easy to manage user access and load balancing with Amazon Route 53 and Elastic Load Balancer.
Using Amazon ECR for Brightics AI’s image registry to manage permissions through AWS Identity and Access Management (IAM) authentication and maintain high availability

Brightics AI can be installed through AWS Marketplace. Additionally, in the case of images provided in AWS Marketplace, detailed inspection of security vulnerabilities has been completed through enhanced scanning.

The installation script is also user-friendly, and version updates such as EKS and NGINX ingress controller are reflected periodically.

The process of installing Brightics AI from AWS Marketplace is as follows:

AWS console
- Download user manual PDF from the “Usage” tab of the Brightics AI product screen.
- Create a role for Brightics installation by referring to the PDF manual.
- (Optional) Create a key-pair for server-to-server communication.
- Create a bastion Amazon EC2 server.
Linux
- Access bastion EC2 and download the installation file.
- Unzip the installation file and run Installer.sh (specifications can be changed with parameter value of shell).
- Confirm installation is complete and connect to the domain.

Tools and Technologies Used

Development of a market forecasting model:

Application of AI/ML prediction framework.
Early improvement of prediction reliability through prediction methodology and data scientist collaboration.

AWS public-based integrated analysis platform configuration:

Flexible resource structure and state-of-the-art neural network algorithm utilization of AWS.
Stable cloud environment setting and operation supporting professional technical personnel.

AI/ML engineer support and stable operation:

Early stabilization of operation by deploying experts in analysis platforms and operating pilot services.
Secure model continuity by configuring model operation process and manpower.

Lessons Learned

The application of Brightics AI-based predictive models automatically detects major variables and optimal factors that affect the market, and accurately predicts market changes in various regions/country in advance.

For the multi-national electronics manufacturing company, collaboration between data scientists was easy. The operation and management were effectively carried out to enable model development and operation application in a short period of time.

With AWS, it was possible to flexibly secure the necessary resources when adjusting the scale. It was also easy to reduce costs, improve security, and process authority to respond quickly and safely in various situations.

In addition, as stable service operation became possible, better analysis results were achieved by focusing on business factors instead of infrastructure factors.

In carrying out the market share short-term prediction project, high prediction accuracy was derived because it was carried out in accordance with systematic analysis processes. This included requirement derivation, model development, customer review and automation, model application and operation, and monitoring.

Even though it was not easy to predict short-term market share in line with rapidly changing market sizes, one of the main factors that allowed the team to apply forecast models and operate stably was AWS’s public cloud environment.

Since Samsung SDS was able to flexibly secure and utilize as many resources as needed, the solution reduced infrastructure and operation costs and responded stably in various environments.

Conclusion

Many business decisions depend on access to near real-time data and you’re facing challenges such as data latency, high data quality, system reliability, and compliance with data privacy regulations. Brightics AI from Samsung SDS accelerates data delivery in large-scale and near real-time applications that require incremental data pipelines and processing.

To learn more, contact Samsung SDS for help with AWS infrastructure requirements.

.

.

Samsung SDS – AWS Partner Spotlight

Samsung SDS is an AWS Advanced Tier Services Partner and Managed Service Provider (MSP). Its solutions have been leading the digital transformation and innovation of clients for over 35 years across a wide range of industries.

Contact Samsung SDS | Partner Overview | AWS Marketplace

AWS Partner Network (APN) Blog