How MHP SOUNCE Enhances Shopfloor Quality Control by Analyzing Acoustic Anomalies

By Johannes Stoermer, Sr. Consultant – MHP
By Simon Weiß, Sr. Consultant – MHP
By Axel Hodler, Manager, Industrial Cloud Solutions – MHP
By Shankar Subramaniam, Hans Schabert, and Robert Meyer, Partner Solutions Architects – AWS

MHP

Recent technological developments in IT, paired with strained supply chains and labor shortages as well as fluctuating and uncertain market demands, have been putting a high innovation and cost pressure on the manufacturing domain.

Industrial quality management processes make no exception there, as customers continue to demand products and goods that adhere to expectations and fulfil their functionality requirements.

A case in point: the quality control and assurance method is the probing of goods via acoustic analysis of noises emitted by sampling the material under testing. While being considered a proven practice, this procedure usually involves human inspectors which rendered its outcome in many cases dependent on an inspector’s subjective perception. It was also non-repeatable to a certain degree.

Furthermore, due to increasingly complex and automated production processes, as well as the natural limitations of human actors, this approach does not scale well when having to respond to accelerating or slowing cycle times. Ultimately, it can entail high follow-up costs if faulty goods or materials are allowed to stream further downwards a production line due to human error.

In this post, we will show how MHP built an artificial intelligence (AI)-supported acoustic testing solution named MHP SOUNCE, which can be deployed on any manufacturing line or shopfloor-like environment to address the aforementioned challenges.

Leveraging the capabilities of Amazon Web Services (AWS), MHP SOUNCE is able to carry out noise-based quality testing operations in a reliable, scalable fashion, while at the same time being completely modular and cost-efficient.

MHP is an AWS Advanced Tier Services Partner founded in 1996 as a subsidiary of Porsche AG. MHP’s approach comprises management and IT consulting for the automotive sector and beyond, as well as providing in-depth process know-how enabling customers to shape a better future for their business.

The Sound of Defects: Listening to What Faulty Materials Can Tell Us

Here is the high-level flow that includes automatic collection, and detection of defects based on a neural network:

• Data collection: The shopfloor and/or test benches are outfitted with minimally invasive sensor technology that record sound and other relevant data.
• Data refinement: Control inputs data is provided by a quality engineer. This comes as the thresholds and criteria for (non-) faulty material the neural network bases its evaluations on.
• Model training: Data collected using the sensor technology installed on the shopfloor and/or test bench is used to train the deep learning neural network.
• Model inference: Sampled data from goods and materials under testing is monitored for noise anomalies in real-time. A quality engineer can further investigate the reported defects by visualizing and validating the captured acoustic data (with spectrograms, for example).
• Monitoring: Based on the evaluation, a quality engineer provides feedback on the neural network’s detection accuracy, and, in doing so, improves the model over time.

This approach, underlying MHP SOUNCE, allows for a more comprehensive evaluation as opposed to quality control by human quality inspectors, since testing criteria are quantified and consistently checked against.

Furthermore, the solution being fully automated allows for scaling operations up to around-the-clock monitoring, coupled with complete documentation of all test procedures. It further allows evaluating a range of different input variables, which helps in determining the right steps to assure quality levels.

In order to be easily adaptable and not require large upfront installation and deployment costs, MHP SOUNCE has been designed as a modular all-in-one software-as-a-service (SaaS) solution, capable of recording, analyzing, alerting, and documenting.

Therefore, the solution only requires easily installable sensory equipment, as well as a network connection for uploading data and access by users.

Figure 1 shows a screenshot of the MHP SOUNCE dashboard as a user would see it when investigating individual acoustic samples by analyzing spectrograms and acoustic maps generated from captured noise data.

Figure 1 – MHP SOUNCE dashboard with spectrogram analysis.

Based on a pilot deployment for quality testing of car doors in the plant of an internationally operating car manufacturer, MHP observed an increase in manufacturing process stability (-10% reduced processing time in FTE compared to previous process), along with a substantial increase in detection accuracy (> 96% accuracy in correctly determining defects).

MHP SOUNCE was able to reduce manufacturing project costs by 5-10% by identifying defects early on, thus avoiding change costs for processes and tools. This resulted in lower rework costs and less warranty claims.

In scope of the car door quality testing scenario, MHP SOUNCE processed more than 10,000 data points per day and provided a quality indication in under five seconds from first noise recording.

MHP SOUNCE: Instrumenting AWS for Analyzing Sound Patterns

To allow for a seamlessly scalable, highly modular design, while at the same time achieving global reach, MHP SOUNCE leverages the AWS Cloud’s global footprint and point-of-presence network, as well as a variety of serverless technologies.

Figure 2 – MHP SOUNCE architecture.

Figure 2 gives an overview on the solution’s high-level architecture from a principal design point of view:

• Amazon CloudFront: Caches and distributes static content from the MHP SOUNCE dashboard single-page-application (SPA). Additionally, CloudFront speeds up session- and bookkeeping-related API calls made by the SPA dashboard by leveraging an edge-optimized API endpoint, and accelerates data submission by proxying the Application Load Balancer, which serves the containers implementing the business logic.
• AWS WAF: A web application firewall endpoint scrutinizes all traffic that goes in and outside of the application by interfacing with the CloudFront distribution.
• Amazon S3: An S3 bucket holds the SPA implementing the MHP SOUNCE dashboard, which also serves as central entry point for users. Amazon S3 is further used to store BLOBs of recordings and generated images.
• Amazon API Gateway: In order to orchestrate session management and bookkeeping-related calls issued by the SPA dashboard, an Amazon API Gateway serves as façade for the AWS Lambda functions staggered behind it.
• AWS Lambda: An array of Lambda functions serves as session management and bookkeeping-backend for the MHP SOUNCE SPA dashboard.
• Amazon DynamoDB: The Lambda functions use Amazon DynamoDB as a storage layer for session data and bookkeeping.
• Application Load Balancer: The containers bearing the MHP SOUNCE business logic are proxied by an Application Load Balancer to shield them from the remaining components and help implement an independent scaling mechanism.
• Amazon ECS on AWS Fargate: Containers running on Amazon ECS with AWS Fargate hold the main business logic of MHP SOUNCE.
• Amazon RDS for PostgreSQL: An Amazon RDS for PostgreSQL database holds all cycle measurements and metadata, as well as accompanying predictions made by the application.
• Amazon Cognito: Authentication of all actors and components happens via an Amazon Cognito tenant.

The containers holding the backend service, prediction model, and spectrogram visualizer are at the heart of MHP SOUNCE and have been deployed on Amazon ECS with AWS Fargate. This is to offload the undifferentiated heavy lifting of managing infrastructure resources for running containers, as well as to be able to scale dynamically.

Furthermore, the container approach allows for modularly extending the solution; for example, by taking into account input data for quality testing and/or implementing additional prediction models.

By leveraging an Amazon CloudFront distribution, MHP ensures low latency and speedy load-times for the dashboard SPA. At the same time, the CloudFront network allows for a speedy upload of measurement data to the MHP SOUNCE backend via edge-optimized Amazon API Gateway endpoints.

Due to the serverless and managed-service nature of all components involved, MHP SOUNCE achieves a high degree of availability, resiliency, and cost-efficiency.

An exemplary user interaction flow for MHP SOUNCE looks like this:

• Piezoelectric sensory fitted in the floor of an end-of-line quality test-bench continuously records vibration noises and acoustic data (so-called “cycle data”) of an individual good or material under test.
• This cycle data is collected by a recording client, which encodes it as FLAC/WAV file and uploads it to the backend container, together with according metadata like temperature and humidity, which stores the recording in another S3 bucket and writes the metadata to an Amazon RDS for PostgreSQL database.
• A prediction model running in a second container on Amazon ECS with AWS Fargate analyzes the audio recording, makes a prediction based on the stored data, and adds it to the cycle data already residing in the Amazon RDS for PostgreSQL database.
• The cycle’s recording data is then fed to a spectrogram visualization service running in a third container on Amazon ECS with AWS Fargate, which creates a spectrogram picture of the corresponding FLAC/WAV file and stores the BLOB on an S3 bucket.
• Via the MHP SOUNCE dashboard (hosted as SPA on an S3 bucket), users authenticated via Amazon Cognito can access and review a cycle’s collected data and corresponding predictions, along with spectrogram visualizations.
• The SPA dashboard makes calls to the solution’s edge-optimized API Gateway endpoint and Application Load Balancer to dynamically provide content about collected data via calls to AWS Lambda and a backend container, running on Amazon ECS with AWS Fargate.
• Communication between MHP SOUNCE and users/recording clients, runs through a CloudFront distribution with integrated AWS WAF.

A Sound Foundation: Tuning into Even More Use Cases

Having already been deployed at multiple customers in the automotive sector, MHP SOUNCE proved itself a robust and flexible SaaS solution, helping improve and transform quality management processes in test benches and on factory shopfloors.

MHP SOUNCE is being evaluated for further, adjacent use cases like workshops and insurance claim verification. For example, a mechanic in a car repair workshop could approach the root cause of a broken down car by letting MHP SOUNCE analyze its operating noise.

In a less controlled environment such as this, where purchasing and installing dedicated sensors for noise capturing might not make economic sense for smaller players, data collection could be handled by generic recording equipment like a mechanic’s smartphone. The feasibility of this approach is currently being tested by MHP engineers.

Furthermore, MHP SOUNCE is considered for tamper detection scenarios where machinery is mission-critical and security is a top priority. Judging by the sound certain machine parts emit during operation, a warning could be issued if any parts of the audio spectrogram are outside defined limits. This approach is under consideration for supporting the verification of insurance claims by examining a faulty machinery’s historical and current sound patterns to confirm if the break-down occurred as stated by the policyholder.

Conclusion

For customers interested in taking MHP SOUNCE for a test drive with their scenarios, MHP offers the possibility to perform a feasibility study with accompanying demo deployment over a 4-6 week timeframe. If required, individual adjustments like integration of additional third-party components or special use cases can be implemented as well.

Check out the MHP SOUNCE website for the latest developments and/or book a demo deployment. Stay in touch with MHP and learn more about the MHP and AWS collaboration.

.

.

MHP – AWS Partner Spotlight

MHP is an AWS Advanced Tier Services Partner founded in 1996 as a subsidiary of Porsche AG. MHP’s approach comprises management and IT consulting for the automotive sector and beyond, as well as providing in-depth process know-how that enables customers to shape a better future for their business.

Contact MHP | Partner Overview