Streamlining Software Development with Data-Driven Insights and Automation Using Apexon COMPASS

By Tanmay Baxi, Practice Head, Cloud & Platform Engineering – Apexon
By Apurv Doshi, Practice Head, Labs, Innovation and R&D, – Apexon
By Shilpa Tyagi, Sr. Partner Management Solutions Architect – AWS

Apexon

Managing software development activities can be complex for enterprises, especially with the need for a single dashboard that provides insights into business, process, and people-centric metrics.

Apexon’s COMPASS solution leverages machine learning (ML) to capture and integrate continuous data and intelligence across the software development lifecycle, providing a comprehensive view of resource performance and its impact on key performance indicators (KPIs).

By streamlining test automation and digital maturity initiatives, COMPASS helps organizations optimize software development activities, improve quality and compliance, and drive business success through actionable insights and automation.

Apexon is an AWS Advanced Tier Services Partner and AWS Marketplace Seller with a digital-first technology mindset. Apexon leverages extensive Amazon Web Services (AWS) expertise to provide enterprises with speed, agility, and performance.

The Challenge

Digitization has presented significant challenges for software development operations. A unified view of progress across multiple projects is crucial for accurate project status and timely completion.

Predictive insights are needed to identify potential patterns and trends, allowing organizations to take preventive action before issues arise. In addition to progress tracking, there’s a growing need for predictive insights to prevent potential issues before they arise. This requires advanced analytics and data-driven decision-making capabilities that can identify potential patterns and allow organizations to take preventive actions.

It’s a real challenge to gain such insights from decentralized, fragmented, and multi-sourced data.

Apexon COMPASS

Apexon COMPASS is a comprehensive data analytics solution that automates the extraction of valuable insights from disparate sources, including issue trackers, code version management systems, code quality control tools, CI/CD platforms, and project management tools.

The solution follows a structured process of data collection, sanitization, and aggregation to extract meaningful insights, which are then presented through a unified dashboard system.

Figure 1 – Conceptual diagram of Apexon COMPASS.

The solution is comprised of several key components, including the project onboarding module, data collectors, data synthetization engine, data visualization microservices, and prediction and prescription models. Each component plays a crucial role in enabling COMPASS to deliver accurate and actionable insights.

Project Onboarding

Apexon COMPASS features a sophisticated project onboarding utility that offers both web and mobile interfaces to enable project managers to onboard existing and running projects on the platform with ease.

The utility provides a user-friendly interface that enables project managers to configure project details, including credentials to access various data sources, field mappings, access rights for intended stakeholders, data fetch intervals, and other governance details.

Figure 2 – AWS architecture for COMPASS project onboarding module.

The project onboarding utility is built on top of Amazon Simple Storage Service (Amazon S3), which is utilized to host static resources such as HTML, CSS, and JavaScript files.

Additionally, the utility leverages a microservice architecture to provide overall coordination of the onboarding process. The microservices are deployed using Amazon Elastic Kubernetes Service (Amazon EKS), which provides a scalable and secure platform for containerized applications.

COMPASS Application Architecture

The Apexon COMPASS architecture uses the following AWS services:

Amazon CloudWatch: Triggers the data collectors to fetch the data from the different sources.
AWS Lambda: Synthesizes the data to unified data models (UDM).
Amazon S3: Creates raw data lakes and stores ML models.
Amazon EKS: For data collectors and visualization microservice deployment.
AWS Secret Manager: To manage, retrieve, and rotate passwords, API keys, and other sensitive information.
Amazon SQS: Simple queue service for queuing of the new arrival of messages before serverless takes over and populates the UDM.
Amazon SageMaker: Trains ML models for the predictive and prescriptive analysis of ingested data.

Figure 3 – AWS architecture for COMPASS.

Data Collectors and Synthetization

The Apexon COMPASS data collectors are a suite of microservices running on an Amazon EKS cluster that periodically collect data from various data sources. These data collectors utilize change data capture (CDC) techniques to fetch incremental data from the sources, enabling Apexon COMPASS to provide up-to-date and accurate insights.

Once the data is collected, it’s stored in the raw data lake area for further processing. Apexon COMPASS supports data collection from a wide range of project management tools, including Jira, Asana, Zoho, and others.

The solution also supports data collection from source code management tools such as Git, Bitbucket, GitLab, and code quality analysis tools like SonarQube, as well as other proprietary internal tools.

The data synthetization component of Apexon COMPASS is a collection of serverless tasks that are triggered upon the arrival of new data in the raw data lake. These tasks are responsible for processing the data and converting it into a UDM, which enables Apexon COMPASS to provide comprehensive and accurate insights into software development projects.

Data Visualization

The data visualization component of Apexon COMPASS is a suite of microservices running on an Amazon EKS cluster. These are responsible for fetching data from various aggregation stores created on top of the UDM, enabling organizations to visualize and analyze software development data in a comprehensive and intuitive way.

Prediction

Apexon COMPASS includes two key components for predictive and prescriptive analysis: the prediction microservice and the prescription modules. The prediction microservice leverages past release data and project management data from project management tools to predict risk levels. It also utilizes static code analyzer data and software configure management (SCM) metadata along with project management data to predict the bug range of product modules for the upcoming release.

Apexon COMPASS uses project management, source code, and quality control data to make predictions and prescriptions. This data may be incomplete, but it uses natural language processing (NLP)-based models to tag the missing components, statistical models to tag the missing release versions, and other ML models to make the data complete.

Anomaly detection, data balancing, extra, transform, and load (ETL), and data splitting are used to check and balance the data. Apexon COMPASS uses pre-trained and active learning ML models to classify project modules into high, medium, and low-risk categories based on features such as release version, module, number of blockers, and critical, major, and minor issues.

The major advantages of classification output are:

Managing risk: Helps identify high-risk modules, making it easier for managers to make informed decisions.
Tracking module journey: Teams can track modules over time to identify any changes in risk profile and allocate resources appropriately.
Identifying skilled team members: Correlating risk profiles with team members helps teams allocate resources more effectively and improve overall software quality.

Apexon COMPASS regression-based models use input features such as planned work (story points), available resources (release date, team members), requirement complexity, and historical module stability to predict the number and severity of issues that may occur in an upcoming release. These models are trained on past release data using proprietary algorithms.

To use the models for the upcoming release, the project manager or product owner provides planned work, available resources, and the complexity of the planned changes.

The major advantages of regression output are:

Managing expectations: Helps set realistic expectations by identifying any limitations or issues that may affect user experience.
Prioritizing development efforts: By identifying modules or features that are more prone to bugs, developers can prioritize their efforts and allocate resources accordingly.
Implementing a robust testing strategy: A comprehensive testing strategy helps catch and address issues early on, minimizing the risk of bugs and errors that could impact user experience.

Prescription

The prescription modules provide recommendations for optimizing test suites, allocating resources, and determining ideal durations to burn out planned story points. By providing prescriptive guidance, organizations can optimize their workflows and ensure maximum efficiency throughout the development process.

The Apexon COMPASS recommendation engine carefully observes the past data and predictions generated for upcoming releases. It comes with the following recommendations:

Regression testing suite: Recommends an ideal test suite for daily and weekly regression cycles, ensuring critical functionality is thoroughly tested without unnecessary overheads.
Task allocation: Recommends team members who are most appropriate for crucial tasks, improving efficiency and productivity.
Story points estimation: Helps estimate the number of story points that are ideal for a given time range or release cycle, helping teams prioritize their work.
Refactoring: Identifies modules that require immediate refactoring, improving software quality and maintainability.
Process improvement: Identifies areas where process adherence can be improved, leading to better efficiency and quality.

Conclusion

Apexon COMPASS brings complete visibility and control over your software development projects with cutting-edge analytics and data-driven decision-making capabilities. The solution’s unified view of progress across multiple projects helps ensure accurate project status and timely completion.

Extended access of Apexon COMPASS dashboards to client helps build healthy and transparent relationships. With Apexon’s advanced predictive insights, you can identify potential issues before they arise and take preventive action, saving time and resources.

Say goodbye to the challenges of decentralized, fragmented, and multi-sourced data—Apexon COMPASS provides a single, integrated solution for all your development operations needs.

You can also learn more about Apexon in AWS Marketplace.

.

.

Apexon – AWS Partner Spotlight

Apexon is an AWS Advanced Tier Services Partner that leverages technical expertise to ensure organizations take full advantage of AWS capabilities.

Contact Apexon | Partner Overview | AWS Marketplace | Case Studies