Listing Thumbnail

    Smart Tests: AI Co-Pilot for Test Intelligence & Optimization

     Info
    Sold by: CloudBees 
    Deployed on AWS
    Transform your software testing with Smart Tests by CloudBees, the AI-augmented tool revolutionizing test impact analysis. Easily integrate it into your workflow with GitHub Actions or test runner plugins like pytest and nose. Smart Test's command-line interface supports a wide range of test runners, including Android Debug Bridge, Ant, Bazel, and more. It intelligently evaluates code changes to run only the most relevant tests, drastically reducing testing time and allowing you to focus on innovation. Setup takes under an hour with a simple four-line code, offering a maintenance-free experience. Customers have saved over 2,000 hours monthly, cut machine hours by 50%, slashed test execution times by 90%, and reduced build times by 40%.
    4.2

    Overview

    Smart Tests is the AI co-pilot for Test Suite Intelligence and Optimization.

    Shorter test runs, smarter test triaging, test health insights, and test failure analysis.

    Tests are one of the biggest bottlenecks in the Dev-Test workflow. Teams face headwinds trying to release high quality products while maintaining swift delivery cycles. With Smart Tests, remove friction from the dev-test loop to iterate & ship faster.

    The Smart Tests suite of AI testing and optimization tools help teams analyze, optimize, and accelerate the pace and precision of testing and improvement.

    Smart Tests helps teams effectively manage and analyze test failures. It offers AI-driven tools for faster root-cause analysis, summarizing complex error logs, and providing actionable insights into test failures. This capability reduces the manual effort involved in identifying and fixing issues, thereby improving overall productivity.

    Smart Tests accelerates the development and testing process by optimizing test runs, triaging test failures intelligently and providing insights into test health. This helps teams iterate and ship software faster by reducing the time spent on unnecessary tests and inefficient workflows.

    Smart Tests improves visibility into test suite performance and test health. It includes features like personalized notifications for test failures via GitHub and Slack, dashboards with charts and analytics, and the ability to prioritize issues based on severity and impact. This ensures that teams can focus their efforts on fixing critical issues quickly and efficiently.

    Highlights

    • Intelligent Test Failure Diagnostics Uses GenAI to summarize and group issues and drives instant clarity on underlying software issues raised by test failures. Smart Tests transforms complex, voluminous error logs into succinct summaries to quickly identify the crux of issues. Provides actionable insights on Day 1
    • Predictive Test Selection Find failures up to 80% faster. Uses ML to train on build and test metadata to help you run less tests based on commits. Eliminate long runtimes for tests that slow feedback signal to developer and QA teams. Know which tests matter, automatically test upon code-changes, find failures faster and prioritize them. Correlate code changes to tests to run an ML-based subset to find failing builds in a fraction of the time.
    • Test Failure Highlights and Notifications Notify engineers of build and test failures caused by them directly in GitHub PR. Get to problems faster with personalized notifications when tests fail. Identify and deep dive into Flaky and Unhealthy tests.

    Details

    Delivery method

    Deployed on AWS
    New

    Introducing multi-product solutions

    You can now purchase comprehensive solutions tailored to use cases and industries.

    Multi-product solutions

    Features and programs

    Buyer guide

    Gain valuable insights from real users who purchased this product, powered by PeerSpot.
    Buyer guide

    Financing for AWS Marketplace purchases

    AWS Marketplace now accepts line of credit payments through the PNC Vendor Finance program. This program is available to select AWS customers in the US, excluding NV, NC, ND, TN, & VT.
    Financing for AWS Marketplace purchases

    Pricing

    Smart Tests: AI Co-Pilot for Test Intelligence & Optimization

     Info
    Pricing is based on the duration and terms of your contract with the vendor. This entitles you to a specified quantity of use for the contract duration. If you choose not to renew or replace your contract before it ends, access to these entitlements will expire.
    Additional AWS infrastructure costs may apply. Use the AWS Pricing Calculator  to estimate your infrastructure costs.

    12-month contract (2)

     Info
    Dimension
    Description
    Cost/12 months
    Launchable - User (Git committer)
    One User (Git committer) - Test Suite Intelligence and Optimization
    $120.00
    Launchable - Test Suite
    One Test Suite - Test Suite Intelligence and Optimization
    $15,000.00

    Vendor refund policy

    Refund Policy No refunds allowed.

    How can we make this page better?

    Tell us how we can improve this page, or report an issue with this product.
    Tell us how we can improve this page, or report an issue with this product.

    Legal

    Vendor terms and conditions

    Upon subscribing to this product, you must acknowledge and agree to the terms and conditions outlined in the vendor's End User License Agreement (EULA) .

    Content disclaimer

    Vendors are responsible for their product descriptions and other product content. AWS does not warrant that vendors' product descriptions or other product content are accurate, complete, reliable, current, or error-free.

    Usage information

     Info

    Delivery details

    Software as a Service (SaaS)

    SaaS delivers cloud-based software applications directly to customers over the internet. You can access these applications through a subscription model. You will pay recurring monthly usage fees through your AWS bill, while AWS handles deployment and infrastructure management, ensuring scalability, reliability, and seamless integration with other AWS services.

    Resources

    Vendor resources

    Support

    Vendor support

    CloudBees Smart Tests Standard CloudBees Gold Support Level via web and email Free online support forums support@cloudbees.com  https://support.cloudbees.com 

    Contact sales for more information:

    AWS infrastructure support

    AWS Support is a one-on-one, fast-response support channel that is staffed 24x7x365 with experienced and technical support engineers. The service helps customers of all sizes and technical abilities to successfully utilize the products and features provided by Amazon Web Services.

    Similar products

    Customer reviews

    Ratings and reviews

     Info
    4.2
    20 ratings
    5 star
    4 star
    3 star
    2 star
    1 star
    45%
    50%
    5%
    0%
    0%
    7 AWS reviews
    |
    13 external reviews
    External reviews are from PeerSpot .
    KishoreKumar4

    Consistent automation has improved deployments and supports efficient client-facing workflows

    Reviewed on Apr 07, 2026
    Review from a verified AWS customer

    What is most valuable?

    Deployment and other processes are feasible for me, making this a smooth process. The integration part is good. We can use multiple things by integrating with CI/CD pipelines, and this is very feasible for us. This allows us to do our work easily. The productivity is very good. CloudBees  is a good and efficient tool. We can work in client-facing scenarios, and since clients have provided these types of tools, we are able to work on them effectively.

    What needs improvement?

    For CloudBees, I can suggest that the log is very high, which is very hectic for us when trying to find which error is coming. This creates complexity. If possible, we could sort the logs so we can get the exact error on one page.

    For how long have I used the solution?

    I have been using ServiceNow  for the last five plus years.

    What do I think about the stability of the solution?

    This is dependent on our JAR files and other code elements. If the code is large, it takes more time. If the code is less, it takes a short time. The deployment can take a couple of hours or a couple of minutes.

    What other advice do I have?

    We can upload the JAR file and deploy at the Jenkins  level during our deployment. These types of labels and IDs can be deployed using CloudBees. Once we click to start the build, it automatically runs and automatically outputs the result, showing whether it succeeded or failed. If anything failed, we can check the logs in CloudBees, which is very interactive. I would rate this review 9 out of 10.

    Which deployment model are you using for this solution?

    Private Cloud

    If public cloud, private cloud, or hybrid cloud, which cloud provider do you use?

    Amazon Web Services (AWS)
    Lokesh Katamaneni

    Streamlined deployment efficiency with excellent integration and comprehensive integration capabilities

    Reviewed on Oct 09, 2024
    Review provided by PeerSpot

    What is our primary use case?

    The primary use case for CloudBees  is for continuous integration and deployment. It is primarily used for production deployments. We use it to trigger Autobahn, such as CloudBees , well in advance to clear out many issues upfront, trigger change requests, and test in various environments to be ready with the production deployment on the actual date.

    How has it helped my organization?

    CloudBees has helped our organization by integrating with various systems like version one, GitHub , and PDSM. It reduces the need for documentation by automating processes, saving time when creating documents and testing the application. This reduces the amount of time spent on deployments and helps pull out all the test results and create change requests, providing a more streamlined process for deployment.

    What is most valuable?

    The most beneficial aspect is that CloudBees integrates with everything, like version one, GitHub , and PDSM. It automates documentation and testing processes, reducing the need to create documents manually. By creating automated processes, it saves time and effort.

    What needs improvement?

    It could improve the document upload process. Currently, documents need to be uploaded manually to the PDSM request. An enhancement could be to have a feature where you can upload documents directly to go into the PDSM request.

    For how long have I used the solution?

    For about two to three years, approximately two and a half to three years.

    What do I think about the stability of the solution?

    CloudBees has improved a lot over time. Initially, there were challenges setting it up, but once we understood its functions and processes, there were no stability issues.

    What do I think about the scalability of the solution?

    CloudBees is scalable as it integrates with various complex systems and offers the flexibility to scale to many other application integrations beyond those currently used.

    How are customer service and support?

    I have not personally interacted with customer service or support since the DevOps team handles template creation and maintenance, and they might have contacted support during the initial phase.

    Which solution did I use previously and why did I switch?

    Before CloudBees, we did not use a similar tool. However, we are currently considering a transition to GitHub Actions .

    How was the initial setup?

    The initial setup involved creating a template that checks pull requests in GitHub. It verifies approvals, mergers, and other criteria necessary for a build. The setup includes extensive integration with other systems, stringent checks, and a series of deployment stages from QA to production.

    What about the implementation team?

    The DevOps team is responsible for creating templates and handling integration with various systems. They play a crucial role in supporting the implementation and deployment processes.

    What was our ROI?

    The measurable benefits of using CloudBees include a significant amount of time saved during deployment because of the automation and integration capabilities it offers.

    What's my experience with pricing, setup cost, and licensing?

    I am not aware of the pricing, setup cost, or licensing details for CloudBees.

    Which other solutions did I evaluate?

    We are currently using GitHub Actions , with an effort to transition from CloudBees to GitHub Actions.

    What other advice do I have?

    Follow the guidelines and step-by-step processes thoroughly. If you encounter integration issues with your tools, seek support from CloudBees to ensure integration. Once set up, CloudBees is a reliable tool for deployment.

    I'd rate the solution nine out of ten.

    Vaddi Yashoda Lakshmi

    Improves deployment speed with automation features

    Reviewed on Oct 07, 2024
    Review provided by PeerSpot

    What is our primary use case?

    We use CloudBees for deploying the code in higher environments, such as QA, C2, staging, and production.

    How has it helped my organization?

    CloudBees has made deployment faster compared to on-premise solutions. It also facilitates working with multiple machines, improving efficiency and performance.

    What is most valuable?

    I find the automation feature most valuable. CloudBees is highly scalable and supports both small and large teams. The deployment process is also faster when compared to on-premise.

    What needs improvement?

    There are connection issues with CloudBees, specifically between Sybase and CloudBees. We often encounter connection problems, and there are issues with the pipelines.

    For how long have I used the solution?

    I have been working with CloudBees for one and a half years.

    What do I think about the stability of the solution?

    There are a lot of connection issues with CloudBees. We experience disconnections frequently.

    What do I think about the scalability of the solution?

    CloudBees supports the deployment of large amounts of data efficiently across multiple machines.

    How are customer service and support?

    I haven't gone through the technical support, so I can't comment on it.

    Which solution did I use previously and why did I switch?

    Before CloudBees, we were using Jenkins on-premises. We switched to CloudBees because we moved our SQL servers to the cloud, which made deployment easier.

    How was the initial setup?

    The initial setup of CloudBees took approximately fifteen to twenty minutes.

    What about the implementation team?

    We used UrbanCode deployment to deploy it in different environments, but I was not directly involved in the implementation part.

    What was our ROI?

    I am not aware of the return on investment metrics for CloudBees.

    What's my experience with pricing, setup cost, and licensing?

    I am not sure about the pricing, setup cost, and licensing for CloudBees. The team takes care of it.

    Which other solutions did I evaluate?

    I am not aware of any other solutions that were evaluated before choosing CloudBees.

    What other advice do I have?

    I would recommend CloudBees to others because building jobs is much easier than with other solutions.

    Sinthusan Thangarajah

    Streamlined deployment and automation with enhanced efficiency

    Reviewed on Oct 03, 2024
    Review from a verified AWS customer

    What is our primary use case?

    We use a pipeline to deploy our application. All our organization checks and requirements are added to the CloudBees  pipelines, ensuring code quality and checking vulnerabilities. We use Jenkins  and SonarQube  as part of the deployment process.

    How has it helped my organization?

    CloudBees  has streamlined our deployment process. Previously, deployment involved manual file transfers and checks, which were time-consuming. Now, with CloudBees, many tasks are automated, making our work easier and more efficient.

    What is most valuable?

    CloudBees assists by automating tasks, previously done manually, in the pipeline setup. This saves time and improves efficiency. The features set rules that are regularly updated, reducing the need for constant oversight. One-click deployment processes also highlight the flexibility and ease of the platform.

    What needs improvement?

    Sometimes, there are performance issues, however, they may be due to our organization's configuration. Currently, I do not have specific improvements in mind as these might be related to our side.

    For how long have I used the solution?

    Previously, our organization had an on-premise solution, but we have moved to the cloud version.

    What do I think about the stability of the solution?

    Due to security constraints within our organization, the performance can appear slow. It is difficult to discern if the issue is with CloudBees or our organization's configuration.

    What do I think about the scalability of the solution?

    CloudBees is scalable, but I cannot provide specific details on how this scalability impacts us.

    How are customer service and support?

    We experience issues with the CloudBees pipelines occasionally, like resource limits or permission errors. The DevOps team generally resolves these, and I would rate CloudBees support eight out of ten.

    Which solution did I use previously and why did I switch?

    Before CloudBees, we used FTP for deployment, which involved no pipelines.

    How was the initial setup?

    The setup was managed by a separate team. My experience starts with using the product, which was straightforward.

    What about the implementation team?

    A separate team, the DevOps team, manages the CloudBees setup and any technical support needed.

    What was our ROI?

    CloudBees saves time and resources by executing automated deployment processes, which would otherwise have been manual and time-consuming.

    What's my experience with pricing, setup cost, and licensing?

    I am not aware of the pricing, setup cost, or licensing details.

    Which other solutions did I evaluate?

    I have not evaluated other solutions, as my interaction began with CloudBees directly.

    What other advice do I have?

    I would recommend CloudBees, especially to those using traditional ways of deployment, because it simplifies and automates many processes.

    Jonatas L. R. Costa

    Efficient feature deployment with robust DevOps capabilities

    Reviewed on Sep 16, 2024
    Review from a verified AWS customer

    What is our primary use case?

    We use CloudBees  Jenkins  with Jenkins  and Sonar  validation for our pipeline. We use it for all microservices, web system payment integrations, and overall pipeline configurations and deployment processes.

    How has it helped my organization?

    CloudBees  has improved our DevOps practices by allowing us to deploy features and fixes efficiently. It supports our business rules, security analysis, and overall deployment process while simplifying the configuration.

    What is most valuable?

    The most valuable features are Java features, microservice communication, payment validation, Jenkins Sonar , management master to CloudBees, Blue Ocean, JobConfig, and support.

    What needs improvement?

    The user interface of CloudBees is good but could be even more intuitive. Improving the user-friendliness of the interface and having simpler setup configurations would greatly benefit new users.

    For how long have I used the solution?

    I have been working with CloudBees for around two to three years.

    What do I think about the stability of the solution?

    CloudBees is very stable and reliable. From one to ten, I would rate the stability a ten.

    What do I think about the scalability of the solution?

    CloudBees offers high scalability. On a scale from one to ten, I would rate its scalability a ten.

    How are customer service and support?

    Customer service and technical support are excellent. I rate the technical support from one to ten as a ten. They provide fast, simple, and efficient support.

    Which solution did I use previously and why did I switch?

    Before using CloudBees, we used Jenkins SonarQube, Nexus, Azure , and AWS . The decision to switch to CloudBees was made for better integration and deployment capabilities.

    How was the initial setup?

    The initial setup of CloudBees is very easy and straightforward. It takes around 20-30 minutes to deploy completely without any complicated steps.

    What about the implementation team?

    Our configuration team handled the setup and deployment. Twenty people were involved in the deployment and maintenance processes, and they ensured everything worked perfectly without delays.

    What other advice do I have?

    CloudBees provides a robust and efficient solution for our DevOps needs. Ensuring an intuitive user interface and simplifying the setup process can further enhance its usability for new users.

    View all reviews