Selenium Grid in the Cloud

My main use case for Selenium Grid in the Cloud  involves the company I work for, Telstra, which is a large-scale enterprise telecom company. Here  we have many capability teams, and each team has its own automation suite. In order to effectively run this automation suite on their need-to-do basis, we use Selenium Grid in the Cloud  using AWS Fargate  instance, so that each team can use their own container to run their automation scripts without any interference.

I have something else to add about my main use case or something unique about how my team uses Selenium Grid in the Cloud, specifically for doing our automation runs to ensure that the regression suite is intact. The main reason that we use Amazon Fargate is because we feel the APIs are very good to understand the progress of the automation runs.

The best features Selenium Grid in the Cloud offers for my team include containerization, which isolates the run of one team from the other. The maintenance of the instances is pretty easier, so that the team doesn't have to focus more on the infrastructure point of view. Additionally, we ensure that we use resources only on the need-to-go basis, so we are not overusing any resources. When there is no automation run happening, there are no active instances running. This way, we control the cost as well.

Selenium Grid in the Cloud has positively impacted my organization, Telstra, because we provide more importance to security. All the environments in which our automation runs happen are thoroughly scanned by the internal audit team. Using a cloud service like AWS , where we have a proper security mechanism in place, has helped a lot. In terms of productivity, we feel creating images and other tasks on the go has improved the productivity of the run. Cost-wise, most of the resources are used only on our need basis, which has reduced the cost of our QA budget.

There are many improvement areas I feel that Selenium Grid in the Cloud can make.

I have been using Selenium Grid in the Cloud for the past one year, when I joined Telstra.

In my experience, Selenium Grid in the Cloud is stable and has been reliable for my team, as we noticed very few to minimal issues regarding its performance, making us quite satisfied about the tool.

Selenium Grid in the Cloud's scalability has handled increased demand and larger workloads smoothly. It auto-scales or auto-degrades based on the flow, which has definitely helped our costs effectively. There are no blockers during peak release days where the automation runs are effectively used by almost all the teams. It has managed to handle the releases on time without any complaints from teams about the non-availability of instances to run their automation scripts.

Customer support for Selenium Grid in the Cloud has been good. There is a dedicated team that communicates with the cloud provider and they have provided a satisfactory experience.

Negative

I have not had experience using a different solution before Selenium Grid in the Cloud during my time at Telstra. However, in a previous organization, we created a Jenkins  agent node setup within the organization itself, using an in-house distribution with a node and agent connection, resembling a master-slave connection.

I have seen a return on investment, as at least 20% of work has been reduced effectively after going for Selenium Grid in the Cloud, which reduces the man-hours required to maintain and manage tasks. This increase in efficiency allows our team to focus on other aspects of automation, including effectively optimizing the CI/CD pipelines.

I did not evaluate other options before choosing Selenium Grid in the Cloud since I joined Telstra and the infrastructure was already in place, so I did not have the chance to explore other products.

Easy maintenance and resource optimization have helped my team significantly, specifically because the automation is pretty less most of the time. We need to effectively use resources, and in that case, setting up our Selenium Grid in the Cloud becomes the best option since we do not run these regression suites on a daily basis. Teams use them on the need-to-go basis, so whenever there is no automation happening, there will not be any instances running, which definitely helped the cost as we do not use the resources. For debugging purposes, there are proper APIs in place that help to track the failures and get them distributed to the mail distribution list.

The other feature I would like to add about Selenium Grid in the Cloud is the availability of resources. Whenever there is a need for a run, there is a container or instance available for us, which ensures that there is no manual overhead required to maintain the resources. I feel we have an effective infrastructure.

I cannot share specific numbers or estimates about time or money saved, but I can definitely say that productivity has improved by at least 20%. We do not have to spend much of our man-hours to maintain the infrastructure of Selenium Grid in the Cloud, which has reduced the number of people in our R&D team who specifically work on infrastructure and instead focus on other areas including effectively creating a common framework and setting up our CI/CD pipeline using the same.

My advice for others looking into using Selenium Grid in the Cloud is to understand its purpose, grasp the concepts and logging mechanism it has, be aware of pricing, and ensure that it matches your requirements and the pricing policies of your organization.

My main use case for Selenium Grid in the Cloud  is testing the applications that I'm developing in an integrated, user-facing sort of environment. I had set up Selenium Grid in the Cloud  on which we were running our application, and we had automated some tests using a common framework. It was Node.js based and we used Selenium Grid in the Cloud to test that across various browsers and various versions of these browsers so that we could ensure compatibility for our end users and also ensure that our app worked as intended.

The best features Selenium Grid in the Cloud offers include being able to test the application from a user perspective. Testing from a user perspective stands out as a feature for me, and I find parallel execution and cross-browser support to be basic—not having those is a deal-breaker for me.

Selenium Grid in the Cloud by itself does not provide any reporting. Mostly it's the frameworks that do that for me. WebdriverIO was the one that we were using, and it allowed us to do reporting while keeping parallel execution and cross-browser support. It also allowed us to scale pretty effectively and gave us debuggability so that we could keep our tests stable.

Selenium Grid in the Cloud definitely saved a lot of time for my organization. We do not have a specific QA team that tested each and every release, so we followed test-driven development. We ensured that tests were written alongside development, and they helped guide our product lifecycle. It definitely reduced costs a lot. Additionally, being able to get the reliability of running the entire test suite at a whim and with each and every release gave us a lot of confidence in each and every release.

Selenium Grid in the Cloud can be improved by providing more ready-to-use solutions and one-click setup. Right now, I had to go through a lot of hoops just to get it set up in a dockerized, containerized environment. Having a single-click sort of solution right in the open-source projects would be a godsend feature.

Being previously in startups, support did not matter as much for Selenium Grid in the Cloud, but integration has definitely been done well because of the W3C standards and very standardized frameworks which implement that. Documentation has definitely been a strong suite for Selenium Grid in the Cloud. However, being open-source, support lacks a lot, which definitely stops a lot of organizations from picking up a solution such as an open-source Selenium Grid. Being open-source, there is no support.

I have been using Selenium Grid in the Cloud for more than six years.

Selenium Grid in the Cloud is as stable as you make it, but overall, we were able to make it pretty stable.

We were able to scale Selenium Grid in the Cloud as per our needs. We did not have to have anything specifically for our needs or do anything specifically. Selenium out of the box does not provide scalability, so because we used some open-source projects which allowed us to do that, we were able to scale it. However, out of the box, I would say it does not rank great on scalability by its own.

I have dabbled in QTP  before choosing Selenium Grid in the Cloud, but being a VB Script based and very old framework, it did not make sense to continue with it. Selenium was the perfect framework for our use since our application was web-based.

We saved about an entire team's worth of salaries with Selenium Grid in the Cloud, which came to about 100k per year. The release confidence is difficult to quantify, but the time saved was significant with each release. I would say at least three to four days were saved per release, and we were doing a release every week, if not every two or three days.

Having not had to hire an entire QA team to test each and every release with Selenium Grid in the Cloud definitely saved us about 100k per year in USD. Additionally, we saved a lot on release timelines. For each release, we saved up to two to three days of manual testing effort at the very least. While it is difficult to quantify that in money terms, this was definitely a huge return on investment.

My experience with pricing for Selenium Grid in the Cloud involved just the infrastructure cost, as being open-source, there was no licensing or setup cost.

I looked at Puppeteer for a while before choosing Selenium Grid in the Cloud, but the cross-browser support wasn't there. Selenium provided me the familiarity because I had worked with it before, so it was an obvious choice.

I would advise others looking into using Selenium Grid in the Cloud to get someone who understands it at a grassroots level. You would be in good standing as long as you're testing a web-based application. It does have support for native apps as well, up to a certain extent, especially if you're also using something like React Native  for testing across Android and iOS for cross-platform development. It could be great in that as well, but for web-based applications, I would say this is the best tool out there right now. I would rate this product seven out of ten overall.