We do load testing with LoadRunner Cloud. LoadRunner itself is cloud-hosted, but we load test the applications that are hosted on-premises or in our application data center.
External reviews
28 reviews
from
and
External reviews are not included in the AWS star rating for the product.
Supports multiple protocols and helps to ensure that our applications are stable at any given point
What is our primary use case?
How has it helped my organization?
By implementing LoadRunner Cloud, we wanted to make sure that our applications are stable when there is a peak load with 3,000 or 4,000 users. We wanted to make sure that our applications are stable at any given point. To validate that, we are using LoadRunner Cloud. We are putting our expected load through LoadRunner and making sure our applications are stable.
We can plan and run tests using LoadRunner Cloud without having to manage testing infrastructure. That is very helpful. One of the reasons why we moved to LoadRunner Cloud was that they manage the infrastructure, and it is up 99% of the time. We used to support Performance Center, which was on-premise, and we also support application servers and all the load generators. It is a lot of work to manage them. Migrations, security scans, and all the patching take a lot of time, whereas, with the cloud option, our work is reduced by 50% to 60%. We can now focus on testing instead of managing the whole infrastructure. LoadRunner Cloud has been very helpful. It is stable and user-friendly. They provide scalability. They have a flexible licensing model, so everything is great.
LoadRunner Cloud has partially saved us money by not having to maintain hardware and the power costs associated with that hardware. In my company, we are still using on-premise load generators, so in our case, the savings are a little bit less, but any other company that has all public-facing or hosted applications does not need to spend any money on on-premise infrastructure. Because we are using a hybrid version, we are still spending some money.
What is most valuable?
The beauty of LoadRunner Cloud is that we can use the load generator that is hosted by us on-premises, and we also have the option to use their hosted load generator. If it is a public-hosted application, we can also use their public-hosted load generator, but in our case, all our applications are hosted in our data center, so we are using the on-premise load generator. We have the option to deploy those load generators as we want. We have deployed our VMs there. We have deployed them in our OCP cluster, and we have also deployed them in our AWS instance. On a day-to-day basis, we do load testing of our applications, and that load is distributed with a different load generator.
What needs improvement?
Initially, there were a couple of things, but they got resolved. When they released it three years back, they were not supporting multifactor authentication. We use Okta. In my business unit, we are using Okta integration or authentication. They were not supporting that earlier, but we requested them, and they implemented it. At this time, I do not see anything that they need to improve in existing features.
In terms of new features, they can natively integrate with Chaos engineering tools such as Chaos Monkey and AWS FIS. With LoadRunner, we can generate load, and if Chaos tools are also supported natively, it will help to get everything together.
For how long have I used the solution?
I have been using LoadRunner for 15 years, but we have been using LoadRunner Cloud for almost three years.
What do I think about the stability of the solution?
I have not seen any issues on that side of it. In three years, I have seen only one unplanned outage. Other than that, everything has been great. I have not seen any performance issues.
What do I think about the scalability of the solution?
Its scalability is great. It is highly scalable. You can put as much load as you want as long as you have the license. They are supporting both licenses. You can pre-purchase a license or you can purchase over the cloud. They support scalability on the user side as well as the load side. In our case, we also have on-premises load generators. They support all types. It is easily scalable.
How are customer service and support?
They are very good. I am supporting multiple tools. Support-wise, they are very good. In HP days, it was not great, but with Micro Focus and now OpenText, the support is better.
We reach out to them when we face any issues. Because it is cloud-hosted, we reach out to them and open a ticket if we are trying to start the load, but we are not seeing any screen or have any other issue. Within SLA, they reach out to us. If it is a new feature request, they reach out to us. We also have monthly calls with their customer success manager. They keep us up to date and give us all the information about the new things that are coming. They also send notifications beforehand when they have any maintenance scheduled. The communication and support have been good.
I would rate their support a nine out of ten. I am taking one point off because sometimes, it can take a few cycles to explain the issue to the support, but that is applicable to any tool. Overall, I am satisfied with their support.
How would you rate customer service and support?
Positive
Which solution did I use previously and why did I switch?
Before this, we had the Performance Center tool, which was a Micro Focus tool. They are in the same line. Performance Center was on-premises, whereas LoadRunner Cloud is vendor-hosted. The base functionalities remain mostly the same. The main difference is the way we access them.
Performance Center only supported IE. Users using a MacBook could not access Performance Center, whereas LoadRunner Cloud is cross-browser. We can access the tool from any browser, which is very helpful, so one of the reasons for moving from Performance Center to LoadRunner Cloud was that LoadRunner Cloud supported different browsers, and accessing it was easy.
How was the initial setup?
I was involved in its onboarding. It was easy. Because we did not plan to migrate anything, in our case, it was easy. We had ten years of data, but when we moved, we decided to not migrate that data because we wanted to start fresh. For us, it was very easy, but I do not know what options people have if they plan to migrate their data.
What was our ROI?
We did not calculate the ROI, but the benefits are definitely there. We are testing most of the customer-facing applications before putting them out there, and we are able to ensure that our applications are able to handle the load.
What's my experience with pricing, setup cost, and licensing?
It is neither costly nor cheap. It is not too high and not too low. I know the price of other tools, and LoadRunner Cloud's price is in the medium range.
Which other solutions did I evaluate?
I did a PoC with other tools such as Flood, NeoLoad, and BlazeMeter. In my company, we are supporting multiple protocols testing. For web and API testing, any tool is good, but when it comes to RDP and Citrix protocols, LoadRunner is at the top in the market for supporting all different protocols.
What other advice do I have?
If you are looking for only web or web service protocols, you can find a cheaper solution, but if you have any other protocol testing, then use LoadRunner because LoadRunner is the only tool available in the market that supports multiple protocol testing. For load testing, there are a number of tools, but they only support web or web services protocols, and not any other, so make sure you know what you are looking for.
Overall, I would rate LoadRunner Cloud an eight out of ten. For performance testing, I have not seen any other tool close to an eight.
- Mark review as helpful
The best feature is that we don't have to build and maintain infrastructure anymore
The best benefit would be budget. It's a really inexpensive if you are testing your application every spring or every month. Even for less frequent applications that you have to test regularly. You can hook up your infrastructure to StormRunner Cloud and you can get the best of both worlds.
In larger enterprises, we have different departments using Performance Center and StormRunner or Selenium. So, StormRunner can act like an umbrella and plug-in everything, get it executed or done across the world. Since they use both HPE Cloud, AWS, and Microsoft, it uses more access points geographically to test it making it best of the class.
What is most valuable?
StormRunner itself is a pretty good hybrid product of Performance Center. Keeping up with DevOps, thus the best feature of StormRunner is that we don't have to build and maintain infrastructure anymore. Whenever we have to test lab scale applications, and in a scenario where we don't have to test every day, we don't have to build the machines and pay for it. Instead, we can get the infrastructure from on-demand from StormRunner, and its ability to run it anywhere just by opening it in browser is the best part. The evolution of StormRunner starting with supporting LoadRunner along with the open source technologies like JMeter and Selenium. Their unit testing tools are actually very advanced in the region of the product. That's why I would recommend anyone to use StormRunner, even though something is not supported now, eventually it's going to be supported by StormRunner. That's the kind of credibility that's needed for any customer when it comes to relying on a product or going for a new product.
What needs improvement?
More insight into test results and allowances. It might be a tailor-made requirement for me, but I would like to download them offline and do my own customizations on the reports. Right now, we have some standard templates that generates reports. But if I had to do some customizations, include something else and create a report, it's not easy. So, if I can an order to download the raw data I can make custom report. That would be the case with every customer because even though they tested one product it might be part of a big project and they need to have other information included in it along with the report, so feasibly it is good to have.
Also, it's evolving, where there's too many features for me to handle and it's too much on my plate to any make ROI out of it.
There is a steep learning curve for the product, too.
What do I think about the stability of the solution?
Over the last couple of years, it's been evolving. We started using more in the last one year and we see it's pretty stable. Adding new features to StormRunner is going to slow down once it has every feature. That has to do with stability and I don't see any drawback in that.
What do I think about the scalability of the solution?
We haven't really hit any scalability issues by using StormRunner, but I just now heard that they can support two million users, which is kind of astonishing. The max I have used is 40,000 users, but two million is good, too. It's big step up from 40,000.
At the same time, you don't have to get the infrastructure built and set it up. For example, let's say you had to test it for one million users. You don't have to procure all of the machines you need for the one test you're going to do. You can do it on demand, that's the best thing. Two million is still very much overboard, but it's good to know. It's good to know that there's no limitation nearby.
How is customer service and technical support?
I'm not a big fan of the tech support to be frank. SaaS tech support (HPE tech support or Micro Focus), there's a gap between the people who access our tickets and the people who know the product. So, there is always a blind exchange of information within Micro Focus and most of the time get frustrated with the kind of ticket updates we get. What happens is the person handling the ticket might not be an expert in the product so we end up redoing everything. Communicating everything about our setup and infrastructure and the customer engagement for each ticket. Therefore, we started involving our technical correspondent with Micro Focus, but we're told that it's a pretty messy situation over there.
Which solutions did we use previously?
I always thought it would be good to have something like this from HPE because we rely a lot on HPE. Then, StormRunner was released and I knew where exactly it was going and what's it would be for us.
How was the initial setup?
I was involved in the initial setup. It was straightforward and there was a lot of good information available, but I did not need it. I didn't go through any support to set it up. The documentation itself was good.
Which other solutions did I evaluate?
The other product we considered was BlazeMeter. We eventually chose StormRunner. I think only those two are pretty much in the market - nothing else.
What other advice do I have?
I like that StormRunner incorporates the idea of accepting and adapting all open sources. It is my understanding that they are planning to continue accepting, supporting, and adapting all open sources.
For someone evaluating StormRunner and similar products, there are two parameters I would tell him to evaluate:
* Is the application under test? Is it customer facing?
* Is how often do you test it?
showing 1 - 2