
LaunchDarkly
Seamless A/B Testing Across Segments
Better Control Deploying Features .
LaunchDarkly helps make feature releases safer and easier.
This makes controlling feature rollouts much easier.
Feature flags have transformed our deployments and empower fast, low-risk experimentation
What is our primary use case?
Recently, we launched a feature where we allowed users to upload documents using a new third-party vendor called files.com. This rollout had several moving parts to it, and since it was a new feature, we wanted to experiment with a targeted set of users first. We only rolled it out to one percent of our existing user base. We found an issue during the rollout, and we discovered a bug that we did not encounter during testing. We quickly flipped the flag back, fixed the issue, put the experimentation out again, and then eventually launched it to our entire user base.
Previously, we used to do deployments based on a cadence. We had to do thorough QA testing for every single change or commit that was rolled into the deployment. With this new cadence, we are doing continuous deployments, and we are putting a lot of experimentation behind LaunchDarkly flags. If we find something, we quickly flip that flag back without having to roll back the entire deployment. The cadence has changed from doing a deployment once a week to continuous deployments.
What is most valuable?
LaunchDarkly allows us to do structured experimentation and safe deployments in a single unified workflow. Running A/B tests is pretty smooth, and the UI is very intuitive, so those are the best features.
The ease of use and the UI being intuitive are valuable aspects. Quickly flipping the flags, knowing what belongs where, and having that part be pretty intuitive have been useful.
Customer support is highly rated, particularly for its technical depth and efficacy, and how quickly they respond back if we have any queries. We had a couple of queries to them in the past, and they were diligent about it and got back to us quickly.
LaunchDarkly has positively impacted our organization by transforming how we deploy software and manage risk. We are now able to separate code from release, which significantly reduces our deployment risk. Previously, if something went wrong, we would have to roll back our entire release, so now that is decoupled. Even the non-technical product teams can easily turn on and turn off features, so we do not have to rely on the product team communicating with the engineering team and waiting for them to get that done. It has empowered our product teams, and it has accelerated our CI/CD pipeline, so we can do more frequent code merges and run more experimentation.
What needs improvement?
In a microservices world, managing flag state changes and propagation across complex or heavily distributed backend architectures can introduce latency or consistency challenges. While the UI is clean and pretty intuitive, tracking conditional flag modifications across large engineering teams requires more robust historical audit logging.
For how long have I used the solution?
How are customer service and support?
Which solution did I use previously and why did I switch?
Which other solutions did I evaluate?
What other advice do I have?
There are some areas of friction and technical risk. This non-human identity and service account overhead means that automating AI deployments via CI/CD requires service accounts. Managing these non-human identities requires strict privilege access management integration to prevent API key leaks or exposing critical data for runtime AI flags.
About LaunchDarkly's accuracy and reliability, it is important to clarify that it does not generate AI content itself. It acts as an operational control plane and a feedback loop. Things that it does really well are targeted rollouts for AI, advanced AI A/B testing, native online evaluations and LLM judges. However, it has limitations such as non-configurable out-of-the-box settings. While custom rubrics are supported, the initial setup of out-of-the-box judges offers limited granular tuning. Teams with highly specialized domain needs must invest time into coding custom evaluation prompt wrappers, which is something that we had to do.
We have been working with several different AI agents as part of our experimentation with LaunchDarkly. It has helped us quickly launch those experiments. Some other critical AI challenges it has helped us solve include safely testing different kinds of prompts in production instead of relying on staging environments completely. LaunchDarkly allows us to run different canary tests, and we can tweak prompts on the fly to see which one works better. It has allowed us to control some hallucination and latency outages, so if a newly deployed system prompt or model starts causing hallucinations or has access latency, we can use an instant kill switch or roll back the AI config in milliseconds before it impacts our broader user base. Additionally, it helps with mitigating model vendor lock-in, allowing us to utilize multivariate flags to easily swap traffic between different providers such as Opus or Sonnet without changing application code. It is something similar to what Bedrock provides in AWS, allowing us to seamlessly optimize for cost, speed, and accuracy.
For our CI/CD code management, GitLab is our primary tool for that. In terms of observability and monitoring, we use Datadog and integrate with that. Feature flags are also directly linked to Jira issues, allowing product managers to track deployment status straight from the ticket, with the status of the flag automatically updating as a feature moves from development to full production release. We also have dedicated Slack channels that receive automated real-time alerts when a flag configuration is changed, turned on, or turned off. All these integrations make our process smoother.
Understand your use case before implementing LaunchDarkly. LaunchDarkly is pretty good at complex flag management, but if your use case is simple flag management and you do not really want to scale to too many users, then it might be overkill for you. A rating of eight out of ten reflects a balance of strong functionality alongside specific operational hurdles, namely the lack of management for stale flags and an aggressive scaling-based cost structure.
Which deployment model are you using for this solution?
If public cloud, private cloud, or hybrid cloud, which cloud provider do you use?
Seamless Setup and Scalable Flag Management, but Pricing Can Be Tricky
If you’re B2B and have higher revenue per user, the pricing tends to work out well and feels more cost-effective.