LangSmith Agent Engineering Platform (SaaS)

What do you like best about the product?

It features robust tools for "LLM-as-a-judge" evaluations, allowing you to create datasets, run A/B testing on prompts, and automatically generate evaluators from production traces to catch regressions before they hit users.

What do you dislike about the product?

While it offers SDKs for non-LangChain code, the platform is fundamentally designed around the LangChain ecosystem. Teams using competing frameworks (like LlamaIndex or CrewAI) often find tools like Langfuse or Future AGI to be a more framework-agnostic fit.

What problems is the product solving and how is that benefiting you?

It has evolved past simple monitoring to include full agent-engineering tools like "Fleet" (for deploying agents), "Sandboxes" for safe code execution, and centralized prompt management (the LangSmith Hub).

What do you like best about the product?

LangSmith makes it easier to trace and debug LLM workflows. I especially like being able to see each step of an agent run, compare outputs and quickly identify where prompts or tools are failing instead of debugging it blindly..

What do you dislike about the product?

LangSmith can feel bit overwhelming at first, especially when there are many traces and runs to inspect. The interface is powerful, but filtering and navigating large volumes of debugging data could be simpler and more intuitive.

What problems is the product solving and how is that benefiting you?

LangSmith helps us understand why an LLM workflow is failing instead of guessing. By tracing each step, comparing runs and evaluating outputs, we can identify weak prompts or tool errors faster, improve response quality, and reduce the time spent debugging AI workflows.

What do you like best about the product?

What I like most about LangSmith is how straightforward it makes tracing and debugging LLM workflows. The interface feels clean and well organized, and it’s easy to inspect prompts, responses, and each execution step all in one place. This saves me time when I’m testing different prompt variations, and it helps me spot problems quickly. I also appreciate how well it integrates with the LangChain ecosystem, which makes it a practical tool for building, refining, and improving AI applications.

What do you dislike about the product?

What I dislike most about LangSmith is that some of its advanced features can feel a bit overwhelming when you’re just getting started. It took me a while to wrap my head around the tracing and evaluation workflow, and I ended up leaning on the documentation more than I expected. I also feel that a few workflows could be streamlined to cut down on the number of steps involved. That said, once I got familiar with how everything fits together, the overall experience became much smoother.

What problems is the product solving and how is that benefiting you?

LangSmith helps me troubleshoot and improve my LLM applications by making it easier to trace requests, inspect outputs, and pinpoint where issues occur. Rather than spending a lot of time manually debugging prompts or model responses, I can quickly see what happened in a given run and adjust accordingly. Overall, it has reduced my development time, improved response quality, and made it much more efficient to test and iterate on new prompt variations.

What do you like best about the product?

The interface is clean, and it saves time when trying to fix issues or compare different responses.

What do you dislike about the product?

There are some parts of the interface that can feel a bit confusing, and it takes little time to figure out where everything is.

What problems is the product solving and how is that benefiting you?

It helps me to find issues in my AI app faster and understand why certain responses are better than others. It saves my time and makes testing much more easier.

What do you like best about the product?

What I like most about LangSmith is its end-to-end debugging and tracing capabilities for LLM applications. It provides complete visibility into prompts, model responses, tool calls, and execution paths, making it much easier to develop, test, and optimize AI workflows. Detailed tracing of prompts, chains, agents, and tool executions. Built-in evaluation tools to measure response quality and compare experiments. Dataset management for testing prompts and models consistently. Easy integration with LangChain and other popular AI frameworks. Clear dashboards for monitoring latency, token usage, and application performance. For me, the most valuable feature is the trace visualization. It allows me to quickly identify where an AI workflow fails, making debugging and optimization much more efficient. The biggest benefit is improved reliability and faster development. LangSmith helps me evaluate AI applications systematically, reduce debugging time, and confidently deploy improvements based on measurable results.

What do you dislike about the product?

The biggest drawback is the initial learning curve. While LangSmith offers powerful debugging and evaluation capabilities, it takes some time to understand how to structure datasets, evaluations, and observability workflows effectively. Large applications can generate a high volume of traces, making organization and filtering important. Some advanced evaluation workflows require additional configuration to match specific use cases.

What problems is the product solving and how is that benefiting you?

LangSmith solves the challenge of developing, debugging, evaluating, and monitoring LLM applications. Instead of treating AI workflows as a black box, it provides detailed traces, evaluation tools, and performance metrics that help identify issues and improve model behavior. Provides end-to-end tracing of prompts, chains, agents, and tool calls. Simplifies debugging by showing exactly where an AI workflow fails. Supports structured evaluations to compare prompts, models, and application changes. Tracks latency, token usage, and execution performance. Helps detect regressions before deploying updates to production. In my workflow, LangSmith helps me validate prompt changes, analyze execution traces, and measure the impact of model updates. Instead of relying on manual testing alone, I can evaluate AI behavior using datasets and metrics, making optimization much more systematic. The biggest benefit is more reliable AI applications and faster development cycles. LangSmith reduces debugging time, improves confidence in production deployments, and enables continuous improvement through structured evaluation and observability.

What do you like best about the product?

LangSmith provides the visibility and control that modern AI workflows sorely need. The observability tools are great – every agent action, every model call, every retrieval step is logged in a nice, searchable trace. This has drastically reduced our debugging time, and improved our ability to diagnose issues such as hallucinations, latency spikes, or unexpected cost behavior.The evaluation workflows are just as impressive. With automated LLM-as-judge testing, side-by-side comparisons, and human review pipelines, it’s easy to keep quality high as you iterate quickly.

What do you dislike about the product?

LangSmith is powerful and the learning curve is a little steep at first. If your team lacks experience with structured observability, then it may take some time to fully adopt the workflow. The cost might be a concern for smaller organizations, but the value becomes evident when operating at scale. A more beginner friendly onboarding content would make the ramp up smoother.

What problems is the product solving and how is that benefiting you?

LangSmith solves the biggest operational challenges in AI deployment: lack of transparency, inconsistent quality, and unpredictable costs. With SmithDB, we can query millions of traces instantly, which is essential for compliance, debugging, and client reporting. The platform ensures our AI systems are reliable, measurable, and continuously improving — which directly strengthens the service quality we deliver to clients.

What do you like best about the product?

LangSmith provides excellent visibility into how AI agents act in production. The observability tools make debugging easy. The evaluation workflows ensure our models keep improving. I particularly appreciate the ability to integrate with different frameworks, whether it’s LangChain, OpenAI, or Anthropic. The platform fits well with all of them. The dashboards are clear and user-friendly. They help us feel confident in both performance and cost management.

What do you dislike about the product?

The platform is powerful, but it can take some getting used to. New users might need some time to fully understand the structured observability and evaluation workflows. Pricing can seem high for smaller teams – but the ROI is clear when you scale. Would be nice to see a few more beginner friendly onboarding resources.

What problems is the product solving and how is that benefiting you?

LangSmith helps us solve the hardest problems in deploying AI: hallucinations, unpredictable costs, and lack of transparency. With SmithDB, we can query millions of traces in seconds, which is critical for debugging and compliance. This has directly improved reliability, reduced downtime and strengthened customer trust in our AI-driven offerings.

What do you like best about the product?

LangSmith has revolutionized our approach to developing and operating AI applications. The observability capabilities of LangSmith cannot be beat being able to see the execution process of each agent run step by step has saved us many hours of troubleshooting. The evaluation functionality is impressive and allows us to perform automated testing and human-in-the-loop testing. I also love the smooth integration with different platforms such as LangChain, OpenAI, and Anthropic.

What do you dislike about the product?

Learning curve can be quite steep when working with new teams. It will take some time before you can fully grasp how to make use of the observability process in its fullest. Prices can also be considered quite high for startups, although they are very valuable once you scale.

What problems is the product solving and how is that benefiting you?

LangSmith assists us in overcoming some of the most difficult issues involved in deploying AI models: hallucination, unpredictable pricing, and lack of visibility. The SmithDB allows us to search through millions of traces in seconds, which is vital for debugging and compliance. LangSmith guarantees the reliability, cost-effectiveness, and consistent improvement of our agents.

What do you like best about the product?

End 2 End visibility - provides for LLM applications. Tracing is easy, token usage, evaluation. Easy to navigate. Performance wise its good.

What do you dislike about the product?

pricing and usage cost is difficult to predict at higher trace volumes

What problems is the product solving and how is that benefiting you?

has reduced troubleshooting from hours to minutes.

What do you like best about the product?

Comprehensive tracing: View prompts, model responses, tool calls, and execution paths in one place, making debugging much faster.
Built-in evaluation: Compare prompts, models, and application versions using automated and human evaluations to improve quality over time.
Production monitoring: Track latency, errors, costs, and application performance to quickly identify issues after deployment.
Dataset management: Create and reuse test datasets to validate changes before releasing updates.
Developer-friendly integrations: Works seamlessly with LangChain and can also be used with custom LLM applications through its SDKs.

What do you dislike about the product?

No dislike from my side. It is very helpful.

What problems is the product solving and how is that benefiting you?

Difficult debugging: Instead of guessing why an AI response was incorrect, LangSmith provides detailed execution traces showing prompts, model outputs, tool calls, and intermediate steps. This makes it much faster to identify and fix issues.
Inconsistent AI performance: Its evaluation framework lets you test prompts and models against benchmark datasets, helping ensure changes actually improve quality before deployment.
Limited production visibility: LangSmith monitors latency, errors, token usage, and application behavior in production, making it easier to detect problems early and maintain reliability.
Regression risk: By comparing application versions and running repeatable evaluations, it helps prevent updates from introducing new bugs or reducing response quality