The most likely best decision is to buy Vellum
What do you like best about the product?
Scenarios:
1. You are new to LLM-based product development: Buy Vellum (no question - build your own tooling later if you even need to).
2. You have started developing LLM-based products but not yet in production: Buy Vellum (get to production faster).
3. You have existing LLM-based products in production, but your tooling is limited: Buy Vellum (you likely need to improve the rate of development for the future - Vellum gets you there faster).
4. You have production LLM-products and extensive tooling: Probably still buy Vellum (focus on building and improving your products or experimenting with new ones. Vellum has the tooling you need in most cases and it will continue to get better while you work on what matters to your business).
Reason to buy:
Vellum is your partner. They work with you to understand your needs to build your products. That makes them great and building the product/tooling that allows you to focus on building your product and not the tooling required to do it.
1. You are new to LLM-based product development: Buy Vellum (no question - build your own tooling later if you even need to).
2. You have started developing LLM-based products but not yet in production: Buy Vellum (get to production faster).
3. You have existing LLM-based products in production, but your tooling is limited: Buy Vellum (you likely need to improve the rate of development for the future - Vellum gets you there faster).
4. You have production LLM-products and extensive tooling: Probably still buy Vellum (focus on building and improving your products or experimenting with new ones. Vellum has the tooling you need in most cases and it will continue to get better while you work on what matters to your business).
Reason to buy:
Vellum is your partner. They work with you to understand your needs to build your products. That makes them great and building the product/tooling that allows you to focus on building your product and not the tooling required to do it.
What do you dislike about the product?
Git-based development/management was a major disadvantage. With the new SDK, that's solved for the most part. There may be some very advanced requirements that aren't supported by the Vellum tooling, but there is likely some 'today' workarounds and 'tomorrow' fully baked solves.
What problems is the product solving and how is that benefiting you?
Prompt/RAG development, evaluation, and production monitoring.