AppFolio Leverages Datadog LLM Observability and Amazon Bedrock to Boost Adoption by 300% and Maximize Efficiency

Supporting customers in the property management industry, AppFolio helps clients manage operations through the entire resident lifecycle. This includes marketing vacancies and handling leads to exceeding the expectations of existing occupants and managing move-outs. AppFolio’s one powerful platform and centralized data structure make it easier for customers to manage all aspects of their operations such as tracking maintenance operations, recording payments, and tying expenses to corresponding accounting and financial records.

Throughout the course of their day, property managers may spend up to 50 percent of their time communicating with residents. AppFolio recognized that users wanted to reduce time spent on communications to focus more on other responsibilities and priorities. So, it set out to improve its communication features to help users be more efficient and focused on leveraging AI to achieve that goal. AppFolio wanted to integrate AI functionality while ensuring transparency and accuracy, with key guardrails in place. It also wanted to maintain reliability and incorporate responsible AI practices while deploying and scaling AI-powered applications.

To do this, AppFolio sought a solution that could identify potential issues and drive improvements during product development. They needed to ensure maintenance of high quality standards as the company expanded its services and to provide continuous usage monitoring for insights into user behavior.

AppFolio wanted its users to have an unmatched customer experience. With comprehensive observability in place, the company could make improvements to its services faster and release new capabilities to more customers with confidence. This was important for integrating new AI-driven features, due to the unpredictable nature of LLMs and the difficulty identifying operational and functional issues in LLM-based application architectures.

An AWS Advanced Technology and DevOps Competency Partner, Datadog presented a strong case for its cutting-edge LLM observability features. Datadog LLM Observability would allow AppFolio to monitor the performance and quality of its LLM-powered generative AI applications. For example, it provided deep tracing for LLM chain steps to identify bottlenecks, optimize logic, and improve LLM response accuracy—a must for AppFolio.

AppFolio had identified Amazon Bedrock as its service of choice to incorporate AI, so the company needed something that would integrate easily with it. With access to foundation models from leading AI companies through a single API, Amazon Bedrock would enable AppFolio to build, customize, and deploy generative AI applications with its data. Datadog’s LLM Observability capabilities can track usage, performance, and error rates for foundation models provided by Amazon Bedrock while ensuring response quality and data security. Therefore, AppFolio was able to use Amazon Bedrock to experiment with and evaluate various models, test different LLMs, and select the ones best suited for its use cases.

Datadog’s unified dashboards provide insight into LLM latency and token usage over time, and real-time alerts detect anomalies, such as error spikes, to ensure optimal application performance. Out of the box LLM evaluations such as Toxicity and Failure-to-answer help prevent negative user experiences and prioritize new features. With hundreds of thousands of messages going through AppFolio’s systems each day, having a solution to track, monitor, and analyze each conversation at a high quality was key.