Building a Scalable and Secure Plugin Platform for Generative AI Using AWS Lambda with Dify

Dify is an open-source development platform for large language model applications. Its intuitive interface combines agentic AI, a retrieval-augmented generation pipeline, agent capabilities, model management, observability features, and more, helping users quickly move from prototype to production. Dify is available as a software-as-a-service (SaaS) platform on AWS and as a self-deployable platform for enterprises and individual users, and it has more than 100,000 GitHub Stars.

Dify uses Amazon Elastic Kubernetes Service (Amazon EKS)—a fully managed Kubernetes service—for its managed SaaS platform and Amazon Elastic Compute Cloud (Amazon EC2)—secure and resizable compute capacity—for its compute needs. Dify also migrated to AWS Graviton processors, which offer companies the best price performance for cloud workloads running on Amazon EC2. In the releases before Dify v1.0.0, models and tools were integrated into the core code base, which was on Amazon EKS. By that arrangement, adding features, making changes, or offering new tools required changing the core code repository, which slowed down development and innovation. With its user base increasing, Dify needed to find a solution that would accelerate deployment and enhance the developer experience.

For the version 1.0.0 release of its SaaS platform, Dify decided to use a plugin-based architecture and host tool and model plugins on AWS Lambda. Development took 3–4 months, and version 1.0.0 and later releases are now available to developers.

On the plugin-based SaaS platform, user-generated code is isolated from Dify’s core code base and runs in its own isolated environment. When a developer uses a Dify workflow for a plugin, the core Dify application uses Amazon API Gateway—which creates, maintains, and secures APIs at any scale—to call an AWS Lambda function. The plugin runs as an AWS Lambda function and accesses Amazon Simple Storage Service (Amazon S3)—an object storage service—to retrieve relevant information or store data. After the user has worked on the plugin, the value gets passed back to Dify’s core app using HTTP. “The plugin architecture using AWS Lambda makes it simpler for developers to extend Dify’s functionalities and build their own plugins,” says Chenhe Gu, head of developer relations at Dify. “It is also more secure, as the plugins are separate from Dify’s core logic.”

Each plugin has its own runtime environment and is separate from other plugin calls. “Our users are reassured that their data is separated in self-contained AWS Lambda functions,” says Gu. The SaaS platform currently uses more than 2,300 AWS Lambda functions and handles millions of plugin calls each month. It scales smoothly for Dify’s more than 600,000 registered users and has over 1 million published applications.

Using AWS Lambda helped LangGenius save tens of thousands of dollars a month in compute costs. “Because tool and plugin calls are sporadic, we wanted to handle them dynamically on AWS Lambda,” says Gu. “It didn’t make sense for us to have multiple Amazon EC2 clusters running 24/7 to handle the plugins.”

The new architecture also lowered operational overhead because AWS manages the maintenance, security updates, and infrastructure of AWS Lambda. “Serverless solutions like AWS Lambda help us save on operational costs, empowering our teams to focus on building products and delivering for users,” says Gu. Dify’s SaaS platform needs only one infrastructure engineer to manage it, so the company can free up resources to build and deploy solution capabilities.

Following the success of its SaaS platform, Dify plans to use the same plugin-based architecture for its self-deployable enterprise platform. Enterprises can use either a locally hosted plugin runtime environment or AWS Lambda on the cloud for the platform. Dify’s enterprise customers, including more than 30 Fortune 500 companies, use the platform to develop generative AI applications that enhance operational efficiency. By using the plugin-based architecture for its enterprise platform, Dify will help those same customers save on resources and costs. “We plan to make plugin runtimes more efficient for enterprise customers who deploy the platform on their own clusters,” says Gu.

Dify will continue using AWS Lambda as it adds new plugins on its platform to extend functionality and help users develop richer applications. On Dify’s road map is integration of the Model Context Protocol so that its users’ large language model applications can interact with external tools, databases, and APIs. The next major upgrade of the platform will include more powerful features for retrieval-augmented generation and the enterprise-level extract, transform, load data pipeline.

Dify Enterprise Edition is available on AWS Marketplace, which helps users discover, deploy, and manage software that runs on AWS. Dify continues to benefit from AWS expertise on both technical and business fronts. “Guidance from AWS is among the deepest and most useful collaborations we have at Dify,” says Gu. “AWS is super helpful in optimizing our SaaS deployment and in marketing our platform and making it frictionless for enterprise users to adopt it.” Dify will continue to use AWS to expand to other regions through workshops, events, and other opportunities for marketing its solutions.