How to strengthen Cloud Security with Pulumi ESC and AWS Secrets Manager

By Josh Kodroff , Principal Customer Success Architect – Pulumi
By Marina Novikova, Senior Partner Solution Architect – AWS
By Jason Janiak, Partner Solutions Architect – AWS

Pulumi

Modern organizations need to manage a multitude of secrets, including Application Programming Interface (API) keys, tokens or credentials. Several factors contribute to organizations using multiple secrets management solutions. These factors include legacy systems, mergers and acquisitions, and various regulatory requirements.

With a single point of control, organizations gain visibility into where secrets are stored and who has access to them, facilitating auditing and compliance efforts. The complexity of managing secrets grows exponentially as organizations scale. This is especially true when adding more AWS accounts and utilizing hybrid environments.

This post explores how to unify secrets management for organizations of any size. We’ll look at AWS tools like AWS Secrets Manager and AWS Systems Manager Parameter Store, combined with Pulumi ESC (Environments, Secrets, and Configuration) to unify secrets management.

The problem of secrets sprawl

A solution of storing credentials in Secrets Manager can evolve into a sprawling ecosystem of secrets distributed across storage locations and systems. The problem is even tougher if your organization’s resource footprint is also present outside of AWS, utilizing other data centers.

Consider a typical enterprise scenario with multiple secret locations. Production database credentials might be stored in Secrets Manager in one account. Meanwhile, third party API keys and other secrets reside in Systems Manager Parameter Store in a different account. Development teams maintain their own sets of credentials in .env files, shell configurations, and local development environments. Secrets also end up duplicated in CI/CD pipeline configurations across different repositories.

The Figure 1 demonstrates an example of an organization that has several AWS development and deployment environments, developer devices, and CI/CD pipelines.

Figure 1: Example of organization with multiple environments, devices, and pipelines

This secrets sprawl creates several critical challenges. Distributing secrets across multiple systems expands the potential attack surface and increases security risk exposure. An unintended access at any single point, such as a developer’s machine, could compromise sensitive credentials. When it’s time to rotate credentials, tracking down every instance of a secret becomes a daunting task. AWS Identity and Access Management (IAM) provides dynamic capabilities for temporary credentials through AWS Security Token Service (AWS STS). However, some tools still rely on static values that need periodic rotation. Consuming services need time to transition to new secret values to avoid downtime. This leads to error-prone and complex rotation management.

Secrets rotation with Pulumi ESC helps to avoid the burden. Another problem which raises operational overhead is fragmented audit trail. While AWS services provide detailed access logs, secrets stored locally on developer machines or in configuration files leave no audit trail. When secrets are spread across multiple AWS accounts and regions, piecing together a comprehensive access history becomes challenging and time consuming.

In addition, to access secrets across multiple storage locations, you’re forced to grant broad IAM permissions to principals. This results in over-privileged access and violates the principle of least privilege. Different storage solutions have different security models and access control mechanisms. Inconsistent access controls makes it challenging to implement and maintain consistent security policies.

This fragmented approach to secrets management not only increases security risks but also creates operational overhead and compliance challenges. Modern organizations need a centralized, secure, and manageable approach to secrets handling.

How Pulumi ESC helps solve secrets sprawl

Pulumi ESC offers a solution to manage secrets securely. It acts as a secure broker between your applications and various secrets providers. These providers include AWS Secrets Manager, HashiCorp Vault, and Kubernetes configuration files. Pulumi ESC also works with the Kubernetes External Secrets Operator.

This Figure 2 illustrates how Pulumi ESC operates, followed by a more in-depth description below in the blog.

Figure 2: Illustration of how Pulumi ESC operates

Setting Up ESC

The implementation begins with an administrator configuring an OIDC provider in IAM. It enables Pulumi Cloud to securely authenticate with your AWS accounts and to establish a trusted relationship between your infrastructure and the secrets management system. Now engineers can author Pulumi ESC environments and use the OIDC role for secret access in AWS accounts.

Pulumi ESC Environments

At the core of Pulumi ESC are environments, YAML-based configurations, that serve as a single source of truth for your temporary credentials, secrets, and configuration values. Environments can store static secrets and configuration values directly in Pulumi Cloud’s secure vault and integrate with cloud-native identity providers through OIDC, like IAM. Other common use cases include using integration with managed services like Secrets Manager and Systems Manager Parameter Store, as shown on Figure 3 below.

Figure 3: Example of an environment configuration

For a full list of supported integrations, see Pulumi ESC Integrations.

Environments are designed with flexibility in mind. Administrators can configure how secrets are delivered to applications: as environment variables, temporary files (such as Kubernetes KUBECONFIG files) or direct inputs to Pulumi Infrastructure as Code programs.

To promote reusability and maintain Don’t Repeat Yourself (DRY) principles, Pulumi ESC environments can import other environments, allowing for hierarchical organization of secrets and configurations.

Access Control and Security

ESC implements robust Role-Based Access Control (RBAC) through Pulumi Cloud. Administrators can precisely define which principals can perform specific actions (read, open, write, delete) on each environment. This granular control ensures that users and services access only the secrets they need, adhering to the principle of least privilege.

Developer and CI/CD Experience

End users, whether they’re developers or automated processes, can access secrets using either the pulumi CLI or the dedicated esc CLI. First, use esc open to specify an environment to open and it allows to define the command to execute with the secrets within this specific environment. ESC handles the rest.

Behind the scenes, ESC authenticates the user via their Pulumi Cloud token and verifies access permissions through RBAC. Next, it retrieves necessary secrets from various providers and creates an ephemeral environment for command execution. ESC also automatically cleans up after command completion.

This process ensures that secrets never persist on local machines and that access is both logged and controlled.

Beyond Basic Secret Management

ESC’s capabilities extend beyond traditional secret management with four key features. Applications can directly integrate with ESC using the Pulumi ESC SDK for secure runtime secret access without local storage. The platform creates new named versions when environments change, allowing consumers to use specific or latest versions. ESC’s integration with Kubernetes External Secrets Operator provides applications with unified access to secrets from various providers. Finally, through IAM integration, ESC enables operators to rotate access keys for workloads requiring long-lived credentials.

By centralizing secret management through ESC, organizations can maintain a single point of control and ensure comprehensive audit logging across all secret access. It simplifies secret rotation and reduce IAM permission complexity while maintains consistent access controls across all environments. As result, it helps teams meet organization’s compliance requirements.

This unified approach to secrets management transforms what was once a scattered and risky landscape into a controlled, secure, and auditable system.

Boost Insurance: Enhancing Security and Developer Productivity with Pulumi ESC

Boost Insurance is a New York-based platform vendor that empowers companies to build, launch, and scale innovative insurance programs. Operating at the intersection of technology and insurance, Boost Insurance requires secure, scalable cloud infrastructure to support their growing business needs. As it expanded cloud operations, the organization needed a solution to manage access credentials securely while maintaining developer productivity and consistent configuration practices. Boost Insurance found Pulumi ESC addressed these needs.

“With Pulumi ESC, our developers get dynamic AWS credentials on-demand. That removes the need for long-lived tokens and enhances security. ESC allows for Pulumi programs to share secure credentials and access secrets in their given cloud environments. ESC is a great configuration and secrets management sharing tool to allow for a DRY approach to development.” says Richard Genthner, VP of Infrastructure Security at Boost Insurance.

Conclusion

Organizations today face complexity in managing secrets across their cloud infrastructure. The risks of secrets sprawl – from security vulnerabilities to operational overhead – are not theoretical. As demonstrated by our customer success stories, these challenges are real, and they demand practical solutions.

Improve your secrets management with Pulumi ESC, regardless of your team’s size. Visit the Pulumi blog for AWS resources and join our workshops for live demonstrations of AWS solutions.

.

Pulumi – AWS Partner Spotlight

Pulumi is an AWS Advanced Technology Partner and AWS Competency Partner that makes it easy for platform engineering teams to automate, secure and manage any cloud deployment with infrastructure as code, centralized secrets management, policy enforcement and analytics capabilities.

Contact Pulumi | Partner Overview | AWS Marketplace | Case Studies