Salesforce Migrates DevOps to AWS with CloudBees CI

Salesforce empowers software developers to create high-quality, secure enterprise apps on its Force.com platform by moving development operations to the cloud using CloudBees CI and Amazon EKS.

Salesforce’s fast growth has encouraged more developers to create applications on the Force.com platform. Currently, the group works with about 1,200 developers across 60 to 70 teams, putting pressure on the DevOps team to keep developers supplied with the latest tools and techniques. "There are all kinds of tools coming into the market all the time,” says Aaron Nassiry, DevOps Engineer at Salesforce. "Our job is to make sure we’re utilizing the right technologies and providing the right guidance to our scrum teams.”

One of Salesforce’s important software development tools is Jenkins. For years, engineers used CloudBees Jenkins Enterprise to automate and accelerate code integration and delivery. Initially, development work was hosted on the group’s own datacenter and used by about 70 development teams at the time.

But the on-premise datacenter environment wasn’t ideal. The underlying server infrastructure – composed mostly of Linux virtual machines (VMs) managed by Puppet – was more than five years old. Changes were increasingly difficult to make, sometimes taking up to a month. "You’d basically have to work through 100 different steps to make it work right,” says Venkat Kothapalli, DevOps Engineer at Salesforce.

Adding capacity was also tough. To keep up with demand, administrators were forced to add more VMs and that meant more management overhead. Other inefficiencies made scaling the system difficult. For example, development teams were stuck running static Jenkins agents that needed constant maintenance. And since the agents were running continuously, they consumed significant energy and other resources, keeping costs elevated.

The growing maintenance burden took a toll on administrators and potentially made it hard to recruit new team members. "That’s not how you attract the best talent,” says Nassiry. "Good developers, good administrators like to add value or do exciting work, so you can’t keep good people around if all they do is maintain 30 or 40 VMs. It’s always our goal to work smart and make sure our teams are doing meaningful work.”

The situation began to turn around after Salesforce directed its IT operations to leave its on-premise environment and move to the public cloud. "It just didn’t make sense anymore for us to run a datacenter and do application hosting and everything else by ourselves,” says Nassiry.

Instead of "lifting and shifting” their datacenter to the cloud, the DevOps teams chose to adopt Amazon Elastic Kubernetes Service (EKS), a managed service that makes it easy to deploy and manage containerized applications at scale. The choice was fortuitous because CloudBees – the DevOps solution the team had been using for software delivery automation and management – also offered Kubernetes integration and support through its new CloudBees CI solution. "It was like a match made in heaven,” says Nassiry.

At first, the prospect of migrating from its on-premise Jenkins environment to CloudBees CI on Amazon EKS seemed a little overwhelming for Nassiry, who didn’t have deep experience with AWS. But he quickly climbed the learning curve with the help of his teammates and timely support from CloudBees. "There were times that I was personally stuck and CloudBees was able to guide us to the right path,” he says. His advice: "Take the plunge. Don’t be afraid.”

Today the vast majority of Salesforce’s Force.com development community builds applications in the cloud using CloudBees CI on Amazon EKS. Moreover, as part of the move, the DevOps team also codified the entire CI/CD platform, simplifying management of software pipelines.

Crucially, the switch to CloudBees CI allowed the DevOps team to leverage efficient on-demand (versus static) agents, saving the time and cost of maintaining dozens of always-running agents. "This eliminated one of the headaches we had to deal with using traditional Jenkins,” says Nassiry.

The switch to configuration as code with CloudBees CI on Amazon EKS has helped the DevOps team complete system upgrades significantly faster. Previously, when the team needed to update its 1,000-developer CloudBees platform, the job took about three weeks of manual testing, enlisting the efforts of four team members. Since moving to CloudBees CI, a single administrator takes just two to three hours to do the same upgrade.

The efficiency boost has enabled the DevOps team to upgrade the platform multiple times a year, giving developers earlier access to the latest functionality, plugins and security features. "All of our plugins are configured as code,” says Nassiry. "We don’t need to go to the operation center and download plugins and install them. It’s all through code. We are much more confident now to do upgrades and it takes very little time for us.”

The team’s move to AWS has helped support the widespread adoption of modern containerized applications. "Kubernetes is our go-to platform for deploying new applications,” says Nassiry. "That’s a huge innovation which was made possible by moving to CloudBees CI on AWS.”

Developers get easy access to new technologies and utilities by pulling Docker images into the organization’s security-scanned repository. "Developers can now leverage a lot of utilities out in the wild and securely bring them to our platform and use them right away,” says Nassiry. "They don’t need to go through the DevOps team, which accelerates innovation.”

Like most major companies, Salesforce puts a premium on security and the DevOps group’s policies and processes reflect this priority. "Getting to production is important, but getting to production in a secure way is more important,” says Nassiry. "It’s built into our culture.”

Security safeguards are embedded throughout the group’s cloud infrastructure, which leverages VPNs, two-factor authentication and security certificates among other measures. Every line of code is scanned for vulnerabilities. The fact that CloudBees easily integrates with all the leading security scanning tools and plugins helps enable faster development lifecycles.

The DevOps team is looking at other security initiatives, including adding CloudBees Role-based Access Control (RBAC) to create an extra layer of security between teams, and bringing the code-scanning solution SonarQube into the pipeline.

Just ahead, the team plans to standardize its CI/CD workflow as part of an enterprise-wide pipeline for the entire business technology group. "The idea is to have the enterprise standard built into the organization so we can deliver CI/CD tooling and processes in a very fast and efficient way,” Nassiry says.

Salesforce developers readily embraced the move to CloudBees CI on AWS. "It’s given them a lot of confidence in us, that we provide solutions and services of this caliber,” says Kothapalli. "CloudBees has enabled us to move to that next level.” The cloud platform has also proven to be solid and reliable. "At any given time we have 20 or 30 builds running 24 by 7 and it’s all handled predictably in a very stable fashion with CloudBees CI,” says Nassiry.

The platform’s dependability allows the DevOps team to spend more time on improving the developer’s experience. "With CloudBees CI up and running, you almost don’t have to monitor or worry about it. It helps the developers keep doing business as usual despite being remote or despite all the other interruptions they might face,” says Nassiry. "It means we can focus on building new features and new services to make the developer’s life easy within Salesforce.”