Integration & Automation

Automating server creation with EC2 Image Builder and AWS Systems Manager: A collaboration between AWS and Ziff Davis

This post was co-written with Tom Newman of Ziff Davis.

Managing servers at scale can be challenging, especially in dynamic environments with legacy applications and inconsistent configurations. In collaboration with AWS, Ziff Davis tackled these challenges by using EC2 Image Builder and AWS Systems Manager to create automated, consistent, and reliable server creation processes.

This post details the challenges faced, the automated solution implemented, and the resulting improvements in efficiency, consistency, and disaster recovery.

Challenges

Ziff Davis offers virtual phone number services through its brands Line2, eVoice, and eReceptionist. These services equip small businesses and individuals with reliable voice and SMS communication tools, so they can separate personal and professional interactions) platform hosts several legacy applications on Internet Information Services (IIS), Microsoft’s web server software, which rely on Component Object Model (COM) components for certain functionalities. These legacy applications present significant challenges due to inconsistent configurations across environments:

  • Inconsistent environments – Development, quality assurance (QA), and production environments had different server configurations. This inconsistency led to unreliable QA testing results, making it difficult to make sure deployments would work as expected in production.
  • Lack of documentation – Server creation wasn’t documented, making it hard to reproduce environments accurately. Dependencies required to run the applications were often unclear.
  • Manual processes – Servers were manually created for each platform, leading to inefficiency and potential for errors. Patching was also a manual process, consuming significant time and effort.

Ziff Davis needed a standardized and automated approach to create and manage servers efficiently, providing consistency, reliability, and straightforward maintenance.

Solution overview

To address the challenges of inconsistent server configurations, undocumented processes, and manual patching, Ziff Davis adopted a streamlined automation approach using EC2 Image Builder and Systems Manager. This solution makes sure web servers are created consistently across environments, with automated configuration and deployment processes.

The solution consists of two primary components:

  • EC2 Image Builder – EC2 Image Builder is a fully managed service that simplifies customization, testing, distribution, and lifecycle management of Amazon Machine Images (AMIs). By defining image pipelines with both AWS provided and custom components, Ziff Davis could standardize the base configuration for their IIS servers, making sure each AMI includes the necessary features, Windows Server roles, and dependencies. This automated approach alleviated the need for manual server creation and reduced configuration drift between environments.
  • Systems Manager – Systems Manager automates the deployment and configuration of instances created from the AMI. Specifically:
    • Automation documents were used to launch instances from the latest AMI dynamically. These documents incorporate API calls to fetch the desired AMI and deploy the server instances with predefined configurations.
    • State Manager, a capability of Systems Manager, schedules these automation documents to run on a regular basis (for example, weekly). This makes sure new instances are created and configured on a predictable schedule, keeping environments up to date with the latest patches and settings.

This automated workflow provides consistent server deployments, improves operational efficiency, and provides a reliable foundation for Ziff Davis’s legacy applications. The following are key features of the solution:

  • Dynamic AMI selection – Instead of manually specifying AMIs, the solution dynamically identifies the latest AMI by filtering based on tags or name patterns. This makes sure instances are always launched from the most recent, validated image.
  • Tag-based workflow – Tags are used to manage and track the progress of instance creation and configuration. For example, tags indicate when a server is ready for postprocessing tasks or when an issue requires attention.
  • Automated configuration – Post-launch configuration tasks, such as joining the instance to a domain, setting static IP addresses, and configuring IIS web applications, are handled by Systems Manager automation steps. This makes sure each instance is fully configured and ready for use with minimal manual intervention.
  • Logging and monitoring – Automation steps generate logs in Amazon CloudWatch, making it straightforward to track the progress and troubleshoot issues. Logs are isolated by task, which simplifies the process of identifying where failures occur.

By combining EC2 Image Builder with Systems Manager, Ziff Davis achieved a robust, automated solution for creating and managing their IIS servers. This approach improved operational efficiency, reduced manual effort, and provided a consistent server environment across the different stages of deployment.

The following diagram illustrates the solution architecture.

To address the challenges of inconsistent server configurations, undocumented processes, and manual patching, Ziff Davis adopted a streamlined automation approach using EC2 Image Builder and Systems Manager. This workflow is divided into three key steps: image creation, instance creation, and post-launch configuration.

Create an image with EC2 Image Builder

The first step involves creating a standardized AMI using EC2 Image Builder. This makes sure the IIS servers are built with a consistent base configuration, which includes necessary components and dependencies.

  1. Identify server requirements:
    1. Document all the necessary Windows Server roles, IIS features, and third-party applications required for hosting legacy applications.
    2. Make sure dependencies like SSL certificates and ODBC drivers are accounted for in the configuration.
  2. Define the image pipeline:
    1. Use a combination of AWS managed components (such as core Windows Server features) and custom components (such as additional IIS roles and drivers).
    2. Configure the pipeline to include these components, making sure the image meets the needs of Ziff Davis’s legacy applications.
  3. Handle SSL certificates:
    1. IIS applications require SSL certificates for bindings. However, images are created under the NT SYSTEM user, which can lead to permission issues if certificates are installed during the image build. To avoid this issue, certificates are installed during the post-configuration step instead of the image build process.
  4. Generate the AMI:
    1. After the pipeline is defined, EC2 Image Builder automates the build, validation, and delivery of the AMI. This alleviates the need for manual server creation and provides consistency.

During the instance creation process, common features were identified and used by various platform servers. At this point, the common features were extracted into a common base image pipeline. The output from this pipeline is a common image that is then used to create the platform-specific AMIs.

Create an instance with Systems Manager Automation

After creating the AMI, the next step is to launch and configure instances dynamically using Systems Manager Automation documents:

  1. Dynamic AMI selection:
    1. Instead of manually specifying AMIs, the Automation document dynamically identifies the latest AMI using the DescribeImages API with filters for name patterns or tags (for example, Tom*). This makes sure instances are created from the most up-to-date image.
  2. Launch the instance:
    1. Use the aws:runInstances action in the Automation document to launch the instance from the selected AMI.
    2. Apply relevant tags to track the server’s progress through various stages of configuration and deployment.
  3. Schedule the Automation document with State Manager:
    1. Automate the instance creation process by scheduling the Automation document using State Manager.
    2. A cron expression (for example, cron(0 0 ? * SUN *)) makes sure instances are launched and configured on a regular basis, keeping environments consistently updated.
  4. Python Script for AMI sorting:
    1. In cases where filtering the AMI list directly with the DescribeImages API is insufficient, a simple Python script sorts the AMIs by creation date and returns the latest AMI ID.

Set up post-launch configuration with Systems Manager

After the instance is launched, Systems Manager Automation handles the post-launch configuration to prepare the server for production use:

  1. Domain join:
    1. Use Systems Manager commands to join the instance to the appropriate Active Directory domain.
  2. Network configuration:
    1. Assign static IP addresses to the network adapters based on environment requirements (for example, development, QA, or production).
  3. IIS configuration:
    1. Configure IIS web applications, including setting up SSL bindings and making sure necessary sites and services are running.
  4. Staggered processing:
    1. Spread configuration tasks over several hours to allow time for checking logs and addressing issues.
    2. If a failure occurs during this phase, apply a failed tag to the instance to flag it for troubleshooting.

Results and impact

By implementing automated server creation with EC2 Image Builder and Systems Manager, Ziff Davis achieved significant improvements in their server management process. The key outcomes of this solution include:

  • Consistent server environments – Servers across development, QA, and production environments are now built and configured using the same process and specifications. This consistency makes sure QA testing accurately reflects production, reducing the risk of deployment issues.
  • Operational efficiency – Automation alleviated the need for manual server creation and patching, saving time and reducing human errors. The scheduled creation of AMIs with patches already applied streamlined the patch management process.
  • Improved disaster recovery – Standardized AMIs and automated deployment workflows make it possible to recreate servers quickly in different Availability Zones or AWS Regions. This simplifies disaster recovery and provides rapid response to outages.
  • Scalability – The solution supports scaling server deployments on demand, whether for planned updates or unexpected outages. Instances can be replaced or added seamlessly based on business needs.
  • Simplified troubleshooting and logging – Centralized logging in CloudWatch provides detailed, task-specific logs. This makes it straightforward to pinpoint and resolve issues without searching through extensive log files. Failures during image creation or instance configuration are clearly highlighted, facilitating quick troubleshooting.
  • Cost-efficiency – By automating server creation and patching, Ziff Davis reduced the overhead of manual maintenance and minimized downtime. The automation also enables better collaboration between teams, making sure infrastructure changes are versioned and documented.
  • Server consistency – Servers are created with the same process, but the process also includes tagging.

These improvements not only streamlined server management but also positioned Ziff Davis to gradually transition their legacy applications to more cloud-based architectures.

Conclusion

Automation is essential for achieving operational efficiency and reducing the risks associated with manual tasks. By using EC2 Image Builder and Systems Manager, Ziff Davis transformed their server creation process into a reliable, repeatable, and scalable workflow.

The solution created through the collaboration between Ziff Davis and AWS addressed key challenges like inconsistent configurations, undocumented processes, and manual patching. Involving the team early in the pipeline creation process helped secure buy-in and fostered a shared understanding of the automation goals.

This automated approach not only enhanced efficiency but also supported business objectives for disaster recovery, patch management, and server consistency. With standardized, version-controlled infrastructure, Ziff Davis can now respond quickly to changes, scale operations effortlessly, and focus on modernizing their applications.

For more information about using EC2 Image Builder and Systems Manager, see the following blog posts:

Ziff Davis

Ziff Davis offers virtual phone number services through its brands Line2, eVoice, and eReceptionist. These services equip small businesses and individuals with reliable voice and SMS communication tools, so they can separate personal and professional interactions. Small businesses benefit from enhanced communication features like call routing, voicemail-to-email, and auto-attendants, delivering professionalism and efficiency without a significant investment. For individual users, these services provide privacy protection, improved security for social interactions, and a straightforward way to manage communications. Ziff Davis makes sure that effective and professional communication remains accessible, organized, and seamlessly integrated into daily operations.

About the authors

Pedram Jahangiri is an Enterprise Solution Architect at AWS with a PhD in Computer and Electrical Engineering. With over 15 years of expertise in the cloud, operational technology, IT, AI/ML, and energy industries, he has a solid history of leading technical teams and developing strategic initiatives at AWS. He is also a distinguished speaker and author, known for his contributions to cloud, energy, and AI/ML technologies.

Tom Newman is a Senior DevOps Engineer at Ziff Davis with 25 years of experience in development, integration, and operations. He started in C++, moved into the .Net framework, then into build management and automation. Tom now advocates the use of IaC and automation principles with the business unit, pushing the standardization of deployment CI/CD for all applications.