How Enterprises Are Migrating to AWS with Confidence with Cloud-Native Visibility

By Areg Alimian, Senior Director of Product Management at Ixia

At Ixia, a Keysight business and AWS Partner Network (APN) Advanced Technology Partner, I am fortunate to have the opportunity to learn from and work with dozens of enterprise IT professionals and executives as they plan to migrate applications and web services to the AWS Cloud.

As the move towards cloud accelerates, unsubstantiated fears about security are inhibiting the use of cloud services by some enterprises. A vital element of a successful cloud migration is understanding shared responsibilities between the organization and cloud service provider. In fact, Gartner predicts that, through 2020, 95 percent of cloud security failures will be the customer’s fault.*

The security and monitoring tools offered by cloud providers include support for networking, database, storage, compute, and availability zones. However, analysis of customer packet data, which is critical for network security, is the customer’s responsibility.

Other customer responsibilities outlined in the Shared Responsibility Model include platform, application, identity and access management, operating system, network and firewall configuration, and the protection of data at rest and data in motion.

Migration Strategies and Security

There are multiple strategies for cloud migration—re-architecting for the cloud, building cloud-native, re-hosting (lift-and-shift), or re-platforming. Stephen Orban, who heads up enterprise strategy at AWS, does a terrific job explaining trade-offs in his Medium post 6 Strategies for Migrating Applications to the Cloud.

Regardless of the approach, a successful migration depends on the security of your cloud environment, as well as application performance and fault tolerance, ease of use, cost, and the availability of vendor solutions for security analytics, applications, or network performance management.

To support a distributed application architecture built to leverage the cloud’s full capabilities, there are several critical requirements for successful visibility and controls designed to provide security—capturing and filtering traffic, and allowing for horizontal scaling while providing pervasive data to tools.

Requirement 1: Access to Packet Data

In a traditional data center, there is physical access to the network so physical taps and network packet brokers can be used to access and filter data. In the cloud, however, the physical infrastructure is the responsibility of the service provider.

Cloud-based applications are generally built as a collection of services that decouple the data from the application and are modeled to scale by spinning up additional instances and leveraging a pool of shared resources, when needed. This means control to the network domain is limited.

A key customer requirement for moving workloads to the cloud is the need to employ independent, application-level monitoring and analytics of workload behavior. But the tools available to monitor the performance of your environment are virtual private cloud (VPC) flow logs or application metadata, both of which address some needs but do not provide the level of visibility that packet data does for complete clarity.

Figure 1 – Migration requires independent, application-level monitoring and analytics.

Requirement 2: Ability to Handle Complexity

Many organizations run tens of thousands of instances in the cloud. Often, these are logically separated into different VPCs, using metadata tags. Different departments or teams within an organization can build their own VPCs, leading to monitoring challenges. For example, it’s possible, even likely, there will be overlapping IP addresses in different VPCs. Managing thousands of instances and the amount of traffic this creates, while dealing with potential IP address overlap, must be considered in a cloud visibility architecture.

Requirement 3: Elastic Scale On-Demand

A well-architected cloud workload is designed to auto-scale to meet peak demand and contract when demand subsides. As applications scale to meet demand, new instances are created and destroyed dynamically. To be effective, a cloud visibility solution should accommodate the dynamic nature of events, without requiring manual VPC architecture changes or significantly modifying security group policies.

Requirement 4: Secure and High Availability Monitoring

To best expose native security capabilities of AWS, a visibility solution needs to integrate with the parameters available. One way to do this is to have visibility tools sit within the same VPC as workload instances. A key consideration for secure monitoring is ensuring that security policies configured for your workload and monitoring tool instances are applied to your AWS Cloud visibility.

If cloud visibility management components are in a separate VPC, like when using a virtual machine, you would have to configure and manage the security of that environment separately, leaving room for error. If that visibility node goes down, all the traffic being monitored and delivered to tools will go missing. This results in missing critical security events or indicators of compromise/breach.

Requirement 5: Cost-Effective Solutions

Organizations migrate and adopt cloud services for many reasons, including cost-effectiveness. It is counterproductive to guess and size visibility for peak demand. A solution for cloud visibility should work like the cloud itself—pay-per-use and cost-effective. A non-cloud-native approach to visibility creates substantial incremental costs, including complexity costs in deploying, operating, and managing, ensuring uptime, and the potential lack of security capability. Each of theses requirements add to the total cost of ownership.

CloudLens—a Solution for Complete Cloud Visibility

Ixia is an AWS Networking Competency Partner that built CloudLens to address cloud visibility with a cloud-native solution that is designed from the ground up to retain the cloud’s elastic scale, flexibility, and agility benefits.

There are two components of CloudLens that work together to enable visibility in the cloud:

• Software-as-a-Service (SaaS) visibility management. This is where users can configure visibility and define filtering. It is accessible from anywhere and is always available. It has a simple, easy-to-use interface with drag-and drop functionality.
• Sensors and connectors. Docker containers of software agents deployed within AWS source and tool virtual machine instances. Source instances are where users run their application workloads, and tool instances are where monitoring, analytics, or security forensics tools are deployed. The sensors and connectors are how CloudLens has full access to rich metadata because they sit within end user instances and communicate with each other over a secure VPN tunnel.

CloudLens addresses the five requirements we started earlier in this post, because it does the following:

• Provides access to packet data with the sensors that sit in source instances to mirror and filter packet data before forwarding to monitoring instances and tools.
• Addresses complexity because sensors operate peer-to-peer, over a secure VPN tunnel, so they don’t require an additional VPC or modifications to any existing construct. This makes it easy without any architectural changes.
• Elastically scales on-demand as instances are created and destroyed. CloudLens uses metadata from cloud platform instances to classify them. Because metadata inherently exists for each new instance created, the cloud platform automatically knows how to treat it and which security and monitoring policies need to be applied. An auto-scaling solution eliminates the burden of manual configuration, therefore mitigating the potential for error.
• Provides secure and highly available monitoring because the sensors and connectors are deployed within a cloud virtual machine instance and adheres to the same security policies associated with that instance. CloudLens sensors always send packet data via a secure VPN tunnel.
• Is cost-effective because it uses a pay-per-use model which tracks instances and can scale incrementally.

All of this is possible in a single platform because CloudLens is built to be cloud-native. It applies the tenets of containerization, dynamic orchestration, and is built as microservices. In contrast, a lift and shift solution is limited in which of these requirements it can address, and to what extent.
.

Figure 2 – CloudLens applies the tenets of containerization and is built as microservices.

Elastic Scale, Security, and Availability with CloudLens

CloudLens meets the five requirements of cloud visibility because of how its designed. Let’s examine each of the components of CloudLens and highlight how we leverage the AWS Cloud to deliver elastic scale, high availability, and a secure management portal.

AWS Lambda is at the core of CloudLens’s serverless architecture. AWS Lambda is designed to use replication and redundancy to provide high availability for both the service and the functions it operates, without maintenance windows or scheduled down times. Events are processed in milliseconds at whatever scale is required.

These combined features are designed to provide high availability, virtually unlimited scale, rapid elasticity, and high reliability of the CloudLens management portal. This allows Ixia to deliver a true software-as-a-service, without requiring customers to provision and manage their own VPC environment for visibility management software.

CloudLens leverages the Amazon API Gateway as a communication proxy, connecting CloudLens Docker agents and customer browser UI instances with our backend business logic. CloudLens Docker agents use API Gateway’s REST calls for management and cloud virtual instance metadata publication, while CloudLens UI session authentication and management also use the API Gateway.

Examples of metadata collected by CloudLens agents include underlying instance architecture, operating system information, CPU and memory utilization, performance metrics, pre-populated user data, and network statistics.
.

Figure 3 – CloudLens meets the five requirements of cloud visibility.

CloudLens uses the AWS IoT Core managed cloud platform to authenticate agents. It employs a bi-directional publish and subscribe model to allow agents to automatically “call home” to identify themselves to the backend portal. It also controls events to dynamically enforce configuration and operational changes on each CloudLens container.

Amazon DynamoDB further supports and reinforces our cloud-native architecture to deliver a reliable, low latency service at any scale. Its flexible data model, reliable performance, and automatic scaling of throughput capacity with single millisecond latency makes DynamoDB a great fit to power our managed cloud visibility that can seamlessly scale to tens or hundreds of thousands instances, on demand, without impacting customer experience. AWS CodeCommit is also used to securely store customer environment configuration data.

Ultimately, our integration with Zuora on AWS allows CloudLens to deliver usage-based pricing with on-demand rapid turn-up and a free 45-day trial.

How CloudLens Works

Configuring visibility begins by logging into CloudLens’s management portal and creating a Project. This creates a unique Project Key that is loaded into the visibility sensors running in the source and tool instances, which will be isolated as part of a Project. Once the key is installed in the visibility sensors, they phone home to the central management platform with metadata about the instances.

The management interface has a smart search capability, which allows users to create source and tool groups based on metadata. This information is auto-populated as search criteria in the management platform. The metadata can also be user defined, allowing maximum flexibility. As new instances are created, they are automatically added to groups based on their metadata. This retains scalability and elasticity in a cloud visibility solution.

Figure 4 – Users can associate source groups with tool groups to create an encrypted secure visibility path.

The next step in configuration is for users to associate source groups with tool groups to create an encrypted secure visibility path. In CloudLens, this is done in a point-and-click visual interface as shown in Figure 4. Once defined, the secure visibility path transfers filtered packet data from source to tool instances.

Together, the sensor, management platform, and secure visibility path address the challenge of providing visibility within AWS.

Next Steps

As enterprises move to AWS, visibility solutions that provide security and compliance are required. To scale, filtering rules cannot be static; rather they must be based on workload attributes and type of traffic. Ixia’s CloudLens has a serverless architecture that scales with distributed software systems built for cloud scale, which delivers intelligent, resilient, and proactive cloud visibility.

Visit the CloudLens website to learn how you can start eliminating the visibility blind spots of your AWS environment and access the data you need, where and when you need it.

*Gartner, Market Insight: Cloud Computing’s Drive to Digital Business Creates Opportunities for Providers, Refreshed: 24 July 2017 | Published: 24 May 2016

The content and opinions in this blog are those of the third party author and AWS is not responsible for the content or accuracy of this post.

.

Ixia – APN Partner Spotlight

Ixia is an AWS Competency Partner. A Keysight business, it helps enterprises, service providers, and governments accelerate innovation to connect and secure the world.

Contact Ixia | Solution Overview

*Already worked with Ixia? Rate this Partner