Adding MACsec security to AWS Direct Connect connections

AWS Direct Connect now supports MACsec security (IEEE 802.1AE), giving you a new option for securing your data from when it leaves your network until it arrives at AWS. With this release, Direct Connect delivers native, near line-rate, and point-to-point encryption for 10 Gbps and 100 Gbps links. Available at select locations for dedicated connections (that is, when a Direct Connect Ethernet port is dedicated to a single customer), MACsec is a great option if you need a secured connection to AWS at the highest speed available.

So first things first, let’s understand why we introduced MACsec support for Direct Connect.

Why use MACsec with AWS Direct Connect?

At AWS we have a motto, we keep repeating to our customers: “Dance like nobody is watching. Encrypt like everyone is.” We already support IPsec VPNs over Direct Connect, and we already use TLS for our applications. So why introduce MACsec?

In short, the answer is that MACsec operates at layer 2 of the TCP/IP stack. This protects Ethernet connections while the two other protocols protect the upper layers of the stack. This has the following benefits:

Benefit 1: Layer 2 Confidentiality and integrity

MACsec Layer 2 encryption and integrity protects ethernet links from threats, such as man in the middle snooping and passive wiretapping. MACsec allows you to secure an Ethernet link—including all control plane protocol packets (such as ARP, DHCP, LLDP, and Layer 3 routing protocols like BGP).

Benefit 2: High speed encryption

MACsec can encrypt data faster, which results in greater speed. Because MACsec encryption is done through hardware (ASIC/PHY) it provides bi-directional line-rate, or near line-rate, encryption. In comparison, IPsec typically relies on a dedicated crypto engine or chip. These deliver a fraction of the overall throughput capabilities of the router or switch, as their performance is largely influenced by the packets size.

With the constantly growing amount of data transiting between on-premises data centers and the cloud, demand for Direct Connect bandwidth is outpacing the encryption rates that a traditional IPsec solution can support. AWS customers in regulated industries, such as financial services or healthcare, and/or customers with high-bandwidth workloads that have strict security requirements, such as media production and autonomous vehicle development, tell us they are running into these limits more and more often.

What is MACsec?

Media Access Control security (MACsec) is an IEEE 802.1AE standard for encrypting packets traveling between two MACsec-capable Ethernet switches/routers. (The current version, at the time of this writing, is IEEE 802.1AE-2018.)

In comparison to upper layer protocols such as IPSec and TLS for instance, MACsec was originally designed as a hop-by-hop encryption technology. Frames are encrypted when they are sent on the wire and are decrypted by the next hop device processing the frames.

MACsec brings these key benefits to Layer 2 Wired communication:

• Confidentiality: MACsec helps ensure confidentiality, providing strong encryption at layer 2 by encrypting the frame’s ethertype and payload.
• Data Integrity: MACsec provides integrity checking to help ensure that data cannot be modified in transit
• Data origin authenticity: MACsec ensures the frame was sent by one of the trusted MACsec peers.
• Replay protection: MACsec ensures frames are not processed out of order

Let’s look at some key MACsec concepts:

• Secure Channel
• Secure Associations
• MACsec encapsulation
• MACSec Key Agreement
• Cipher Suite

Secure Channel

Each MACsec participant creates a Secure Channel that is used to send traffic to the other participant. These secure channels are unidirectional, one channel is used to send traffic to the other participant, the second channel is used to received traffic from the other participant.

The secure channels are assigned an 8 bytes identifier, the Secure Channel Identifier (SCI).

Secure Associations

Communication on each secure channel take place as a series of transient sessions called secure associations. Each secure channel contains up to two Secure Associations (SAs) at any point in time.

In general a channel only has one SA, however when a SA needs to be replaced the channel will have two SAs while they are swapped from one to the other, such as when keys are rotated.

The Secure Association contains two critical pieces of information, which are required to encrypt and protect the frames.

1. A Secure Association Key (SAK), which is an encryption key derived from a pre-shared Key
2. Counters related to Packet Numbers (PN); these Packet Numbers are used during the encryption and verification process and for replay protection. At the transmission end of the channel, the counter is used to record what is the next Packet Number and at the receiving end, the SA keeps a record of which Packet Number is expected next.

Packet Numbers are used by the MACsec replay protection mechanism. When replay protection is enabled, MACsec drops frames whose numbers are too far from the expected order. The peer receiving a frame compares its number with its lowest acceptable PN and drops any frames with lower numbers.

What actually defines the lowest acceptable PN is the replay window, the lowest acceptable PN is basically derived from the next expected number – the configured replay window.

MACsec encapsulation

MACsec modifies the ethernet frame by inserting a 16 bytes MACsec tag (SecTAG) just before the EtherType and by adding a 16 bytes Integrity Check Value (ICV) at the end of the frame. The original EtherType and frame payload are encrypted and inserted between the SecTAG and the ICV. (Including the original Dot1q tag.)

Figure :1 MACsec Encapsulation

MACSec Key Agreement Protocol (MKA)

The MACsec Key Agreement protocol (MKA) manages peer discovery, authentication, and encryption key generation. This includes discovering peer devices and verifying their legitimacy by checking if they are configured to match the following items:

The Secure Connectivity Association Key (CAK): The CAK is a static key that is associated with each MACsec-enabled interface. In AWS’ implementation, and in fact in most of the implementations, a pre-shared key is used and initially exchanged offline.

The Connectivity association Key Name (CKN): A name that’s included, in the form of a string, in each MKA message to identify which CAK is being used. MACsec authentication is based on mutual possession and acknowledgment of the pre-configured CAK and Connectivity Association Key Name (CKN). MKA uses the Connectivity Association Key (CAK) to derive the Secure Association Key (SAKs are the session keys). The Secure association keys have a lifetime and are getting refreshed over a period of time, that time period is defined as a certain amount of frames. SAKs, along with other essential control information, are distributed in MKA protocol control packets, also referred to as MKPDUs.

Note: The CKN and CAK each need to be 32 Octets expressed as 64 Hex digits

Cipher suite

MACsec originally used GCM-AES-128 as a cipher suite. It was later amended to add an option to use GCM-AES-256. GCM-AES-256 extends packet numbering and adds support for the GCM-AES-XPN-128 and GCM-AES-XPN-256 cipher suites. These additional ciphers were introduced to deal with fast packet number exhaustion (PN field). At high link speeds, the original 32-bit packet number (PN) field was not able to handle higher speed interfaces and could cause a new security association to occur every 5 minutes for a 100G interface (for example, establishing new SA de facto forces a key rollover).

Note: AWS Direct Connect MACsec is using GCM-AES-XPN-256. UPDATE: AWS Direct Connect MACsec now supports GCM-AES-XPN-256 on 100Gbps connections, and GCM-AES-XPN-256 and GCM-AES-256 on 10Gbps connections.”

A specific cipher is used to encrypt the MKA control packets, in AWS implementation we currently support AES-CMAC-256.

Configuring MACsec for AWS Direct Connect

To enable the Direct Connect MACsec feature, you must migrate to a MACsec capable Direct Connect circuit.

This is done by selecting the “request MACsec” option in the AWS Direct Connect console.

Figure 2: Direct Connect Console MACsec Request

Last Mile Provider

Since MACsec protects links on a hop-by-hop basis, you must have a dedicated connection that is transparent to layer 2 traffic and therefore MACsec compatible. Your device must have a direct layer 2 adjacency with our Direct Connect device. Your last-mile provider can help you verify that your connection will work with MACsec.

Pre-Requisites

One of the first steps in configuring MACsec for Direct Connect is to choose the on-premises equipment that connects to the Direct Connect device, and make sure your devices supports MACsec. It is important that both ends of the connection agree on the following AWS default configuration options:

• CKN Length: 32 Octets expressed as 64 Hex digits.
• CAK Length: 32 Octets expressed as 64 Hex digits.
• Cryptographic Algorithm: AES-CMAC-256
• SAK Cipher Suite: GCM-AES-XPN-256
• Confidentiality Offset: 0
• ICV Indicator: No
• SAK Rekey Time: PN Rollover

Topology

Because MACsec protects links on a hop by hop basis, your device must have direct layer 2 adjacency with the AWS Direct connect device. This leads to two likely topologies:

Topology 1: Router – Switch – Direct Connect Device

Figure 3: Router-Switch-Direct Connect Device Topology

In this topology MACsec is configured between your on-premises switch and an AWS Direct Connect device. IP routing is handled by the on-premises router connected to your switch.

Topology 2: Router – Direct Connect 

Figure 4: Router – Direct Connect Device Topology

In this topology MACsec is configured between your on-premises Router and the AWS Direct Connect device.

Note: In both topologies, a Link Aggregation Group (LAG) can be used to consolidate links to achieve higher capacity.

AWS Direct Connect MACsec Configuration

You configure Direct Connect MASsec using the AWS Management Console or the Direct Connect API. We are introducing new Direct Connect APIs as part of our MACsec rollout, and some currently available APIs are extending to support MACsec features. We are also delivering an updated AWS CLI version that supports these new API methods.
The configuration workflow is pretty straightforward:

1. Create a new connection with MACsec support
2. Associate the CKN/CAK with the connection
3. Verify the connection status
4. Migrate traffic to new connection as appropriate

Step 1: Create a new connection with MACsec support

The Direct Connect CreateConnection API has been extended with a new requestMACSec field. You can use the AWS CLI to request MACsec support for a new connection with the following:

aws directconnect create-connection --location IAD66 --bandwidth 10Gbps --connection-name "MACsec connection" --request-mac-sec

Step 2: Associate the CKN/CAK with the connection

A new Direct Connect API AssociateMacSecKey has been introduced to associate a CKN/CAK pair with a MACsec-enabled port (or LAG).

You can associate the CKN/CAK with the connection in two ways, either you have already defined the pair in AWS Secret Manager (and you are referencing the secret ARN), or you directly specify the CKN/CAK values.

In the latter case we create secrets in AWS Secrets Manager to store the CKN/CAK values you entered and associate them with the connection.

Here are two examples of how to use the AWS CLI to associate a CKN/CAK with a connection:

• Referencing a pre-defined secret:

aws directconnect associate-mac-sec-key --secret-arn secretARN --connection-id connectionID

• Creating the CKN/CAK pair:

aws directconnect associate-mac-sec-key --ckn ckn --cak cak --connection-id connectionID

You cannot modify a MACsec secret key after you associate it with a connection. If you need to modify the key, disassociate the key from the connection, and then associate a new key with the connection.

Step 3: Verify the connection status

The Direct Connect DescribeConnections API is extended to include status of MACsec related configurations in API response such as:

• The connection MACsec support
• The encryption port status and the CKN in use
• The encryption mode
• The MACsec keys associated with the port

Here is a snippet of the aws directconnect describe-connection output, highlighting the MACsec specific values:

<snip>
"macSecCapable": true,
"portEncryptionStatus": "Encryption Up with CKN 444CF2486F0B7890FE88279F15DA27C31DB301C47DE95A2FE0E2F42A5D68107F",
"encryptionMode": "should_encrypt",
"macSecKeys": [
           { "secretARN": "arn:aws:secretsmanager:useast-1:123456789012:secret:directconnect!example/us-east-1/ dxmacsec/222CF2486F0B7890FE88279F15DA27C31DB301C47DE95A2FE0E2F42A5D68107F-7lsJMj",
            "ckn": "222CF2486F0B7890FE88279F15DA27C31DB301C47DE95A2FE0E2F42A5D68107F",
            "state": "associated", "startOn": "2020-11-10 21:57:42" },
            { "secretARN": "arn:aws:secretsmanager:useast-1:123456789012:secret:directconnect!example/us-east-1/ dxmacsec/444CF2486F0B7890FE88279F15DA27C31DB301C47DE95A2FE0E2F42A5D68107F-Yf88hh",
            "ckn": "444CF2486F0B7890FE88279F15DA27C31DB301C47DE95A2FE0E2F42A5D68107F",
            "state": "associated", "startOn": "2020-12-17 21:46:45" } ]
</snip>

In this output you can see the port is associated with two keys as shown by the “macSecKeys” and the port encryption status is up and using the CKN starting with 444CF2…., and the encryption mode is “should_encrypt”.

The encryption mode parameter defines how the port participates in MKA and MACsec. The possible values are:

• should_encrypt: The connection attempts MKA and, if successful, the connection only sends and receives encrypted traffic. If MKA times out or fails, the connection permits unencrypted traffic.
• must_encrypt: The connection attempts MKA and, if it succeeds, the connection only sends and receives encrypted frames. If MKA times out or fail the connection goes down with encryption down. The authentication is retried after a while
• no_encrypt: The connection does not perform MKA. Any MKA frames received are ignored. The connection will only send and receive unencrypted frames.

The default value for the encryption mode is “should_encrypt”, and this can be changed using the new DirectConnect API UpdateConnection.

Here is an example using the AWS CLI:

aws directconnect update-connection --connection-id dxcon-ffs0c7qr --encryption-mode must_encrypt

Step 4: Migrate Traffic to new connection as appropriate

You can migrate existing VIFs that are associated with a connection using the existing AssociateVIF AWS Direct connect API. This process involves downtime as the change is provisioned on our end.

You can also build out additional VIFs and migrate to the new links by updating your routing configuration. The Using the AWS Direct Connect Resiliency Toolkit to get started page in our documentation is a useful resource as you go through the process. For a quick and very high-level introduction to the Resiliency Toolkit, a blog post on Exploring the AWS Direct Connect Resiliency Toolkit is available as well.

AWS Direct Connect MACsec and Link Aggregation Group (LAG)

MACsec is also supported on Direct Connect Link Aggregation Group (LAG).

A key point with regards to LAG and MACsec, is that the MACsec-specific configuration is done at the LAG level and not at the Direct Connect Connection level.

The configuration workflow consists of:

• Creating the LAG and requesting MACsec capable ports (note that all connections must support MACsec)
• Associate the CKN and CAK to the LAG (All member connections will use the same CKN/CAK)
• Set the LAG encryption mode (Currently the default mode is no encryption)

AWS Direct Connect MACsec Key Management

Proper key management is critical for any MACSec implementation. That said, it can be challenging to securely store, distribute, rotate, and consume the CKN/CAK pairs used with MACsec. To reduce the operational overhead required for these tasks, MACsec for Direct Connect integrates with AWS Secrets Manager using a service-linked role.

AWS Secrets Manager helps you protect the secrets needed to access your applications, services, and IT resources. This service enables you to easily rotate, manage, and retrieve database credentials, API keys, and other secrets throughout their lifecycle. A service-linked role is a unique type of AWS Identity and Access Management (IAM) role that is linked directly to an AWS Service. For more information, see Service-Linked Role Permissions in the IAM User Guide.

As previously explained, to associate a new or previously stored MACsec key with a connection, you use the associate-mac-sec-key command (or the AWS Management Console) and either provide the CKN/CAK pair or the ARN of the secret already stored in AWS Secrets Manager. The first time you execute the associate-mac-sec-key command, AWS creates an IAM service-linked role that allows AWS Direct Connect to interact with AWS Secrets Manager to store and interact with the MACsec Keys. Therefore, you must ensure the account executing the associate-mac-sec-key command has access to create the service-linked role in your account. See details in the Using service-linked roles section of the IAM User Guide.

When it comes time to rotate keys, hitless key rollover is supported with MACsec keychains. Direct Connect MACsec supports MACsec keychains with capacity for storing up to 3 CKN/CAK pairs. You use the associate-mac-sec-key command to associate the CKN/CAK pair with the existing MACsec enabled connection. You then configure the same CKN/CAK pair on the device on your end of the AWS Direct Connect connection. The Direct Connect device will attempt to use the last stored key for the connection. If that key does not coincide with the key on your device, Direct Connect continues to use the previous working key.

The current Direct Connect MACsec implementation does not support key rotation based on key lifetime, to begin the key rotation you must associate a new stored key to the connection using the console or the API.

Summary

MACsec is currently supported on new dedicated 10 Gbps and 100 Gbps Direct Connect links in select Direct Connect locations.

Customers that require layer 2 confidentiality and integrity or high speed encryption for their Direct Connect connections will benefit from the new MACsec feature.

If you are interested in using MACsec, check the availability in your preferred Direct Connect location. If you do not have equipment at an AWS Direct Connect location, reach out to the AWS Partner Network who can assist you in accessing the AWS Direct Connect service from your location.

An update was made on January 12, 2023: An earlier version of this post stated that AWS Direct Connect MACsec is using GCM-AES-XPN-256. The post has been updated to reflect changes in the product offering: AWS Direct Connect MACsec now supports GCM-AES-XPN-256 on 100Gbps connections, and GCM-AES-XPN-256 and GCM-AES-256 on 10Gbps connections.

A correction was made on February 6, 2024: An earlier version of this post used the words “cipher” and “cypher” inconsistently. The post has been updated to consistently use the word “cipher” instead of  “cypher”.