Verify enclave counterparties with reproducible builds and cryptographic attestation using AWS Nitro Enclaves

In the post Establishing verifiable security: Reproducible builds and AWS Nitro Enclaves, we introduced deterministic and reproducible builds, highlighting their critical role in securing the software supply chain. We also explored how software reproducibility connects to remote attestation, demonstrating how AWS Nitro Enclaves enable decentralized remote verification of runtime and software integrity using cryptographic attestation.

In this post, we take a deeper look at building a trusted enclave build environment, focusing on how to prevent attackers from tampering with build infrastructure components, such as continuous integration (CI) runners. We show how you can establish trust between enclaves in synchronous and asynchronous communication scenarios, by combining reproducibility and cryptographic attestation. Finally, we address the challenge of distributing hash measurements securely, introducing the concept of public and private notaries.

The Fireblocks CI Enclave Builder

Bit-by-bit reproducible builds allow others to verify you’re running the same code you claim that you are, but they don’t protect you from building and signing the wrong thing in the first place, as discussed by Ken Thompson in Reflections on Trusting Trust.

Nitro Enclaves provide strong isolation and tamper-resistance at runtime through memory and CPU isolation. However, if the system used to build those enclaves doesn’t offer similar guarantees, it becomes an obvious weak link. A malicious actor could target a component in your build infrastructure (such as a CI runner) instead of the enclave itself.You can address this problem in different ways:

• Air-gapped, secured build machine: A dedicated, physically secured and access-restricted machine used to build and sign enclave images. It’s not as agile as a cloud-based CI runner, but it provides isolation and makes sure secrets like signing keys never leave the machine.
• Enclave Builder: An enclave that builds other enclaves. When triggered from CI, it requires multiple independent signatures from trusted maintainers to authorize building a specific commit, enforcing intent verification on top of other safeguards (for example, only allowing recent commits on the main branch).

Fireblocks combined these primitives to create a secure CI build pipeline, depicted in the following process flow diagram.

The workflow consists of the following steps:

1. An Enclave Builder Enclave Image File (EIF) is created one time in an air-gapped build machine with strict access control.
2. Whenever CI needs a build, it spawns that Nitro Enclave, which clones the target code and verifies maintainers’ signatures to authorize the build.
3. To obtain build-time secrets (for example, EIF signing keys), the Enclave Builder sends an attestation document to a secrets store pre-loaded with the Enclave Builder’s Platform Configuration Register (PCR) measurements. PCR measurements are cryptographic measurements (hash values) that are unique to an enclave such as PCR0, which is a contiguous measure of the contents of the image file excluding the data section. These PCR measurements are calculated during the Enclave Builder’s air-gapped EIF build. Only the build-time secrets required for that specific build are provisioned after verifying the attestation document.
4. The Enclave Builder builds the artifact inside its tamper-proof enclave and pushes the signed results to the internal registry. The internal registry is a server or service hosted within an organization’s own infrastructure that stores, manages, and distributes software artifacts such as container images, binaries or packages.

How to establish trust between enclaves

When a reproducible EIF is running as an Nitro Enclave, the cryptographic attestation feature allows an independent party to verify that it is interacting with the expected code and that the code is running in a trusted AWS enclave runtime environment.

As an example use case, consider two MPC cosigners that work together to sign a payload. Multi-party computation (MPC) cosigners are independent services executing MPC algorithms, such as MPC-CMP, to collaboratively sign payloads by passing messages between each other over several communication rounds. Each cosigner in this setup plays both roles as depicted in the following flow chart.

1. The attesting party peer, presents its own attestation document to the relying peer.
2. The relying party peer validates the received attestation document and compares values against set of trusted PCR{0,1,2} measurements.
3. Relying party initiates same check in the opposite direction, sending its own attestation document.
4. Former attesting party, now serves as the relying party and validates the received attestation doc against own set of trusted measurements.

Three immediate questions come to mind:

• How can we link the PCR measurements to the applicative data?
• How do we maintain the set of trusted measurements, given the measurements are only known after the EIF is built (and therefore the EIF can’t hardcode its own measurement)?
• How can we adapt to dynamically changing measurements, for example due to new versions of the cosigner or deprecation of older ones?

How to link PCR measurements

For the first question, the answer depends on whether the communication is synchronous or asynchronous.

AWS KMS based approach

A straightforward way to link PCR measurements to applicative data is to use the cryptographic attestation feature together with AWS Key Management Service (AWS KMS). The idea is simple: two enclaves that want to establish a trust relationship first must decrypt a common shared secret using AWS KMS. This secret could be a temporary symmetric key. To do so, both enclaves authenticate themselves against AWS KMS and have their identity validated. Due to the transitive trust relationship of both enclaves with the AWS KMS symmetric key, the enclaves can now trust each other and start to communicate, for example using the temporary symmetric key. Though technically simple, this approach always requires AWS KMS, which introduces additional API calls overhead.

Synchronous communication

For synchronous communication channels, you can extend Transport Layer Security (TLS) to include the attestation document in the handshake, which some drafts aim to standardize.

Alternatively, after a connection is established, an applicative authorization layer could require a valid attestation document before any further processing. The attestation document should include the expected PCR measurements, nonces or other replay-protection techniques and must be signed by AWS.

For a conceptual design of such an applicative authorization layer, refer to Nitriding: A tool kit for building scalable, networked, secure enclaves, section 3.3.2 “Authenticating secure channels”.

After the authentication step, communication is protected by TLS with TLS termination happening inside the enclave.

Asynchronous communication

For asynchronous communication channels based on message passing using a (usually untrusted) message broker, you can embed a hash over the applicative message payload (for example, the payload of one of the MPC rounds’ response) in the attestation document’s user_data field. The attestation document is then distributed along with the payload.

The asynchronous communication flow is depicted in the following process diagram.

The communication flow consists of the following steps:

1. The attesting cosigner’s enclave sends an MPC round N payload including the attestation document to the message broker.
2. The relying cosigner’s enclave retrieves the attesting cosigner’s message from the message broker.
3. The relying cosigner verifies the attestation document.
4. The relying cosigner hashes the received payload and compares it to user_data, verifying it was neither altered nor produced by an untrusted party.

Refer to Build secure multi-party computation (MPC) wallets using AWS Nitro Enclaves, to learn more about the asynchronous communication pattern and the Fireblocks approach to implement secure MPC Cosigner communication.

How to decide which measurements to trust

Now let’s explore the second and third question on how to decide which measurements (PCR values) to trust and how to adapt to dynamically changing measurements.

Closed set of measurements

The simplest scenario is when the relying party knows, prior to building the EIF, the set of (static) PCR measurements it wants to trust and that set remains relatively stable over time. In that case, those measurements can be hardcoded into the relying party’s EIF, for example as a source file with a set of constants.

When an attester is handing out its attestation document (using either a synchronous or asynchronous communication channel), the PCR values can be looked up in that constant, hard-coded set to decide whether to continue the communication or cut it.

Open set of measurements – Measurement Notary Service Architecture

Confidential computing environments require a trusted mechanism for managing software artifact measurements, particularly when the set of trusted measurements cannot be predetermined at build time. A notary service addresses two critical scenarios: enabling relying parties to trust their latest build measurements without circular dependencies and supporting dynamic updates to trusted measurement sets without requiring constant rebuilds and redeployments.

Core Capabilities

A notary service provides three essential functions:

• It validates and accepts new measurements or updates existing measurements for trusted Enclave Image Files (EIFs).
• It maintains comprehensive audit capabilities to ensure transparency around measurement submissions and their sources.
• It exposes interfaces accessible from public or private continuous integration/continuous deployment (CI/CD) pipelines to enable seamless integration into existing development workflows.

Authorization Models

Organizations can implement authorization through multiple approaches. The signature-based model requires K out of N authorized signers to approve new trusted measurements before the notary stores them in its database. Alternatively, organizations can execute the build process inside a secure enclave, such as AWS Nitro Enclaves, and deliver resulting measurements to the notary through an authenticated channel using either synchronous or asynchronous communication patterns as discussed in the previous section.

Implementation Options

For implementation, organizations can leverage existing infrastructure and security primitives. AWS based implementations can utilize AWS KMS for cryptographic operations, Amazon S3 for measurement storage, and AWS Secrets Manager for credential management. These implementations can operate within private or public CI/CD pipelines depending on organizational security requirements. Integration with sigstore.dev provides public verification capabilities specifically designed for open-source software supply chain security.

Reproducibility Considerations

The notary serves trusted measurements to enclaves during their bootstrap process. For notaries with infrequent code changes, embedding the notary’s PCR values directly into the relying party’s EIF provides a straightforward solution. However, when the notary undergoes frequent updates, verification should focus on PCR8, which represents the EIF signer’s identity. This approach ensures that any EIF signed with the same cryptographic key produces identical PCR8 values, enabling consistent verification despite code changes.

End-to-End Runtime Verification Flow

The following diagram illustrates the workflow from initial notary EIF build, to requesting or updating measurements on the secrets store to serving trusted measurements to relying parties.

The workflow consists of the following steps:

1. A notary EIF is created one time in an air-gapped build machine with strict access control.
2. An optional human or Enclave Builder sends new trusted measurements to be stored by the notary.
3. The notary requests and stores trusted measurements in the secrets store.
4. The MPC cosigner A initiate’s communication with MCP cosigner B. Cosigner A sends an attestation document including PCR values.
5. MPC cosigner B requests measurements from the notary to verify MCP cosigner A.

Conclusion

In Establishing verifiable security: Reproducible builds and AWS Nitro Enclaves, we explored the fundamental concepts of deterministic and reproducible builds, how it is a crucial aspect of secure software supply chains, and how it can be combined with Nitro Enclaves cryptographic attestation to enable remote verification of software and runtime environments.

In this post, we explored how Fireblocks implements trusted build environments and demonstrated the powerful combination of reproducible builds and Nitro Enclaves cryptographic attestation to establish secure trust relationships between enclaves. We presented various enclave communication patterns, with special focus on asynchronous communication mechanisms for securing MPC blockchain cosigner interactions. Additionally, we examined robust approaches for distributing hash measurements through a notary system, providing a comprehensive framework for maintaining security across your enclave infrastructure.

Now go, explore hardened CI/CD systems and AWS Nitro Enclaves cryptographic attestation capabilities and start establishing verifiable trust relationships between your applications!

About the authors