Client-Side Data Encryption with the AWS SDK for Java and Amazon S3

Many applications process sensitive data and have requirements to securely store and retrieve that data. The AWS SDK for Java provides an easy-to-use Amazon S3 client that allows you to securely store your sensitive data in Amazon S3. The SDK automatically encrypts data on the client side when uploading to Amazon S3, and automatically decrypts it for the client when data is retrieved. Your data is stored encrypted in Amazon S3 - no one can decrypt it without your private encryption key.

You control the private encryption keys used for the encryption/decryption, and those keys never leave your application. It is your responsibility to protect your encryption keys just like you protect your AWS security credentials. If you lose your encryption keys, you will not be able to unencrypt your encrypted data. Since all the encryption/decryption happens inside your application, AWS has no knowledge of your encryption keys and cannot help you recover your private encryption keys if you lose them.

The AmazonS3EncryptionClient implements the same interface as the standard AmazonS3Client, allowing you to easily switch to the encryption client, without your application code having to be aware of the encryption and decryption happening automatically in the client.

If you haven't already done so, you can download the AWS SDK for Java from https://aws.amazon.com/sdkforjava. If you're an Eclipse user, you can also get the SDK by installing the AWS Toolkit for Eclipse from https://aws.amazon.com/eclipse. (With Eclipse, you can get updates to the AWS SDK for Java automatically through the AWS Eclipse update site.) To create an instance of AmazonS3EncryptionClient, you need to supply:

• Your AWS security credentials - BasicAWSCredentials
• Your keys for performing the encryption/decryption - EncryptionMaterials

// Generates a sample asymmetric key pair.
//
// IMPORTANT: In a real application, you need to save your encryption
//            key pair somewhere so you don't lose it when the JVM exits.
//            Your encryption keys NEVER leave your application,
//            so it's important that you safely manage them.
//            AWS doesn't know anything about your encryption keys, so
//            if you lose them, AWS can't help you recover them, or help
//            you decrypt any stored data.
//
// Several good online sources explain how to easily create an RSA key pair
// from the command line using OpenSSL, for example:
// https://en.wikibooks.org/wiki/Transwiki:Generate_a_keypair_using_OpenSSL
KeyPairGenerator keyGenerator = KeyPairGenerator.getInstance("RSA");
keyGenerator.initialize(1024, new SecureRandom());
KeyPair myKeyPair = keyGenerator.generateKeyPair();


// Construct an instance of AmazonS3EncryptionClient
AWSCredentials credentials = new BasicAWSCredentials(myAccessKeyId, mySecretKey);
EncryptionMaterials encryptionMaterials = new EncryptionMaterials(myKeyPair);
AmazonS3EncryptionClient s3 = new AmazonS3EncryptionClient(credentials, encryptionMaterials);


// Then just use the encryption client as normal...
//
// When we use the putObject method, the data in the file or InputStream
// we specify is encrypted on the fly as it's uploaded to Amazon S3.
s3.putObject(bucketName, key, myFile);


// When you use the getObject method, the data retrieved from Amazon S3
// is automatically decrypted on the fly.
S3Object downloadedObject = s3.getObject(bucketName, key);

All encryption/decryption happens exclusively in your application using a process called "envelope encryption." Your private encryption keys and your unencrypted data are never sent to AWS, so it's very important that you safely manage your encryption keys. If you lose your encryption keys, you won't be able to unencrypt your data, and you can't recover your encryption keys from AWS, since AWS doesn't know anything about them.

The goal of envelope encryption is to combine the performance of fast symmetric encryption while maintaining the secure key management that asymmetric keys provide. A one-time-use symmetric key (the envelope symmetric key) is generated by the Amazon S3 encryption client to encrypt your data, then that key is encrypted by your master key and stored alongside your data in Amazon S3. When accessing your data with the Amazon S3 encryption client, the encrypted symmetric key is retrieved and decrypted with your real key, then the data is decrypted. Another benefit of envelope encryption is that if your master key is compromised, you have the option of just re-encrypting the stored envelope symmetric keys, instead of re-encrypting all the data in your account.

• Generate a one time use envelope symmetric key.
• Encrypt the file data using this envelope key.
• Encrypt that envelope key using a master public key or symmetric key.
• Store this encrypted envelope key with the encrypted file.
• Store a description of the master key alongside the envelope key to uniquely identify the key used to encrypt the envelope key.

• Retrieve the encrypted envelope key you stored with the encrypted file.
• Retrieve the description of the original master key.
• If the description of the master key on hand does not match the description of the original master key, use the unique description to fetch the original master symmetric key or private key.
• Decrypt the envelope key using the master key.
• Decrypt the file data using the envelope key.

For more advanced scenarios, such as when you want to support rotating the keys you use to encrypt/decrypt data, the AWS SDK provides the EncryptionMaterialsAccessor interface.

When decrypting data from Amazon S3, the AWS SDK will first check whether the description of the original encryption key (stored either as object metadata or as a separate object), matches the description for the current EncryptionMaterials. If the description does not match, then the SDK will ask the EncryptionMaterialsAccessor for the EncryptionMaterials with the matching description. Because of this verification, it's important that the descriptions you provide from your custom EncryptionMaterials subclass uniquely identify the EncryptionMaterials so that they can be correctly retrieved later.

By implementing the EncryptionMaterialsAccessor interface and a subclass of EncryptionMaterials, it's easy to integrate the Amazon S3 encryption support in the AWS SDK with your own key management system. For example, the following code implements EncryptionMaterialsAccessor to integrate with an existing private key management system.

public class MyEncryptionMaterialsAccessor implements EncryptionMaterialsAccessor {
    EncryptionMaterials getEncryptionMaterials(Map materialsDescription) {
        // Use the information in the materialsDescription to lookup the corresponding key
        // in your existing private key management system, then return an EncryptionMaterials object.
        String uniqueKeyPairId = materialsDescription.get(MyEncryptionMaterials.INTERNAL_KEY_PAIR_ID);
        keyPair = callOutToExistingKeyManagementSystem(uniqueKeyPairId);
        return new MyEncryptionMaterials(keyPair, uniqueKeyPairId);
    }
}

public class MyEncryptionMaterials extends EncryptionMaterials {
    private static final String INTERNAL_KEY_PAIR_ID = "uniqueKeyPairId";
    private final String uniqueKeyPairId;

    public MyEncryptionMaterials(KeyPair keyPair, String uniqueKeyPairId) {
        super(keyPair);
        this.uniqueKeyPairId = uniqueKeyPairId;
    }

    EncryptionMaterialsAccessor getAccessor() {
        return new MyEncryptionMaterialsAccessor();
    }

    @Override
    Map getMaterialsDescription() {
        Map map = new HashMap();
        map.put(INTERNAL_KEY_PAIR_ID, uniqueKeyPairId);
        return map;
    }
}

You can choose between using asymmetric or symmetric encryption materials. We recommend using asymmetric encryption materials because of the increased security.

Asymmetric

The more secure of the two encryption mechanisms. It uses your asymmetric keys to encrypt and decrypt the envelope key, which is used to encrypt and decrypt object data. With an asymmetric key pair, you can give a party one of the keys, and that party will only have access to one of the encrypt or decrypt operations. For example, a data generator could be given a public key to encrypt data and store it in Amazon S3. If this data generator were ever compromised, the data stored would remain safe since the private key is still needed to decrypt the data. To use asymmetric encryption to store your data, simply pass an RSA asymmetric key pair to the constructor of the EncryptionMaterials object. Then use this EncryptionMaterials object to construct the AmazonS3EncryptionClient:

EncryptionMaterials materials = new EncryptionMaterials(myKeyPair);
AmazonS3 encryptionClient = new AmazonS3EncryptionClient(credentials, materials);

Symmetric

The faster of the two encryption mechanisms. It uses your symmetric key to encrypt and decrypt the envelope key, which is used to encrypt and decrypt the actual object data stored in S3. To use symmetric encryption to store your data, simply pass an AES symmetric key to the constructor of the EncryptionMaterials object. Then use this EncryptionMaterials object to construct the AmazonS3EncryptionClient:

EncryptionMaterials materials = new EncryptionMaterials(mySymmetricKey);
AmazonS3 encryptionClient = new AmazonS3EncryptionClient(credentials, materials);

The encrypted envelope symmetric key must be stored somewhere with the encrypted file so that the envelope key can be used to decrypt the file later on. We can store the envelope symmetric key in two places:

• Object Metadata - The envelope symmetric key is stored directly in the encrypted file's metadata.
• Instruction File - The envelope symmetric key is stored in a separate instruction file which is then placed in the same location as the encrypted file.

Storing encryption metadata in an instruction file will result in extra PUT and GET operations, which can be expensive for some use cases. However, if an instruction file is used, then the object metadata is completely open for use and will not contain any encryption information.

In "Object Metadata" storage mode, encryption metadata is stored in the object metadata during the encryption process. Depending on the strength of the keys you provide, between 200 bytes and 1 KB of encryption information will be stored in the metadata. Be careful that you do not store more than 2 KB of information in the metadata. You will get MetdataTooLarge error if the size of the metadata is greater than 2 KB.

The Object Metadata storage mode is used by default, but if you would like to explicitly select the storage mode, use the CryptoConfiguration object.

CryptoConfiguration cryptoConfig = new CryptoConfiguration().withStorageMode(CryptoStorageMode.InstructionFile);
AmazonS3 encryptionClient = new AmazonS3EncryptionClient(credentials, materials, cryptoConfig);

If you want to use a crypto provider other than the default JCE crypto provider, you can specify your own provider with the CryptoConfiguration object:

// Bouncy Castle is a third party crypto provider, used as an example.
Provider bcProvider = new BouncyCastleProvider();
CryptoConfiguration cryptoConfig = new CryptoConfiguration().withCryptoProvider(bcProvider);
AmazonS3 encryptionClient = new AmazonS3EncryptionClient(credentials, materials, cryptoConfig);