AWS Developer Tools Blog
AWS SDK for C++ Version 1.8 is Now Generally Available
We’re happy to share that version 1.8 of AWS SDK for C++ is now generally available. AWS SDK for C++ provides a modern C++ (version C++ 11 or later) interface for Amazon Web Services (AWS). It is performant and fully functioning with low- and high-level SDKs, and minimizes dependencies. The AWS SDK for C++ also provides platform portability, including Windows, OSX, Linux, and mobile. To learn more visit the AWS SDK for C++ site.
Version 1.8 release is a minor version bump based on version 1.7.x. and introduces several new features based on customer feedback and modifications to the default build configurations. In this post, we overview a list of changes for version 1.8 of the AWS SDK for C++.
Below, items with an * include a breaking change from the previous version:
ENABLE_CURL_LOGGINGis nowONby default in CMake. *ENABLE_UNITY_BUILDis nowONby default in CMake. *- The deprecated version of function
MakeRequest()was removed. This function takes a reference to anHttpRequestobject as its argument, which is unsafe. * - Client configuration now reads environment variables, configuration file and EC2 metadata to get the default AWS region. *
- Exceptions may include more service and operation specific details now.
- New pseudo region:
aws-globalto make cross region requests. * - S3 client in
us-east-1now uses regional endpoint by default. * - Improvements on the underlying HTTP client override.
Updates in version 1.8 include: changes in the CMake default configuration, bug fixes, and new features.
Changes in the CMake default configurations
Turn on ENABLE_UNITY_BUILD by default
When ENABLE_UNITY_BUILD is turned on, most SDK libraries will be built as a single, generated .cpp file. This can significantly reduce static library size as well as speed up compilation time.
The following examples compare the compile time and binary size with ENABLE_UNITY_BUILD set to ON and OFF.
First, with the following Cmake flags to turn off ENABLE_UNITY_BUILD and then build the SDK:
cmake <path-to-source> -DENABLE_UNITY_BUILD=OFF -DBUILD_ONLY=s3 -DBUILD_SHARED_LIBS=OFF
make -j8It takes around 280 seconds and the binary size of libaws-cpp-sdk-s3.a is 9.3MB.
And then turn on this option:
cmake <path-to-source> -DENABLE_UNITY_BUILD=ON -DBUILD_ONLY=s3 -DBUILD_SHARED_LIBS=OFF
make -j8It takes around 145 seconds and the binary size of libaws-cpp-sdk-s3.a is 4.6 MB.
The results depends on the build machine, but it can still show the differences here.
Turn on ENABLE_CURL_LOGGING by default
When ENABLE_CURL_LOGGING is turned on, Curl’s internal log will be piped to the SDK’s logger if the logging level is greater than or equal to DEBUG. As this is turned on by default in version 1.8, your logs will be similar to the following:
With this now enabled, you get more raw data about headers, bytes read/written, secure channels and so on to help you debug and track the communication with server.
Bug Fixes
Remove unsafe version of MakeRequest()
In the class Aws::Http::HttpClient, a version of member function MakeRequest() is defined as following:
virtual std::shared_ptr<HttpResponse> MakeRequest(HttpRequest& request,
Aws::Utils::RateLimits::RateLimiterInterface* readLimiter = nullptr,
Aws::Utils::RateLimits::RateLimiterInterface* writeLimiter = nullptr) const = 0;This is unsafe, as it takes a reference to an HttpRequest object as a parameter and when this object is out of scope outside the function, the HTTP client will no longer be able get access to the original request.
We fixed that by adding another safe version of this function by passing a shared pointer to the HttpRequest object and marked the legacy one as “deprecated”, starting from version 1.4.46.
For version 1.8 we removed the deprecated functions. This will break your code if you have a subclass of HttpClient that has your own implementation of MakeRequest().
New Features
Client configuration now reads environment variables, configuration file and EC2 metadata to get default AWS region
With previous version of AWS SDK for C++, there are only two ways to specify regions for service clients with client configurations:
1. By specifying region in ClientConfiguration explicitly:
Aws::Client::ClientConfiguration config;
config.region = Aws::Region::US_WEST_2;
Aws::S3::S3Client s3Client(config);2. By specifying profile in ClientConfiguration explicitly and define the region in the configuration file:
Aws::Client::ClientConfiguration config("default");
Aws::S3::S3Client s3Client(config);Where your default configuration file (~/.aws/config) looks like:
In version 1.8, the region is automatically determined by checking environment variables, configuration files and EC2 metadata. This has been a top request by customers.
With version 1.8, here’s how ClientConfiguration determines the AWS region:
- If you specify a region in Client configuration, it will always override region from other sources.
- If you specify a profile with Client configuration, it will search configuration file for region associated with that profile. So far it’s the same as that of the previous version of SDK.
- If neither are specified, the SDK will try to determine the AWS region automatically based on environment variables. It will check
AWS_DEFAULT_REGIONfirst, thenAWS_REGION. In some cases, the environment variables could be pre-defined by something other than the SDK (for example, in a CodeBuild project or a Lambda function). - Next, the SDK will check the configuration files. The default profile is
defaultand the default configuration file is~/.aws/config. You can also specify these with environment variables.AWS_DEFAULT_PROFILE,AWS_PROFILEspecify the profile name, andAWS_CONFIG_FILEspecifies the configuration file. - As the last step, the SDK will check EC2 metadata for AWS region information, if your application is running on an EC2 instance. You can set environment variable:
AWS_EC2_METADATA_DISABLEDtotrueto disable it. - And finally, if you do nothing with Client configuration, environment variables or configuration files, and your code is not running in any pre-configured environments, like EC2 instances or CodeBuild projects, the default region is
us-east-1.
Exceptions may include more service and operation specific details now
In the previous version of the SDK, the data structure of AWSError, only returned limited information about exceptions. This included error type, exception name, and the error message. Some services return other useful information that could not be deserialized. For example, the Amazon Elastic File System operation CreateFileSystem returns the following response body for the FileSystemAlreadyExists exception:
{
"ErrorCode": "FileSystemAlreadyExists",
"FileSystemId": "fs-some-id",
"Message": "File system 'fs-some-id' already exists with creation token 'basic-file-system-creation-some-token'“
}With previous version, the SDK can only get the ErrorCode and Message from the payload.
To support modeled exceptions, we introduced a similar interface, without breaking changes. The interface is easy to use. Just provide the type when getting the error: outcome.GetError<ERROR_TYPE>(). Here is an example of how to use this new interface:
CreateFileSystemRequest createFileSystemRequest;
createFileSystemRequest.SetCreationToken(FILE_SYSTEM_CREATION_TOKEN);
auto createFileSystemOutcome = efsClient.CreateFileSystem(createFileSystemRequest);
if (createFileSystemOutcome.IsSuccess())
{
fileSystemId = createFileSystemOutcome.GetResult().GetFileSystemId();
std::cout << "Succeeded to create file system with ID: " << createFileSystemOutcome.GetResult().GetFileSystemId() << std::endl;
}
else if (createFileSystemOutcome.GetError().GetErrorType() == EFSErrors::FILE_SYSTEM_ALREADY_EXISTS)
{
std::cout << "File system with ID: " << createFileSystemOutcome.GetError<FileSystemAlreadyExists>().GetFileSystemId() << " already exists." << std::endl;
std::cout << "Failed to create file system. Error details:" << std::endl;
std::cout << createFileSystemOutcome.GetError() << std::endl;
}
else
{
std::cout << "Failed to create file system. Error details:" << std::endl;
std::cout << createFileSystemOutcome.GetError() << std::endl;
}The most interesting part is createFileSystemOutcome.GetError<FileSystemAlreadyExists>() , which returns a FileSystemAlreadyExists object. Then you can call GetFileSystemId() to get the existing file system id.
The SDK can provide more common information about the exception, including remote host IP address and request ID if available. For the example code, if you are reusing FILE_SYSTEM_CREATION_TOKEN to create a file system, then you will get the output that is similar to the following, with an FileSystemAlreadyExists error:
You can find the complete version of this example here: https://github.com/awsdocs/aws-doc-sdk-examples/blob/master/cpp/example_code/elasticfilesystem/create_file_system_with_modeled_exceptions.cpp
New pseudo region: aws-global to make cross region requests
For most AWS services, you have to know the region before accessing the resource, or it may encounter a signature mismatch error. The new pseudo region aws-global provides the flexibility to get resources without knowing the region. We are reusing the interface to specify a regular region in ClientConfiguration:
Aws::Client::ClientConfiguration config;
config.region = Aws::Region::AWS_GLOBAL;Use cases include operations on an S3 bucket (such as ListObjects) without knowing the region. Also you don’t need to call GetBucketLocation to get the region. Instead, the SDK will help you solve the region issue by making two requests. With the first request, the SDK will get a response with status code 307 as well as the correct region and endpoint. Then the SDK will re-calculate the signature with the correct region and will use the correct endpoint to make the second request. If you don’t want the SDK to make a cross-region request, don’t specify aws-global as the client region. See the following example:
Aws::Client::ClientConfiguration config;
config.region = Aws::Region::AWS_GLOBAL;
S3Client s3Client(config);
// Create a bucket in us-west-2.
CreateBucketRequest createBucketRequest;
createBucketRequest.SetBucket(BUCKET_NAME);
CreateBucketConfiguration createBucketConfiguration;
createBucketConfiguration.SetLocationConstraint(BucketLocationConstraint::us_west_2);
createBucketRequest.SetCreateBucketConfiguration(createBucketConfiguration);
auto createBucketOutcome = s3Client.CreateBucket(createBucketRequest);
// S3 client with aws-global region should be able to get access to bucket in any region.
ListObjectsRequest listObjectsRequest;
listObjectsRequest.SetBucket(BUCKET_NAME);
auto listObjectOutcome = s3Client.ListObjects(listObjectsRequest);You can find the complete version of this example here: https://github.com/awsdocs/aws-doc-sdk-examples/blob/master/cpp/example_code/s3/list_objects_with_aws_global_region.cpp
Another breaking changes is that the type of followRedirect in the client configuration changed from a true or false boolean to the enum, FollowRedirectsPolicy, including the following options:
DEFAULT: Let the SDK decide if the underlying HTTP client should redirect a request if it receives a response with a 30x status code. With theaws-globalregion, the HTTP client will handle the 30x response manually. Otherwise, it will redirect the request by default.ALWAYS: The underlying HTTP client will always redirect the request if it receives a 30x response. (It’s equivalent to true before version 1.8.)NEVER: The underlying HTTP client will never redirect the request. (It’s equivalent to false before version 1.8.)
We made this breaking change because of the underlying implementation of the aws-global region. If your service client tries to hit a resource in another region, for example getting a bucket in us-west-2 with a client in us-east-1, sometimes the service will return a response with status code 30x. Then we need to handle the request redirect carefully. For example, with the global region, the SDK will handle the 30x response manually, rather than depending on the underlying HTTP client doing redirects.
S3 client in us-east-1 now uses regional endpoint by default.
In the previous version of AWS SDK for C++, an S3 client with region us-east-1 will make requests to the legacy global endpoint: bucket-name.s3.amazonaws.com to hit the bucket with virtual hosted-style requests by default. With version 1.8, we are changing this behavior by making requests to regional endpoint instead, for example:
You can see S3 Legacy Global Endpoint for more details.
However, you could still use legacy global endpoint for S3 client in us-east-1 with one of the following three approaches:
- In environment variables, set
AWS_S3_US_EAST_1_REGIONAL_ENDPOINTtolegacy - In the configuration file, under the profile you are using, specify
s3_us_east_1_regional_endpoint=legacy - Specify
Aws::S3::US_EAST_1_REGIONAL_ENDPOINT_OPTION::LEGACYwhen creating an S3 client, for example:
Aws::Client::ClientConfiguration config;
S3Client s3Client(config, Aws::Client::AWSAuthV4Signer::PayloadSigningPolicy::Never, true, US_EAST_1_REGIONAL_ENDPOINT_OPTION::LEGACY);Improvements on the underlying HTTP client override
With AWS SDK for C++, you can plug in your own implementation of the underlying HTTP client by extending HttpClientFactory. However, sometimes you want a minor code change in the configuration of the default HTTP client. There was not easy way to do that before version 1.8. We’ve improved the existing functionality by providing the virtual function: OverrideOptionsOn*Handle() so that you can add or override any configuration they want. The following is an example for disabling DNS caching for requests sent to S3 with curl HTTP clients.
First, we will need to create a subclass of CurlHttpClient, and then override the virtual function OverrideOptionsOnConnectionHandle() to configure CURLOPT_DNS_CACHE_TIMEOUT with curl API:
class MyCurlHttpClient : public Aws::Http::CurlHttpClient
{
public:
MyCurlHttpClient(const Aws::Client::ClientConfiguration& clientConfig) : Aws::Http::CurlHttpClient(clientConfig) {}
protected:
void OverrideOptionsOnConnectionHandle(CURL* connectionHandle) const override
{
std::cout << "Disable DNS caching completely." <<
std::endl;
curl_easy_setopt(connectionHandle,
CURLOPT_DNS_CACHE_TIMEOUT, 0L);
}
};Then, as with the previous SDK version, create a custom HTTP client factory extending class HttpClientFactory:
class MyHttpClientFactory : public Aws::Http::HttpClientFactory
{
std::shared_ptr<Aws::Http::HttpClient> CreateHttpClient(const Aws::Client::ClientConfiguration& clientConfiguration) const override
{
return Aws::MakeShared<MyCurlHttpClient>(ALLOCATION_TAG, clientConfiguration);
}
};And use it in your application:
SetHttpClientFactory(Aws::MakeShared<MyHttpClientFactory>(ALLOCATION_TAG));When making requests with this custom HTTP client, the DNS caching will be disabled.
You can find the complete version of this example here: https://github.com/awsdocs/aws-doc-sdk-examples/blob/master/cpp/example_code/s3/list_buckets_disabling_dns_cache.cpp
Feedback and Contribution Back
To provide feedback, leave a comment in this GitHub issue, or create a pull request in our Github repository.