AWS Developer Tools Blog
AWS SDK for Go 2.0 – Generated Marshalers
The AWS SDK for Go 2.0 has released generated marshalers for the restjson and restxml protocols. Generated marshalers will help with the performance and customer issues the SDK had been receiving.
To better understand what was causing the performance hit, we used Go’s benchmark tooling to help us determine the main bottleneck—reflection. The reflection package was consuming a large amount of memory and performance, as shown below.
Roughly 50% of the time is spent in the JSON marshaler, which uses a lot of the reflection package. To improve both memory and CPU performance, we implemented generated marshalers. The idea was to bypass the package that was affecting performance, like reflection, and set values directly in the query, header, or body of requests.
The following benchmark data was gathered from restjson and restxml benchmark tests on GitHub.
REST JSON Benchmarks
The restjson protocol shows great performance gains. We see that for the Complex_ETCCreateJob benchmark, the speed improved by 59.66%. However, the overall gains in memory allocation were far less than expected, and some benchmarks took even more memory
REST XML Benchmarks
The restxml protocol greatly improved both memory and speed. The RESTXMLBuild_Complex_CFCreateDistro benchmark had a about a 70% improvement in both.
Overall for both protocols there were massive speed improvements in more complex shapes, but only small improvements in the simple shapes. There are some outliers in the data that have minor performance or memory hits. However, there are more optimizations we can do to potentially eliminate them.
Try out the developer preview of the AWS SDK for Go 2.0 here, and let us know what you think in the comments below!