Category: AWS SDK for Go

My colleague Loren Segal shared the blog post below. As you can see, he’s really excited about the new AWS SDK for Go!

— Jeff;

Today we are announcing the first Developer Preview Release of the AWS SDK for Go (v0.6.0). If you have not been following along with development, the AWS SDK for Go now has full AWS service support, exponential backoff retry handling, and more. Since our last announcement, we’ve also added a concurrent streaming upload and download manager for Amazon S3, and built-in support for response pagination, with resource waiter support in the works. We have also added a Getting Started Guide, which we will continue to update as we add new features to the SDK.

This release also comes with a few organizational and process changes. Firstly, we have moved our GitHub repository from the awslabs organization to aws/aws-sdk-go. This signifies that this SDK is no longer in an experimental state. Please make sure to update your “import” statements to reflect this new location accordingly (see the example below for the full statement). Secondly, being in Developer Preview mode means that we will now be providing releases for new services and service updates as they are made available, which should allow users to keep up-to-date with all of our new AWS functionality.

We’re excited to get this SDK in your hands and to hear what you think about the changes and new features we’ve been working on. The goal of our Developer Preview cycle is to get feedback about what works and what does not so that we can make tweaks to our API before locking it down for a stable 1.0 release, so don’t be shy about letting us know what you do and don’t like on our GitHub Issues page!

To end this post, I thought I’d share a small example code snippet using response pagination and our Amazon S3 download manager to pull down a series of objects in S3 to your local file system. For simplicity, each file is downloaded sequentially, though it would be possible to wrap these operations in goroutines and have them download concurrently as well. Note that, while using the download manager, each individual download operation will retrieve data concurrently in chunks.

package main
 
import (
    "fmt"
    "os"
    "path/filepath"
 
    "github.com/aws/aws-sdk-go/service/s3"
    "github.com/aws/aws-sdk-go/service/s3/s3manager"
)
 
var (
    Bucket         = "MyBucket" // Download from this bucket
    Prefix         = "logs/"    // Using this key prefix
    LocalDirectory = "s3logs"   // Into this directory
)
 
func main() {
    manager := s3manager.NewDownloader(nil)
    d := downloader{bucket: Bucket, dir: LocalDirectory, Downloader: manager}
 
    client := s3.New(nil)
    params := &s3.ListObjectsInput{Bucket: &Bucket, Prefix: &Prefix}
    client.ListObjectsPages(params, d.eachPage)
}
 
type downloader struct {
    *s3manager.Downloader
    bucket, dir string
}
 
func (d *downloader) eachPage(page *s3.ListObjectsOutput, more bool) bool {
    for _, obj := range page.Contents {
        d.downloadToFile(*obj.Key)
    }
 
    return true
}
 
func (d *downloader) downloadToFile(key string) {
    // Create the directories in the path
    file := filepath.Join(d.dir, key)
    if err := os.MkdirAll(filepath.Dir(file), 0775); err != nil {
        panic(err)
    }
 
    // Setup the local file
    fd, err := os.Create(file)
    if err != nil {
        panic(err)
    }

    defer fd.Close()

    // Download the file using the AWS SDK
    fmt.Printf("Downloading s3://%s/%s to %s...\n", d.bucket, key, file)
    params := &s3.GetObjectInput{Bucket: &d.bucket, Key: &key}
    d.Download(fd, params)
}

— Loren Segal, Software Development Engineer

Following up on his recent guest post, my colleague Peter Moon has more news for Go developers!

— Jeff;

Since our initial kickoff announcement in January, we have been revamping the internals of the AWS SDK for Go in the project’s ‘develop’ branch on GitHub, laying out a solid foundation for a well-tested, robustly generated SDK that meets the same high quality bar as our other official SDKs.

Today, with complete support for all AWS protocols and services, the develop branch has been merged to the master branch of the project. At this point the SDK’s architecture and interfaces include the initial set of key changes we have envisioned, and we’re excited to announce our progress and humbly invite customers to try out the SDK again.

While collecting and responding to your valuable feedback, we will also continue to work on additional improvements including various usability features and better documentation. We are immensely grateful for the amount of engagement and support we’ve been getting from the community and look forward to continue making AWS a better place for Go developers!

— Peter Moon, Senior Product Manager