What’s New for AWS Storage & Ingestion Services from re:Invent 2016

We hope you have had a chance to catch up on the security and compute services announced at re:Invent. Next up, we have the re:Invent updates on storage and ingestion that will benefit our public sector customers.

AWS Snowball Edge – Petabyte-scale Data Transfer with On-Board Compute

AWS Snowball Edge is our newest 100TB data transfer device, offering highly secure, on-board storage and in-flight compute capabilities with AWS Greengrass. Organizations can use AWS Snowball Edge to move massive amounts of data into and out of the AWS Cloud, use the device as a temporary storage tier for large local datasets, or seamlessly support edge workloads in remote or offline locations.

Snowball Edge connects to your organization’s existing applications and infrastructure using standard storage interfaces, streamlining the data transfer process, minimizing setup and integration, and helping ensure that the applications continue to run even when they are not able to access the cloud.

How does Snowball Edge accelerate data transfer to the cloud?

It has four times the network speed of the original AWS Snowball, built-in WiFi and cellular wireless communication, a Network File System (NFS) interface, and an Amazon S3-compatible endpoint. The device automatically encrypts all data stored. Encryption keys are managed with the AWS Key Management Service (KMS) and never stored on the device, ensuring that your most sensitive data is secure on site and in transit to AWS.

The AWS Snowball Edge device also comes with AWS Greengrass embedded, so you can execute AWS Lambda functions and process data locally, making it possible to collect and analyze sensor data streams, transcode multimedia content, compress images in real time, or run a local Amazon S3-compatible file server.

How it works

Jobs are created right from the AWS Management Console. Once a job is created, AWS automatically ships a Snowball Edge device to you. When you receive the device, simply attach it to your local network and then connect your applications. Once the device is ready to be returned, the E Ink shipping label will automatically update to the correct AWS facility, and the job status can be tracked via Amazon SNS generated text or email messages, or directly in the console.

AWS Snowball & HIPAA Compliance

AWS has expanded its HIPAA compliance program to include AWS Snowball, allowing you to transfer large amounts of data, including Protected Health Information (PHI), into and out of AWS securely and cost-effectively. Read the HIPAA Compliance whitepaper.

AWS Snowmobile – Move Exabytes of Data to the Cloud in Weeks, Not Years

Even with high-end connections, moving petabyte and exabyte-scale data to the cloud is challenging. Now migrating financial and regulatory records, scientific archives, and satellite imagery to the cloud won’t take years or decades. The AWS Snowmobile secure data truck stores up to 100 PB of data so customers can migrate data to the AWS Cloud in weeks.

Tamper-resistant AWS Snowmobile shipping containers attach to your network and appears as a local, NFS-mounted volume. Each AWS Snowmobile consumes about 350 kW of AC power and includes a network cable connected to a high-speed switch, capable of supporting 1 Tb/second of data transfer spread across multiple 40 Gb/second connections.

Snowmobile also incorporates multiple layers of logical and physical protection, including chain-of-custody tracking, 24/7 video surveillance and GPS tracking with cellular or satellite connectivity back to AWS. AWS Snowball offers 2560-bit encryption and encrypts with AWS Key Management Service (KMS) keys. We can even arrange for a security vehicle escort when the AWS Snowmobile is in transit and dedicated security guards while your AWS Snowmobile is on-premises.

New Amazon S3 Features

• Amazon S3 CloudWatch Metrics – Understand and improve the performance of your applications that use Amazon S3 by monitoring and alarming on 13 new Amazon S3 CloudWatch metrics. For web and mobile applications that depend on cloud storage, these metrics allow you to quickly identify and act on operational issues. You can receive one-minute Amazon S3 CloudWatch Metrics, set CloudWatch alarms, and access CloudWatch dashboards to view real-time operations and performance.
• Amazon S3 Object Tagging – With S3 Object Tagging, you can manage and control access for Amazon S3 objects. Amazon S3 Object Tags are key-value pairs applied to Amazon S3 objects which can be created, updated, or deleted at any time during the lifetime of the object. You’ll also have the ability to create Identity and Access Management (IAM) policies, set up Amazon S3 Lifecycle policies, and customize storage metrics.
• Amazon S3 Analytics, Storage Class Analysis – With storage class analysis, you can analyze and visualize storage access patterns and transition the right data to the right storage, optimizing costs. You can configure a storage class analysis policy to monitor an entire bucket, a prefix, or object tag. This new Amazon S3 Analytics feature automatically identifies the optimal lifecycle policy to help you transition less frequently accessed storage to Standard Infrequent Access SIA and save.
• Amazon S3 Inventory – You can simplify and speed up business workflows and big data jobs using Amazon S3 Inventory, which provides a scheduled alternative to Amazon S3’s synchronous List API. Amazon S3 Inventory provides a CSV (Comma Separated Values) flat-file output of your objects and their corresponding metadata on a daily or weekly basis for an Amazon S3 bucket or a shared prefix.

Learn More.

New Amazon EFS (Elastic File System) Features

Amazon EFS (Elastic File System) offers storage for use with Amazon EC2 instances and allows you to access file data from on-premises datacenters. You can now migrate file data to and from on-premises into Amazon EFS to support cloud bursting workloads and backups to the cloud when connected to your Amazon VPC with AWS Direct Connect.

New AWS Storage Gateway Provides File Interface to Objects in Amazon S3 Buckets

AWS Storage Gateway now provides a virtual on-premises file server, which enables you to store and retrieve Amazon S3 objects through standard file storage protocols. With file gateway, existing applications or devices can use secure and durable cloud storage without modification. File gateway simplifies moving data into Amazon S3 for in-cloud workloads, provides cost-effective storage for backup and archive workloads, or expands your on-premises storage into the cloud.

File gateway is available as a virtual machine image which you download from the AWS Management Console. To start using the new AWS Storage Gateway, click here.

Contact us to get started today with these new services.