AWS Storage Blog

Tag: AWS Cloud Storage

Amazon S3

Cross-account bulk transfer of files using Amazon S3 Batch Operations

As customers scale their business on AWS, they can have millions to billions of objects in their Amazon S3 buckets. Customers often run operations on a large number of these objects in their buckets, including copying objects across accounts, encrypting objects, or tagging. Running operations on a large number of objects in S3 involves listing […]

Read More

How Qubole optimizes cost and performance by managing shuffle data

Ad hoc analytics, data exploration, data engineering, and machine learning (ML) workloads are often run at a massive scale and require significant computing power, which can be costly. Amazon EC2 Spot Instances, which enable you to use unused Amazon EC2 computing capacity in the AWS Cloud, offer up to 90% in savings over On-Demand Instances. […]

Read More

Introducing the Storage Migration Acceleration Program

At AWS, we understand migrating your data is not trivial, especially migrating out of tenured, familiar data storage solutions. It can involve serious and tedious work in designing, planning, and executing your data migration project plan, plus having a contingency plan in place in case of any hiccups. There are both technical and business risks […]

Read More
AWS Transfer Family Featured Image

How Liberty Mutual uses AWS Transfer Family to manage financial data

Liberty Mutual is the sixth largest property and casualty insurer in the world based on 2019 gross written premium. We rank 77th on the Fortune 100 list of largest corporations in the US based on 2019 revenue, and as of December 31, 2019 we had $43.2 billion in annual consolidated revenue. As a part of […]

Read More

Automatically import Amazon S3 object updates into Amazon FSx for Lustre

Many enterprises and other AWS customers often store their datasets and build their data lakes on Amazon S3. Since Amazon FSx for Lustre is deeply integrated with S3, customers can create a new FSx for Lustre file system linked to their S3 bucket in minutes. FSx for Lustre file systems transparently present S3 objects as […]

Read More
Amazon EFS Featured Image

Deploying Jenkins on Amazon EKS with Amazon EFS

UPDATE (5/17/2021):  It looks like the Jenkins Helm repos have moved and the configurable parameters have been updated as well! You will need to modify the commands listed in the blog accordingly. Check out the new Jenkins helm repos and configurable parameters at the following links: https://github.com/jenkinsci/helm-charts/blob/main/charts/jenkins/README.md https://github.com/jenkinsci/helm-charts/blob/main/charts/jenkins/VALUES_SUMMARY.md Also, make sure your EKS cluster control […]

Read More

Online Tech Talk July 27: Explore high-performance shared file storage for compute clusters

Don’t miss our AWS Online Storage Tech Talk on July 27, where an AWS expert covers reducing the cost of running compute-intensive workloads through high-performance storage using Amazon FSx for Lustre. This Tech Talk is at 11:00 AM – 12:00 PM PT (2:00 PM – 3:00 PM ET). Fast parallel file systems can help your […]

Read More
Amazon S3

Auditing Amazon S3 encryption methods for object uploads in real time

Encryption of data at rest is increasingly necessary for maintaining compliance and providing another layer of protection for data against unauthorized access. Amazon S3 offers multiple methods for server-side encryption (SSE) of new objects in your bucket. As not all encryption options are equal, customers often ask how to track and control the method of […]

Read More
AWS Transfer Family Featured Image

Using Okta with multi-factor authentication for AWS Transfer for SFTP

Enterprises often use SFTP to provide third parties like vendors, partners, or offsite laboratories access to their data lakes for things like uploads, downloads, or distributing data exports to clients. AWS Transfer Family provides a fully managed SFTP (now expanded to offer FTPS and FTP) service for Amazon S3. The service frees you from managing […]

Read More
Amazon EFS Featured Image

Making it even simpler to get started with Amazon EFS

Today, we launched a significant update to the Amazon EFS management console. In this post, I talk about some of the new capabilities in the console, what you can do with them, and how they make it even easier for you to create and manage your EFS resources. Console overview Our main focus with this […]

Read More