AWS Import/Export

General

Billing

Using the Service

Using Amazon Import

Using Amazon Export

Hardware Information

Security

3rd Party Data Loads

↑ Top

General

Q. What is AWS Import/Export?
AWS Import/Export accelerates moving large amounts of data into and out of AWS using portable storage devices for transport. AWS transfers your data directly onto and off of storage devices using Amazon’s high-speed internal network and bypassing the Internet. For significant data sets, AWS Import/Export is often faster than Internet transfer and more cost effective than upgrading your connectivity.
Q. What can I do with AWS Import/Export?
AWS Import/Export makes it easy to quickly transfer large amounts of data into and out of Amazon S3. You can use AWS Import/Export for:
  • Data Migration – If you have data you need to upload into the AWS cloud for the first time, AWS Import/Export is often much faster than transferring that data via the Internet.
  • Offsite Backup – Send full or incremental backups to Amazon S3 for reliable and redundant offsite storage.
  • Direct Data Interchange – If you regularly receive content on portable storage devices from your business associates, you can have them send it directly to AWS for import into your Amazon S3 buckets.
  • Disaster Recovery – In the event you need to quickly retrieve a large backup stored in Amazon S3, use AWS Import/Export to transfer the data to a portable storage device and deliver it to your site.
Q. Which Amazon S3 Regions does AWS Import/Export support?

AWS Import/Export supports buckets stored in the US Standard and EU (Ireland) Regions. Data loads to and from US Standard Region buckets must be sent to a specified AWS address in the US, while data loads to and from EU (Ireland) Region buckets must be sent to a specified AWS address in the EU.

Q. How can I get started with AWS Import/Export?
To start using the service, click the “Sign up for AWS Import/Export” button on the AWS Import/Export detail page. You must have an Amazon S3 account to access AWS Import/Export; if you do not already have one, you will be prompted to create one when you begin the AWS Import/Export sign-up process.
Q. When should I consider using AWS Import/Export?
If loading your data over the network would take 7 days or more, then you should consider using AWS Import/Export. To determine the best case for loading your data over the network use this formula:

Number of Days = (Total Bytes)/(Megabits per second * 125 * 1000 * Network Utilization * 60 seconds * 60 minutes * 24 hours)

For example if you have a T1 connection (1.544Mbps) and 1TB (1024 * 1024 * 1024 * 1024 bytes) to move in or out of AWS the theoretical minimum time it would take to load over your network connection at 80% network utilization is 82 days.

Available Internet Connection Theoretical Min. Number of Days to Transfer 1TB at 80% Network Utilization When to Consider AWS Import/Export?
T1 (1.544Mbps) 82 days 100GB or more
10Mbps 13 days 600GB or more
T3 (44.736Mbps) 3 days 2TB or more
100Mbps 1 to 2 days 5TB or more
1000Mbps Less than 1 day 60TB or more

Q. How long does it take to load my data?
AWS typically begins loading your data the business day following receipt of your package. Customers with single drive USB devices with large files have achieved data loading rates over 20 MB/sec. eSATA devices have achieved rates over 40 MB/sec. For more details regarding data loading speeds and cost see the AWS Import/Export Calculator.
Q. Which factors influence data loading speed?
The factors that have the biggest influence on data transfer rates are device speed and average file size. Though other factors such as file system type and number of files per directory may impact the time it takes to load your data. Faster devices and larger files will tend to result in decreased transfer times. For more details regarding data loading speeds and cost see the AWS Import/Export Calculator.
Q. How much data can I load?
The total amount of data you can load using the AWS Import/Export is limited only by the capacity of the devices that you send to AWS. Individual files, which will be loaded as objects in Amazon S3, may range up to 5 gigabytes in size.
Q. How much data can I fit onto my device?
It depends. The amount of data that will fit on your device is determined by the specific exported file set, your formatted device block size, the amount of file system overhead, and actual formatted capacity. For example, a 1TB drive with a raw capacity of 1,000,000,000,000 bytes, when formatted, could be left with only 910,000,000,000 usable bytes. Block size could be an issue for file sets with smaller files. If your device is formatted with 4KB block sizes, but your files are all 1KB, you would have 3KB of unusable space per file. To estimate how many bytes block size could potentially impact, multiply the number of files for export by your formatted block size.
Q. I am using AWS Export. How many devices should I send to AWS?
That depends on the size of your devices and amount of data to export. You can figure out how many jobs you need by using the CREATE EXPORT PLAN command, which includes a manifest file and your device sizes. AWS will respond back with a set of manifest files that splits your export across your devices. The CREATE EXPORT PLAN command incurs Amazon S3 charges for LIST operations.
Q. Can I use AWS Import/Export to transfer data to Amazon EC2 or another AWS product?
Yes, once your data is in Amazon S3, it can be accessed by any AWS product. Data loaded through AWS Import/Export is treated just like data loaded through the network.

↑ Top

Billing

Q. How will I be charged and billed for my use of AWS Import/Export?
With AWS Import/Export, you pay only for the resources that you use. Pricing is per device, data-loading-hour, and package shipped. Partial data-loading-hours are billed as full hours. Return shipping charges are dependent on the location of your Amazon S3 bucket and your return shipping destination. There is no Data Transfer charge for data transferred between AWS Import/Export and Amazon. Usage for Amazon S3 is billed separately from AWS Import/Export.


For AWS Import/Export pricing information, please visit the pricing section on the AWS Import/Export detail page. For Amazon S3 pricing information, please visit the pricing section of the Amazon S3 detail page.

Q. What other charges am I responsible for?
You are responsible for purchasing your storage device(s) and the shipping charges to AWS (including any applicable duty and taxes). Packages with unpaid duty and/or taxes will be denied upon arrival. If your Amazon S3 bucket is located in the US Standard Region, you will also be responsible for any applicable duty and taxes on return shipments sent to addresses outside of the US. These duty and taxes will be charged directly by our return shipping carrier upon delivery of your package.
Q. When will I be billed for my use of AWS Import/Export?
The device fee is charged when your package arrives at AWS. The data-loading-hour billing starts once AWS Import/Export begins transferring data to or from your storage device. If there are any interruptions to the data load, you will not be charged during the period of interruption. Any applicable return shipping expenses will be charged once your package is ready for shipment to your return shipping address.
Q. What am I charged for return shipping?

Return shipping expenses vary depending on your return shipping destination and the location of your Amazon S3 bucket:

US Standard Region

  • If you have specified a return address within the US, AWS will pay reasonable return shipping charges, but you will be responsible for any return shipping expenses that AWS determines to be unreasonable or excessive.
  • If you have specified a return address outside of the US, shipping charges based on your shipment destination and weight will apply.

EU (Ireland) Region

  • If you have specified a return address within Ireland, AWS will pay reasonable return shipping charges, but you will be responsible for any return shipping expenses that AWS determines to be unreasonable or excessive.
  • If you have specified a return address outside of Ireland, shipping charges based on your shipment destination and weight will apply. At this time, devices shipped to AWS for import into or export from EU (Ireland) Region buckets must originate from and be returned to an address within the European Union.

To estimate your return shipping charges, please refer to the AWS Import/Export Calculator. PO Boxes are not a valid return shipping address.

↑ Top

Using the Service

Q. Do I need any special software?
No, but the AWS Import/Export resources include an example Java application for generating signatures. You should either create your own signature tool or download the Java Standard Edition 2, 1.4.2 or newer to run this example. For more details refer to the AWS Import/Export documentation.
Q. Do jobs expire?
Yes, if AWS has not received your storage device within 30 days of creating the job, it automatically expires. AWS will not accept packages for expired jobs. To resubmit, simply create a new job and update the SIGNATURE file, hard drive label, packing slip, and JOBID on the ship-to label.
Q. Can a job have more than one storage device?
No, each job is associated with exactly one manifest file and one customer device. If your data spans multiple devices, you must create multiple jobs.
Q. Can I enclose multiple jobs in the same package?
No, each job must be sent in a separate package.
Q. Can my job span multiple partitions?
No, each job has exactly one partition.
Q. What will happen if I ship my device to AWS in the US for import or export from my EU (Ireland) Region bucket?

Data loads to and from US Standard Region buckets must be sent to a specified AWS address in the US, while data loads to and from EU (Ireland) Region buckets must be sent to an AWS address in the EU. If you ship your device to the wrong AWS address, AWS will return your package to you without performing the data load. You should then ship your device to the correct AWS address. You will be responsible for any applicable return shipping charges and device-handling fees.

Q. Can I use a courier service?

You can use a courier service to send packages to AWS. For imports into and exports from US Standard Region buckets, AWS will return all US-bound packages via standard UPS ground and all international-bound packages via DHL Express Worldwide. For imports into and exports from EU (Ireland) Region buckets, AWS will return all packages via An Post. For additional information regarding return shipping fees and services, please refer to the AWS Import/Export Calculator.

Q. What locations does AWS accept shipping from?

For imports into and exports from US Standard Region buckets, AWS accepts packages from all countries, except those that are listed as under embargo by the US or countries not serviced by DHL, our return shipping carrier in the US. For imports into and exports from EU (Ireland) Region buckets, AWS accepts packages from all countries within the European Union.

Q. What will happen if I ship my device to AWS in the EU from outside of the European Union?

At this time, devices shipped to AWS for import into or export from EU (Ireland) Region buckets must originate from and be returned to an address within the European Union. If you ship your device to AWS in the EU from a location outside of the European Union, AWS will return your package to you without performing the data load. You will be responsible for any applicable return shipping charges and device-handling fees.

Q. Can I ship my package from one address and have it returned to another address?

Yes, you can ship your package to AWS from one address and have it returned to another address.

Q. Are there specific device and content regulations for shipments to or from the US?

If you are importing or exporting from a US Standard Region bucket and are shipping your device from or requesting that AWS return your device to a non-US based address, your shipment must comply with the following US and international import and export regulations:

  1. The import and export of your device and/or data should not require a license under the regulations of any applicable country, including the United States Export Administration Regulations or the International Traffic in Arms Regulations. Only devices and data that are classified as EAR99 under US Export Administration Regulations and are not subject to the International Traffic in Arms Regulations are eligible for AWS Import/Export.
  2. The import and export of your device and/or data should not be prohibited by the regulations of any applicable country, including the United States Export Administration Regulations or the International Traffic in Arms Regulations.
  3. If the data on your device contains encryption software, then the software must be properly classified as 5D992 under US Export Administration Regulations (please confirm classification with the software manufacturer).
  4. Only devices valued at less than US $2,500 are eligible for AWS Import/Export.
Q. What happens if I change my AWS keys before a pending job completes?
The signature on your storage device will be invalid. As a result the job will not load your objects and your device will be returned. You’ll need to create a new job and generate a valid signature for your storage device.
Q. Can I cancel a job?
Yes, you can cancel a job anytime before we finish the data load. If AWS has not yet received your package, then we will deny delivery of the package. If AWS has already received your package you will be charged the per device fee. If the data load has started you will be charged for all data- loading-hours prior to processing the cancel job command. To cancel a job, simply send an e-mail to awsimportexport@amazon.com with the subject line “CANCEL JOB” and with the referenced JOBID as the first part of the e-mail body. You will incur any applicable return shipping charges if you cancel a job after you have shipped your device to us.
Q. Can I undo a job?
No, there is no option to restore your bucket to its previous state. If you want to delete the loaded data, you’ll need to write a program to traverse your import log and issue a DELETE operation on each object.
Q. Can you import my exported data?
Yes, but the exported files won’t map back to the same location in Amazon S3. If this is important to your use case, please contact AWS with the details at awsimportexport@amazon.com
Q. How does AWS Import/Export work with Amazon S3’s Versioning capability?
When importing your data via AWS Import/Export into a Versioning-enabled bucket, the objects within your bucket will not be permanently overwritten. The import process creates a new version for each object in your bucket. When exporting your data via AWS Import/Export from a Versioning-enabled bucket, you can only export the most recently written versions of your objects. Versioning is currently available in beta across all Amazon S3 Regions.

↑ Top

Using Amazon Import

Q. How do I send my data to AWS?
To use import, start by loading your data onto your storage device, such as an external USB or eSATA hard drive. Then create a simple manifest file that includes your Amazon S3 bucket, AWS Access Key Id, and return shipping address. Next send an e-mail to awsimportexport@amazon.com with the subject line “CREATE JOB” and the manifest file attached. If you are importing into a US Standard Region bucket and are shipping your device from or requesting that AWS return your device to a non-US based address, you must also attach an AWS Import/Export Declaration Form with your CREATE JOB e-mail.


After we validate and store your manifest file, a job is created. We then send you a “RE: CREATE JOB” success e-mail containing a JOBID and a link to a website specifying an AWS address. An example JOBID is X452D. Following this, you should generate a signature file named “SIGNATURE” in the root directory of your storage device by signing the manifest file and JOBID with your AWS Secret Access Key.

Next, label your media with the JOBID and fill out the information on the packing slip. Pack your media, clearly label the JOBID on your package, and ship it to the correct AWS address specified in the “RE: CREATE JOB” success e-mail.

After we load your data to Amazon S3, we send your device to the Return Shipping Address specified in your manifest file. PO Boxes are not a valid return shipping address. For more information about the manifest file format, SIGNATURE file, Declaration form, AWS addresses, or the process to create the AWS Import/Export job, please refer to the AWS Import/Export Developer Guide.

Q. Can my data be loaded into more than one bucket?
No, data loads are restricted to one bucket per job.
Q. How are my files mapped to Amazon S3 objects?
The job’s manifest file specifies how to map files to key names and can be used to set the following object attributes: Content-Disposition, Content-Type and x-amz-acl.
Q. What happens if a key already exists?
Just like a standard Amazon S3 PUT, if a new object uses the same key as an existing object, the AWS Import/Export overwrites the old object with the new object.
Q. Can I write to someone else’s bucket? And who pays for it?
Yes, as long as your AWS account has write access to the bucket. The AWS account specified in the manifest file will pay all AWS Import/Export charges, while the bucket owner will pay the applicable Amazon S3 request and storage fees.
Q. Can I write to a requestor pays bucket?
No, requestor pays buckets are not supported at this time. You’ll need to load your data to a non-requestor pays bucket and then either make that bucket a requestor pays bucket or copy the data to the requestor pays bucket.
Q. Do I get a checksum or any kind of receipt on what was loaded into Amazon S3?
Yes, AWS Import/Export saves an import log report to your bucket. This report contains per file information including the date and time of the upload, Amazon S3 key, MD5 checksum, and number of bytes.
Q. I just loaded over a million objects into Amazon S3, how do I manage those objects?
Amazon S3 users create custom tools to manage millions of objects. We recommend reviewing the Best Practices for Using Amazon S3 article before creating your own tools.
Q. What happens when a file is larger than 5GB?
AWS Import/Export rejects all files larger than 5GB. We will provide the names of those files in the import log report.
Q. Do I need to backup my data before sending it to AWS?
Yes, you should have another copy of your data before shipping it to AWS. Your device and data could be damaged or lost in transit.

↑ Top

Using Amazon Export

Q. How do I get my data from AWS?
To use export, start by creating a manifest file that includes your Amazon S3 buckets, AWS Access Key Id, and return shipping address. If all of your data won’t fit on one device, or you’re unsure, you should issue a CREATE EXPORT PLAN command. This command takes a manifest file and allows you to specify information such as size about one or more storage devices. AWS will return a set of manifest files that splits your job up among those devices. Next send an e-mail to awsimportexport@amazon.com for each device you’re ready to send with the subject line “CREATE JOB” and the corresponding manifest file attached. If you are exporting from a US Standard Region bucket and are shipping your device from or requesting that AWS return your device to a non-US based address, you must also attach an AWS Import/Export Declaration Form with your CREATE JOB e-mail.


After AWS validates and stores your manifest file, a job is created. We then send you a “RE: CREATE JOB” success e-mail containing a JOBID and a link to a website specifying an AWS address. An example JOBID is X452D. Following this, you should generate a signature file named “SIGNATURE” in the root directory of your storage device by signing the manifest file and JOBID with your AWS Secret Access Key.

Next, label your media with the JOBID and fill out a packing slip. Pack your media, clearly label the JOBID on your package, and ship it to the correct AWS address specified in the “RE: CREATE JOB” success e-mail.

After AWS finishes transferring the data to your storage device, the device is sent to the Return Shipping Address specified in your manifest file. PO Boxes are not a valid return shipping address. For more information about the manifest file format, SIGNATURE file, Declaration form, AWS addresses, or the process to create the AWS Import/Export job, please refer to the AWS Import/Export Developer Guide.

Q. Can I retrieve data from more than one bucket?
Yes, you can specify more than one bucket in your manifest file; however, all buckets must be located within the same Amazon S3 Region. In addition, you must have read permissions on each bucket and each object.
Q. How are my Amazon S3 objects mapped to files?
Each key is copied to your device in a directory tree that starts with the bucket’s name. For example, if the key was “images/chewie.jpg” and the bucket was “starwars”, then the object would be saved to /starwars/images/chewie.jpg. Meta data associated with each object is not copied to your storage device.
Q. What about my user metadata, ACLs, or headers?
At this time export does not include this data, however if this is important for your use case, please contact AWS with the details at awsimportexport@amazon.com.
Q. Do I get a checksum or any kind of receipt on what was copied to my storage device?
Yes, AWS saves an export log report to your specified log bucket. The report contains per file information which includes the date and time of the transfer, location on your storage device, MD5 checksum, and number of bytes.
Q. What happens if a key name doesn’t map to a valid filename?
Your file will be written to the recovery directory. Your log file will contain a redirect status code.
Q. Can I specify the name of the recovery directory?
Yes, you can specify the name of the recovery directory in your manifest file. The default name is “EXPORT-RECOVERY”.
Q. Can AWS format or erase my device before starting the export?
No, AWS only copies additional information to your device.
Q. What happens if a file already exists on my device?
The existing file will be overwritten with the new file. However before performing the export, AWS checks all keys for case sensitive name collisions on case insensitive file systems and saves duplicates to the recovery directory. For example, on FAT32 “gumby.txt” and “GUMBY.TXT” refer to the same file.
Q. What happens if my device is full?
An error will be flagged in your export log for each attempted key. Those objects will not be written to your storage device. You’ll need to download those files over the network or create a new job.

Hardware Information

Q. What are the storage devices requirements?

AWS Import/Export supports storage devices that can be mounted on Red Hat Linux. The storage device requirements are based on the location of your Amazon S3 bucket:

US Standard
 
EU (Ireland)
 
Item Requirement
Power
  • US wall socket plug*
  • 120Volts @60Hertz
  • Max Power Consumption 2,000Watts
Interface Type
  • eSATA
  • USB 2.0
  • 2.5 inch and 3.5 inch internal SATA hard drives
Dimensions
  • The maximum device size is 14 inches high by 19 inches wide by 36 inches deep (8Us in a standard 19 inch rack).
Weight
  • The maximum device weight is 50 pounds.
File System Type
  • NTFS
  • ext2, ext3
  • FAT32
Device Capacity
  • Up to 4TB

* An adaptor must be included for plugs other than the ones supported above

AWS can support larger and heavier devices upon request. If you have a device that does not meet these requirements, please contact us with the specifics. Make sure to include your AWS account(s) and the number of terabytes to load.

Q. Do I need to include my USB/eSATA cable, power cable, and power supply?
Yes, for non internal SATA hard drives, your package must include your connector cables, power cables and power supply. If any of these are missing, we will return your package unprocessed. You will be charged the per device fee, but not for any data-loading-hours. You will also be charged for any applicable return shipping charges.

For internal SATA hard drives, no cables are necessary.

Q. How do I determine if my power supply supports 120Volts at 60Hz or 230Volts at 50 Hz?
If you have concerns, the power supply typically will have a label that specifies voltage and frequency. For example, your power supply might specify, Input: 100-240V~ 50-60Hz. This label indicates that the power supply supports voltages from 100 to 240 volts at frequencies between 50 and 60Hz. If you have additional questions, please contact your device manufacturer.

↑ Top

Security

Q. How is my device authenticated?
The signature file on your storage device is created with your AWS Secret Access Key. This signature is used to securely identify and authenticate your device.
Q. Where does the data load occur?
You data will be loaded over our internal high-speed network at one of our data centers.
Q. Can I use a courier service?
Yes, for inbound shipment you can use a courier service. However at this time all outbound shipments are standard UPS ground.
Q. Can AWS wipe my device?
Yes, if you set the eraseDevice field to Yes in your manifest file, after completing the data load all writable blocks on your storage device will be overwritten with zeroes. You will need to repartition and format your device after the wipe. Data-wiping-hours, which are equivalent in price to data-loading-hours, will be charged during this process. Partial data-wiping-hours are billed as full hours. For more details regarding data wiping costs see the AWS Import/Export Calculator.
Q. Who handles my package and device?
AWS uses Amazon employees and 3rd party contractors to securely process and transport your package. For example, packages shipped to our specified AWS addresses are transported to our data centers by 3rd party contractors. All 3rd party contractors and Amazon employees have gone through extensive background checks.
Q. Does AWS provide a chain of custody for my package?
Yes, once a shipment arrives at the AWS address we provide you, AWS maintains a chain of custody for your package up until the package is delivered to your return address.
Q. What happens if my device is lost or damaged in transit?
If your device is lost in transit, you’ll need to follow up with the shipping carrier. If AWS Import/Export is unable to process your device, because it was damaged in transit, AWS will return your device to the Return Shipping address specified in your manifest file. Return shipping charges may apply depending on the location of the return shipping address you have provided. You will need to contact your shipping carrier regarding any damage to your device.
Q. What happens if AWS loses or damages my device?
Although AWS has a number of internal controls and procedures to prevent loss, damage or disclosure of your data, AWS is not responsible for damages associated with loss or inadvertent disclosure of data, or the loss, damage or destruction of your storage device. You should always retain a back-up copy of your data.
Q. Do I need to encrypt my data?
There is no requirement to encrypt your data. However AWS generally recommends encrypting any sensitive data sent to AWS on your storage device or stored in Amazon S3. You should determine whether your use case warrants encryption.
Q. Can AWS Import/Export encrypt my exported data?
No, if you want your exported data encrypted, then you’ll need to save it to Amazon S3 in an encrypted format prior to export. If this is important to your use case, please contact AWS with the details at awsimportexport@amazon.com.

↑ Top

3rd Party Data Loads

Q. Can I load data on behalf of another AWS account or have data sent to me using AWS Import/Export?
Yes, provided that the AWS account specified in the manifest file is signed up for AWS Import/Export, and the signature on the device is produced using that account’s credentials.
Q. What is the best way to load data for another account?
If your agreement with the other account holder does not provide you access to their AWS credentials, you should have them produce the signature for the storage device using their keys. If your agreement provides you access to their AWS credentials, you can produce both the manifest and signature yourself.


AWS recommends loading data directly into the intended target bucket rather than using staging buckets.

↑ Top

©2010, Amazon Web Services LLC or its affiliates. All rights reserved.