Another S3 Bash Interface

Sample Code & Libraries>Another S3 Bash Interface
Community Contributed Software

  • Amazon Web Services provides links to these packages as a convenience for our customers, but software not authored by an "@AWS" account has not been reviewed or screened by AWS.
  • Please review this software to ensure it meets your needs before using it.

Put, get, delete, and list files using nothing more than curl, openssl, and coreutils.

Details

Submitted By: nescafe5
AWS Products Used: Amazon S3
License: GNU General Public License (GPL version 2)
Source Control Access: git clone git://fnordovax.org/another-­s3-­bash/
Created On: December 4, 2007 9:41 PM GMT
Last Updated: September 21, 2008 8:56 PM GMT
Download
Originally designed as a code library for a larger project, this contains all the tools needed to do basic single-user file management on S3. Also includes a standalone hmac script to make up for the lack of hmac functionality exposed in the openssl command interface. Only tested on Ubuntu 7.10, should work in any recent UNIX environment that has curl and openssl installed.

Installation: Download, unpack, and copy the s3 and hmac commands to a location in your path.

Usage:
  • s3 ls bucket -- lists the contents of a bucket
  • s3 put bucket file -- uploads file into bucket using the same name.
  • s3 get bucket file -- downloads file from bucket using the same name.
  • s3 rm bucket file -- deletes a file on S3
  • s3 rmrf bucket -- deletes all files from bucket.


UPDATED: Replaced GNUism in using the date(1) command, and replaced use of the non-POSIX stat(1) command with wc(1). This should greatly improve cross-platform portability.

Comments

Great but ...
If this script is executed on a machine whose LANG is not C such as ja_JP, it cannot make a proper date header and signature. Thus, it will end with ACCESS DENIED error in the respose of S3. I took a couple of hours to figure this out... So plz add "LANG=C" at the top of the script or state the info in this page. Thanks !!
wolfgang127us on February 22, 2009 6:46 PM GMT
I just got warning: 'header'
Sorry, this are my first steps. During calling the script (mac os x server) i just got: /dev/fd/63:1: warning: 'header' requires parameter Ok, i thougt it's just a warning, but nothing more happens.
sportics on January 29, 2009 5:50 AM GMT
Works great on Mac OS X
Had the same 41-character access file problem in Mac OS X (Tiger), but managed to create a 40-character file with the following command: perl -e "print 'TYPE_YOUR_ACCESS_KEY_HERE'" > access_key_file After that, everything worked fine - thank you very much! Does anyone know why there aren't official Amazon versions of these S3 scripts?
mckenfra on June 27, 2008 9:38 AM GMT
I also had problems on OSX
I'm not sure why, but my S3 secret access key comes to 41 bytes when saved to a file. There are no newlines or other odd characters in place. Just the key. So I tweaked the script to check for the secret access key to be `wc -c $sak_file` == 41. I then did `s3 ls $mybucket` and nothing printed out. No error, nothing at all. There are definitely files in the bucket. Also, `s3 --help` doesn't return anything. This would be a nice addition. Good start though. All the heavy lifting is done. Nice work.
Laran Evans on April 14, 2008 1:33 PM GMT
Nice work
Love the simplicity of it!
T. Reeder on March 1, 2008 6:08 PM GMT
Helpful but not portable
I am confident that these scripts will prove very useful, but they are not portable to all Unices without some modification first. Under NetBSD 3.1_RC3, the arguments to "date" will require modification, as well as those to "stat" - and that is as far as I have gotten; I am still working on the latter and waiting to find out what other obstacles, if any, will pop up.
alb@quandary.org on January 16, 2008 5:09 AM GMT
Great on Linux, some problems on OSX
This is a great utility, but I don't like the way it forces you to keep your secret key in a file as opposed to an environment variable. Also, the script checks the secret key's existence and length before it runs, and the arguments to "stat" which it uses to check the file size do not work on Mac OS X (tiger). Other than those minor nitpicks, this is a great and very useful utility.
David Salgado on December 30, 2007 4:27 PM GMT
Off site backups in 5 minutes
This is an excellent way to perform off site backups of critical data such as subversion repositories, database dumps, etc. It took about 5 minutes to configure and use.
johnpfeifer4 on December 29, 2007 9:44 PM GMT
We are temporarily not accepting new comments.
©2013, Amazon Web Services, Inc. or its affiliates. All rights reserved.