Amazon S3 PHP Stream Wrapper

As of the 2.3.0 release, the AWS SDK for PHP now provides an official Amazon S3 PHP stream wrapper. The stream wrapper allows you to treat Amazon S3 like a filesystem using functions like fopen(), file_get_contents(), and filesize() through a custom stream wrapper protocol. The Amazon S3 stream wrapper opens up some interesting possibilities that were either previously impossible or difficult to implement.

Registering the stream wrapper

Before you can use the Amazon S3 stream wrapper, you must register it with PHP:

use AwsS3S3Client;

// Create an Amazon S3 client object
$client = S3Client::factory(array(
    'key'    => '[aws access key]',
    'secret' => '[aws secret key]'
));

// Register the stream wrapper from a client object
$client->registerStreamWrapper();

After registering the stream wrapper, you can use various PHP filesystem functions that support custom stream wrapper protocols.

$bucket = 'my_bucket';
$key = 'object_key';

// Get the contents of an object as a string
$contents = file_get_contents("s3://{$bucket}/{$key}");

// Get the size of an object
$size = filesize("s3://{$bucket}/{$key}");

Stream wrappers in PHP are identified by a unique protocol; the Amazon S3 stream wrapper uses the “s3://” protocol. Amazon S3 stream wrapper URIs always start with the “s3://” protocol followed by an optional bucket name, forward slash, and optional object key: s3://bucket/key.

Streaming downloads

The Amazon S3 stream wrapper allows you to truly stream downloads from Amazon S3 using functions like fopen(), fread(), and fclose(). This allows you to read bytes off of a stream as needed rather than downloading an entire stream upfront and then working with the data.

The following example opens a read-only stream, read up to 1024 bytes from the stream, and closes the stream when no more data can be read from it.

// Open a stream in read-only mode
if (!($stream = fopen("s3://{$bucket}/{$key}", 'r'))) {
    die('Could not open stream for reading');
}

// Check if the stream has more data to read
while (!feof($stream)) {
    // Read 1024 bytes from the stream
    echo fread($stream, 1024);
}
// Be sure to close the stream resource when you're done with it
fclose($stream);

Seekable streams

Because no data is buffered in memory, read-only streams with the Amazon S3 stream wrapper are by default not seekable. You can force the stream to allow seeking using the seekable stream context option.

// Create a stream context to allow seeking
$context = stream_context_create(array(
    's3' => array(
        'seekable' => true
    )
));

if ($stream = fopen('s3://bucket/key', 'r', false, $context)) {
    // Read bytes from the stream
    fread($stream, 1024);
    // Seek back to the beginning of the stream
    fseek($stream, 0);
    // Read the same bytes that were previously read
    fread($stream, 1024);
    fclose($stream);
}

Opening seekable streams allows you to seek only to bytes that have been previously read. You cannot skip ahead to bytes that have not yet been read from the remote server. In order to allow previously read data to be recalled, data is buffered in a PHP temp stream using Guzzle’s CachingEntityBody decorator.

Streaming uploads from downloads

You can use an Amazon S3 stream resource with other AWS SDK for PHP operations. For example, you could stream the contents of one Amazon S3 object to a new Amazon S3 object.

$stream = fopen("s3://{$bucket}/{$key}", 'r');

if (!$stream) {
    die('Unable to open stream for reading');
}

$client->putObject(array(
    'Bucket' => 'other_bucket',
    'Key'    => $key,
    'Body'   => $stream
));

fclose($stream);

Uploading data

In addition to downloading data with the stream wrapper, you can use the stream wrapper to upload data as well.

$stream = fopen("s3://{$bucket}/{$key}", 'w');
fwrite($stream, 'Hello!');
fclose($stream);

Note: Because Amazon S3 requires a Content-Length for all entity-enclosing HTTP requests, the contents of an upload must be buffered using a PHP temp stream before it is sent over the wire.

Traversing buckets

You can modify and browse Amazon S3 buckets similar to how PHP allows the modification and traversal of directories on your filesystem.

Here’s an example of creating a bucket:

mkdir('s3://bucket');

You can delete empty buckets using the rmdir() function.

rmdir('s3://bucket');

The opendir(), readdir(), rewinddir(), and closedir() PHP functions can be used with the Amazon S3 stream wrapper to traverse the contents of a bucket.

$dir = "s3://bucket/";

if (is_dir($dir) && ($dh = opendir($dir))) {
    while (($file = readdir($dh)) !== false) {
        echo "filename: {$file} : filetype: " . filetype($dir . $file) . "n";
    }
    closedir($dh);
}

You can recursively list each object and prefix in a bucket using PHP’s RecursiveDirectoryIterator.

$dir = 's3://bucket';
$iterator = new RecursiveIteratorIterator(new RecursiveDirectoryIterator($dir));

foreach ($iterator as $file) {
    echo $file->getType() . ': ' . $file . "n";
}

Using the Symfony2 Finder component

The easiest way to traverse an Amazon S3 bucket using the Amazon S3 stream wrapper is through the Symfony2 Finder component. The Finder component allows you to more easily filter the files that the stream wrapper returns.

require 'vendor/autoload.php';

use SymfonyComponentFinderFinder;

$finder = new Finder();

// Get all files and folders (key prefixes) from "bucket" that are less than
// 100K and have been updated in the last year
$finder->in('s3://bucket')
    ->size('< 100K')
    ->date('since 1 year ago');

foreach ($finder as $file) {
    echo $file->getType() . ": {$file}n";
}

You will need to install the Symfony2 Finder component and add it to your project’s autoloader in order to use it with the AWS SDK for PHP. The most common way to do this is to add the Finder component to your project’s composer.json file. You can find out more about this process in the Composer documentation.

More information

We hope you find the new Amazon S3 stream wrapper useful. You can find more information and documentation on the Amazon S3 stream wrapper in the AWS SDK for PHP User Guide.

AWS Developer Tools Blog