Tag: symfony


Amazon S3 PHP Stream Wrapper

by Michael Dowling | on | in PHP | Permalink | Comments |  Share

As of the 2.3.0 release, the AWS SDK for PHP now provides an official Amazon S3 PHP stream wrapper. The stream wrapper allows you to treat Amazon S3 like a filesystem using functions like fopen(), file_get_contents(), and filesize() through a custom stream wrapper protocol. The Amazon S3 stream wrapper opens up some interesting possibilities that were either previously impossible or difficult to implement.

Registering the stream wrapper

Before you can use the Amazon S3 stream wrapper, you must register it with PHP:

use AwsS3S3Client;

// Create an Amazon S3 client object
$client = S3Client::factory(array(
    'key'    => '[aws access key]',
    'secret' => '[aws secret key]'
));

// Register the stream wrapper from a client object
$client->registerStreamWrapper();

After registering the stream wrapper, you can use various PHP filesystem functions that support custom stream wrapper protocols.

$bucket = 'my_bucket';
$key = 'object_key';

// Get the contents of an object as a string
$contents = file_get_contents("s3://{$bucket}/{$key}");

// Get the size of an object
$size = filesize("s3://{$bucket}/{$key}");

Stream wrappers in PHP are identified by a unique protocol; the Amazon S3 stream wrapper uses the "s3://" protocol. Amazon S3 stream wrapper URIs always start with the "s3://" protocol followed by an optional bucket name, forward slash, and optional object key: s3://bucket/key.

Streaming downloads

The Amazon S3 stream wrapper allows you to truly stream downloads from Amazon S3 using functions like fopen(), fread(), and fclose(). This allows you to read bytes off of a stream as needed rather than downloading an entire stream upfront and then working with the data.

The following example opens a read-only stream, read up to 1024 bytes from the stream, and closes the stream when no more data can be read from it.

// Open a stream in read-only mode
if (!($stream = fopen("s3://{$bucket}/{$key}", 'r'))) {
    die('Could not open stream for reading');
}

// Check if the stream has more data to read
while (!feof($stream)) {
    // Read 1024 bytes from the stream
    echo fread($stream, 1024);
}
// Be sure to close the stream resource when you're done with it
fclose($stream);

Seekable streams

Because no data is buffered in memory, read-only streams with the Amazon S3 stream wrapper are by default not seekable. You can force the stream to allow seeking using the seekable stream context option.

// Create a stream context to allow seeking
$context = stream_context_create(array(
    's3' => array(
        'seekable' => true
    )
));

if ($stream = fopen('s3://bucket/key', 'r', false, $context)) {
    // Read bytes from the stream
    fread($stream, 1024);
    // Seek back to the beginning of the stream
    fseek($stream, 0);
    // Read the same bytes that were previously read
    fread($stream, 1024);
    fclose($stream);
}

Opening seekable streams allows you to seek only to bytes that have been previously read. You cannot skip ahead to bytes that have not yet been read from the remote server. In order to allow previously read data to be recalled, data is buffered in a PHP temp stream using Guzzle’s CachingEntityBody decorator.

Streaming uploads from downloads

You can use an Amazon S3 stream resource with other AWS SDK for PHP operations. For example, you could stream the contents of one Amazon S3 object to a new Amazon S3 object.

$stream = fopen("s3://{$bucket}/{$key}", 'r');

if (!$stream) {
    die('Unable to open stream for reading');
}

$client->putObject(array(
    'Bucket' => 'other_bucket',
    'Key'    => $key,
    'Body'   => $stream
));

fclose($stream);

Uploading data

In addition to downloading data with the stream wrapper, you can use the stream wrapper to upload data as well.

$stream = fopen("s3://{$bucket}/{$key}", 'w');
fwrite($stream, 'Hello!');
fclose($stream);

Note: Because Amazon S3 requires a Content-Length for all entity-enclosing HTTP requests, the contents of an upload must be buffered using a PHP temp stream before it is sent over the wire.

Traversing buckets

You can modify and browse Amazon S3 buckets similar to how PHP allows the modification and traversal of directories on your filesystem.

Here’s an example of creating a bucket:

mkdir('s3://bucket');

You can delete empty buckets using the rmdir() function.

rmdir('s3://bucket');

The opendir(), readdir(), rewinddir(), and closedir() PHP functions can be used with the Amazon S3 stream wrapper to traverse the contents of a bucket.

$dir = "s3://bucket/";

if (is_dir($dir) && ($dh = opendir($dir))) {
    while (($file = readdir($dh)) !== false) {
        echo "filename: {$file} : filetype: " . filetype($dir . $file) . "n";
    }
    closedir($dh);
}

You can recursively list each object and prefix in a bucket using PHP’s RecursiveDirectoryIterator.

$dir = 's3://bucket';
$iterator = new RecursiveIteratorIterator(new RecursiveDirectoryIterator($dir));

foreach ($iterator as $file) {
    echo $file->getType() . ': ' . $file . "n";
}

Using the Symfony2 Finder component

The easiest way to traverse an Amazon S3 bucket using the Amazon S3 stream wrapper is through the Symfony2 Finder component. The Finder component allows you to more easily filter the files that the stream wrapper returns.

require 'vendor/autoload.php';

use SymfonyComponentFinderFinder;

$finder = new Finder();

// Get all files and folders (key prefixes) from "bucket" that are less than
// 100K and have been updated in the last year
$finder->in('s3://bucket')
    ->size('< 100K')
    ->date('since 1 year ago');

foreach ($finder as $file) {
    echo $file->getType() . ": {$file}n";
}

You will need to install the Symfony2 Finder component and add it to your project’s autoloader in order to use it with the AWS SDK for PHP. The most common way to do this is to add the Finder component to your project’s composer.json file. You can find out more about this process in the Composer documentation.

More information

We hope you find the new Amazon S3 stream wrapper useful. You can find more information and documentation on the Amazon S3 stream wrapper in the AWS SDK for PHP User Guide.

AWS at Symfony Live Portland 2013

by Jeremy Lindblom | on | in PHP | Permalink | Comments |  Share

A few weeks ago, I had the pleasure of attending the Symfony Live Portland 2013 conference. This year, Symfony Live co-located with the very large DrupalCon, and though I did not attend any of the DrupalCon sessions, I did get to talk to many Drupal developers during lunches and the hack day. It was awesome to be among so many other PHP developers.

I had the honor of being selected as a speaker at Symfony Live, and the topic of my session was Getting Good with the AWS SDK for PHP (here are the slides and Joind.in event). In this talk I did a brief introduction about AWS and its services, taught how to use the AWS SDK for PHP, and demonstrated some code from a sample PHP application that uses Amazon S3 and Amazon DynamoDB to manage its data.

How does the SDK integrate with Symfony?

Since I was in the presence of Symfony developers, I made sure to point out some of the ways that the AWS SDK for PHP currently integrates with the Symfony framework and community.

The SDK uses the Symfony Event Dispatcher

The SDK uses the Symfony Event Dispatcher component quite heavily. Not only are many of the internal details of the SDK implemented with events (e.g., request signing), but users of the SDK can listen for events and inject their own logic into the request flow.

For example, the following code attaches an event listener to an SQS client that will capitalize messages sent to a queue via the SendMessage operation.

use AwsCommonAws;
use GuzzleCommonEvent; // Extends SymfonyComponentEventDispatcherEvent

$aws = Aws::factory('/path/to/your/config.php');
$sqs = $aws->get('sqs');

$dispatcher = $sqs->getEventDispatcher();
$dispatcher->addListener('command.before_send', function (Event $event) {
    $command = $event['command'];
    if ($command->getName() === 'SendMessage') {
        // Ensure the message is capitalized
        $command['MessageBody'] = ucfirst($command['MessageBody']);
    }
});

$sqs->sendMessage(array(
    'QueueUrl'    => $queueUrl,
    'MessageBody' => 'an awesome message.',
));

We publish an AWS Service Provider for Silex

For Silex users, we publish an AWS Service Provider for Silex that makes it easier to bootstrap the AWS SDK for PHP within a Silex application. I used this service provider in my presentation with the sample PHP application, so make sure to check out my slides.

You can use the Symfony Finder with Amazon S3

In my presentation, I also pointed out our recent addition of the S3 Stream Wrapper to our SDK and how you can use it in tandem with the Symfony Finder component to find files within your Amazon S3 buckets.

The following example shows how you can use the Symfony Finder to find S3 objects in the bucket "jcl-files", with a key prefix of "family-videos", that are smaller than 50 MB in size and no more than a year old.

use AwsCommonAws;
use SymfonyComponentFinderFinder;

$aws = Aws::factory('/path/to/your/config.php');
$aws->get('s3')->registerStreamWrapper();

$finder = new Finder();
$finder->files()
    ->in('s3://jcl-files/family-videos')
    ->size('< 50M')
    ->date('since 1 year ago');

foreach ($finder as $file) {
    echo $file->getFilename() . PHP_EOL;
}

Others talked about AWS

One of my co-workers, Michael Dowling, also presented at the conference. His presentation was about his open source project, Guzzle, which is a powerful HTTP client library and is used as the foundation of the AWS SDK for PHP. In his talk, Michael also highlighted a few of the ways that the AWS SDK for PHP uses Guzzle. Guzzle is also being used in the core of Drupal 8, so his presentation drew in a crowd of both Drupal and Symfony developers.

Aside from our presentations, there were various sessions focused on the Symfony framework as well as others on various topics like Composer, caching, and cryptograpy. David Zuelke and Juozas Kaziukėnas both mentioned how they use AWS services in their talks: Surviving a Prime Time TV Commercial and Process any amount of data. Any time, respectively. It was nice to meet in person many PHP developers I’ve talked with online and to participate in Symfony Live traditions such as PHP Jeopardy and karaoke.

While at the conference, I talked to several developers about what would make a good AWS Symfony bundle or Drupal module, but I’m also curious to find out what you think. So… what would you like to see in an AWS Symfony Bundle? What would make a good AWS Drupal module? Let us know your thoughts in the comments.