Monster Muck Mashup - Video Conversion Service AMI

items>Monster Muck Mashup Video Conversion Service AMI
Community Contributed Software

  • Amazon Web Services provides links to these packages as a convenience for our customers, but software not authored by an "@AWS" account has not been reviewed or screened by AWS.
  • Please review this software to ensure it meets your needs before using it.

This is the public AMI referenced and used in the AWS article "Monster Muck Mashup - Mass Video Conversion Using AWS".

Details

Submitted By: M. Garnaat
US East AMI ID: ami-2eba5f47
AMI Manifest: aes-images/convertvideo.manifest.xml
Operating System: Other
Created On: March 19, 2007 2:39 AM GMT
Last Updated: January 3, 2009 11:36 PM GMT

This AMI is a companion to the AWS article "Monster Muck Mashup - Mass Video Conversion using AWS".  It is a scalable video conversion service as described in the article.  It is built from the Base Core Fedora 4 AMI and adds ffmpeg for the video conversion and boto for the service framework.  This particular conversion service can only convert AVI format files to MPeg4, for use on a Video iPod although the service could be extended to include other conversions supported by ffmpeg.

Building This Image from Scratch

First, create a group that we can use for controlling access to the services:

$ ec2-add-group -d "A group for controlling access to autonomous services." auto_service

Now, authorize ssh access (port 22) to instances in this group:

$ ec2-authorize auto_service -p 22
GROUP auto_service
PERMISSION auto_service ALLOWS tcp 22 22 FROM CIDR 0.0.0.0/0

Now, let's create a new instance of the Fedora Core 4 Base public image (ami-20b65349) using our new security group:

$ ec2-run-instances -k gsg-keypair -g auto_service ami-20b65349
RESERVATION r-dd52b7b4 084307701560 auto_service
INSTANCE i-10876079 ami-20b65349 pending gsg-keypair 0

We can check on the boot progress for the instance by periodically running the ec2-describe-instances command like this:

$ ec2-describe-instances
RESERVATION r-dd52b7b4 084307701560 auto_service
INSTANCE i-10876079 ami-20b65349 domU-12-31-34-00-03-E2.usma2.compute.amazonaws.com running gsg-keypair 0

So, we can see that our new instances is now up and running, so let's log in and begin to configure our video conversion service:

$ ssh -i ~/Projects/id_rsa-gsg-keypair root@domU-12-31-34-00-03-E2.usma2.compute.amazonaws.com
The authenticity of host 'domu-12-31-34-00-03-e2.usma2.compute.amazonaws.com (216.182.239.46)' can't be established.
RSA key fingerprint is 8c:4f:6d:74:84:51:59:a0:9e:14:98:4b:6c:14:33:d9.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'domu-12-31-34-00-03-e2.usma2.compute.amazonaws.com,216.182.239.46' (RSA) to the list of known hosts.

__| __|_ ) Rev: 2
_| ( /
___|\___|___|

Welcome to an EC2 Public Image
:-)
Base

[root@domU-12-31-34-00-03-E2 ~]#

We are going to need a few extra packages installed on our instances so let's use yum, the Fedora package manager to do that right now:

[root@domU-12-31-34-00-03-E2 ~]# yum install subversion gcc
...
Complete!
[root@domU-12-31-34-00-03-E2 ~]#

The easiest way to install the ffmpeg package is to use the yum package manager just as we did above to install subversion and gcc. However, there is small problem. The ffmpeg package is not in the normal list of yum repositories that a standard EC2 images searches. So, we need to add a new repository, freshrpms, to our list of repositories. To do that, follow these steps:

[root@domU-12-31-34-00-03-E2 ~]# wget http://ftp.freshrpms.net/pub/freshrpms/fedora/linux/4/freshrpms-release/freshrpms-release-1.1-1.fc.noarch.rpm
[root@domU-12-31-34-00-03-E2 ~]# rpm -i freshrpms-release-1.1-1.fc.noarch.rpm
[root@domU-12-31-34-00-03-E2 ~]# yum update
[root@domU-12-31-34-00-03-E2 ~]# yum install ffmpeg

Okay, we have all of our system packages installed. We also need to install the boto library but before we do that, let's create a new user on our instance. We will use this new user's home directory as a base to install all of our service software. We will also run the services as this new user rather than running the services as root. To add a new user:

[root@domU-12-31-34-00-03-E2 ~]# useradd -m autoserv
[root@domU-12-31-34-00-03-E2 ~]# passwd autoserv
Changing password for user autoserv.
New UNIX password:
Retype new UNIX password:
passwd: all authentication tokens updated successfully.
[root@domU-12-31-34-00-03-E2 ~]#

I would recommend using a strong password for this user, just to be safe. Once the user is created, let's su to that user and checkout the boto code from the subversion respository:

[root@domU-12-31-34-00-03-E2 ~]# su - autoserv
[autoserv@domU-12-31-34-00-03-E2 ~]$ pwd
/home/autoserv
[autoserv@domU-12-31-34-00-03-E2 ~]$ svn checkout http://boto.googlecode.com/svn/trunk/ boto
...
[autoserv@domU-12-31-34-00-03-E2 ~]$ ls
boto
[autoserv@domU-12-31-34-00-03-E2 ~]$

Now we need to edit our .bash_profile to set up our environment for Python. Add the following lines to the end of the .bash_profile file located in /home/autoserv:

export PYTHONPATH=$PYTHONPATH:/home/autoserv/boto

Comments

ffmpeg ...
Good article. I've re-created your image, and I've tried your public AMI. ffmpeg consistently segfaults, when trying to convert video to flv. I've tried several input formats. ffmpeg -y -i 'SANY0073.MP4' -ar 22050 -ab 16k -f flv -s 320x240 'processed.flv' 2>&1
t1500 on February 17, 2008 11:15 PM GMT
Some what incomplete
The article didn't mention how the service is started, so here it is; $ tail -3 /etc/rc.local # startup our autonomous service sudo -b -i -u autoserv /home/autoserv/run_server $ cat ~autoserv/run_server #!/bin/sh cd /home/autoserv/boto svn update cd /home/autoserv python /home/autoserv/boto/boto/services/start_service.py
bltjockstar on April 10, 2007 3:59 AM GMT
We are temporarily not accepting new comments.
©2017, Amazon Web Services, Inc. or its affiliates. All rights reserved.