Category: Cool Sites

Tradier – A Brokerage Cloud Running on AWS and NASDAQ FinQloud

Late last year I wrote about the new NASDAQ OMX FinQloud platform. FinQloud provides customers with cost-effective and efficient management, storage and processing of financial data.

Today I’d like to tell you about an even higher-level platform that has been built on top of FinQloud and AWS.

Tradier’s Brokerage in a Box
Tradier has built the first “brokerage cloud” platform for tool providers, online banks, and wealth management firms. These organizations will no longer have to lease data center space or procure their own technology infrastructure, giving them the ability to run their business within a secure, scalable, and cost-effective framework.

Tradier was able to take advantage of FinQloud to build this platform. For example, Tradier Brokerage (a subsidiary of Tradier) will use FinQloud’s Regulatory Records Retention (R3) to meet data storage requirements for books and records. Notably, R3 is the first cloud-based Write Once Read Many (WORM) compliant data storage system, as required by SEC Rule 17a-4 and CFTC Regulation 1.31. They’ll also use FinQloud’s search and retrieval application for that data.

Mobile app providers, developers of tools to implement trading and options strategies, along with algorithmic and robotic traders will all find the Tradier platform to be of interest. As is often the case with the announcements that I have made in this blog, this platform frees up resources that would otherwise be devoted to the development of lower-level facilities, and enabling innovation at the higher levels.

Tradier’s platform includes a number of foundation components including a Brokerage Cloud API, access to real-time market data, streaming and request/response APIs, support for browsers and mobile devices, and a marketplace / ecosystem for investment tools.

— Jeff;


JumpBox for the AWS Free Usage Tier

We’ve teamed up with JumpBox to make it even easier and less expensive for you to host a WordPress blog, publish a web site with Drupal, run a Wiki with MediaWiki, or publish content with Joomla. You can benefit from two separate offers:

  • The new JumpBox free tier trial for AWS customers lets you launch and run the applications listed above at no charge. There will be a small charge for EBS storage (see below).
  • If you qualify for the AWS free usage tier it will give you sufficient EC2 time, S3 storage space, and internet data transfer to host the application and to handle a meaningful amount of traffic.

Any AWS user (free or not) can take advantage of JumpBox’s offer, paying the usual rates for AWS. The AWS free usage tier is subject to the AWS Free Usage Tier Offer Terms; use of AWS in excess of free usage amounts will be charged standard AWS rates.

Note: The JumpBox machine images are larger than the 10 GB of EBS storage provided in the free usage tier; you’ll be charged $1.50 per month (an additional 10 GB of EBS storage per month) if you run them in the free usage tier.

The applications are already installed and configured; there’s nothing to set up. The application will run on an EC2 instance of its own; you have full control of the configuration and you can install themes, add-ins, and the like. Each application includes a configuration portal to allow you to configure the application and to make backups.

Here’s a tour, starting with the 1-page signup form:

After a successful signup, JumpBox launches the application:

The application will be ready to run in a very short time (less than a minute for me):

The next step is to configure the application (I choose to launch Joomla):

And I am up and running:

You can access all of the administrative and configuration options from a password-protected control panel that runs on the EC2 instance that’s hosting the application:

Here are the links that you need to get started:

As you can probably see from the tour, you can be up and running with any of these applications in minutes. As long as you are eligible for and stay within the provisions of the AWS free usage tier, you can do this for free. I’m looking forward to hearing your thought and success stories; leave me a comment below.

— Jeff;

HeyWatch – AWS-Powered Video Encoding

Earlier this week I spent some time on the phone with Eric Fontaine of HeyWatch. Eric gave me some background information on their video encoding and sharing platform and I thought I would share it with you. HeyWatch has been making use of AWS since 2006 and I am a bit embarrassed that it has taken me this long to find the time to write about it.

Anyway, they offer a professional level video encoding platform. The platform is accessible through the HeyWatch API and can encode videos of any duration or file size. It supports all formats and all devices and is ready to support HTML5. You can even create your own video format by specifying the bitrate, size, codec, and container.

HeyWatch uses Amazon S3 as its primary storage system for encoded videos. An existing in-house server handles some of the encoding work, with automatic spillover into Amazon EC2 when things get busy.

In order to use HeyWatch you purchase credits and then use them to pay for your encoding tasks. Each credit costs $0.10. Encoding costs between 1 and 4 credits ($0.10 to $0.40) depending on the quality (SD or HD), the duration, and the use of 1-pass or 2-pass encoding. You can sign up here and take advantage of a 15-day trial.

Eric told me that they have a number of high-profile customers including the very popular Tumblr blogging platform. All user videos uploaded to Tumblr are automatically routed to HeyWatch and encoded for viewing on Tumblr and on mobile devices.

— Jeff;

Sauce Labs – OnDemand Testing Service on EC2

Late last month I spent some time on the phone with John Dunham and Steve Hazel of Sauce Labs to learn more about their Sauce OnDemand testing service. The product is built around the popular Selenium testing tool and can actually make use of existing Selenium scripts for functionality and performance testing. John and Steve said that I can think of it as “robotically controlled browsers as a service.”

We talked about the fact that the cloud is the natural place for testing resources, since the amount of usage within a particular organization is subject to extreme fluctuations. John told me that the dedicated test resources in a typical organization are idle 99% of the time. This inefficient use of capital is of great concern to CIOs and CTOs, since the return on idle resources is zero.

By allocating EC2 instances on demand, their customers can now test in hours instead of days. Steve said that many customers see a 10- to 20-fold improvement (reduction) in test times because they can simply throw additional resources at the problem. He told me that this works really well until it becomes an “accidental load test” of the system under test! They perform a continuous screen capture from each test machine and store all of the video in Amazon S3. To date they have run over 3 million tests and have retained all of the videos. This feature is called Sauce TV.

Architecturally, the system uses a single Extra Large (m1.xlarge) EC2 instance to coordinate an elastic pool of High CPU Medium (c1.medium) test servers. They’ve built 5 or 6 different Amazon Machine Images (AMIs) with different combinations of operating systems and browsers (IE6, IE7, Firefox, and so forth — complete list is here). They keep a few instances idle so that they can start testing as soon as the need arises. They make use of historic usage data to guide their pre-scaling process, further adjusted to reflect actual demand. They are able to run high-scale tests in short order without charging extra.

Scale-wise, the largest test has consumed about 250 instances and the overall pool has grown as large as 400 instances.

You can sign up for a free 30-day trial of Sauce OnDemand. After that, you can get 1,000 minutes of testing per month for $49 (additional minutes cost $0.05 each). There are also some enterprise plans.

Sauce Labs has implemented a number of improvements to Selenium including SSL and Unicode support, cross-browser file uploads, and an OS-level popup eliminator. They plan to make these enhancements available to the Selenium project.

John told me that The Motley Fool is really sold on the idea of cloud-based testing with Sauce OnDemand. Here’s a presentation from Dave Haeffner, Quality Assurance Manager at The Motley Fool:


FameTown – A New AWS-Powered Facebook Game

Amir of Diversion, Inc. wrote to tell me about his company’s newest release, an AWS-powered Facebook game called FameTown.

FameTown lets you play the role of a movie star in a digital version of Hollywood. You can start on the D-List (a total unknown) and attempt to progress to the A-List, earning points by completing tasks such as meeting with cast members and directors. You can also improve your social standing by attending parties and charity events, and you can hire agents, assistants, and publicists to further boost your career.

Under the hood, this Facebook game makes good use of AWS and a number of other technologies. Here’s a summary:

  • The game is written in Sinatra, a DSL (domain specific language) used to create Ruby apps with minimal effort. The code runs on Amazon EC2.
  • Traffic to the EC2 instances is load balanced using the Nginx load balancer.
  • Membase is used for data storage, hosted on a number of Elastic Block Storage (EBS) volumes.
  • Scalr is used to scale and manage the application.

Amir and I chatted about scalability. He told me that each application server runs on High CPU Extra Large (c1.xlarge) instance and can process 3000 to 3500 requests per second. Membase runs on a set of three Extra Large (m1.xlarge) instances and can handle over 100,000 requests per second.

I’ve not yet played FameTown (but I will), and I hope that Amir and company have a lot of success with it.

— Jeff;


Restore The Gulf – US Government Site Hosted on Amazon EC2

Ho hum – another web site (pictured at right) running on Amazon EC2. No big deal, right?

Actually, it is a pretty big deal.

Take look at the top left of the site. What does it say?

An Official Website of the United States Government

A number of US Government regulations, including an important one called FISMA (Federal Information Security Management Act), establish stringent information security requirements that had to be satisfied before this site was brought on line.

The prime contractor for this project was a company called Government Acquisitions. They worked with Acquia for hosting, and SiteWorx (an Acquia partner) to build the site.

The site itself was built with the very popular Drupal content management system (CMS). You can read more about Acquia Hosting and their use of AWS in this update from Drupal founder Dries Buytaert. Ths is a nice step forward for Drupal and an unmistakable sign that the US Government is paying attention to open source software like Drupal.

If your application requires FISMA certification and you’d like to learn more about running it on AWS, please use the AWS Sales and Business Development form to get in touch with us.

Speaking of the US Government, we’ll be participating in the Adobe Government Assembly on November 3rd, 2010. At this event, government IT innovators will discuss ways to engage, innovate, and improve government efficiency. We hope to see you there!

— Jeff;



TKLBAM – Turnkey Linux Backup and Migration

Liraz from Turnkey Linux sent me some information about their newest product TKLBAM – Turnkey Linux Backup and Migration.

TKLBAM was designed to be a painless and fully automated backup and restore system that “magically” knows what to backup and what not to backup. They wanted to make sure that testing of backups was easy, to avoid painful surprises (read this story to see how painful such a surprise can be). They also designed it to function as a migration system with the goal of allowing it to move a fully working system to another host in minutes.

They were able to meet all of these goals by storing the backups in Amazon S3, taking advantage of its eleven nines of durability and automatically routing requests to the closest S3 location. All backups are AES encrypted, and you can even use an escrow key for extra protection.

The backup and restore mechanism can also be used to migrate application from your desktop to the cloud, from cloud to desktop, or between different Linux distributions. All online starts at the Turnkey Hub:

You can manage backups:

And you can launch Turnkey Linux appliances on Amazon EC2:

You can learn more about TKLBAM by watching the screencast.

— Jeff;

FindTheBest – AWS in Action

Mike from Cirrhus9 sent me some information about their recent implementation of FindTheBest. With his permission, I am sharing some of the details with you so that you can get a better idea of how a sophisticated developer brings together a wide variety of technologies in order to construct a complete web application.

What is FindTheBest?

FindTheBest is an objective comparison search engine that allows people to choose a topic, compare options and decide what’s best for them. It makes for faster and more informed decisions by allowing for easy comparison between all of the available options.

FindTheBest is organized into nine broad categories including: Arts and Entertainment, Business and Economy, Education, Health, Reference, Science, Society, Sports and Recreation and Technology. Each category includes dozens of Apps from Adventure Travel Vacations to Job Websites. Each App consists of a variety of related listings from Vail and Whistler ski resorts under the Ski Resorts App to Barack Obama and Angelina Jolie under the Celebrities App and each listing can be sorted by numerous key filters.

FindTheBest’s structured search allows the user to quickly sort through factors and filters important to them, ultimately helping them make more objective and more informed decisions to important questions and answers.

The user interface blends sorting, filtering, and comparison to produce fast, data-driven pages. Here’s a page with information on over 9000 libraries:

How Was it Implemented?

FindTheBest runs on a 64-bit Ubuntu server. The server boots from an EBS volume and has an XFS RAID-0 two volume EBS storage configuration, with daily snapshot backups via cron-driven calls to Eric Hammond’s ec2-consistent-snapshot script. The server runs a classic LAMP stack with the addition of a lightweight email server and a full-text search engine, all monitored by a custom-tuned Nagios/Groundwork setup.

— Jeff;


This Is A Stick-Up!

The AWS Marketing Team recently moved into a shiny new building in Seattle’s South Lake Union area.

We’d like to spiff up and personalize our space, and thought that our AWS users and fans could help us to do so! We’ve got some really nice AWS stickers that we can trade for any or all of the following:

  1. Some stickers from your company or group.
  2. A picture of your team, perhaps enhanced with your company or product logo.
  3. An interesting piece of SWAG.
  4. A blog post detailing the ways in which your company puts AWS to use, complete with an architecture diagram.

If you’d like some stickers, send us your offering and include a self-addressed envelope (we’ll take care of the postage) to the following address:

Amazon Web Services
Attn: AWS Stickers
P.O. Box 81226
Seattle, WA

— Jeff;

Developers Support the AWS Asia Pacific Region With Cool Applications

Here’s what I know about so far:

Here are screen shots and logos from all of the applications featured above (created by AWS-powered Animoto):

— Jeff;