AWS Case Study: Optensity
Optensity was formed in 2010 when the company’s founders wanted to create a flexible, low-cost environment where users could easily create executable applications and workflows from new and existing algorithms and applications to analyze large volumes of data. The company’s flagship product, AppSymphony, is a Platform as a Service (PaaS) offering that enables the development of applications and big data processing in the cloud.
Headquartered in Virginia, Optensity provides the AppSymphony platform and associated services for companies and government agencies that build, deploy and use cloud-based analytics for business intelligence, finance, social media, computer network defense, and intelligence, surveillance, and reconnaissance (ISR).
Optensity works extensively with customers in the US Department of Defense (DoD) and Intelligence agencies. The company’s principals observed that while federal agencies made extensive investments to capture and store data, much of the data remained underutilized. Pamela Arya, co-founder and CEO of Optensity, explains: “Fully exploiting big data often requires expertise in computer programming, high performance computing, mathematics, statistics, as well as specific operational knowledge. Not all agencies or companies have the ability to do this quickly for each new problem or data source. In addition, the data is often distributed across multiple locations, which requires a tool that can run in a hybrid cloud environment.”
Optensity believed that a PaaS solution could rapidly adapt to changing needs, new sources, and massive data volumes, allowing users to easily build and use new analytic techniques and exploitation algorithms. AppSymphony was conceived and built specifically to run on a distributed elastic compute infrastructure and for hybrid cloud environments.
Why Amazon Web Services
As a bootstrap start-up working from home offices, the Optensity team had no space and insufficient funds to purchase and set up a dedicated infrastructure. “We originally learned about Amazon Web Services (AWS) in 2008 through IT magazines and started experimenting with AWS in 2009,” explains Arya. “AWS has a variety of pre-built image sizes and operating systems, and an SDK that made it easy to control the services. We didn’t look at alternatives because the cost was so low and AWS met all of our requirements. AWS was the only way that we could affordably build, test, and demonstrate our product.”
Initially, it took Optensity about two months to develop the first AppSymphony prototype. Optensity’s principals spent that time learning about AWS, creating base Amazon Machine Images (AMIs) and installing code. From the initial prototype to the official product launch, the team built, tested, and demonstrated AppSymphony on the AWS Cloud.
Optensity developed AppSymphony using several languages, primarily Java. The company uses Amazon Elastic Compute Cloud (Amazon EC2) and Amazon Simple Storage Service (Amazon S3) for its software development and test environments. Using Amazon EC2 instances in multiple regions and Availability Zones makes development and testing capabilities available to Optensity’s geographically distributed software engineering team. AppSymphony is installed across AMIs backed by Amazon Elastic Block Store (Amazon EBS) volumes attached to Amazon EC2 instances.
AppSymphony’s component libraries access AWS APIs for Amazon EC2, Amazon S3, Amazon Simple Notification Service (Amazon SNS), and Amazon Simple Queue Service (Amazon SQS) and Amazon Virtual Private Cloud (Amazon VPC). Using a graphic interface, AppSymphony users can create sophisticated distributed analysis agents that can communicate and synchronize complex real-time and archived data analysis strategies.
Optensity also built AppSymphony components so that users can graphically compose and run map-reduce applications using the Hadoop framework. Arya explains, “We are currently using a Hadoop solution that we built on our AMIs from Apache distributions, which we use to dynamically deploy in clusters within Amazon VPCs.”
The AppSymphony platform running on AWS allows analysts and subject matter experts to exploit the power of big data in scalable, federated, heterogeneous cloud infrastructures without requiring them to be experts in cloud software development. Arya adds, “We have built discreet components that interact with Amazon SNS, Amazon SQS and Amazon S3 from our flow diagrams. This means that our analyst users can compose calls to use services as an integral part of their applications without software engineering skills.” Figure 1 illustrates an Optensity flow diagram.
Optensity takes advantage of AWS to offer the following benefits to their customers:
- Reduced time to market for new analytic capabilities because AppSymphony applications running on AWS can often be composed in minutes instead of weeks;
- Increased comprehensiveness of the resulting analyses because application components can leverage AWS to screen and incorporate more relevant data sources; and
- Responsiveness to highly varying analytic demand because AppSymphony analysis applications running on Amazon EC2 instances can be rapidly scaled up or down to meet demand.
“We simply would not have been able to afford to successfully launch AppSymphony without AWS,” states Arya. “We estimate that it would have cost us $50,000 in hardware and software, one man-year in system engineering and administrative labor, and $25,000 in facility costs (rent and utilities) to build, test, and launch the first version of the product with an on-premises data center. With AWS, we were able to charge all our infrastructure computing costs to a credit card for about $200 per month.”
Optensity expects to continue to add AWS products to power discreet components of the AppSymphony platform as customers request services. Optensity is also considering using AWS to host AppSymphony as an on-demand app development platform as a service in the commercial space. “As we continue to innovate, we revisit our product roadmap and say, wouldn't it be nice if AWS had this service or capability,” comments Arya. “Then there’s a press release announcing just what we need.”
To learn more about how AWS can work with public sector customers, visit our Public Sector details page: http://www.aws.amazon.com/publicsector/.