As a startup, Essess sought to find a way to easily and cost-effectively store thermal-imaging data from its vehicles’ imaging systems. “Each vehicle, deployed across large geographic areas, collects more than a petabyte of data each year,” says John Morrissey, the company’s director of software engineering. “We need technology that makes it easy to store that much data.”
The company also needed to be able to scale its application quickly. “We sometimes need to scale up to run several hundred instances at once, depending on the project and the customer,” says Morrissey. Additionally, Essess must be able to scale the application down when it is not being heavily used. “Our data processing happens in bursts. We don’t want to pay for hardware sitting around when it’s not needed.”
Essess also wants to give its software developers flexibility. “We are a young company, and we’re trying to be as agile as we can in our development processes,” says Morrissey.
To address its business needs, Essess decided to launch its business in the cloud. “The cloud is ideally suited for our business model, because of our need to scale up and down on demand,” says Morrissey. Essess chose to run its data-acquisition application on Amazon Web Services (AWS). “We’ve been on AWS from day one, because Amazon gives us the right mix of features, support, and flexibility for our business.”
Each Essess vehicle collects data using AWS Import/Export Snowball, a purpose-built appliance for data transfer and storage. From there, the data is imported into Amazon Simple Storage Service (Amazon S3) for storage in the cloud. The company’s scientific analysis pipeline consumes the raw data and extracts high-resolution thermal images, storing them in Amazon S3 for further analysis.
The organization bases its data-acquisition application on Amazon Relational Database Service (Amazon RDS) for PostgreSQL, which is used for operating and scaling PostgreSQL databases. “We like that Amazon RDS gives us multiple availability zones for better redundancy, and we also like that it’s cheaper than other alternatives we’ve seen,” says Morrissey.
The company also uses AWS CloudFormation templates to automatically deploy database instances. “We can deploy an Amazon RDS instance in less than a minute using CloudFormation templates,” says Morrissey.
Using AWS, Essess vehicles can store large volumes of thermal-imaging data every year. “We can use Amazon S3 for both long-term archival storage and storage of intermediate metadata,” says Morrissey. “It gives us the performance and flexibility we require from a storage solution.”
With AWS Snowball, the company also has a reliable way to quickly transfer huge volumes of data from its application to the cloud. “Using AWS Snowball, we can very quickly move terabytes of data every day to the cloud without worrying about maintenance and management,” says Morrissey. “Previously, we had to invest in and manage up to 40 hard drives in each of our vehicles. That wasn’t optimal. The hard drives often got damaged as we shipped them back and forth, and their lifespan was short. Snowball makes the process more reliable and simple. Our remote teams simply drop the appliance in the mail once it’s full, and we get the data into Amazon S3 within a few days.”
Essess also has the scalability it needs using AWS. “We can scale our machine-learning projects, for example, up to hundreds of instances at a moment’s notice whenever we want to. We don’t have to pay to have hardware sitting around not being used,” says Morrissey. “And in general, the cloud is ideally suited to the ‘bursty’ nature of our analysis pipeline. We only pay for what we use, and we can access those instances immediately, without waiting weeks to procure new hardware from vendors. We don’t get into situations where our capital expenditures hinder what we’re able to deliver to our clients.”
The organization’s developers also have the agility they require. “We want to be as lightweight as possible as a company, and we can do that by relying on AWS,” says Morrissey. “Our developers can spin up test clusters whenever they need to, without waiting.”
In addition, Essess can provide rapid, actionable insights to its customers. “Because we don’t have the overhead of hardware management, we can turn products around to our customers very quickly,” says Morrissey. “As a startup company, the clock is always ticking. Using AWS, we can spend more time improving our solutions and turning them around faster. We can let our business needs dictate our underlying technology and not the other way around. AWS gives us the kind of fast project turnaround that impresses our customers, and that helps us grow faster as a company.”
To learn more about how AWS can help you manage your data center migration, visit our AWS Cloud Data Migration details page.