Forage Improves Data Analytics Speeds and Gains Real-Time Metrics with AWS
The Rise of Virtual Internships
As offices worldwide increasingly embrace flexible work-from-home arrangements, the nature of internships is undergoing a profound shift, with many companies moving their programs online.
Forage, a startup technology company that connects students with Fortune 500 companies and other organizations offering virtual internships, experienced rapid growth in 2020 as this trend accelerated. Enrollments on its platform surged from 80,000 interns in December 2019 to more than 1 million in December 2020. Over the same period, companies providing internships on Forage tripled, from 20 to more than 70.
Faster analytics speeds with AWS represent a huge improvement on product quality and usability for our clients.
Cofounder and Chief Technology Officer, Forage (left)
Improved Data Analytics Capabilities
When Forage first launched in 2017, then known as InsideSherpa, it was set up on MongoDB Atlas, a managed hosting provider for MongoDB which used servers and databases that ran on Amazon Web Services (AWS). As the business grew, Forage has expanded its use of AWS to create more value for its customers by enabling them to execute faster, high-quality data queries, such as analyses of student retention rates and comparisons of survey responses across countries.
“In the past, some queries would take 30−40 seconds. Now, it’s gotten down to 1 second or half a second. For us, the faster analytics speeds with AWS represent a huge improvement on product quality and usability for our clients,” says Pasha Rayan, cofounder and chief technology officer at Forage.
Forage’s customers can access a dashboard that allows them to sort and analyze information about interns, including their backgrounds and performance in the program. With faster query speeds on AWS, Forage structures its data so its clients can quickly identify reporting and analytics for diversity. This enables its clients to reach underrepresented groups, especially in the technology industry. Recruiters could previously visit only 20–30 university campuses a year in the US, but with Forage, they can reach thousands of international campuses via a single online platform.
Analytics Solution Provides Real-Time Metrics
Forage worked with AWS to build a data analytics solution that takes advantage of AWS Database Migration Service (AWS DMS) to replicate data from a legacy database into a serverless data lake on AWS Lambda. And Forage used AWS Glue to transform, process, and publish the data to Amazon Relational Database Service (Amazon RDS) for the dashboards that its applications display.
This analytics solution provides real-time quantitative metrics for use on client dashboards, including student completion rates for programs, user demographic and geographic breakdowns, usage patterns, and more. It has also been useful for Forage’s internal data science, with ad hoc data queries in Postgres being much faster and easier to implement than MongoDB equivalents.
Global Infrastructure Paves Way for Expansion
From its roots in Australia and the US, Forage has quickly expanded internationally to the UK, Europe, Singapore, India, and South Africa, with more opportunities on the horizon. This growth into new geographic regions has driven the company’s need for infrastructure that is truly global. Data sovereignty and data privacy are vital to Forage’s clients. Forage is using multiple AWS Infrastructure Regions to support its global footprint and store client data in local servers across the UK, US, Europe, and Asia. “The security across multiple AWS Regions means we can grow into different countries and areas that we wouldn’t have been able to without AWS,” says Rayan.
Additionally, some of Forage’s customers require security and compliance certifications—including General Data Protection Regulation (GDPR), California Consumer Privacy Act (CCPA), and SOC 2—all of which AWS has been able to support. “We often get asked if our infrastructure meets the highest standards. With AWS, we have documentation that allows us to say, ‘Yes, we have the highest standards,’” Rayan says.
Scalability and Flexibility as Enrollments Grew Tenfold
With Forage’s technology solution running on AWS, the company scaled with relative ease into a period of rapid growth in 2020. AWS also enabled Forage the flexibility to continue to innovate at a fast pace, including creating customizable graphs and analytics for corporate partners and providing better feedback by surfacing the right users faster. “Because AWS is both cost-driven and rich in its tool set, we can experiment a lot faster than most other infrastructure providers. We’re quite a high experimentation team. For us, having the freedom to play around with our infrastructure, and set it up to do what we need to do, has been invaluable,” Rayan says.
AWS Support “Incredibly Important” to IT Team
As its business has grown, so too has Forage’s IT team. When the company was formed, it had just two IT specialists. By the end of 2020, the company had 30 employees, including more than a dozen IT specialists. According to Rayan, AWS Support has been “incredibly important” in helping facilitate growth alongside new additions to Forage’s internal tech team. “We wouldn’t have been able to roll out key features, such as our enterprise dashboard and our reporting and emailing features, with our previous data infrastructure. With our data analytics solution running on the AWS Cloud, plus 24/7 support from the AWS team, we’ve been able to release key features, get a leg up, and move much faster. It would’ve been very hard for us to build a company if not for AWS.”
Forage is a technology startup that connects students with Fortune 500 companies and other organizations offering virtual internships. The Forage platform helps these companies attract, train, and hire the best talent for entry-level roles.
- Reduces data analytics query time from 30–40 seconds to 0.5–1 second
- Adheres to international security and compliance standards
- Enables cost-efficient scalability
- Provides flexibility to innovate
- Facilitates internal IT team growth
AWS Services Used
AWS Glue is a serverless data integration service that makes it easy to discover, prepare, and combine data for analytics, machine learning, and application development. AWS Glue provides all of the capabilities needed for data integration so that you can start analyzing your data and putting it to use in minutes instead of months.
Amazon Relational Database Service
Amazon Relational Database Service (Amazon RDS) makes it easy to set up, operate, and scale a relational database in the cloud. It provides cost-efficient and resizable capacity while automating time-consuming administration tasks such as hardware provisioning, database setup, patching and backups.
AWS Database Migration Service
AWS Database Migration Service helps you migrate databases to AWS quickly and securely. The source database remains fully operational during the migration, minimizing downtime to applications that rely on the database.
AWS Lambda is a serverless compute service that lets you run code without provisioning or managing servers, creating workload-aware cluster scaling logic, maintaining event integrations, or managing runtimes.
To learn more, visit aws.amazon.com/big-data/datalakes-and-analytics.
Companies of all sizes across all industries are transforming their businesses every day using AWS. Contact our experts and start your own AWS Cloud journey today.