Foursquare Modernizes Data Analytics Using Amazon EMR Serverless and Reduces Costs by 45%

Founded in 2009, Foursquare is a geospatial technology company headquartered in New York. Its cloud offering collects near real-time data points from users in more than 200 countries.

In 2015, Foursquare began offering enterprise products to developers and marketers, giving advertising agencies and companies the opportunity to process information from these insights to attract and better serve customers. The information Foursquare gathers also benefits urban planners who use behavior patterns to formulate emergency response plans.

The company processes billions of data points daily. Before it performs analyses, it uses data pipelines to scrub personal identifiable information to protect user privacy.

In 2019 and 2022, Foursquare completed a two-stage cloud migration, transferring its open-source data pipelines to Amazon EMR, a cloud big data solution for petabyte-scale data processing, interactive analytics, and machine learning using open-source frameworks.

In 2023, to further optimize its big data offering, Foursquare decided to pursue another migration—this time from Amazon EMR to the solution’s serverless option. It chose to migrate its data pipelines that scrub personal identifiable information to Amazon EMR Serverless to simplify cluster management. “In Amazon EMR Serverless, you don’t have to define specific instance types, which makes it even simpler to use,” says Robert Joseph, vice president of engineering at Foursquare. The process of deploying its data scrubbing pipelines to Amazon EMR Serverless took 2 months.

The primary driver behind Foursquare’s modernization with Amazon EMR Serverless was productivity optimizations, but the company also achieved significant cost savings. “Amazon EMR Serverless allocates just enough RAM CPU and can spin up or down faster than instances, so you get more efficient usage, which saves money,” says Joseph. Overall, Foursquare’s use of Amazon EMR Serverless reduced costs by 45 percent.

The modernization to a serverless AWS solution also increased staff productivity. Using Amazon EMR Serverless removes the need to define instance types, so Foursquare’s developers can spend more time scrubbing user data as it’s collected and less time dealing with cluster settings and other granular optimizations. “There is a lot of efficiency to be gained by the rapid scaling and the rightsizing of clusters that happens automatically with Amazon EMR Serverless,” says Joseph.

As a result of these benefits, Foursquare saw improvements in its ability to enhance the experience of its customers. “We have confidence that we can efficiently meet our customer-facing service-level agreements for delivery of data with Amazon EMR Serverless,” says Joseph. “Our data deliveries are now even more reliable and cost effective, which translates to benefits for our customers.”

The increased efficiency achieved with the company’s modernization comes with the opportunity for Foursquare to evaluate current practices and to plan for future projects. “The modernization of the data scrubbing pipelines with Amazon EMR Serverless has been great,” says Joseph. “It ended up being a sort of proof of concept for us.”

The company continues to build data pipelines and is taking time to consider which ones will take advantage of serverless solutions on AWS in the future. “Where Amazon EMR Serverless is applicable, it makes jobs more cost effective,” says Joseph. “We think it’s best practice to use serverless solutions where we can.”