AWS Business Intelligence Blog

How SmugMug accelerates business intelligence with Amazon QuickSight scenarios

This post was co-written with Kip Farmer and Dr. Geoff Ryder from SmugMug.

SmugMug serves as the trusted platform of choice for elite photographers worldwide. As the Data Science and Engineering Team (Team Prophecy), we manage our company’s data lake and data warehouse operations, helping experts across the organization discover and act on business insights from data. Since 2002, our team of approximately 200 employees has supported and served a community of 100 million photographers and photo enthusiasts each year. In this post, we demonstrate how SmugMug uses the scenarios capability of Amazon Q in QuickSight to enable self-service AI-powered analysis while maintaining data security.

Challenges running AI analysis without compromising security or quality

For the past 5 years, Amazon QuickSight has been our go-to business intelligence (BI) solution. It’s now the main BI platform at SmugMug, used by 60% of our employees across seven departments: Customer Support, Engineering, Finance, Marketing, Operations, Product Management, and Sales. These teams rely on QuickSight for reporting on operational data that drives critical business decisions.

As a lean team running multiple lines of business at scale, we face unique challenges. We have only a few dedicated data engineers supporting a business with tens of millions of active users and tens of billions of photos. This reality requires us to find solutions that maximize efficiency while maintaining data integrity and security.

Before implementing the scenarios capability of Amazon Q in QuickSight, our business users who wanted AI-powered analysis faced two significant security challenges:

  • Data governance – External AI services could store our data on their servers, potentially in different jurisdictions with varying privacy laws. This created compliance challenges, especially for General Data Protection Regulation (GDPR) and Sarbanes-Oxley Act (SOX) requirements.
  • Data quality – Export size limitations and selection bias meant analyses were based on incomplete data, leading to potentially flawed insights.

Amazon Q in QuickSight solved these challenges for us, allowing us to enable AI-powered analytics while maintaining our uncompromising security standards. Unlike other AI tools that required us to compromise on data protection, QuickSight Scenarios empowers us to harness cutting-edge AI capabilities within our existing secure infrastructure. This means we can innovate for our customers without ever putting their data at risk. With tens of billions of photos and millions of active users each month, QuickSight Scenarios provides the direct connectivity to our data that we need to get clear insights without worrying about piecemeal exports.

Solution overview

QuickSight scenarios revolutionized our approach to data analysis by addressing these challenges head-on. The solution enables comprehensive agentic analysis across multiple data sources, making it possible to analyze up to 10 million rows combined. This capability alone dramatically expanded the scope and depth of our analytical capabilities.

Moreover, the scenarios feature provides real-time integration with our AWS data warehouse, making sure our analyses are based on the most current data available. Perhaps most importantly, it enables secure AI-powered analysis within our existing environment, alleviating the need for risky data exports.

The democratization of advanced analytics has been another game changer for our organization. Scenarios offer natural language interfaces for complex queries, making it possible for non-technical users to ask sophisticated questions of our data. The feature also provides guided statistical analysis, helping users navigate complex analytical concepts without requiring extensive training.

Key use cases for QuickSight scenarios

We’ve implemented several high-impact use cases with QuickSight scenarios:

  • Computing customer retention curves for arbitrary cohorts – Previously, only our analysts could develop the complex SQL required for retention reports. With QuickSight scenarios, business users can now create these reports independently, democratizing access to critical customer insights. In the following example, we use the scenarios functionality to analyze customer retention based on sign-up date. The first screenshot depicts a natural language description of the problem. The second screenshot shows the results from the analysis of retention by customer cohort over time (based on example data).

  • Discovering variables that influence key metrics – Before the scenarios feature, we used machine learning (ML) in other tools to identify variables affecting engagement and churn, but we saw gaps in access and explainability. With scenarios, QuickSight Reader users can do deep-dive analysis. In addition, we can see the assumptions used by the AI agent and the generated analysis in a step-by-step manner.
  • Bringing statistical analysis features to QuickSight – The guided exploration available in the scenarios feature helps users discover what kind of statistical and forecasting features they need to answer specific problems. In the following example, we use the scenarios functionality to show sales over time in a particular product category. The first screenshot shows a natural language prompt to forecast sales in a particular category. The second screenshot shows the chart presenting forecasted sales (based on example data). Although standalone insights have powerful capabilities, they weren’t widely adopted at SmugMug because users had to predefine their analysis needs. Scenarios bridge this gap by guiding users through the analysis process.

Our data sources

Our implementation of QuickSight scenarios uses a diverse array of data sources, reflecting the complex nature of our business operations. The base of our data architecture is Amazon Redshift, which houses an impressive 99% of our total data volume analyzed through QuickSight.

Complementing our Redshift data warehouse, we also integrate planning data from JIRA and operational metrics from AWS logs. This multi-source approach helps us create a holistic view of our business, combining high-level strategic planning with granular operational details.

Benefits realized with QuickSight

QuickSight has become essential for analyzing KPIs across seven departments and enabling root cause analysis for their metrics. Key benefits include:

  • Improved decision-making – We’ve observed recent increases in our most important KPIs, including a double-digit increase in our conversion rate for new subscriptions. This improvement can be attributed to the more nuanced, data-driven marketing and product strategies we’ve been able to implement using insights from QuickSight scenarios.
  • Faster analysis – Compared to alternatives, the visual system in QuickSight speeds up the turnaround time by several hours for each report. This has helped our teams be more agile, responding quickly to market changes and customer needs.
  • Cost-efficiency – We’re achieving the same value from QuickSight that we would from tools costing two to five times more, representing significant savings for our organization. Additionally, by consolidating our AI-powered analytics within the QuickSight environment, we’ve reduced costs associated with multiple external AI tools.
  • Democratized analytics – We’ve successfully expanded our analysis capabilities to 60% of our employees, democratizing data analysis across our organization. This has reduced the burden on our specialized data team and allowed for more widespread data-driven decision-making.
  • Enhanced data security – By keeping AI-powered analysis within our AWS environment, we’ve minimized the security and compliance risks previously associated with exporting data to external AI tools. This has strengthened our ability to maintain consistent data governance and meet regulatory requirements like GDPR and SOX.

Conclusion

At SmugMug, we operate with a lean team running multiple lines of business at scale. We only have a few dedicated data engineers supporting a business with tens of millions of active users and tens of billions of photos. The QuickSight scenarios feature helps our business users become more self-sufficient and helps our entire team accomplish more with fewer resources.

We couldn’t perform smart AI-guided exploration against our full datasets without a tool like QuickSight scenarios. With years of curated datasets in QuickSight, this feature is by far the best way to bring the power of AI to the task of discovering business insights from them.

For organizations looking to democratize data analysis while maintaining security and data quality, QuickSight scenarios offer a compelling solution that balances accessibility with governance, which we’ve found essential to our success at SmugMug.

Start your journey with QuickSight scenarios today:

About the authors

Kip Farmer serves as a Data Analyst in the Data Science department at SmugMug, where he delivers business reporting data analysis and dashboards to support the company’s product, marketing, and finance organizations. He develops and maintains reporting aspects of the data warehouse, ensuring data integrity and availability for business intelligence purposes. Kip has over 30 years of experience in database development and management roles at Hewlett-Packard and holds a Master’s degree in Computer Science from Santa Clara University.

Dr. Geoff Ryder serves as the Manager of Data Science and Engineering at SmugMug, where he leads Team Prophecy in managing the company’s cloud-based data warehouse and analytics platforms. With a focus on leveraging the best AI tools, his team empowers photography clients to enhance their sales of both physical and digital photographic products. Geoff brings over two decades of experience in technical and business roles across Silicon Valley companies, and holds a PhD in Computer Engineering from UC Santa Cruz.

Kevin Bell is a Senior Solutions Architect at AWS based in Seattle. He has been building things in the cloud for over 10 years. You can find him online as @bellkev on GitHub.

Corey Keane is a Media and Entertainment (M&E) Sr. Account Manager at AWS. Corey has held a number of positions at Amazon and AWS throughout his 8 years with the company across M&E—including technical business development for strategic partnerships with international game developers, in addition to his current role managing AWS customers in the Media vertical. He leans on his pan-Amazon experience from working on other teams to identify new partnerships between our customers and other Amazon businesses to bring disruptive products to market.

Eddie Yao is an Enterprise Support Lead at AWS. He guides AWS customers build and run production workloads at scale in the cloud. With over a decade experience in tech, from web application engineering and consulting, to digital platform solutions architecture, Eddie currently focuses on Media & Entertainment industry and AI/ML (including generative AI).