Streamline SaaS application audit costs with AWS AppFabric and Amazon S3

In the ever-evolving landscape of modern business operations, organizations increasingly rely on a multitude of software-as-a-service (SaaS) applications to streamline processes and enhance productivity. While the adoption of diverse SaaS tools brings undeniable advantages, managing the collection and storage of audit logs from these applications becomes a crucial challenge for maintaining security, compliance, and overall health of organizations.

Each vendor provides audit logs in custom formats stored in proprietary cloud storage services. This fragmented approach creates siloed storage systems, as each unique log format requires a custom approach to access and analysis. Organizations must build custom integrations to centralize these logs, often involving costly data pipelines.

But the challenge doesn’t stop there. Complying with various data privacy and security regulations often mandates retaining audit logs for extended periods, sometimes years. This requirement becomes particularly problematic when dealing with siloed storage and custom integrations. Managing and accessing this data for compliance purposes becomes a complex and potentially expensive task. As the volume of audit logs accumulates over time, the cost of maintaining them in siloed formats across various proprietary cloud services can become immense. This makes finding a cost-effective and scalable solution for long-term log storage crucial for organizations navigating the complex landscape of SaaS applications and regulatory compliance.

In this blog post, we will explore how to utilize AWS AppFabric and Amazon Simple Storage Service (Amazon S3) to tackle these challenges of centralizing and reducing SaaS audit log management overheads. AppFabric helps aggregate, normalize, and enrich logs from different SaaS applications and formats. Amazon S3 provides a scalable and cost-optimized object store for long-term audit log retention. Combined, these services offer an efficient way for organizations to stay compliant while controlling costs.

Simplified SaaS connectivity and normalization with AppFabric

AppFabric quickly connects multiple SaaS applications without the need to configure point-to-point integrations or maintain any code. AppFabric normalizes the disparate SaaS audit log data from these applications into the Open Cybersecurity Schema Framework (OCSF) format – an open-source schema for cybersecurity events and logs that makes it easier to analyze and correlate security data across different sources. Other supported data formats include JSON, and Apache Parquet. AppFabric supports Amazon S3 and Amazon Data Firehose as destination locations for audit logs.

AppFabric is an ideal fit for centralized and simplified SaaS audit log management. With AppFabric, you no longer need to build custom connectors or normalization logic for every SaaS app you onboard. It solves the following key challenges of SaaS audit log centralization out-of-the-box:

• Quickly connecting new SaaS application connections without engineering effort
• Ingesting schema-agnostic audit log data from each connected SaaS application
• Normalizing disparate log data to Open Cybersecurity Schema Framework (OCSF)
• Routing normalized audit logs to curated storage and analytics services

You can then send normalized logs to analytic services like Amazon Redshift, Amazon OpenSearch or a security tool like Splunk, Rapid7, and more. This provides a fully managed SaaS interoperability solution to funnel logs from connected SaaS applications into security analytics systems (see Figure 1).

Figure 1: General architecture of sending audit logs from multiple SaaS applications through AWS AppFabric.

In the next section, we will look at how Amazon S3 allows you to manage the storage of these normalized SaaS audit logs efficiently to meet regulatory compliance requirements.

Cost-optimized long-term audit log storage with Amazon S3

Retaining growing volumes of audit logs over years, as mandated by regulations, can get expensive. This is where leveraging Amazon S3’s storage management capabilities in conjunction with AppFabric’s log aggregation allows organizations to optimize costs. Some organizations use AppFabric to ingest large volumes of normalized audit logs from multiple SaaS applications into a centralized and durable datastore in Amazon S3. This enables security analytics for threat detection across applications.

With 11 9’s durability, Amazon S3 offers a reliable long-term data store to meet stringent regulatory retention policies cost-efficiently. Amazon S3 allows you to store any amount of data and pay only for what you use, eliminating the need to over-provision storage. As audit log data grows from terabytes to petabytes, cost optimization becomes vital. Amazon S3 provides a range of techniques for optimizing storage costs, including storage classes tailored for different access patterns. Analyzing audit log access patterns allows organizations to align data with the most cost-effective storage class through lifecycle policies. For instance, rarely accessed audit logs older than one year can benefit from Amazon S3 Glacier Flexible Retrieval, providing up to 68% savings over Standard-Infrequent Access. For more information about S3 storage classes see, Amazon S3 Storage Classes.

Additionally, Amazon S3 lifecycle policies enable seamless transitioning between storage classes based on predefined schedules. For example, recent audit logs from AppFabric can reside in Amazon S3 Standard for real-time access, while logs older than 30 days with lower access can move to Standard-Infrequent Access, and logs older than 90 days with rare access can move to Amazon S3 Glacier Deep Archive storage tier which delivers archive storage at less than one-tenth of a cent per gigabyte. Monitoring tools like Amazon S3 Storage Class Analysis and Cost Explorer offer visibility into usage patterns, aiding in informed decision-making. Insights gained from usage monitoring allow organizations to right-size storage class and lifecycle policy choices over time. Amazon S3 Intelligent-Tiering further automates the optimization process, moving audit logs to a more cost-effective storage tier based on access frequency without introducing operational complexities.

Combining these capabilities allows you to build an Amazon S3-based audit log repository that scales securely and cost-efficiently over years.

Conclusion

As organizations adopt more SaaS applications, managing and retaining growing audit logs from these systems efficiently becomes critical but challenging for security teams. The fragmented approach of handling logs individually for each application results in security vulnerabilities, excessive costs, and compliance risks.

In this blog post, we explored how you can leverage AppFabric and Amazon S3 to develop a unified audit log architecture for multi-SaaS environments. AppFabric simplifies the aggregation of disparate application logs and normalizes them into standard formats, facilitating downstream analytics. Amazon S3 offers a durable, secure, and cost-optimized object store, ensuring effective long-term retention of audit logs. Together, these services address the complexities associated with cross-application log management.

To learn more about AppFabric see, Cross-application audit log analysis with AWS AppFabric and How AWS AppFabric and Splunk work together to improve your security observability of SaaS applications. To get started visit AWS AppFabric console.