AWS Storage Blog
How Mycom optimized its SaaS data fabric with Amazon S3 Access Points and Amazon FSx for NetApp ONTAP
Modern data environments increasingly need to serve the same dataset through more than one access protocol. Analytics engines expect object storage semantics and the Amazon Simple Storage Service (Amazon S3) API, while the applications that produce and consume operational data often expect a shared file system. Maintaining two physical copies of the same data (one on a file system for the applications and one in an object store for analytics) inflates storage costs, adds synchronization complexity, and introduces the risk of the two copies drifting apart.
Mycom is an AWS Specialization Partner that offers assurance, automation, and analytics applications as a cloud service for communications service providers (CSPs). Mycom’s Assurance Cloud Service (ACS) provides network performance, fault, and service quality management. It also supports AI and machine learning (ML)-driven closed-loop assurance for hybrid, physical, and virtualized networks, delivered either as a cloud service or in a self-managed model.
This post discusses how Mycom evolved its SaaS Data Fabric, the shared data layer of ACS that collects network data and makes it available to Mycom’s applications and analytics tools. By attaching Amazon S3 Access Points to an Amazon FSx for NetApp ONTAP volume, Mycom now keeps a single copy of its telco assurance data. Applications read and write that data as files, while analytics services consume the same data through the S3 API without moving data or maintaining duplicate stores.
Data management challenges
The Mycom Data Fabric is the analytics backbone of ACS. It ingests high-cardinality telco assurance data, time-series performance metrics, events, and topology and catalog data modeled to TM Forum Open APIs (such as TMF634, TMF639, and TMF628) and makes it available to Mycom’s AI applications as well as customer data lakes.
Historically, the Data Fabric persisted its TMF-formatted parquet objects in native Amazon S3 buckets. This served the analytics side well: an AWS Glue crawler catalogs the object prefixes, and Amazon Athena queries the resulting tables. However, the operational applications that produce and work with this data are file-oriented microservices running on Amazon Elastic Kubernetes Service (Amazon EKS), backed by NFS storage on Amazon FSx for NetApp ONTAP.
This split created a familiar tension:
- Two stores for one dataset: Operational data lived on the FSx for ONTAP file system, and a copy was maintained in a native S3 bucket for analytics. Every terabyte was, in effect, paid for and managed twice.
- Synchronization overhead: Keeping the object copy current with the file copy required data movement, which consumed bandwidth, added latency between “data written” and “data queryable,” and created operational surface area to monitor.
- Consistency risk: Any pipeline that copies data between two systems can drift, fail partway, or fall behind. This is a particular concern for the near real-time triage use cases that telco assurance depends on.
As telco customers build larger assurance repositories and retain historical data for longer, duplicating the analytics footprint became increasingly hard to justify on cost and sustainability grounds. This motivated an initiative to serve one physical copy of the data through both the file and object interfaces.
Proposed solution
The team focused on a specific capability of Amazon FSx for NetApp ONTAP: Amazon S3 Access Points attached directly to an FSx for ONTAP volume. An S3 access point attached to an ONTAP volume presents the file data through an Amazon S3 endpoint, supporting standard S3 object operations such as GetObject, PutObject, and ListObjectsV2, while the data continues to reside on the ONTAP volume and remains accessible over NFS and SMB at the same time. This can remove the need for a separate object copy.
Some of the key features evaluated include:
- Single copy, dual protocol: The same data on the ONTAP volume is reachable as files (NFS/SMB) by the operational applications and as objects (S3 API) by the analytics services. There is no second bucket to keep in sync.
- No application changes: Analytics services and AWS services that already speak to S3, such as the AWS Glue crawler and Athena, can work against the access point endpoint without modification.
- Layered authorization: Mycom keeps its existing ONTAP file permissions as the single source of truth for who can access the data, while standard IAM and access point policies control which S3 consumers can call in. Because the file system independently enforces its permissions after S3 authorizes a request, Mycom doesn’t have to recreate or keep in sync a separate set of bucket permissions.
- Private-by-default network posture: Access points attached to an FSx for ONTAP volume can be restricted to a specific virtual private cloud (VPC), and Amazon S3 enforces Block Public Access for these access points and it cannot be disabled.
In the ACS implementation, the Data Fabric volume is exposed through an access point. Objects written through the access point are owned by a dedicated ONTAP UNIX identity, which the access point authorizes as. Because the access point is VPC-restricted, any consumer that reads through it must run inside the tenant VPC.
The team wanted to preserve customer choice rather than force a single pattern. Telco customers come in all shapes and sizes, and there is rarely a one-size-fits-all solution. The design supports TMF-formatted data served through an S3 access point as the service endpoint, with the FSx for ONTAP volume as the backing store, as well as alternative data formats depending on customer preferences and data lake requirements.
Implemented solution
The Data Fabric analytics pipeline was adapted so that its consumers can read from either the native bucket or the ONTAP S3 access point, with the switch driven by configuration rather than code changes to the consuming services.
A configuration-driven consumer switch
The AWS Glue crawler is the primary analytics consumer. Rather than hard coding a data source, the deployment process resolves the crawler’s data source at deployment time. When Mycom provisions the S3 access point attached to the ONTAP volume, they point the crawler at it. If not, the crawler falls back to the native bucket. This makes enabling the access point a reversible configuration change.
Because the access point is VPC-restricted, the crawler connects to it through an AWS Glue VPC connection in the same subnet as the Amazon FSx file system. This connection is created only when the access point is in use. A native-bucket crawl does not need it.
Least-privilege access across two ARN namespaces
S3 access points have their own Amazon Resource Names (ARNs), separate from bucket ARNs, so access to them is granted with dedicated IAM policy statements. Mycom grants the AWS Glue crawler and the EKS-based Data Fabric consumers narrowly scoped read permissions on the specific access point, alongside their existing bucket permissions. This way, any authorized Data Fabric consumer can use the access point when it is enabled.
Two-pass provisioning
An S3 access point can only be attached to an ONTAP volume once that volume exists, and the ONTAP UNIX identity the access point authorizes as must be created first. Provisioning is therefore a deliberate two-pass infrastructure as code (IaC) process: the first pass creates the volume and the ONTAP user. The second pass attaches the S3 access point to the new volume.
Migrating existing data onto the volume
For tenants that already had data in the native bucket, the existing objects are migrated onto the ONTAP volume before consumers are cut over. The migration runs from an EC2 instance where the Data Fabric volume is NFS-mounted, and streams objects from the native bucket directly onto the mounted volume with aws s3 sync --delete (a one-way mirror as the source bucket is not modified), then sets file ownership to the ec2-user identity so it matches the access point.
It’s worth noting that the migration deliberately does not copy from S3 to the access point directly. The S3 access point attached to the ONTAP volume does not use a server-side CopyObject from an external bucket, so syncing bucket-to-mount (which streams through GetObject) is both the working and the disk-efficient path. A short write pause on the producers during the final delta sync helps produce a consistent copy, after which the consumers are pointed at the access point.
Availability considerations
As with the file-system storage pattern, the ONTAP volume behind the access point can be deployed as a single-AZ file system while remaining reachable from applications across the tenant, because the access point and NFS mounts are reached from within the VPC. This keeps the EKS topology and deployment template unchanged while offering the storage at a cost- and sustainability-optimized availability tier appropriate to the dataset. The trade-off, as always, is between availability, cost, and sustainability. This is evaluated per customer and per dataset.
Results
Innovating on behalf of customers can be a key differentiator. By serving one copy of telco assurance data through both file and object interfaces, Mycom reduced its analytics storage footprint by 20% alongside its associated cost and carbon, tightened the consistency between operational and analytics views of the data, preserved a consistent SaaS architecture across a diverse telco customer base, and improved its overall total cost of ownership (TCO) for assurance applications on ACS.
What’s next: Extending the pattern across Mycom’s cloud service
Following the success of the Data Fabric implementation, Mycom is actively applying the same single-copy, dual-protocol pattern to other components of its cloud service.
The first is Mycom’s CODAP analytics layer, which refines telco assurance data in stages, from raw to processed to curated, business-ready datasets used by Mycom’s analytics services and AI agents. With S3 access points attached to Amazon FSx for NetApp ONTAP volumes, each stage can be kept as a single copy. CODAP’s processing services write the data as files. Analytics services and AI agents then read the same data through the S3 API. This brings the cost, consistency, and sustainability benefits of the Data Fabric to more of ACS.
Conclusion
Mycom’s operational data already lived on Amazon FSx for NetApp ONTAP. The native S3 bucket existed only as a second copy so that analytics services could use the S3 API. By attaching Amazon S3 Access Points to the existing FSx for ONTAP volumes, Mycom made that same data available through the S3 API directly, allowing it to drop the duplicate S3 copy. The operational microservices continue to read and write files over NFS, while AWS Glue, Amazon Athena, and other S3-native services read the very same data through an S3 endpoint. This requires no application changes and no duplicate storage to synchronize or reconcile.
The design keeps customer choice front and center. Each tenant can run against the native S3 bucket or the ONTAP-backed access point, and the switch is a validated, reversible configuration change rather than a code fork. Mycom grants access using least-privilege IAM across both the bucket and access point ARN namespaces, the access point is VPC-restricted and private by default, and provisioning follows a controlled two-pass flow with a mirror-based data migration.