Enhancing observability for AI workloads using AWS with Mediacorp

For a media organization of Mediacorp’s scale, metadata is not a technical detail. It is the foundation that determines whether content reaches the right audience at the right moment. Editors searching for archival footage, recommendation systems surfacing relevant programming, and advertising systems targeting appropriate audiences all depend on accurate, machine-readable descriptions attached to every piece of content. With a catalog of approximately 5 million to 6 million images, 150,000 hours of video, and 16,000 hours of audio, and with volumes growing every month, manual tagging was no longer feasible. Without automation, the organization faced a growing gap between the content that it produced and the content that its teams and audiences could effectively find.

As Mediacorp’s AI capabilities expanded across its content pipelines, its models grew increasingly sophisticated, progressing from trained baseline models to large language models and multimodal systems. With this growing sophistication, Mediacorp saw an opportunity to further strengthen visibility into model performance, cost-efficiency, and workflow optimization. This included assessing whether outputs were accurate, understanding compute and token consumption across workflows, and identifying which models performed best for different content types. A more integrated view of these dimensions would support teams making faster, more informed decisions on the right balance between performance and cost.

Mediacorp selected SoftwareOne through a competitive procurement process, choosing the AWS Partner for both technical depth and adaptability. The Partner used the AWS Business Value Realization (BVR) framework to define and track customer outcomes. Because the AI landscape shifted during the engagement, with new large language models and multimodal systems becoming available, SoftwareOne continually adjusted the solution architecture to reflect what produced the best results. Together, the team built its observability solution on Amazon CloudWatch. Rather than using Amazon CloudWatch for infrastructure monitoring alone, SoftwareOne extended it to cover AI-specific signals, including model accuracy, token consumption, latency, and cost per workflow. This made the engagement one of the first known uses of Amazon CloudWatch applied to AI workloads.

The solution was structured around four pillars. The first was infrastructure rightsizing, where Amazon CloudWatch metrics tracked CPU and memory usage across the two key services running its AI pipelines: AWS Lambda, a serverless compute service, and Amazon Elastic Container Service (Amazon ECS), a fully managed container orchestration service. Across more than 8,900 AWS Lambda invocations over 30 days, memory use on key functions averaged below 25 percent, with one function using just 8.5 percent of its allocated memory. Rightsizing those resources required zero code changes and produced infrastructure cost reductions of up to 87.5 percent on individual functions.

The second pillar was model selection. Amazon CloudWatch tracked performance, latency, and token cost across models available through Amazon Bedrock, a service for building generative AI applications and agents. SoftwareOne also implemented dynamic model selection by using Amazon CloudWatch to track historical performance, cost, and quality metrics across models simultaneously. With this tracking data, Mediacorp could identify which model produced the best results for a specific task. As a result, 60 percent of entity-linking workloads were routed from a higher-cost model to a lower-cost alternative that performed equally well on those tasks, cutting token costs by 55 percent with no loss in output quality.

The third pillar—continuous accuracy monitoring and A/B testing—was also supported by dynamic model selection. The named entity recognition (NER) workflow’s F1 score, a score that reflects the accuracy and recall of a model, registered at 0.80, prompting the team to build an improved model that achieved a score of 0.886, reflecting a 10.5 percent F1 score gain. “Using Amazon CloudWatch, we can observe which model works better and then make informed decisions based on the data we actually see,” says Abhinav Ramesh Kashyap, machine learning engineer lead at SoftwareOne.

The fourth pillar was issue response, built on AWS X-Ray—which collects data about requests that applications serve—so that developers can analyze and debug production and distributed applications. Engineers received a structured path from alert to root cause, moving through the service topology map, segment timeline, and exception detail. A root cause analysis cycle that previously took up to 30 hours is now completed in minutes—a roughly 99 percent improvement in resolution time.

Using Amazon CloudWatch, the observability layer delivered measurable results across the dimensions that Mediacorp cared about most. Infrastructure costs for the knowledge graph processing pipeline fell 64.5 percent overall, with individual functions achieving reductions as high as 87.5 percent. Token costs on the entity-linking workflow dropped 55 percent through intelligent model routing. AI accuracy on the NER workflow improved from an F1 score of 80.2 to 88.6 percent. System uptime reached 99 percent, up from a 95 percent baseline. The confidence generated by measurable, dashboard-visible results led Mediacorp to expand the monitoring scope mid-engagement from four workflows to nine, reflecting a shift from cautious adoption to active investment.

The value of the solution extended beyond Mediacorp. SoftwareOne built the Amazon CloudWatch observability framework as a set of reusable assets, including dashboards, alerting configurations, A/B testing infrastructure, and evaluation pipelines, designed to be replicated for other customers running AI workloads on AWS. At AWS re:Invent, the engagement drew significant attention from other partners and customers who had not previously considered applying Amazon CloudWatch to AI systems. Monitoring accuracy continuously, comparing models systematically, rightsizing infrastructure without engineering effort, and resolving incidents in minutes rather than hours are now standard operating practice for Mediacorp’s AI workflows.