AWS Partner Network (APN) Blog

Taking control of agents on Amazon Bedrock AgentCore with Dataiku

By: Ankit Mathur, Partner Solutions Architect – AWS
By: Mike Osborne, Field Engineering Lead, North and Central Europe – Dataiku

Dataiku Logo
Dataiku Connect Button

Agent estates are growing from a handful of pilots to hundreds of agents in production. At that scale the question is no longer whether a team can build an agent on Amazon Bedrock, but which agents to stop and which to promote. Answering it takes control and accountability, not just visibility.

Dataiku built Agent Management for that job. It turns the signals Amazon Bedrock AgentCore emits through Amazon CloudWatch into one inventory, an audit trail, and enforceable controls.

This post looks at how Dataiku Agent Management and Amazon Bedrock AgentCore work together, and why that matters for production agents on AWS.

Operational challenges for scaling production agents

A proof of concept has one agent and one owner. Production has many, spread across teams, and the questions change. Which agents are still live? What did each one actually do on a given request? How is it performing, and when something goes wrong, can you show what happened and who signed off?

Traditional observability answers infrastructure questions: is the service up, is latency within bounds. Those still matter, but they say little about an agent that stays healthy on every dashboard while making a decision you would not have approved. Accountability for agents means seeing the decisions, not just the uptime, and that is the gap Agent Management fills on AWS.

A control layer for Amazon Bedrock agents

Agent Management is a monitoring and control layer, not an agent runtime. It does not host your production agents; it observes them, connecting to Amazon Bedrock through the AWS SDK to build a live picture of your agent estate from the metadata, telemetry, and metrics AWS already produces.

Your agents stay where they run, and when you self-host, your data stays in your account too.

Building on AgentCore’s native observability

Amazon Bedrock AgentCore gives you a strong foundation. AgentCore Observability surfaces session-level metrics such as latency, token usage, and error rates in Amazon CloudWatch, along with spans and traces once you enable instrumentation. AgentCore Evaluations scores agent quality from those traces, on demand or continuously against live traffic, and Amazon Bedrock Guardrails, applied directly to model calls or enforced at the AgentCore Gateway through AgentCore Policy, screens inputs and outputs for prompt injection, harmful content, and sensitive information. For agents running on a single AgentCore runtime, much of what you need for visibility and control is already in place.

Agent Management reads from that foundation rather than replacing it. It consumes the same Amazon CloudWatch telemetry and adds breadth on top: one place to view and control agents across AgentCore, Amazon Bedrock, and other environments. It adds agent cost and return on investment (ROI) oversight, risk certification, and recurring testing and alerts. AgentCore gives you depth for each agent it runs. Agent Management gives you one system of record for agents, wherever they operate.

Solution overview

Agent Management runs on AWS in two shapes.

Figure 1 shows the self-hosted reference architecture, with collected telemetry written to an Amazon S3 bucket in your account.

Figure 1 – Self-hosted reference architecture

Figure 1 – Self-hosted reference architecture – Agent Management on Amazon EC2 in the customer’s Amazon VPC, writing normalized telemetry to Amazon S3.

How the integration works

Agent Management works through the AWS SDK and the APIs that AgentCore and Amazon CloudWatch already expose, using an IAM role you control. It discovers the agents in the account, checks that each one is healthy, and reads the traces and metrics AgentCore writes to CloudWatch to reconstruct what each agent did: the request, the steps, and the outcome. Because it reads common trace formats, a mixed estate lands in one consistent view. Figure 2 shows the resulting inventory.

Figure 2 – Agent Inventory

Figure 2 – Agent Inventory – agents discovered on Amazon Bedrock, showing purpose, activity, and organizational metadata.

For customers with existing Amazon Bedrock Agents Classic workloads, Agent Management provides discovery, metadata, health status, and CloudWatch metrics. Trace-level observability is not available, because Agents Classic does not persist traces. AgentCore, the recommended path for new agents, does, and this post centers on it.

In the self-hosted pattern, heavier trace and session data lands in a connected store in your own account: Amazon S3 by default, for scalable, low-cost storage, or Amazon Relational Database Service (Amazon RDS) for PostgreSQL where a relational store is preferred.

Security by design

A tool that provides oversight of agents reads their traces, so it has to meet a higher bar than the agents it watches. By default, the IAM role you create gives Agent Management read access to Amazon Bedrock, AgentCore, and Amazon CloudWatch. If you turn on uptime tests or emergency actions, you add permissions to invoke, disable, or delete the agents in scope. You can audit the role’s API activity with AWS CloudTrail, and Agent Management uses AWS mechanisms to encrypt data at rest and in transit.

Business value for AWS customers

Illustrative scenario – a retail bank has put dozens of agents into production on Amazon Bedrock across several teams. Beyond a small number of high-profile agents, no one can say with confidence which agents are still live, what each is doing, or who owns the outcome. A single, estate-wide view is what brings that sprawl back under control.

Industry benchmark – in the Global AI Confessions Report: CIO Edition, 2026, a Dataiku and Harris Poll survey of 685 CIOs, 83% lack standardized agent lifecycle management and 72% cannot consistently confirm their agents deliver the business outcomes they were built for.

  • Faster oversight – once agent sessions are reconstructed and searchable in one place, reviews that span the estate take less time.
  • Wider visibility – agents across AgentCore, existing Amazon Bedrock Agents Classic deployments, and other environments can be brought into one inventory.
  • Earlier detection – trace-level visibility and risk certification surface issues earlier.

Figure 3 shows the risk matrix. Teams define risks from their own risk taxonomy and assign them to agents, and the matrix plots them by likelihood, impact, and mitigation status.

Figure 3 – Risk matrix

Figure 3 – Risk matrix – risks across the managed estate by likelihood, impact, and mitigation status.

Key takeaways

  • Start from existing telemetry – read the signals AWS already emits rather than re-instrumenting agents, so new controls add little operational overhead.
  • Keep data in the account – the self-hosted pattern keeps agents and trace data inside the customer’s AWS account.
  • Analyze decisions, not health – review reconstructed sessions and business impact, not just uptime, to see what an agent did and what it delivered.
  • Target AgentCore – AgentCore is the primary integration target and the recommended path for new agents, with existing Amazon Bedrock Agents Classic deployments supported where their architecture allows.

Conclusion

If you are running agents on Amazon Bedrock AgentCore, providing visibility and control does not have to mean building a second system. Agent Management connects to your AWS account, reads what AgentCore already emits, and gives you one inventory and audit trail as your system of record, while you keep building on Amazon Bedrock.

To explore the AWS services in this post, visit Amazon Bedrock and Amazon Bedrock AgentCore. The SaaS edition of Dataiku Agent Management is available in AWS Marketplace. To learn more about the Dataiku approach to agent management, visit Dataiku.

Connect with Dataiku

Dataiku – AWS Partner Spotlight

Dataiku is an AWS Technology Partner in the Advanced tier, with its Software Path at the Differentiated stage. It is ACE eligible and a member of ISV Accelerate and the AWS ISV Workload Migration Program. It holds three AWS Competencies: the Machine Learning ISV Competency, the AI Software Competency, and the Healthcare ISV Competency. Dataiku Agent Management extends the Dataiku platform to the discovery, monitoring, and control of AI agents running on Amazon Bedrock AgentCore and beyond.

Contact Dataiku | Partner Overview | AWS Marketplace

Disclaimer

This post is for educational purposes. The architecture and configuration shown here are illustrative and should not be deployed to production environments without additional security testing. Work with your security and legal teams to meet your organization’s security, regulatory, and compliance requirements before deployment. The content and opinions in this blog are those of the third-party author and AWS is not responsible for the content or accuracy of this blog.