Large enterprises across the world are looking to minimize effort, time, and cost for Site Reliability Engineering (SRE) and operations for their huge workloads, so they can channel resources towards innovation and productivity instead. At Daimler Truck, one of the largest commercial automotive manufacturers of the world, the IT Aftersales team worked with AWS Partner CGI to build a novel and effective solution that diagnoses and fixes issues automatically by leveraging the latest AI-offerings from AWS.

This post explains how Amazon Bedrock AgentCore, AWS DevOps Agent, and AWS Lambda power a fully automated solution that detects SRE-relevant errors, performs root cause analysis, and creates prepared pull requests with proposed fixes. The system includes a human-in-the-loop step for review and approval before deployment. By removing the time-consuming debugging and code-fix generation phase, this approach reduces incident resolution time by approximately 88%, cutting what previously took 2 to 3 hours down to just minutes. This translates into a potential 10 to 20% reduction in support-level effort, freeing engineering teams from reactive firefighting to focus on higher-value work, while auto-generated pull requests with full incident timelines improve documentation and audit readiness as a natural byproduct.

The challenge

The Truck After Sales Portal (TASP!) is the central aftersales portal for Daimler Truck, catering to the Mercedes-Benz Truck markets. It offers a Platform-As-A-Service built on AWS, featuring over 36 applications run on it in their own sub-accounts and namespaces across 7 Amazon Elastic Kubernetes Service (Amazon EKS) clusters . More than 130,000 users around the world access TASP! for various phases of after-sales operations, and more than 80 developers work on various modules of TASP! everyday.

With this scale of operations, there is a risk of deploying incorrect configurations or setups that cause disruptions. For example, the allocated memory for a service could be predicted by the developer to be a certain value, however, when it is deployed, it consumes more and therefore results in an Out of Memory error.

When an error is identified, monitoring and alerting mechanisms trigger an alert, and a SRE developer investigates it, ascertains the root cause, makes a minor code-fix to fix the error, and deploys the fix. The process is time-consuming, taking developers on average 2-3 hours per incident.

Solution overview

The team wanted an automated solution that conducts root cause analysis (RCA) of the error and delivers the code changes, reducing the amount of human effort throughout the process.

The team used AWS DevOps Agent to build an end-to-end, automated multi-agent, self-healing SRE solution that detects, analyzes, and prepares fixes for application errors and bugs. The developer reviews all fixes in a review step before deployment. The system also measures key performance indicators including time-to-resolution and cost impact of fixes.

Solution architecture

Figure 1: End-to-End multi-agent self-healing SRE solution

Process flow

The AI-assisted operations workflow automatically connects a live production issue to a proposed, validated fix. When a defective deployment occurs, Amazon CloudWatch monitoring detects failure and sends an alert to the SRE Agent. The agent validates and normalizes the alert, investigates the root cause through the DevOps Agent, and notifies engineers in Microsoft Teams. If remediation is appropriate, the Dev Agent applies the proposed change, runs validation checks, and automatically opens a detailed GitHub pull request. The engineering team is then notified for review and approval, completing a self-healing loop with human oversight.

Example workflow: Incorrect memory limits

Defective change deployed: A developer creates a code change with memory limits that are too low and deploys to the development Kubernetes cluster through the CI/CD pipeline.

Runtime failure detected: During a traffic spike, the pod exceeds its memory allocation. The Linux kernel terminates the container, causing Kubernetes to report an OOMKilled condition and enter a CrashLoopBackOff cycle.

Alert ingested: Instana detects the repeated restarts and sends a structured alert payload via HTTP POST via Amazon API Gateway to the SRE Agent’s /invocations endpoint running on Amazon Bedrock AgentCore Runtime.

Incident validated and normalized: The SRE Agent validates required fields such as alert_id, severity, service, and environment. It then normalizes the alert into an SRE Skill Envelope and invokes the read-only DevOps Agent.

Incident triaged: The AWS DevOps Agent analyzes the telemetry and returns an investigation_summary to the SRE Agent, along with a unified diff proposing a fix to the memory limit.

Figure 2: RCA report and fix recommendation

Remediation scheduled: The SRE Agent feeds the structured investigation_summary output directly into the Dev Agent’s invocation endpoint.

Remediation gated and tested: The Dev Agent confirms that the repository is on its allowed-repositories list and that fix_applicable = true. It authenticates through a GitHub App using an RS256 JWT exchange, applies the proposed patch in a sandbox, and runs checks such as Terraform fmt or pytest.

Pull request generated: If validation succeeds, the Dev Agent creates a working branch, commits the change using its bot identity, pushes it, and opens a detailed GitHub pull request containing the incident timeline, root cause, proposed fix, and test results.

Figure 3: Pull Request generated notification

Engineering review notified: The Dev Agent records the completion event in Amazon CloudWatch and sends a Microsoft Teams notification containing the pull request link, allowing engineers to review and approve the change before merging.

Figure 4: Self-healing remediation flow

Safeguards in action

The team incorporated safeguards in the system to make sure the system can run smoothly at scale. These safeguards include:

1. The AWS Lambda function, that triggers the SRE Agent, has a de-duplication mechanism using Amazon DynamoDB that can ignore same/similar alerts in quick successions and avoid multiple runs for the same issue.
2. The Dev agent will only make changes for allowed repos, therefore not wasting effort on non-relevant repos.
3. The Dev agent validates the proposed fix in a sandbox before making the pull request, adding major confidence level for a fix.
4. We utilize the latest feature from Amazon Bedrock AgentCore leveraging temporal policies for stateful agent authorization and rate limiting for AI traffic as an added layer of safety.
5. We added a cost-triggered circuit breaker using Amazon CloudWatch alarm and AWS Lambda that stops the entire setup if the overall cost goes above a reasonable threshold, therefore making a hard fuse against any unusual usage.

Key Decisions

The team made key decisions on the development of this solution based on the following information:

1. At the time of development, the AWS DevOps agent did not support code fixes, therefore we used a multi-agent architecture and have a dedicated coding and validation agent.
2. Amazon Bedrock AgentCore now offers in-built temporal policies and rate limiting, but this was introduced after the proof-of-concept and prototyping phase of the project. Before it was released, we used a separate limiter via the triggering AWS Lambda function. We’ve since updated the solution to use the Amazon Bedrock AgentCore feature.
3. The solution is only setup to act on rigorously tried and tested error cases, resulting in high success rates, reliably. In the future, we will expand the domain of errors that the solution handles.

Measuring impact and next steps

Prior to this solution, manual investigation and code-fix preparation took approximately 2-3 hours per incident on average to resolve. Using TASP!, the automated workflow and human review and approval time requires 15 minutes, reducing time-to-resolution by approximately 88%.

In the coming weeks we plan to expand the domain of the solution to include more varieties of SRE-related incidents, including:

1. Automating application-level error handling
2. Predictively block erroneous codes
3. Build custom knowledge base including Daimler Truck Coding Standards
4. Run tests inside the sandbox to validate pull requests before developer approval

Conclusion

Daimler Truck’s multi-agent self-healing SRE solution demonstrates how to automate repetitive incident triage and remediation using AWS AI services while preserving human oversight. By combining monitoring, specialized agents, policy controls, sandbox validation, and automated pull-request generation, the TASP! platform reduces operational effort and accelerates recovery from well-understood failures. With safeguards in place and plans to expand into additional incident types, this approach provides a scalable foundation for more efficient and reliable enterprise operations.

Explore the AWS DevOps Agent documentation to get started with your own autonomous pipeline.