Build a self-healing ETL pipeline with Amazon Bedrock AgentCore

Extract, transform, and load (ETL) pipelines break silently. Schema drift, missing fields, and type mismatches corrupt downstream analytics. The typical fix is manual: detect the error, triage it, patch the record, update the code, redeploy. As data volumes grow, this detect-triage-fix-deploy cycle doesn’t scale. A self-healing pipeline built on Amazon Bedrock AgentCore, a platform to build, connect, and optimize agents at scale with any framework or model, can repair itself.

The self-healing pipeline in this post fixes its own data, persists the corrections, and can propose pull requests to help prevent the same errors from recurring (optional, requires repository credentials).

In this post, you will build a self-healing ETL pipeline that uses AWS Glue and Amazon Bedrock AgentCore. The pipeline automatically corrects bad records using domain knowledge and writes them back to the data lake. When you configure the optional repository integration, it also generates code reviews with recommended fixes. The result is a closed loop on data quality that reduces the need for human intervention. You’ll cover the architecture, deploy the infrastructure with the AWS Cloud Development Kit (AWS CDK), and test the data correction workflow with sample data, including a before-and-after record comparison.

Amazon Bedrock Knowledge Bases with pre-computed correction strategies stored in Amazon S3 Vectors, a capability of Amazon Simple Storage Service (Amazon S3), reduce the time to create and deploy corrective measures. A manual workflow involves an engineer diagnosing the error pattern, writing a fix, and deploying it through a release cycle. This system applies known corrections within seconds of detection. It also surfaces new patterns to the team with full context, including a proposed code fix. When records fail validation, automatic correction keeps them flowing to consumers instead of sitting in a dead-letter queue awaiting manual triage. Confidence threshold and human-notification fallback provide built-in oversight governing this autonomous agentic behavior. For additional responsible AI controls, consider Amazon Bedrock Guardrails, grounding, and content filtering.

The problem: Data quality at scale

Consider a hypothetical weather data system ingesting National Oceanic and Atmospheric Administration (NOAA) records across 15 schema types: temperature, precipitation, wind, pressure, humidity, visibility, cloud cover, solar radiation, and more. In a system like this, sensor failures, schema drift, and producer bugs can cause a meaningful percentage of records to fail validation.

Today, that means:

• Engineers get paged about bad data hours after it arrives.
• Someone manually inspects the error, determines the fix, and patches the record.
• A developer traces the root cause back to upstream code and submits a fix.
• The pipeline is redeployed, until the next variant of the same error appears.

Each step requires a person, a tool, and a context switch. A self-healing pipeline captures that correction knowledge and applies it automatically the next time a similar error appears.

Prerequisites

To implement this pattern, you need:

• An AWS account with permissions to create resources in the following services:
  • Amazon S3 (including Amazon S3 Vectors), AWS Glue, Amazon Bedrock.
  • Amazon Elastic Container Registry (Amazon ECR), AWS CodeBuild, AWS Lambda.
  • Amazon Simple Notification Service (Amazon SNS), AWS Secrets Manager, AWS Identity and Access Management (IAM).
• Amazon Bedrock model access enabled for Anthropic Claude (agent reasoning) and Amazon Titan Embeddings V2 (knowledge base indexing) in your target AWS Region. For model availability by Region, refer to supported models by AWS Region in Amazon Bedrock (https://docs.aws.amazon.com/bedrock/latest/userguide/models-regions.html).
• AWS CDK CLI installed (npm install -g aws-cdk@latest) with Python 3.10+.
• Finch or Docker for building agent container images (CDK builds ARM64 images through CodeBuild, but local testing requires a container runtime).
• Familiarity with Amazon S3 data lake patterns and basic understanding of multi-agent orchestration concepts.

The sample implementation uses the Strands Agents SDK for agent development and AWS CDK (Python) for infrastructure. Deploy all resources within a single AWS Region.

Solution overview

This solution replaces that manual cycle with a multi-agent system that automates three things: correct the data, persist the fix, and remediate the code. The following diagram shows the complete architecture:

Figure 1: End-to-end architecture of the self-healing ETL pipeline

Here’s how data flows through the pipeline:

1. Ingest – Data producers write weather records directly to an Amazon S3 raw landing bucket, organized by schema type under the raw/ prefix. Records are preserved unmodified.
2. Transform – An AWS Glue ETL job validates each record against its declared weather schema and writes valid records to the transformed zone. Records that fail validation are classified by error type (missing field, type mismatch, schema violation, invalid format) and routed to the AgentCore system.
3. AI correction – A four-agent system on Bedrock AgentCore handles the error:
   • Orchestrator agent coordinates the sequential workflow: correction to persistence to remediation.
   • Data correction agent queries domain knowledge to find and apply a high-confidence fix.
   • Data persistence agent writes the corrected record back to the data lake.
   • Code remediation agent identifies the root cause and proposes a pull request to help prevent recurrence.
4. Notify – When no high-confidence correction is found, the orchestrator publishes a notification through Amazon Simple Notification Service (Amazon SNS) for manual intervention rather than applying a bad fix.

How validation and error routing work

The AWS Glue ETL job acts as the quality gate. It validates each record against a dispatcher that maps each of the 15 schema types to a dedicated validator. Each validator checks required fields, data types, enum values, and numeric constraints:

def validate_record(record: dict) -> dict:
    """Validate a single weather record against its declared schema_type."""
    schema_type = record.get("schema_type")
    if schema_type not in VALID_SCHEMA_TYPES:
        return {"valid": False, "error_type": "invalid_format",
                "error_details": f"Unknown schema_type: '{schema_type}'"}

    data = record.get("data")
    validator = SCHEMA_VALIDATORS.get(schema_type)
    error_details = validator(data)

    if error_details:
        # Validators emit only three message shapes, so once missing-field and
        # type-mismatch are ruled out, the else is always a constraint/enum/range failure
        if "Missing required field" in error_details:
            error_type = "missing_field"
        elif "Type mismatch" in error_details:
            error_type = "type_mismatch"
        else:
            error_type = "schema_violation"
        return {"valid": False, "error_type": error_type,
                "error_details": error_details}

    return {"valid": True, "error_type": None, "error_details": None}

When a record fails, the job packages it as an error record, preserving the original payload, the classified error type, and the validation details. It then routes the error to the AgentCore orchestrator through the InvokeAgentRuntime API. This is the handoff point where the pipeline stops being a passive data mover and becomes an active self-healing system.

Multi-agent orchestration with Bedrock AgentCore

The orchestrator agent coordinates the entire correction workflow. It receives an error record from the AWS Glue ETL job and drives a sequential path through the specialized agents, degrading gracefully to a human notification when it can’t correct with confidence of at least 70 percent. If a step fails, it stops rather than propagating bad data downstream. The following diagram shows that flow and how each agent connects to its knowledge sources and outputs:

Figure 2: Multi-agent orchestration workflow on Amazon Bedrock AgentCore

The diagram shows the orchestrator receiving an error record from AWS Glue and invoking the correction agent, which draws on the Amazon Bedrock knowledge base and the S3 correction files. The orchestrator then checks the correction’s confidence against the threshold: if it clears, the orchestrator routes the record to the persistence agent (which writes to both data lake zones) and the remediation agent (which opens a repository pull request). If it falls short, the orchestrator sends the record to Amazon SNS for manual review.

Each agent is a containerized application built with the Strands Agents SDK and deployed as an AgentCore runtime, a capability of Amazon Bedrock AgentCore.

The orchestrator doesn’t contain correction logic itself. It delegates to specialized agents and makes routing decisions based on their responses. This separation means you can update correction strategies without touching orchestration logic. You can also swap in a different persistence backend without affecting the correction agent.

Two-source correction: Breadth meets precision

The correction agent’s job is to find the right fix with high confidence. It uses two complementary knowledge sources:

Amazon Bedrock Knowledge Bases. The agent sends a natural language query describing the error to a knowledge base backed by Amazon S3 Vectors and Amazon Titan Embeddings V2. Amazon Bedrock Knowledge Bases provides fully managed Retrieval Augmented Generation (RAG) capabilities. In this case the knowledge base provides semantic search over indexed domain documents (NOAA data guides, schema definitions, error pattern catalogs) and returns ranked results with relevance scores.

def _query_knowledge_base(error_record: dict) -> dict:
    """Query Amazon Bedrock KnowledgeBase for correction strategies."""
    query_text = (
        f"How to correct a {error_record.get('error_type')} error "
        f"in a {error_record.get('schema_type')} weather record? "
        f"Error details: {error_record.get('error_details')}"
    )
    response = bedrock_agent_runtime.retrieve(
        knowledgeBaseId=KNOWLEDGE_BASE_ID,
        retrievalQuery={"text": query_text},
        retrievalConfiguration={"vectorSearchConfiguration": {"numberOfResults": 3}},
    )

    results = response.get("retrievalResults", [])
    top_result = results[0]
    score = top_result.get("score", 0.0)
    content = top_result.get("content", {}).get("text", "")
    source = top_result.get("location", {}).get("s3Location", {}).get("uri", "unknown")
    return {"strategy": content, "score": score}

Amazon S3 Vectors. In parallel, the agent does a direct lookup for a pre-computed correction strategy at correction/{schema_type}/{error_type}.json. These files contain specific fix instructions with confidence scores for known error combinations:

{
  "correction_strategy": "For missing temperature fields, check if the complementary unit field is present (celsius/fahrenheit) and compute the missing value using the conversion formula.",
  "confidence": 0.85
}

The agent compares both results and picks whichever has the higher confidence score. If the selected strategy meets the threshold (default 0.7), the correction is applied. If not, the orchestrator sends an SNS notification for manual intervention.

This design gives you:

• Breadth – The knowledge base can reason about novel error patterns using domain knowledge from indexed documents, even for combinations it hasn’t seen before.
• Precision – S3 Vectors provide instant, high-confidence fixes for known combinations without requiring Amazon Bedrock model inference.
• Graceful degradation – If neither source returns a confident answer, the system notifies humans rather than applying a bad fix.

The correction agent decides which fix to apply by weighing two knowledge sources against a confidence threshold. The following flowchart shows that decision logic, with example confidence scores:

Figure 3: Two-source correction decision flow with a confidence threshold

The flowchart shows the correction agent querying the Amazon Bedrock knowledge base and the S3 correction files in parallel, then selecting the higher-confidence strategy (for example, an S3 score of 0.85 over a knowledge base score of 0.78). If the selected score meets the 0.7 threshold, the agent applies the correction and returns the corrected record. If it falls short, the agent returns a failure. This triggers an Amazon SNS notification for manual review.

Closing the loop: Automated code remediation

Most data quality systems stop at fixing the record. This pipeline goes further: the remediation agent traces the error back to its source and proposes a code fix.

After a successful correction, the orchestrator invokes the code remediation agent, which:

1. Reads the source repository – Connects to the repository using credentials stored in AWS Secrets Manager.
2. Analyzes root cause – Correlates the error type, schema type, and correction applied to identify which upstream code produced the bad data.
3. Generates a code change – Produces recommended changes that preserve existing logic (it does not recommend disabling validation or removing functionality).
4. Submits a pull request – Uses GitHub MCP Server to create a branch, commits the changes, and opens a pull request with the root cause analysis in the description.

def _create_mcp_client(auth_token: str):
    return MCPClient(
        lambda: streamable_http_client(
            GIT_MCP_URL,
            http_client=create_mcp_http_client(headers=headers),
        ),
        startup_timeout=GIT_MCP_TIMEOUT,
    )

This means recurring errors get fixed at the source rather than only patched in the data lake. Over time, the pipeline’s error rate decreases as upstream code improves.

The data lake: Three zones, one truth

The S3 data lake uses prefix-based organization across three zones. Downstream consumers see complete data for processed records, whether a record was valid on first pass or corrected by the agents:

The persistence agent writes corrected records to both zones. The corrected zone provides auditability, capturing what was wrong and how it was fixed. The transformed zone gives downstream consumers a complete dataset without needing to union multiple paths.

Deploy the pipeline

With the architecture understood, deploy the pipeline. The complete sample implementation is available in the GitHub repository. The entire system is defined in AWS CDK (Python). A single cdk deploy command provisions all infrastructure, builds container images through CodeBuild, creates the knowledge base, and sets up the AgentCore runtime resources.

Step 1: Clone the repository and install dependencies

git clone https://github.com/aws-samples/sample-self-healing-etl-pipeline.git
cd sample-self-healing-etl-pipeline

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

Step 2: Bootstrap CDK and deploy infrastructure

Deploy all stacks with a single command. CDK handles dependency ordering, container image builds, knowledge base creation, and runtime provisioning automatically:

# Bootstrap CDK (first time only)
cdk bootstrap aws://$(aws sts get-caller-identity --query Account --output text)/us-east-1

# Deploy all stacks
cdk deploy --all \
  --require-approval broadening \
  -c aws_region=us-east-1 \
  -c aws_account=$(aws sts get-caller-identity --query Account --output text) \
  -c notification_email=your-team@example.com \
  -c git_repo_url=https://github.com/aws-samples/sample-self-healing-etl-pipeline

CDK deploys the following stacks in dependency order:

To pass repository credentials for the optional remediation step:

./scripts/set_git_token.sh --token <your_github_token>

The stacks pass resources to each other through constructor arguments in app.py. For example, the AgentCore runtime stack receives the IAM roles and ECR repository URIs from the AgentCore stack. It also receives the knowledge base ID from the knowledge base stack:

# app.py: cross-stack wiring
agentcore_stack = AgentCoreStack(
    app,
    "WeatherAgentCoreStack",
    notification_email=config["notification_email"],
    git_repo_url=config["git_repo_url"],
    knowledge_base_id=knowledge_base_stack.knowledge_base.attr_knowledge_base_id,
    correction_vectors_bucket=knowledge_base_stack.correction_vectors_bucket.bucket_name,
    persistence_vectors_bucket=knowledge_base_stack.persistence_vectors_bucket.bucket_name,
    corrected_bucket=data_lake_stack.output_bucket.bucket_name,
    transformed_bucket=data_lake_stack.output_bucket.bucket_name,
    log_retention_days=config["log_retention_days"],
    env=env,
)

# Runtime stack deploys last, after CodeBuild pushes the agent images to ECR
runtime_stack = AgentCoreRuntimeStack(
    app,
    "WeatherAgentCoreRuntimeStack",
    agent_roles=agentcore_stack.agent_roles,
    ecr_repo_uris={name: repo.repository_uri for name, repo in agentcore_stack.ecr_repos.items()},
    notification_topic_arn=agentcore_stack.notification_topic.topic_arn,
    git_secret_arn=agentcore_stack.git_secret.ref,
    knowledge_base_id=knowledge_base_stack.knowledge_base.attr_knowledge_base_id,
    correction_vectors_bucket=knowledge_base_stack.correction_vectors_bucket.bucket_name,
    persistence_vectors_bucket=knowledge_base_stack.persistence_vectors_bucket.bucket_name,
    corrected_bucket=data_lake_stack.output_bucket.bucket_name,
    transformed_bucket=data_lake_stack.output_bucket.bucket_name,
    env=env,
)

One wiring detail deserves mention: the AWS Glue stack deploys before the runtime stack exists. This means the orchestrator ARN can’t be passed as a CDK reference without creating a circular dependency. Instead, the runtime stack writes the ARN to an AWS Systems Manager (SSM) parameter (/weather-etl/orchestrator-runtime-arn) and the AWS Glue job reads it at job runtime.

Step 3: Verify knowledge sources

The knowledge sources populate automatically during cdk deploy. Domain documents (schema definitions, error patterns, NOAA data guides) upload from the repository’s knowledge_base/ folder to the knowledge base source bucket. A custom resource then triggers the knowledge base ingestion job. Correction strategy files and storage mappings are generated from the same documents and uploaded to the agent vector buckets.

Verify the population completed:

# Resolve the auto-generated vector bucket names from stack outputs (run once)
CORRECTION_BUCKET=$(aws cloudformation describe-stacks \
  --stack-name WeatherKnowledgeBaseStack \
  --query "Stacks[0].Outputs[?ExportName=='WeatherKB-CorrectionVectorsBucketName'].OutputValue" \
  --output text)

PERSISTENCE_BUCKET=$(aws cloudformation describe-stacks \
  --stack-name WeatherKnowledgeBaseStack \
  --query "Stacks[0].Outputs[?ExportName=='WeatherKB-PersistenceVectorsBucketName'].OutputValue" \
  --output text)

# Check correction strategy files (one per schema-type/error-type combination)
aws s3 ls "s3://$CORRECTION_BUCKET/correction/" --recursive | head

# Check storage mapping files
aws s3 ls "s3://$PERSISTENCE_BUCKET/persistence/" --recursive | head

The correction vectors bucket contains one JSON file per schema-type/error-type combination, each with a correction strategy and confidence score. The persistence vectors bucket contains storage mapping files that tell the persistence agent where and how to write each schema type.

Step 4: Test the correction workflow with sample data

Generate sample weather data including intentional errors and upload directly to S3:

# Generate 15 valid records + 4 error records and upload to S3 raw zone
python3 scripts/generate_sample_data.py --upload --region us-east-1

After uploading data, the AWS Glue crawler catalogs the new records on its next scheduled run. You can then trigger the AWS Glue ETL job to validate records, write valid ones to the transformed zone, and route errors to the orchestrator.

# Trigger the Glue ETL job manually
aws glue start-job-run \
  --job-name weather-etl-job-$(aws sts get-caller-identity --query Account --output text)-us-east-1 \
  --region us-east-1

# Resolve the auto-generated data lake bucket name from stack outputs
OUTPUT_BUCKET=$(aws cloudformation describe-stacks \
  --stack-name WeatherDataLakeStack \
  --query "Stacks[0].Outputs[?ExportName=='WeatherDataLake-OutputBucketName'].OutputValue" \
  --output text)

# Check results
aws s3 ls "s3://$OUTPUT_BUCKET/transformed/" --recursive
aws s3 ls "s3://$OUTPUT_BUCKET/corrected/" --recursive

You can watch the correction agent’s decision flow in Amazon CloudWatch Logs (log group /aws/bedrock-agentcore/runtimes/weather_correction_agent-*). This excerpt from a real run shows the two-source lookup and strategy selection for a missing_field error:

[CORRECTION] Step 1: Querying Amazon Bedrock knowledge base for semantic correction guidance
  (domain docs, error patterns, schema definitions)
[KB] Querying Amazon Bedrock knowledge base (ID=XXXXXXXXXX) for correction strategy.
  Query: 'How to correct a missing_field error in a precipitation_measurement
  weather record? Error details: Missing required field: data'
[KB] Received 3 results from knowledge base
[KB] Top result: score=0.784, source=s3://weatherknowledgebasestack-kbsourcebucket29781297-xxxxxxxxxxxx/error_patterns/missing_field.md,
  content_preview='# Error Pattern: Missing Field ## Description Records where one or
  more required fields in the data payload are absent...'
[CORRECTION] Step 2: Querying S3 Vectors for pre-computed correction strategy
  (exact match by schema_type + error_type)
[S3V] Querying S3 Vectors for pre-computed correction strategy.
  Bucket=weatherknowledgebasestack-correctionvectorsbucket27485eb2-xxxxxxxxxxxx, Key=correction/precipitation_measurement/missing_field.json
[S3V] Found correction strategy: score=0.85, strategy='**Field name correction**:
  Check for similar field names that may be typos and rename them.'
[CORRECTION] Step 3: Strategy selection — KB strategy=found (score=0.78),
  S3V pattern=found (score=0.85) → selected confidence=0.85, threshold=0.70

Both sources returned a strategy. The pre-computed S3 file scored higher (0.85 compared to 0.78), cleared the 0.7 threshold, and the correction was applied.

To see the correction itself, compare an error record with its corrected version. This sample record failed validation because temperature_celsius contained a string instead of a number:

{
  "record_id": "fd04d2d9-ab37-4baf-869b-33a52ef9c574",
  "schema_type": "temperature_observation",
  "data": {
    "temperature_celsius": "not-a-number",
    "temperature_fahrenheit": 74.3,
    "measurement_type": "surface",
    "quality_flag": "valid",
    "sensor_id": "TEMP-ERR-001"
  }
}

The corrected record written to the data lake preserves the original fields and annotates the record with the selected correction strategy. This gives downstream consumers and auditors visibility into which fix was chosen:

{
  "record_id": "fd04d2d9-ab37-4baf-869b-33a52ef9c574",
  "schema_type": "temperature_observation",
  "data": {
    "temperature_celsius": "not-a-number",
    "temperature_fahrenheit": 74.3,
    "measurement_type": "surface",
    "quality_flag": "valid",
    "sensor_id": "TEMP-ERR-001"
  },
  "_correction_applied": "**Type coercion**: For invalid numeric formats,
    check if the complementary unit field is present and compute the value
    using the conversion formula."
}

The sample implementation annotates records with the strategy rather than mutating field values, keeping the correction transparent and reversible. To apply transformations directly (for example, deriving temperature_celsius from the valid temperature_fahrenheit value), extend the correction agent’s _apply_correction function with your domain’s transformation logic.

After the agents finish, the corrected records appear in both data lake zones. This listing from a test run shows the four intentionally corrupted records repaired and written to the corrected zone within minutes of the AWS Glue job routing them:

$ aws s3 ls s3://$OUTPUT_BUCKET/corrected/ --recursive

2026-08-04 11:51:12  720  corrected/precipitation_measurement/2026-08-04/0bde2ae3-....json
2026-08-04 11:56:28  758  corrected/pressure_observation/2026-08-04/ce2ea549-....json
2026-08-04 12:01:44  783  corrected/temperature_observation/2026-08-04/24050a62-....json
2026-08-04 12:06:59  716  corrected/wind_observation/2026-08-04/d894ac43-....json

The same corrected records also land in the transformed zone, so downstream consumers see them alongside the records that passed validation on first pass. If a correction had fallen below the confidence threshold, the orchestrator would instead publish an SNS notification to the configured email address with the error details for manual review.

The remediation step requires repository credentials (git_repo_url context values at deploy time, git token set using script). When configured, the remediation agent creates a branch, commits the proposed fix, and opens a pull request with the root cause analysis in the description. Without credentials, the agent logs that remediation was skipped and the workflow completes with data correction and persistence only.

What this pattern automates

The following table compares a standard ETL error-handling workflow with the self-healing approach:

To see how the self-healing automations combine for a single record, follow one error record through the pipeline from end to end. The following diagram traces that path:

Figure 4: End-to-end trace of a single error record through the pipeline

The diagram traces a single error record through the pipeline: validation failure in the AWS Glue ETL job, routing to the orchestrator agent, two-source correction with confidence scores from the Amazon Bedrock knowledge base and the S3 correction files, persistence to both the transformed and corrected data lake zones, and a repository pull request that addresses the root cause in the upstream code.

Key design decisions

Four agents enforce separation of concerns – You can update the correction agent with new strategies without touching persistence logic. In environments without repository access, skip configuring the remediation agent’s credentials. Failure modes such as low-confidence notifications live in the orchestrator, independent of the correction logic.

The confidence threshold helps prevent incorrect corrections – Applying a wrong correction is worse than no correction. With the 0.7 threshold, the system acts when it’s confident, and notifies humans otherwise. You can tune this per-schema or per-error-type based on your tolerance.

The two knowledge sources serve distinct, complementary roles – The knowledge base excels at novel patterns. It can reason about error combinations it hasn’t seen before by drawing on domain documentation. Amazon S3 Vectors excel at known patterns, returning instant, deterministic results without model inference latency. Together, they cover both the long tail and the common case.

Clean up resources

To avoid ongoing charges, tear down the deployed resources when you’re done:

source .venv/bin/activate
cdk destroy --all

CDK destroys all stacks in reverse dependency order. The cleanup handles:

• Deleting all CDK stacks and their resources.
• Emptying S3 buckets (by using autoDeleteObjects).
• Removing ECR container images (by using emptyOnDelete).
• Deleting the S3 Vector bucket and index.
• Force-deleting the Secrets Manager secret for repo credentials.

If you prefer to clean up manually, delete the stacks in this order:

1. WeatherAgentCoreRuntimeStack.
2. WeatherAgentCoreStack.
3. WeatherKnowledgeBaseStack.
4. WeatherGlueStack.
5. WeatherIngestionStack.
6. WeatherDataLakeStack.

CDK-created S3 buckets have autoDeleteObjects enabled and empty automatically during stack deletion. The cleanup does not delete the CDK bootstrap stack (CDKToolkit). Remove it separately if no longer needed.

Conclusion

A multi-agent system turns a standard ETL pipeline into a self-healing one. The system corrects and persists errors automatically, and the system can propose pull requests to fix the source code. The key components:

• AWS Glue acts as the quality gate, validating data and routing failures to the agent system instead of a dead-letter queue.
• Amazon Bedrock AgentCore with the Strands Agents SDK provides managed multi-agent orchestration without building custom workflow infrastructure.
• Amazon Bedrock Knowledge Bases + S3 Vectors give agents domain expertise through both semantic reasoning and deterministic lookups.
• The remediation agent is designed to close the loop when repo credentials are configured. It proposes fixes to the code that produced the bad data, not only the data itself.

To get started, clone the sample repository and follow the deployment steps in this post. To learn more about the services used, see the Amazon Bedrock AgentCore documentation, the Strands Agents SDK documentation, and the AWS Glue documentation. Adapt the pattern to your own data formats by replacing the weather schema validators with your domain’s validation logic. Update the knowledge base documents with your domain expertise, and populate S3 Vectors with correction strategies for your known error patterns. You can extend the pipeline by adding fine-tuned models for domain-specific corrections or implementing approval policies to automatically merge low-risk remediations. You can also connect additional knowledge sources as your error catalog grows.

About the authors