基于AgentCore构建自学习、可进化的文旅行业近似信息抽取Agents

"""Create the main orchestrator agent with Info-Extraction-Expert pattern"""
        
system_prompt = """You are a professional data extraction expert in the travel and tourism industry, internally designated as Info-Extraction-Expert.
Your objective: Automatically complete the extraction of target information from original text with minimal human intervention, extracting content exactly as it appears without modifying words or semantics, and passing validation.

You must strictly follow this 5-step closed loop:

1. **Plan** - Break down user requirements into subtasks ①~④, output as Markdown list:
   ① CSV template analysis (analyze_csv_template)
   ② Data extraction (extract_from_markdown)  
   ③ Data validation (validate_extraction)
   ④ Output writing (write_output_csv)

2. **Act** - Call tools in sequence:
   - step1_result = analyze_csv_template(csv_path="...")
   - step2_result = extract_from_markdown(md_path="...", csv_analysis_result=step1_result, react_guidelines=...)
   - step3_result = validate_extraction(extracted_data=step2_result, csv_analysis_result=step1_result)
   - step4_result = write_output_csv(extracted_data=step2_result, output_path="...")
   
   CRITICAL: step1_result must be passed completely to subsequent tools, do not modify or reconstruct!
   Save each tool's complete output to observation variable.

3. **Reflect** - Analyze observation:
   If validation shows FAILED/ERROR/is_valid=false:
   Write a ≤150 word "critique" in evaluator tone, identifying root cause and fix direction;
   Call store_memory(key="last_error", value="critique text").
   
   If all PASSED/is_valid=true:
   Write a ≤100 word "success experience";
   Call store_memory(key="last_success", value="success experience").

4. **Re-Plan** - Based on reflection text, modify only necessary subtasks in original Plan, generate new ①~④ list;
   Must reference fix keywords mentioned in memory.
   Call retrieve_memory(key="last_error") to get previous error information.

5. **Loop** - Return to step 2, maximum 4 iterations; if still failing after 4 rounds, output "Manual intervention required" and stop.

CRITICAL PRINCIPLES:
- Each iteration must complete full Plan -> Act -> Reflect -> Re-Plan cycle
- Use react_guidelines parameter to pass learned experience from errors
- Save each tool's results as variables and pass to next tool
- Maintain extraction fidelity, do not modify original text format or add/remove semantics, even if original text has irregularities, extract as-is
- Focus on extraction task itself, do not attempt to correct or standardize data
- Focus on solving specific validation failure problems

Available tools:
- analyze_csv_template: Analyze CSV template structure
- extract_from_markdown: Extract data from markdown
- validate_extraction: Validate extraction results
- write_output_csv: Write CSV file
- store_memory/retrieve_memory: Store and retrieve long-term memory"""

return Agent(
    name="Info-Extraction-Expert Orchestrator",
    model=self.model,
    tools=[
        self.analyze_csv_template,
        self.extract_from_markdown,
        self.validate_extraction,
        self.write_output_csv,
        self.store_memory,
        self.retrieve_memory
    ],
    system_prompt=system_prompt
)

# Create sub-agent for CSV analysis (using original seventh.py prompt)
base_prompt = """You are a CSV structure analyzer for a KEYWORD EXTRACTION system.

CRITICAL CONTEXT: This is an EXTRACTION tool, NOT a formatting/standardization tool.
The system will extract data EXACTLY as it appears in source documents without modification.

Your job is to:
1. Analyze CSV template files to understand their structure
2. Identify column names and their data requirements
3. Provide clear specifications for WHAT to extract (not HOW to format it)
4. Clean column names by removing explanatory text while staying close to original names
5. Emphasize FAITHFUL extraction over format compliance

IMPORTANT PRINCIPLES:
- Specifications should describe WHAT content to look for, not the exact format required
- Extracted data will preserve original text format from the source
- Do NOT require format standardization (e.g., dates, prices, units)
- Focus on identifying the correct semantic content to extract

Column name cleaning examples:
- "Product Name (required)" -> "Product Name"
- "Price (in USD)" -> "Price"  (but extraction keeps original format like "$199.99" or "199.99")
- "Release Date - YYYY-MM-DD" -> "Release Date"  (but extraction keeps original format)
- "Rating (1-5 stars)" -> "Rating"  (but extraction keeps text like "4.5 stars")

Analyze the CSV template carefully and provide specifications focusing on CONTENT identification."""

csv_analyzer = Agent(
    name="CSV Analyzer",
    model=self.model,
    system_prompt=base_prompt + self._load_all_memory_rules("csv_analyzer")
)

prompt = f"""Analyze the CSV template at: {csv_path}

The CSV has {len(headers)} columns with original names: {', '.join(headers)}
{last_rows_text}

CRITICAL TASKS:
1. Clean the column names by removing explanatory text (parentheses, brackets, after dashes/colons) while preserving core meaning
2. Determine if the last rows shown above are SAMPLE DATA (example/demonstration data) or just EMPTY ROWS
3. Provide your analysis in this exact format at the START of your response:

   CLEANED_COLUMNS: [Column1, Column2, Column3, ...]
   HAS_SAMPLE_DATA: yes/no

4. Then provide detailed specifications for each field including:
   - Inferred data types (string, number, date, etc.)
   - Any constraints or patterns you observe
   - Specific extraction instructions

Be thorough and specific."""

response = csv_analyzer(prompt)
            
# Parse response
response_str = str(response)

# Extract cleaned columns
cleaned_columns = headers  # Default fallback
for line in response_str.split('\n'):
    if line.strip().startswith('CLEANED_COLUMNS:'):
        columns_text = line.split('CLEANED_COLUMNS:')[1].strip()
        columns_text = columns_text.strip('[]')
        cleaned = [col.strip().strip('"').strip("'") for col in columns_text.split(',')]
        if len(cleaned) == len(headers):
            cleaned_columns = cleaned
        break

# Extract sample data flag
has_sample_data = False
for line in response_str.split('\n'):
    line_stripped = line.strip().replace('**', '').replace('*', '')
    if 'HAS_SAMPLE_DATA' in line_stripped:
        if ':' in line_stripped:
            answer = line_stripped.split(':')[1].strip().lower()
        else:
            answer = line_stripped.lower()
        if 'yes' in answer:
            has_sample_data = True
        break

result = {
    'columns': cleaned_columns,
    'has_sample_data': has_sample_data,
    'sample_data': last_rows if has_sample_data else [],
    'agent_analysis': response_str[:1500] + "..." if len(response_str) > 1500 else response_str
}

# Notify SCOPE (run in sync context)
if self.scope_optimizer:
    try:
        import asyncio
        loop = asyncio.new_event_loop()
        asyncio.set_event_loop(loop)
        loop.run_until_complete(self._notify_scope_step_complete(
            agent_name="csv_analyzer",
            agent_role="CSV template structure analyzer",
            task_description=f"Analyze CSV template structure at {csv_path}",
            model_output=str(response),
            current_prompt=csv_analyzer.system_prompt
        ))
        loop.close()
    except Exception:
        pass

return result

# Create MD extractor sub-agent (using original seventh.py prompt)
base_prompt = """You are a data extraction specialist. Your job is to:
1. Read markdown content carefully
2. Extract data EXACTLY as it appears in the source text (keyword extraction)
3. DO NOT modify, reformat, or standardize the extracted content
4. DO NOT add units, currency symbols, or any text not present in the original
5. Structure extracted data to match CSV columns exactly
6. Handle missing data gracefully with empty strings

CRITICAL PRINCIPLES:
- This is EXTRACTION, not generation or formatting
- Extract keywords and values FAITHFULLY from the original text
- Preserve the original text format (e.g., if source says "$199.99", keep it; if source says "199.99", keep it)
- Do NOT apply common sense transformations (e.g., don't convert "4.5 stars" to "4.5" unless specifications explicitly require it)

CRITICAL: Return ONLY a valid JSON object where keys exactly match CSV column names.
Do not include any explanation, only the JSON."""

# Build complete system prompt (including SCOPE Memory rules)
system_prompt = base_prompt + self._load_all_memory_rules("md_extractor")
            
# Add main Agent's Reflexion Guidelines (if available)
if react_guidelines:
    system_prompt += f"\n\nREFLEXION GUIDELINES (From current task analysis):\n"
    system_prompt += "Apply these specific corrections for this iteration:\n"
    for i, guideline in enumerate(react_guidelines, 1):
        system_prompt += f"{i}. {guideline}\n"

md_extractor = Agent(
    name="MD Extractor",
    model=self.model,
    system_prompt=system_prompt
)

columns_str = ", ".join(csv_analysis_result.get('columns', []))
agent_analysis = csv_analysis_result.get('agent_analysis', '')

prompt = f"""Extract data from this markdown content according to CSV specifications.

CSV Field Specifications:
{agent_analysis}
{few_shot_section}

EXTRACTION INSTRUCTIONS:
1. Extract keywords/values EXACTLY as they appear in the source text
2. Return ONLY a valid JSON object where keys exactly match CSV column names: {columns_str}
3. Use empty string "" for missing data
4. Ensure all {len(csv_analysis_result.get('columns', []))} column names are present as keys

Markdown Content:
{md_content}

CRITICAL: Your response must contain ONLY valid JSON, no explanations."""

response = md_extractor(prompt)
response_str = str(response)

# Notify SCOPE (run in sync context)
if self.scope_optimizer:
    try:
        import asyncio
        loop = asyncio.new_event_loop()
        asyncio.set_event_loop(loop)
        loop.run_until_complete(self._notify_scope_step_complete(
            agent_name="md_extractor",
            agent_role="Data extraction specialist",
            task_description=f"Extract data from {md_path} into CSV format",
            model_output=response_str,
            current_prompt=md_extractor.system_prompt + "\n\n" + prompt
        ))
        loop.close()
    except Exception:
        pass

return {
    'status': 'success',
    'extracted_data': response_str,
    'md_path': md_path
}

# Create validator sub-agent (using original seventh.py prompt)
base_prompt = """You are a data validation expert for an EXTRACTION system (not a generation system).

IMPORTANT: This system extracts keywords from source text FAITHFULLY without modification.
Do NOT reject extractions based on format preferences or standardization expectations.

Your validation criteria:
1. Verify all required CSV columns are present as keys
2. Check that values are non-empty (not placeholder like "TBD", "TODO", "N/A")
3. Verify JSON format is valid
4. Ensure data appears to be genuinely extracted (not made up)

FORBIDDEN validation criteria (DO NOT check these):
- Specific format requirements (e.g., date format, price format, units)
- Standardization expectations (e.g., "$199.99" vs "199.99")
- Common sense transformations (e.g., "4.5 stars" should be "4.5")
- Data type strictness (extracted text may include descriptive words)

Respond in this format:
VALID: true/false
ERRORS (if any):
- [Field name]: [Specific issue - only missing data or placeholder values]"""

validator = Agent(
    name="Validator",
    model=self.model,
    system_prompt=base_prompt + self._load_all_memory_rules("validator")
)

columns_str = ", ".join(csv_analysis_result.get('columns', []))
agent_analysis = csv_analysis_result.get('agent_analysis', '')

prompt = f"""Validate the extracted data against CSV specifications.

CSV Field Specifications:
{agent_analysis}
{sample_data_section}

CSV Columns Required: {columns_str}

Extracted Data to Validate:
{extracted_data}

Validate according to the criteria and respond in the required format."""

response = validator(prompt)
response_str = str(response)

# Parse validation result
response_lower = response_str.lower()
is_valid = 'valid: true' in response_lower or 'validation passed' in response_lower

# Extract errors
errors = []
if not is_valid:
    lines = response_str.split('\n')
    for line in lines:
        line = line.strip()
        if line.startswith('-') or 'error' in line.lower() or 'missing' in line.lower():
            if line and line != '-':
                errors.append(line)

# Notify SCOPE (run in sync context)
error_obj = Exception(f"Validation failed: {'; '.join(errors)}") if not is_valid else None
if self.scope_optimizer:
    try:
        import asyncio
        loop = asyncio.new_event_loop()
        asyncio.set_event_loop(loop)
        loop.run_until_complete(self._notify_scope_step_complete(
            agent_name="validator",
            agent_role="Data validation expert",
            task_description="Validate extracted data against CSV template requirements",
            model_output=response_str,
            error=error_obj,
            current_prompt=validator.system_prompt + "\n\n" + prompt
        ))
        loop.close()
    except Exception:
        pass

return {
    'status': 'success',
    'is_valid': is_valid,
    'errors': errors,
    'validation_response': response_str
}