Overview

Agent evolution is the core mechanism in evolveRL that allows agents to improve their behavior over time. The process involves:
  1. Creating base agents
  2. Generating variations
  3. Testing and evaluating
  4. Selecting the best performers

Creating Base Agents

Basic Template

# Generate base prompt template
base_prompt = await evolution.make_base_agent_prompt_template(
    domain="mathematics",
    description="Solve complex math problems with detailed explanations"
)

# Example output:
"""You are an expert AI agent specialized in mathematics.
Your role is to solve complex mathematical problems by:
1. Understanding the {task} thoroughly
2. Breaking down the solution into clear steps
3. Providing detailed explanations
4. Checking your work carefully

When responding:
- Restate the problem to confirm understanding
- Show all steps clearly
- Explain your reasoning
- Verify your solution

{task}: [The specific problem to solve]
{context}: [Additional relevant information]"""

Advanced Template

# Generate specialized prompt template
base_prompt = await evolution.make_base_agent_prompt_template(
    domain="mathematics",
    description="""Create a mathematics tutor that:
    1. Adapts explanations to student level
    2. Uses multiple solution methods
    3. Identifies common misconceptions
    4. Provides helpful visualizations
    5. Suggests practice problems
    
    Focus areas:
    - Algebra
    - Calculus
    - Geometry
    - Probability"""
)

Generating Variations

Mutation Strategies

# Create variants with different focuses
variants = []

# Clarity-focused variant
clarity_prompt = """You are a mathematics educator who excels at clear explanations.
Always:
1. Use simple language
2. Break complex concepts into digestible parts
3. Provide real-world examples
4. Check understanding at each step

{task}: [Problem description]
{context}: [Student level and background]"""

# Rigor-focused variant
rigor_prompt = """You are a precise mathematics instructor who ensures accuracy.
For each problem:
1. State all assumptions
2. Define variables clearly
3. Justify each step mathematically
4. Verify solution validity

{task}: [Problem description]
{context}: [Mathematical context]"""

# Create agents with different prompts
variants.append(Agent(AgentConfig(
    llm_config=llm_config,
    prompt_template=clarity_prompt
)))
variants.append(Agent(AgentConfig(
    llm_config=llm_config,
    prompt_template=rigor_prompt
)))

Testing and Evaluation

Comprehensive Testing

# Test variants across different scenarios
async def test_variant(variant, scenarios):
    scores = []
    for scenario in scenarios:
        # Run interaction
        history = await evolution.run_interaction(
            variant,
            scenario['adversary']
        )
        
        # Get score
        score = await evolution.get_agent_score(
            history,
            scenario['judge']
        )
        scores.append(score)
    
    return sum(scores) / len(scores)

# Define test scenarios
scenarios = [
    {
        'name': 'Basic Algebra',
        'adversary': basic_adversary,
        'judge': basic_judge
    },
    {
        'name': 'Complex Calculus',
        'adversary': calculus_adversary,
        'judge': advanced_judge
    },
    {
        'name': 'Word Problems',
        'adversary': word_problem_adversary,
        'judge': comprehension_judge
    }
]

# Test all variants
for variant in variants:
    score = await test_variant(variant, scenarios)
    variant.update_score(score)

Prompt Optimization

Iterative Improvement

# Analyze successful prompts
def analyze_prompt_patterns(variants, min_score=800):
    successful_prompts = [
        v.config.prompt_template
        for v in variants
        if v.config.score >= min_score
    ]
    
    # Extract common elements
    common_elements = {
        'structure': [],
        'instructions': [],
        'examples': []
    }
    
    # Use these patterns in new variants
    return common_elements

# Create improved variant
async def create_improved_variant(patterns):
    new_prompt = f"""You are an expert mathematics agent that combines:
    {patterns['structure']}
    
    Follow these instructions:
    {patterns['instructions']}
    
    Example approaches:
    {patterns['examples']}
    
    {task}: [Problem description]
    {context}: [Relevant information]"""
    
    return Agent(AgentConfig(
        llm_config=llm_config,
        prompt_template=new_prompt
    ))

Best Practices

  1. Clear Structure: Use consistent prompt formatting
  2. Specific Instructions: Be explicit about expected behavior
  3. Dynamic Elements: Use placeholders for flexibility
  4. Balanced Focus: Cover both process and outcome
  5. Regular Testing: Continuously validate improvements

Common Pitfalls

  1. Over-complexity: Making prompts too long or convoluted
  2. Rigid Structure: Not allowing for adaptation
  3. Missing Context: Failing to provide necessary information
  4. Unclear Expectations: Not specifying desired output format
  5. Poor Evolution: Not learning from successful variants