Overview
The evolution process in evolveRL is inspired by genetic algorithms and natural selection. It involves:
- Creating a population of agent variants
- Testing them against adversarial challenges
- Evaluating their performance
- Selecting the best performers for the next generation
Evolution Configuration
The evolution process is configured using the EvolutionConfig class:
from evolverl.evolution import Evolution, EvolutionConfig
from evolverl.llm import LLMConfig
# Configure evolution
config = EvolutionConfig(
population_size=5,
generations=10,
mutation_rate=0.1,
crossover_rate=0.8,
min_score_threshold=0.7,
tournament_size=2,
max_interaction_attempts=5,
output_dir="agents",
llm_config=LLMConfig()
)
# Create evolution instance
evolution = Evolution(config, experiment_id="math_solver")
Configuration Options
Number of agent variants per generation
Number of evolution cycles to run
Probability of mutation during reproduction (0.0 - 1.0)
Probability of crossover during reproduction (0.0 - 1.0)
Minimum score required for survival (0.0 - 1.0)
Number of agents in each tournament round
Maximum number of back-and-forth interactions in testing
Evolution Process
1. Initialization
# Create base agent
base_agent = evolution.build_agent()
base_prompt = await evolution.make_base_agent_prompt_template(
domain="mathematics",
description="Solve complex math problems"
)
base_agent.set_default_prompt(base_prompt)
2. Variant Generation
# Create population of variants
variants = await evolution.mutate_agents(base_agent, count=5)
3. Testing
# Create adversary and judge
adversary = await evolution.build_adversary(domain, description)
judge = await evolution.build_judge(domain, description)
# Test variants
for variant in variants:
# Run interaction
history = await evolution.run_interaction(variant, adversary)
# Score performance
score = await evolution.get_agent_score(history, judge)
variant.update_score(score)
4. Selection
# Find best variant
best_variant_idx = scores.index(max(scores))
best_variant = variants[best_variant_idx]
# Save generation results
evolution.save_generation_results(
generation=0,
variants=variants,
best_idx=best_variant_idx,
scores=scores
)
Output Structure
The evolution process generates several files:
agents/
├── {experiment_id}_gen0.json # Best agent from generation 0
├── {experiment_id}_gen0_full.json # All variants from generation 0
├── {experiment_id}_gen1.json # Best agent from generation 1
├── {experiment_id}_gen1_full.json # All variants from generation 1
└── {experiment_id}_best.json # Best agent overall
Best Practices
- Population Size: Start with a small population (3-5) and increase based on results
- Generations: Run enough generations to see meaningful improvement (10+)
- Mutation Rate: Keep between 0.1-0.3 to balance exploration and stability
- Testing: Use diverse adversarial scenarios to ensure robust evolution
- Monitoring: Track scores across generations to verify improvement
