Exploiting Failure in Evolution
Algorithms typically try to minimize errors and failures. However, this research seeks to challenge this paradigm by investigating the potential advantages of failure. We explore failure-preserving evolutionary algorithms in the context of the Lunar Lander game environment, where the purple lander is the agent. Variants of FI-2Pop in the Lunar Lander environment are tested whether preserving "failed" individuals improves evolutionary search, revealing reward-hacking behaviors.
Summary
multiprocessing, batched rollouts (≈6.7× output speedup).
Methods
We ran these algorithms with 100,000 agents and selected the highest fitness agents:
- Random Search (RS): (baseline 1) Search a randomly-initialized group of agents.
- Evolution Strategy (ES): (baseline 2) Over successive generations, regenerate the entire population by mutating the previous generation’s fittest agents.
- MAP-Elites: Maintain “elites”, which represent the fittest agents of a feature niche in a 2D map. + mortality: old agents are removed from the map
- FI-2Pop: Modify ES by maintaining two populations (feasible/infeasible) based on some fitness-based threshold.
-
- MAP-Elites with FI-2Pop: Modify MAP-Elites by maintaining two feature maps (feasible/infeasible).
-
Findings
- Best agents routinely exceeded fitness 200; a characteristic “tap-dancing” reward hack maximized score without landing between flags.
- Top-performing agent achieved a fitness score of 237.278 with endpoint features
(0.591, −0.215). - Aggregated heatmaps of elites across 20 trials revealed distinct coverage patterns for each evolutionary method (see poster figure set).
Key Contributions
- Added mortality to MAP-Elites; implemented heap-based parent selection for ES; built reproducible visualization pipelines.
- Parallelized evaluations with Python
multiprocessing, achieving approximately 6.7× faster trial throughput. - Validated RS/ES/MAP-Elites baselines against prior results to ensure correctness.
Poster
Presented a research poster at the Lida Orzeck '68 Poster Session at Barnard College on July 31, 2024 and at the Barnard Computer Science Senior Research Symposium on April 25, 2025.