← cs
$ cat projects/LLM-Agent-Memory.md

LLM Agent Memory in Repeated Social Dilemmas

How much memory do LLM agents really need to sustain cooperation in a repeated public-goods game?

Final project for 6.7960 - Deep Learning

2025-12-09
Python
view on github

LLM Agent Memory in Repeated Social Dilemmas

MIT 6.7960 Deep Learning final project. Research question: how much memory do LLM agents actually need to maintain cooperation in repeated social dilemmas, and which memory representation matters?

What I Built

An environment and agent harness for a repeated public-goods game (3 GPT-4o-mini agents, 10 rounds, starting budget 20.0, contribution multiplier α=1.8) with five interchangeable memory modules:

  • None: no history at all
  • Full History (k=5): last k rounds verbatim
  • Summary: LLM-generated rolling 50-word summary
  • Structured: a numerical trust table
  • Hybrid: trust table plus a free-text strategy note

All metrics (welfare, mean contribution, Gini coefficient, token cost) computed with 95% bootstrap confidence intervals over 10,000 resamples.

Key Finding

Hybrid memory (trust table + strategy note) achieves 2× the welfare of every other condition (240.0 vs. ~120-124) by unlocking perfect cooperation:

| Memory Type | Welfare (95% CI) | Mean Contribution | Tokens/Episode | |---|---|---|---| | None | 120.3 [120.0, 120.7] | 5.01 | ~11,200 | | Full History (k=5) | 120.5 [120.0, 121.4] | 5.02 | ~16,900 | | Summary (50w) | 124.0 [122.1, 126.2] | 5.17 | ~13,600 | | Structured | 123.0 [120.7, 126.0] | 5.13 | ~14,700 | | Hybrid | 240.0 [240.0, 240.0] | 10.00 | ~15,500 |

Results

  • Full history provides no benefit over no memory: identical welfare, despite 50% more tokens
  • The strategy note is the critical component, not the trust table; structured memory alone barely beats the baseline
  • Cooperative priming is necessary: a neutral strategy note (~124 welfare) performs like the no-memory baseline, confirming the effect isn't just "more context"
  • Robust across contribution multipliers tested (α = 1.5, 1.8, 2.1)

Implementation

  • Ablations isolating the strategy-note component from the trust table
  • An alpha sweep for robustness across game-payoff parameters
  • Automated blog-consistency checks (tools/check_blog_consistency.py) verifying the write-up's numbers always match the underlying results JSON

Lessons Learned

  • Memory quantity (full history) doesn't drive cooperation: memory shape does
  • A structured summary that encodes intent ("keep cooperating if they do") does more work than raw trust scores
  • Bootstrapped CIs on a small-N multi-agent setup were necessary to trust the "2×" claim rather than a lucky seed

Future Work

  • Scale beyond 3 agents to test whether hybrid memory holds under larger group sizes
  • Test on adversarial/defecting agent populations, not just cooperative GPT-4o-mini agents
  • Explore whether the strategy note's benefit transfers to other repeated games (prisoner's dilemma, ultimatum game)