Agora-Opt Code Package

What This Directory Contains

./code/Agora-Opt/ is the source directory for the Agora-Opt method. It retains two categories of assets:

• the Agora-Opt implementation
• prebuilt memory assets used by the method

Historical run outputs are not stored here.

For compatibility with the original stage naming, the main reproduction script maintains two convenience paths:

• generated_with_memory
• debate_runs

Important Subdirectories

The most important components are:

• src/debate_memory/: core Agora-Opt implementation
• scripts/: command-line wrappers
• memory_storage/: solution memory
• debug_case_memory/: debug memory retrieval bank
• debate_memory_storage/: debate memory retrieval bank
• memory_variants/: retained alternative memory variants
• memory_backups/: retained memory backups

Multiple memory versions are intentionally kept. They were prepared during different stages of the project and can all be treated as available assets for generation, debugging, and debate.

Core Workflow

Agora-Opt runs in two stages.

Stage 1: Generate Initial Solutions

generate_with_memory.py generates candidate solutions, optionally using solution memory and debug memory.

Primary entry script:

• scripts/generate_with_memory.py

This stage:

• reads benchmark problems
• retrieves similar solved cases from memory_storage/
• generates candidate modeling code
• uses debug memory during self-repair when execution fails

Stage 2: Run Debate

run_memory_debate.py takes two sets of initial solutions and runs the decentralized debate stage.

Primary entry script:

• scripts/run_memory_debate.py

This stage:

• loads both sides' initial solutions
• retrieves historical debate cases from debate_memory_storage/
• performs iterative comparison, revision, and convergence
• executes and evaluates the final consensus solution

Memory Types

1. Solution Memory

Directory:

• memory_storage/

Purpose:

• retrieves similar successful modeling cases during generation
• supplies formulation templates and structural priors

Build path:

• extract (problem description, correct code, objective value) from correctly evaluated runs
• build cases.jsonl plus its retrieval index

Related script:

• scripts/build_memory_from_eval_results.py

2. Debug Memory

Directory:

• debug_case_memory/

Purpose:

• retrieves similar execution failures and repair experience
• supports automatic self-debugging during generation

Build path:

• extract unique error signatures from debug_memory.jsonl and its backups
• normalize the error text, repair hints, and metadata into a retrieval bank

Related script:

• scripts/build_debug_memory.py

Note:

• raw debug logs are stored in memory_storage/debug_memory.jsonl
• that log file is one of the inputs used to build debug memory

3. Debate Memory

Directory:

• debate_memory_storage/

Purpose:

• stores examples of how disagreements were resolved during debate
• helps later debates converge more efficiently

Build path:

• select historical runs where the two initial solutions disagreed
• keep cases where debate eventually converged successfully
• extract the dispute, key arguments, and final converged code

Related scripts:

• scripts/build_debate_memory.py
• scripts/process_all_debate_cases.sh

Suggested Memory Construction Order

When preparing memory from scratch, the recommended order is:

1. run generation and evaluation to obtain evaluation_results
2. build solution memory from correct cases
3. build debug memory from accumulated debug_memory.jsonl
4. build debate memory from historical debate runs

The dependency flow is:

• evaluation_results -> solution memory
• debug_memory.jsonl -> debug memory
• debate run artifacts -> debate memory

Retained Memory Assets

This directory intentionally keeps:

• the three primary memory stores
• memory variants
• memory backups

These are treated as static method assets.

Historical run outputs are not retained here, which keeps source code, memory assets, and new results clearly separated.

To rebuild the three memory types, use:

bash ./code/Agora-Opt/scripts/build_memory_assets.sh /path/to/eval_dir1 /path/to/eval_dir2

That script attempts to:

• rebuild solution memory from evaluation directories
• rebuild debug memory from debug_memory.jsonl and its backups
• rebuild debate memory from debate run artifacts

Recommended Entry Points

For paper reproduction, use the outer scripts rather than manually assembling commands in this directory:

• main table: ./code/scripts/run_agora.sh
• 5.1: ./code/experiments/5.1_compatibility_backbone_llms/
• 5.2: ./code/experiments/5.2_ablation_study/
• 5.3.1: ./code/experiments/5.3.1_centralized_judge_selection/
• 5.3.2: ./code/experiments/5.3.2_impact_of_debate_rounds/
• 5.3.3: ./code/experiments/5.3.3_generalization_of_decentralized_debate_protocol/

Direct Source-Level Usage

For direct method-level use, the main wrappers are:

python scripts/generate_with_memory.py
python scripts/run_memory_debate.py
python scripts/execute.py
python scripts/build_memory_from_eval_results.py
python scripts/build_debug_memory.py
python scripts/build_debate_memory.py

Path Conventions

Within the open-source package, the intended layout is:

• benchmark data: ./data/benchmarks/
• Agora-Opt source code and memory: ./code/Agora-Opt/

This separation makes the boundaries between code, memory assets, and newly generated outputs explicit.