# IdeaMaze Starter Kit

Get started with autonomous ML research in 3 steps.

## Quick Start

```bash
# 1. Copy files to your project
cp program-template.md your-project/program.md
cp maze.py your-project/

# 2. Edit program.md to describe your task
# - Set your training script name
# - Define your evaluation metric
# - List data constraints

# 3. Start an experiment run
cd your-project
git checkout -b autoresearch/run1
echo "commit\tavg_metric\tstatus\tdescription" > results.tsv
python maze.py sync
```

Then give your AI agent (Claude, GPT-4, etc.) the `program.md` and let it run the experiment loop.

## What's Included

| File | Purpose |
|------|---------|
| `program-template.md` | Generic program.md template. Customize for your task |
| `maze.py` | Experiment knowledge base manager (SQLite-backed) |

## How It Works

1. **program.md** defines the rules: what file to modify, what metric to optimize, what constraints to follow
2. The AI agent reads program.md and runs experiments in a loop: modify code -> commit -> run -> keep/discard
3. **maze.py** tracks all experiments in a SQLite database, auto-classifies them into categories, detects stagnation, and suggests what to try next
4. Results are logged to `results.tsv` and synced to `maze.db`

## maze.py Commands

```bash
python maze.py sync              # Sync results.tsv -> maze.db
python maze.py status            # Best metric, stagnation, categories
python maze.py next              # Suggest what to try next
python maze.py insight "text"    # Record a pattern learned
python maze.py strategy "text"   # Set research direction
python maze.py history           # Improvement timeline
python maze.py insights          # List all insights
```

## Customizing program.md

Open `program-template.md` and replace the placeholders:

- `{{PROJECT_DESCRIPTION}}` - What you're optimizing
- `{{TRAIN_SCRIPT}}` - The file the agent modifies (e.g., `train.py`)
- `{{RUN_COMMAND}}` - How to execute training
- `{{METRIC_NAME}}` - Primary metric (e.g., `val_loss`, `accuracy`)
- `{{METRIC_DIRECTION}}` - "lower" or "higher" is better
- `{{EXTRACT_COMMAND}}` - How to extract the metric from logs
- `{{CONSTRAINTS}}` - Files/data the agent must not modify

Or use the [online generator](../starter-kit.html) to create a customized program.md interactively.

## Requirements

- Python 3.8+ (maze.py uses only standard library: sqlite3, argparse, os, re)
- Git (for the experiment loop)
- An LLM agent (Claude Code, GPT-4, etc.)

## Learn More

- [Methodology](../methodology.html) - Deep dive into the four innovations
- [Visualizer](../visualizer.html) - Interactive experiment maze visualization
- [Research Paper](https://ideamaze.in) - Full academic paper on ideamaze.in