Results logging

Specification for autoresearch-results.tsv — the per-iteration record of every experiment in a run.

File format

Tab-separated values. Headers on row 1. One row per iteration.

iteration\tcommit\tmetric\tdelta\tstatus\tdescription

Field definitions

Field	Type	Description
`iteration`	integer	0-indexed. Never resets — if you run multiple sessions, continue from the last number.
`commit`	string	7-char git short SHA for kept commits. `-` for discards and crashes.
`metric`	float	Raw metric value from the Verify command.
`delta`	float	`metric − previous_best`. Sign convention: positive = improvement (regardless of higher/lower goal).
`status`	enum	One of: `baseline`, `keep`, `discard`, `crash`
`description`	string	The hypothesis, one sentence. Include the change type and the expected mechanism.

Example file

iteration	commit	metric	delta	status	description
0	-	85.2	0.0	baseline	initial measurement — test coverage 85.2%
1	a1b2c3d	87.1	+1.9	keep	add tests for auth middleware edge cases
2	-	86.5	-0.7	discard	refactor test helpers (broke 2 existing tests)
3	-	0.0	0.0	crash	add integration tests (postgres connection failed — fix in iter 4)
4	b2c3d4e	88.3	+1.2	keep	add tests for error handling in API routes
5	-	88.1	-0.2	discard	add tests for rate limiter (metric within variance, treated as regression)
6	c3d4e5f	89.0	+0.7	keep	add boundary value tests for form validators
7	d4e5f6g	89.8	+0.8	keep	add tests for session expiry edge cases
8	-	89.2	-0.6	discard	mock external API calls (test isolation but metric regressed)
9	e5f6g7h	90.6	+0.8	keep	add tests for concurrent request handling
10	f6g7h8i	91.1	+0.5	keep	add tests for malformed JSON input handling

Progress summary format

Print every 10 iterations. Use this exact format:

=== Autoresearch progress — iteration <N> ===
Goal:         <original goal statement>
Baseline:     <iteration 0 metric>
Current best: <best metric so far> (<total delta> from baseline)
Keeps:        <count> (<keeps/total * 100>%)
Discards:     <count>
Crashes:      <count>
Top pattern:  <the change type that has produced the most total delta>
Last 5:       <sequence of keep/discard/crash for iterations N-4 through N>
Est. to goal: <if goal metric is known, N iterations at current rate>
===

Interpreting the log

Healthy run signature

Keep rate 40-60%
Delta per keep: consistent small positive gains
No long crash streaks
Discards are evenly distributed (not clustered)

Warning signs

Pattern	Meaning	Action
Keep rate < 20%	Hypothesis quality is poor	Re-read full scope, re-read lessons, change direction
Keep rate > 80%	Metric may be too easy or Verify too lenient	Tighten the goal
Long crash streak (5+)	Verify command is fragile or scope is too risky	Fix Verify or narrow scope
Delta per keep shrinking toward 0	Approaching local optimum	Try more radical changes or declare victory
Metric oscillating	Non-deterministic Verify or contradictory changes	Run Verify twice and average; tighten scope

Declaring success

Stop the loop when one of these is true:

Metric has reached the stated goal
Delta per keep has been below 0.1% for 20 consecutive iterations (local optimum with current scope)
All directions have been exhausted (lessons file confirms this)

In all cases, print a final summary and write a lessons entry covering the full run before stopping.

File hygiene

Add autoresearch-results.tsv to .gitignore. It is a working file.
Do not edit it manually during a run.
Between runs, you may archive it: mv autoresearch-results.tsv autoresearch-results-<date>.tsv and start fresh, but keep the lessons file — that is the persistent memory.

3.8 KiB Raw Blame History Unescape Escape