Personal project by Ulf Fäldt Reifenberg · unofficial, not affiliated with Bose or any other company
audio-exp-agent
An LLM agent that launches small audio ML experiments through a fixed set of tools and safeguards, and diagnoses runs that went wrong. Measured on CPU, on the Free Spoken Digit Dataset, against simple baselines. The agent does not win everywhere; the numbers below say where it does and does not.
Results
- Loading…
The setup
Task and data
Model and runs
Each run takes about half a second on CPU, so every experiment in this page was precomputed and stored in an append-only registry. The page makes no live model calls. The source data is Free Spoken Digit Dataset (CC BY-SA 4.0).
Tools and safeguards
| Tool | What it does | Mode |
|---|---|---|
launch_run(config) | Validates the config against an allow-list, checks the budget, trains one run, appends it to the registry, returns validation accuracy. | search |
list_runs, run_status, get_metrics, compare_runs | Read runs from the registry: status, metric history, config diff and data/code hash comparison between two runs. | both |
read_log(run_id, tail) | Last lines of a run's training log. | diagnose |
get_split_info, get_feature_stats, get_audio_info | Speakers per split and overlap; feature scale and normalisation; file vs declared sample rate. | diagnose |
write_report(text) | Stores the agent's written summary in the registry. | search |
Safeguards, each covered by a unit test
- Allow-list. Only 9 config keys, each with a range. Data paths, sample rate, filter range and split mode cannot be set by the agent. A rejected config costs no budget.
- Budget and approval gate. At most 10 runs and 400 training epochs per search. A launch over budget needs an approval callback; the benchmarks run with no human, so it is denied and nothing runs.
- Sealed test set. No tool returns test accuracy. The harness reads it once per run it chooses to report, and every read is logged. The agent selects on validation accuracy only.
- Append-only registry. SQLite triggers block UPDATE and DELETE on runs. Each run stores config hash, data hash, code hash, seed, status and full log.
- Read-only diagnosis mode. The diagnosing agent cannot launch or change anything.
Result 1: diagnosing faulty runs
Correct diagnoses by fault type
Result 2: hyper-parameter search at equal budget
Test accuracy of each search session (same training seed for paired sessions)
Result 3: do the numbers in the agent's report match the registry?
Example report and its check
Example agent traces
A good diagnosis
A failure
Limits
About. Personal project by Ulf Fäldt Reifenberg, built for a job application with coding agents. Unofficial, not affiliated with Bose. It shows how to wrap an LLM agent around training scripts with allow-lists, budgets, a sealed test set and a reproducible registry, and how to measure the agent against baselines. It does not show GPU or cluster scheduling, large-scale training, DSP or acoustic research, or anything about any company's internal systems. LLM calls were made once, offline, through Cloudflare Workers AI ( neurons); the page only reads results.json.