Trace reader
Every run, step by step
Gemma 4 E4B was asked to find the variant behind a melon trait among 3,061 candidates, twenty times under each of three toolboxes. These are the recordings. Pick a condition, pick a run, and read what the agent did, call by call. Under each answer, our own check, which Gemma never sees, shows where the experimentally validated CmEIN3 variant lands and whether every variant listed exists.
The raw file with 3,061 variants defined by their reference letter, alternate letter and other sequencing information, and nothing else. It reads the beginning of the file, and on that information alone it cannot tell the variants apart, so it stops.
The same file plus a computational biologist toolkit: a sandbox, bcftools and samtools, Ensembl VEP, and local copies of the reference genome, gene annotation and gene descriptions; nothing is fetched from the internet. After the first answer, three fixed follow-ups we wrote, as an expert bioinformatician would send them. About a third of its tool calls error out, most often on Ensembl VEP flags borrowed from human-genome usage. In its best runs (run 10, run 20) it ends with about twenty candidates, only two of which are coding changes, and nothing it computes tells those two apart.
The same file and one tool: BOTANIC-1, which returns the log-likelihood ratio between the alternate and the reference base in sequence context. Gemma discards the 567 candidates that are not single-nucleotide substitutions, scores the remaining 2,494 and ranks them.
How to read a tracewhat comes from Gemma, the tools, and us
These runs test whether the setup works, not how good Gemma can be made to look. The harness is deliberately plain: one prompt, the same scripted follow-ups in every run of a condition, the tools as installed, and no tuning or clean-up of what Gemma writes. Its answers appear exactly as it produced them, formatting quirks included.
- Gemma its thinking, the commands it writes, and its answers.
- The tools what came back when Gemma’s commands ran (a file, a bcftools or VEP run, BOTANIC-1 scores), errors included.
- Us, before and during the run the system prompt, the task and the follow-ups, identical in every run of a condition.
- Us, after the run our checks and notes, including “How to read this answer” under each answer. Gemma never saw them.
Gemma was asked for a ranked list, one variant per line, with a reason. Under each answer, “How to read this answer” sums up what the list contains and what it does and does not show, then gives the full table: where the validated variant lands, whether every variant exists, and what each one is (from Ensembl VEP and the read counts). Only the reason column is Gemma’s.
Use ← and → to step, Home and End to jump. The address bar follows your position, so a link to a step can be shared. Tool output over 6,000 characters and thinking over 4,000 are trimmed in the recordings, and say so where they are.
Both halves on the whole genome
Every run above starts from the 3,061 chromosome-2 candidates, which is a substantial head start. In real-world settings the starting point would be the full set of variants across the entire genome. There, neither bioinformatics tools nor BOTANIC-1 recover the causal variant on their own. Rank all 39,656 scorable variants by BOTANIC-1 alone and the causal one lands sixteenth. In a separate set of twenty runs, not included in this reader, Gemma was given that full set with BOTANIC-1 as its only tool, and in 18 of them it ranks the causal variant sixteenth too, exactly where the score puts it. Fifteen dramatic-looking changes sit above it, a lost stop codon, a broken splice site, and not one of them is carried by every plant with the trait.
The steps a bioinformatician would take for a recessive trait in a cross remove them. Keep the single-letter substitutions, drop positions with fewer than ten reads in either bulk, keep the variants carried by every plant in the mutant bulk, and drop those also fixed in the wild-type bulk, which separate the parents from the reference rather than the trait. None of these steps orders what survives. Handing the survivors to BOTANIC-1 does. This is not an agentic result: no run has yet had the full set together with the bioinformatics tools. It is the behaviour one would expect from the agentic system in that setting, since every step here is one the agent already performs in the conventional-bioinformatics condition, and the scoring is what it does in the BOTANIC-1 condition.
- The same study, for a plant biologistWhat BOTANIC-1 contributes to the search, without the agent.
- Technical reportMethod, evaluation, and the 545-study causal-variant benchmark this case sits inside.