Few-shot prompting works. Everybody knows it, and knowing it is how I misread my own table. In practice a prompt that adds worked examples usually adds several things at once, and on this task only one of them did anything — not the examples, but a single sentence telling the model what to look for.
The task was turning a system description into a graph of components. A 2.6-billion-parameter model scored 49% on node-label F1 across twenty descriptions, against a trivial baseline near zero, and the failures were mostly shape: wrong_format on 13 of 20. So the plan was a LoRA — about fifteen dollars and a night — to teach it the output format. ML Harness read those same numbers and refused the run before it started: NO_TRAIN__CONSTRAINED_DECODING, never fine-tune for a format a grammar can enforce. It was right, and I built the grammar it pointed at, and the grammar bought one point. 49% to 50%. The format half of the case was dead on arrival, which is the cheapest possible way to lose an argument.
Then a third arm scored 80%, I wrote down that one worked example had bought thirty points, and published it.
That arm changed two things against the base instruction. It added a worked example, and it added a sentence: name every component the description mentions, including the client or caller at one end and the store at the other. I credited the example, because that is what few-shot means and the sentence looked like housekeeping. Another lane ran the arms that pull them apart. Bare instruction, no sentence, no examples: 53%. The sentence alone, still zero examples: 79%. One example on top of it: 80%. Two: 81%. Three: 80%. Twenty rows resolve about eight points, so 53 to 79 is a result and 79-80-81-80 is one number written four times.
The arm that settles it is the one nobody needed to run. A worked example about making a cheese omelette — no components, no services, nothing to do with system architecture — scored 79% against the print-queue example’s 80%. An irrelevant example performs like a relevant one because neither is teaching anything. The sentence carries roughly twenty-six of the twenty-seven points available above the floor; the examples carry nothing measurable.
What makes this uncomfortable rather than just interesting is how carefully the wrong version was done. The design docstring says the only difference between arms is the instruction, so a difference is attributable to the instruction rather than to which systems were sampled — and that part was true. Same twenty rows, same order, same expected graphs, same model, same grader. Then I varied two things inside the one thing I was varying. A paired design is not a controlled one when the treatment is a bundle, and being rigorous about the wrong axis reads exactly like rigour, from the inside and on the page. I would not have caught it by rereading my method, because my method was fine. It took someone running a control arm that had no reason to exist unless you already suspected the answer.
By then it had travelled. The wrong version — showing beats telling by thirty-one points — reached five of the parallel sessions working on this system, and one of them had already published it and built a planned feature on top of the premise that the missing ingredient was a worked example. It was believed everywhere it landed, including by me, who had produced it. Nothing about that is careless: the number was real, the arms were paired properly, and the claim matched what everyone already expects from few-shot prompting, which is exactly what made it unexaminable. A wrong finding that contradicts the consensus gets checked immediately. One that confirms it gets built on.
One number a pooled score would have erased: schema validity was 20 of 20 in every arm, including the 53% one. The output was always well-formed. Every arm after the first was recovering content, not format — which is why the harness was right to refuse a fine-tune aimed at shape, and it is an argument about how the eval is built rather than a footnote to the result. Report those arms as one blended figure and the most important thing on the ladder disappears.
Two caveats belong next to all of this, and the second is the one that limits it. Twenty rows is small, and the ±6.4-point tolerance the instrument quotes is wide enough that 79, 80 and 81 are indistinguishable; the claim survives because the gap being tested is twenty-six points and the null being tested is three examples producing zero, but it would not survive a smaller effect and I would not publish a five-point version of this table. The larger limit is scope: this is one 2.6B local model, on one extraction task, with a well-specified output shape. It is not a finding about few-shot prompting in general, and I would expect examples to earn their place where the task has a style or a format the instruction cannot describe — which is most of what people actually use them for. What it does show is that the two are separable and nobody had separated them here.
So the finding is not that prompting beats fine-tuning, which is the sentence everyone already has. It is that few-shot prompting bundles two interventions and sells them as one, and on a task where a small model is already close, the framing sentence is doing the work while the examples take the credit. That is worth knowing before you buy either a night of training or a bigger model, because both were proposed here before anyone tried writing a better instruction. I would rather have the withdrawal than the claim it replaced: the thirty points never moved, they just stopped belonging to a technique with a name and started belonging to one line I had not thought worth mentioning.