"One costs forty thousand pounds. The other is a sentence. I cannot tell which does what."— Office Man
What is the difference?
Prompting means giving a model written instructions — a system prompt that tells it what to produce, how to behave, what tone to use, what to avoid. Fine-tuning means taking an existing model and training it further on specific examples, so the behaviour is baked in rather than instructed each time. Both produce a model that behaves more consistently for a particular task. One requires a good document and an afternoon. The other requires a labelled dataset, significant compute cost, ongoing maintenance, and a technical team. For most use cases, start with the document.
When does prompting fail?
A well-written system prompt can include: a detailed persona, specific formatting requirements, multiple examples of ideal output, explicit rules for edge cases, and contextual information about your organisation. Most teams that conclude "prompting is not enough" have never written a serious prompt — they have typed two sentences and drawn conclusions. Prompting genuinely fails when you are running the same task at very high volume (thousands of times a day), when a long system prompt adds too much latency or cost at that scale, and when results are still inconsistent after serious prompt engineering. These are real situations. They are also specific ones.
The fine-tuning checklist
Fine-tuning is the right call when three things are true simultaneously: you run the same task type at significant scale (consistently 100+ times a day); you have tried detailed prompt engineering and the results are still not reliable enough; and you have a good-quality labelled dataset of at least a few hundred high-quality examples. If any of these is no, prompt engineering and iteration is the right next step. The dataset question is often the most important and most underestimated. Poor training data produces a poorly fine-tuned model. The quality of examples matters more than the volume.
What to ignore
"Fine-tuning will give you your own AI." Technically, a fine-tuned model is customised to a specific task. It is not a model that knows everything about your organisation or produces all outputs perfectly. It is a model that is better at one well-defined task — when trained well on good data. Useful, specific, and modest. Not the general solution to making AI work. Similarly: ignore vendor claims that fine-tuning will transform your business. Ask instead whether it solves a specific, well-defined problem you have — and whether a better prompt would solve it first.