Fine-tuning
Further training of a pre-trained model on your own examples so it adopts a specific style, format or task behaviour without starting from scratch.
Definition
Fine-tuning takes an existing LLM and continues training it on a curated set of input, output pairs. The model's weights shift slightly so that it reproduces the pattern in those examples: a house style, a strict output schema, a classification scheme, a domain's terminology. Supervised fine-tuning (SFT) is the most common form; RLHF and related preference methods are a further stage.
Parameter-efficient techniques such as LoRA and QLoRA train a small set of additional weights rather than the whole model, which cuts cost and lets several variants share one base model. Fine-tuning is usually compared with prompt engineering (cheaper, faster to iterate) and RAG (better for changing knowledge).
Example: an insurer needs claim summaries in a fixed 6-field format that adjusters trust. After fine-tuning a small open model on 1,500 approved summaries, the format is followed without a long prompt, and the smaller model runs at a fraction of the cost.
What it means in practice
- Can make a small, cheap model match a large one on a narrow task, cutting inference cost.
- Locks in format and tone, which reduces prompt length and post-processing.
- Creates a maintenance obligation: data pipelines, retraining and regression evals.
- Often the wrong first move; most teams should exhaust prompting and retrieval before training anything.
How Teksmobile uses it
Our fine-tuning and evals service starts with a baseline measurement and only trains when prompting and retrieval have been shown to fall short. The decision framework is set out in our guide prompt engineering vs fine-tuning.
Related terms
Part of the Teksmobile AI Glossary. For longer reads, see our guides.
Frequently asked questions
How much data does fine-tuning need?
For style and format, a few hundred to a few thousand high-quality examples is a common starting range. Quality and consistency matter far more than volume.
Does fine-tuning teach the model our facts?
Unreliably. It shapes behaviour well but is a poor way to store knowledge that must be accurate and current. Use RAG for facts and fine-tuning for style and format.
Can we fine-tune hosted models like Claude or GPT?
Some providers offer fine-tuning for selected models; the options change frequently. Open-weight models (Llama, Mistral) can always be fine-tuned and self-hosted, which also helps with data residency.