Fine-tuning
We adapt small language models to your domain, your tone and your specific tasks.
REQUEST A DIAGNOSTICWhat fine-tuning is
Fine-tuning means continuing to train an existing language model with your own examples, so that it solves a specific task better. The model keeps what it knew and learns the vocabulary, the format and the judgment of those examples. With small models, the result can be a specialized model that is cheaper and faster to run than a large general one.
Fine-tuning, RAG or a good prompt
Prompt
Instructions guide the model without modifying it. It is the quickest to try and the reasonable first step.
RAG
It gives the model access to your documents at the moment of answering, without changing the model. It is the right choice when the information changes often or the source has to be cited.
Fine-tuning
It changes in a lasting way how the model behaves: its style, its output format and its judgment on a task. It is not a reliable way of adding new data to it.
Combined
Using both is common: fine-tuning for behavior and RAG for up-to-date knowledge.
What it adapts to
Domain
The vocabulary and concepts of your sector or your company.
Tone
The way your team writes and answers.
Specific tasks
A narrow, repeated job, instead of answering everything.
Examples of what we have already done
Company tone
A model that writes replies in the company's style and vocabulary.
Classification
A model that sorts emails, incidents or documents into the categories the company uses.
Industry vocabulary
A model adapted to the terminology of a specific field, such as legal, medical or industrial.
Data extraction
A model that extracts specific fields from texts in the company's own format.
Fixed-structure summaries
A model that summarizes long reports or minutes, always with the same outline.
Light local model
A small model that runs on the company's infrastructure for repetitive tasks.
When it helps
When a task repeats often, has a stable output format and there are examples of how it is solved well. Also when a small model is of interest because of cost, speed or where it has to run.
When it is not worth it
- You need your own high-quality examples. With little data, or with bad data, the result does not improve.
- Sometimes a general model with good instructions or access to your documents is enough. If so, it is the simplest option.
- A fine-tuned model has to be evaluated again when the data or the task changes.
- Fine-tuning teaches behavior, not facts. For information that changes, the model needs access to the documents.
- A fine-tuned model can memorize its examples and fail on new cases, or lose general capabilities it had. That is why it is evaluated with examples it has not seen.
- If the examples contain personal data, data protection rules apply. It is advisable to anonymize them before training.
FREQUENTLY ASKED QUESTIONS
What is fine-tuning?
It means continuing to train an existing language model with your own examples, so that it solves a specific task better.
What is needed to do fine-tuning?
Your own high-quality examples of how the task is solved well. With little data, or with bad data, the result does not improve.
Which base models do you fine-tune?
We work with models such as Qwen, Mistral and Llama. The choice depends on the task, the language and the license.
When is fine-tuning not worth it?
When a general model with good instructions or access to your documents already solves the task. In that case it is the simplest option.
Fine-tuning or RAG?
They answer different problems. RAG gives the model access to your documents and works when the information changes or the source has to be cited. Fine-tuning changes how the model behaves: its tone, its format and its judgment on a task. They can be combined.
How many examples are needed for fine-tuning?
There is no single figure: it depends on the task. You can start evaluating with a few dozen well-prepared examples, and a stable result usually needs hundreds or more. Quality matters more than quantity.
Does fine-tuning add new knowledge to the model?
Not reliably. It teaches patterns of behavior, format and style, not facts that can be looked up. For the model to answer with up-to-date information it has to be given access to the documents.
Does the whole model have to be retrained?
Not necessarily. Techniques such as LoRA adjust only a small part of the model's parameters, which reduces the compute cost and makes it possible to work with more modest hardware.
Who owns the fine-tuned model?
It stays in the hands of the client. Its use remains subject to the license of the base model it was tuned from, which should be reviewed.
What does the cost of fine-tuning depend on?
Less on compute than on preparing the data and evaluating the result. Gathering, cleaning and reviewing the examples is usually the most laborious part of the project.
From your examples to your model
We start with a proof of concept on real data and only scale if the value is demonstrated.