Out of the box, a language model knows nothing about your contracts, your product catalog or last week's policy change. There are three main ways to close that gap, and teams often pick the wrong one.
The three approaches
| Approach | What it is | Strengths | Watch-outs |
|---|---|---|---|
| RAG (retrieval-augmented generation) | Search your documents at question time and give the relevant passages to the model. | Fresh data, citations, access control, no retraining. | Quality depends on retrieval, chunking and ranking. |
| Long context | Paste large amounts of material straight into the prompt. | Simple to start; good for a bounded set of documents. | Cost, latency and accuracy can suffer as the window fills. |
| Fine-tuning | Further train a model on examples. | Teaches style, format and specialised behaviour. | Poor for facts that change; needs data and upkeep. |
A simple decision guide
- Need answers grounded in changing company knowledge, with sources? Start with RAG.
- Small, fixed document set, one-off analysis? Long context may be enough.
- Need a consistent tone, format or specialised task behaviour? Consider fine-tuning, often alongside RAG.
In practice, strong systems combine them: RAG for knowledge, good context engineering for what reaches the model, and occasionally fine-tuning for behaviour.
What makes RAG actually work
- Quality ingestion: clean extraction from PDFs, tables and wikis, with sensible chunking and metadata.
- Hybrid retrieval: combine semantic (vector) search with keyword search, then re-rank.
- Access control: retrieve only what the asking user is allowed to see, enforced at query time.
- Citations: show sources so users can verify answers.
- Evaluation: a test set of real questions that measures retrieval and answer quality, and runs on every change.
- Graceful refusal: say "I don't know" when the evidence is not there.
We build private, permission-aware assistants as part of our custom LLM & RAG service.