Machine learning & custom LLMs

When generic stops being good enough

Off-the-shelf models know language; they do not know your claims codes, your product taxonomy, or the strange way your industry abbreviates things. Fine-tuning on your own data closes that gap - often with less data than people expect - and can cut running costs at the same time. Your data stays under your control throughout.

Prompt, retrieve, fine-tune - in that order

There is a ladder of intervention, and cost climbs with every rung: better prompting and structured outputs, then retrieval over your own documents, then fine-tuning, then - rarely - training something custom. We start at the bottom and only climb when the benchmark says the current rung cannot reach. It is remarkable how often a well-built retrieval system over honest data beats an expensive fine-tune over messy data, and how much cheaper it is to discover that in this order.

• Prompting and structured outputs first - often enough on their own
• Retrieval (RAG) when the knowledge is yours and changes often
• Fine-tuning when the task is stable and the examples exist
• Custom models only where the economics genuinely demand them

The problems this is for

Classification and triage on your own categories. Extraction from the documents your industry writes and nobody else reads. Retrieval and question-answering over a document estate. Scoring and forecasting on your own history - the classic machine learning that predates the hype and still quietly earns its keep. If “our data, our terminology, our edge cases” describes the problem, it belongs here; if a general-purpose model with a good prompt would do, we will tell you so and charge you considerably less.

Benchmark first, then build

You cannot improve what you have not measured, so the first artefact we produce is a quality benchmark: what does good look like, on your problem, on your cases? The model is then iterated against that bar, and the final report covers the failures as candidly as the wins. A model trusted in the wrong places is worse than no model.

• An agreed benchmark before any training
• Iteration measured against it, release by release
• Failure modes documented, not glossed over

How an engagement runs

First, the benchmark: a set of real cases, graded, that defines what good looks like on your problem - assembled with your domain experts, because they are the only people who can mark the answers. Then a baseline: the best general-purpose model, measured against that set, so every pound spent afterwards has to beat a number rather than a feeling. Then iteration up the ladder - retrieval, fine-tuning, whichever the evidence justifies - with results reported release by release, failures included. Most clients have their first honest read within a few weeks.

The benchmark outlives the model

A quietly valuable by-product of working this way: the evaluation set - your cases, graded by your experts - is an asset that outlasts any particular model. When a better model ships next year, you can measure it against your problem in an afternoon instead of re-running a pilot. When a vendor makes a claim, you can check it. Clients who have been through one engagement here stop asking “is the new model better?” and start asking “what does it score?” - which is the healthiest question in the field.

Yours: the model, the weights, the pipeline

Everything produced in the engagement is yours in the fullest sense - the fine-tuned weights, the training pipeline, the evaluation sets, the deployment configuration - documented and handed over like any other software we build. A model tuned on your data is a competitive asset, and an asset you rent is not one. If you leave, everything needed to keep the model alive leaves with you.

What it costs

A benchmark-and-baseline engagement is the usual starting point: a fixed-scope project from £8,000 that tells you what a general-purpose model already achieves on your problem and what the next rung of investment would buy. It is deliberately self-contained - if the baseline is good enough, you stop there, better informed. Ongoing model work runs as an embedded team from £4,500 a month per engineer, reported against the benchmark throughout, so you can always see what the spend bought.

Deployed like software, watched like software

Deployment comes with monitoring for drift and degradation, because model quality slips as the world changes around it. You get a model that measurably beats the generic option on your problem - and an honest account of where its edges are.

And when a model is the wrong tool

We once spent two weeks teaching a language model to read a file whose schema had never changed, because the flow was signed off before the engineers were asked. The parser that eventually replaced it took an afternoon and has never got a row wrong. The moral is baked into how we scope now: models where the input is genuinely unstructured and a good guess beats no answer - plain code everywhere else. If your problem is the second kind, expect to hear it early.