Train a private model on the data only your firm holds.
General models have read the public internet. They have not read your matter files, process logs or call recordings. A bespoke training run takes an open-weight model and trains it on those records, by supervised fine-tuning, reinforcement fine-tuning against a grader, distillation, or a sequence of them, until it passes a held-out test your own experts wrote. The weights are delivered to you.
The test is fixed before anything is trained, and an untuned model is scored on it first. If prompting or retrieval already passes, we tell you and stop.
Scope a run
Three answers decide most of a training plan: what you hold, where it has to stay, and where the model will run. The run sheet fills in as you choose, in the order the stages would execute.
Run sheet
- 1.Baseline. Score an untuned base model, and a strong general model with prompting and retrieval, on your held-out set.
- 2.Supervised fine-tuning. Trained to reproduce your experts’ output. LoRA adapters first; full-weight updates only if the adapter plateaus below the pass mark.
- 3.Acceptance. The held-out test you kept back, a regression check on general ability, and a probe for verbatim recall of training records.
- Data
- Not yet decided. Settled at scoping, before any record moves.
- Model
- Size chosen at scoping, from the baseline scores.
- You receive
- Weights or adapters, the evaluation harness, and the run report.
Opens your own email client with the sheet filled in. Nothing is sent or stored by this page.
What a run can include
Each method needs something different from you and changes something different in the model. Most runs use two or three in sequence.
| Method | You supply | What it does | Fits when |
|---|---|---|---|
| Supervised fine-tuning | Worked examples: inputs paired with the output your experts produced. Around a hundred is enough to start; thousands is common. | Updates the model to reproduce expert output for your inputs. | The task has a right form you can demonstrate: drafting in house style, extraction, classification, structured output. |
| LoRA and QLoRA adaptersHu et al., 2021Dettmers et al., 2023 | The same examples as supervised fine-tuning. | Freezes the base weights and trains small low-rank matrices beside them. The original paper cut trainable parameters by a factor of 10,000 and GPU memory by 3; QLoRA fine-tuned a 65B-parameter model on a single 48GB GPU. | The default first pass. Cheap to iterate, and several task adapters can share one base model. |
| Continued pretrainingGururangan et al., 2020 | Unlabelled domain text in volume: millions of tokens and up. | A second phase of next-token training on your corpus, so the model absorbs vocabulary, notation and document structure. | Your domain’s language sits far from general web text. |
| Preference optimisation (DPO)Rafailov et al., 2023 | Pairs of outputs where one was preferred: redlines, rankings, accept or reject. | Shifts the model towards the preferred output directly from the pairs, with no separate reward model. | Quality is a matter of judgment, tone or completeness, not one correct answer. |
| Reinforcement fine-tuningShao et al., 2024 | A grader that returns a score: code, a simulator, a reference check or a rubric. | The model generates attempts, the grader scores them, and policy-gradient updates (PPO, GRPO) raise the expected score. | Several answers can be right and you can score them reliably, including multi-step reasoning and tool use. The base model must already succeed some of the time, or there is no signal to climb. |
| RL policy trainingSchulman et al., 2017 | An environment or simulator, or logged decisions with their outcomes. | Trains a decision policy on reward, not on imitation of text. | Sequential decisions: scheduling, routing, control, execution. |
| DistillationHinton et al., 2015 | A strong teacher model and a pool of unlabelled inputs. | Trains a small student to match the teacher’s outputs. | The capability already exists; its latency, cost or hardware footprint does not work. |
What a run can target, by sector
These are examples of tasks that suit training, with the way each would be scored. They are illustrations of fit, not a client list.
| Sector | Records | A model trained to | Scored by |
|---|---|---|---|
| Law firms | Closed matter files, precedents, redlines | Draft clauses and first-pass review in house style; flag departures from the firm’s positions | Held-out matters marked by partners; redline acceptance |
| Hardware engineering | Design reviews, test reports, change orders, CAD metadata | Review designs against the firm’s own rules; predict test failures from a design change | Recorded outcomes held back by date |
| Manufacturing | Work orders, alarm logs, inspection reports, process data | Diagnose faults from alarm sequences; draft setup sheets | Agreement with the recorded root cause and fix |
| Construction | RFIs, submittals, change orders, daily logs | Answer RFIs from the specification; flag change-order risk | Held-out projects, against the response actually issued |
| Life-sciences labs | Lab notebooks, protocols, run records, assay results | Propose protocols; predict run outcomes; structure notebook entries | Measured results the model never saw |
| Contact centres | Call recordings, transcripts, dispositions | Resolve or route to policy, in the firm’s tone and languages | QA rubric scores; resolution on held-out calls |
| Consulting and advisory | Deliverables, drafts, workpapers, models | Draft to house structure; check models and decks against firm standards | Reviewer preference against the delivered version |
| Finance and research | Research notes, models, trade and risk records | Extract, classify and draft to house methodology; decision policies trained on reward | Out-of-sample periods held back by date |
How a run goes
- Scoping call. The task, the records, the constraints. No data changes hands, and an NDA is signed before records are discussed in detail.
- The test comes first. Before anything is trained we fix a held-out set drawn from your records, a metric and a pass mark, in writing. The held-out set never enters training.
- Baseline. An untuned base model, and a strong general model with prompting and retrieval, are scored on that test. If either already passes, we tell you and stop.
- Data preparation. De-duplication, removal of personal identifiers, formatting, and splits cut by matter, client or date so that near-duplicates cannot leak from training into the test.
- Training. Adapters first, then heavier methods only where the scores call for them. Checkpoints are selected on a validation split, never on the acceptance set.
- Acceptance. The held-out test, a regression check for lost general ability, and a probe for verbatim recall of training records. With a grader in the loop, scores are confirmed on data the grader never touched: a rising reward proves nothing if the model has learned to game the grader.
- Delivery. Weights or adapters in standard open formats, the evaluation harness so you can rerun every number, and a run report covering data lineage, hyperparameters and scores.
What published results show
None of these are our results. Each was published by the group named, on the task named, and together they show the size of effect domain training can produce when the task suits it. Whether yours does is what the baseline step measures.
Across 31 narrow tasks, 4-bit LoRA fine-tunes of small open models outperformed their own base models by 34 points and GPT-4 by 10 points on average. GPT-4 stayed ahead on 6 of the 31, and the authors find that narrower tasks gain the most.
A fine-tuned 770M-parameter model outperformed a few-shot-prompted 540B-parameter model, using 80% of the available data on the benchmark.
Human raters preferred the output of a 1.3B-parameter model trained on human feedback to that of a 175B-parameter model, 100 times its size.
Reinforcement learning alone, with no human-labelled reasoning examples, lifted one model’s score on a competition mathematics benchmark (AIME 2024, first attempt) from 15.6% to 71.0%.
On clinical text, relatively small specialised models substantially outperformed in-context learning across models of up to 175B parameters, even when fine-tuned on limited annotated data.
Distillation cut one model’s size by 40% and made it 60% faster while retaining 97% of its language-understanding capability.
When training is the wrong answer
Most firms do not need a trained model, and a page that sells training should say so. Three cases come up most often.
- The facts change every week. Use retrieval. Training is a poor way to load facts into a model: in one published comparison, retrieval consistently outperformed unsupervised fine-tuning for injecting knowledge (Ovadia et al., 2023).
- A prompt with examples already passes your test. Stop there. The baseline step is designed to find this out before you pay for a run.
- There are no worked examples and no way to score an answer. Build the test set first. That is a data project, and it comes before any training.
Training earns its cost when the task is narrow and repeated thousands of times, when a house style or a judgment will not hold in a prompt, when latency, cost or hardware rule out a large general model, or when the records cannot be sent to a third-party API at all.
The two approaches also stack. One study measured an accuracy gain of more than 6 percentage points from fine-tuning and a further 5 from adding retrieval on top (Balaguer et al., 2024).
Your data, your weights
- One client, one run. Your records are used for your run and nothing else. They are not pooled with another client’s data and not used to train any model but yours.
- Training data is never offered for licence. RoboMacro also helps firms license archives to AI developers. That is a separate arrangement, entered only at your request and under its own contract.
- Where the run executes follows your constraint. Some records can travel under NDA. Others cannot leave your cloud account or your building. Say which at the start, because it decides where the work is done.
- Memorisation is tested, not assumed away. Language models can reproduce training text verbatim (Carlini et al., 2020). Personal identifiers are removed before training, and the trained model is probed for verbatim recall before it is delivered.
- You hold the result. A fine-tune that lives on a vendor’s API lasts only as long as the vendor keeps that model and that service running. Weights on your own hardware have no end-of-life date but the one you set.
Questions
How much data do we need?
For supervised tuning, fewer examples than most firms expect and more care. Google’s published guidance is to start with 100 examples and scale to thousands if needed, with quality counting for more than quantity. Reinforcement fine-tuning needs a reliable grader in place of gold answers. Continued pretraining wants volume: millions of tokens of domain text.
Which base models do you train?
Open-weight models whose licences permit commercial use, chosen at scoping for the task, the languages and the hardware the model has to run on. Open weights are the default because the result can be handed to you outright.
Who owns the trained model?
You do, subject to the base model’s licence. Ownership, permitted use, retention and deletion are written into the agreement before any data moves.
What if the model misses the pass mark?
The pass mark, and what follows if it is missed, are agreed in writing before training starts. The baseline step exists so that a run with no realistic chance of clearing the bar is never commissioned.
Does fine-tuning make us a model “provider” under the EU AI Act?
Usually not. The European Commission’s guidelines treat a downstream modifier as the provider of a general-purpose model only where the modification significantly changes it, and give as an indicative criterion that the modification used more than a third of the original model’s training compute. The same guidelines say few modifications currently meet it. A model built into a high-risk system carries separate obligations. Take your own legal advice.
European Commission, guidelines for providers of general-purpose AI models
What does a run cost, and how long does it take?
Each run is priced after scoping. The drivers are data preparation, model size, the number of training stages and where the run has to execute. Data readiness usually decides the schedule more than compute does.
Can we license our data to AI developers as well?
Yes, as a separate arrangement that happens only at your request. Data supplied for a training run is never offered for licence.
Start with one email
I’m Chris Apostolou, founder of RoboMacro. We publish AI-written macro research every day, behind automated checks that block a note before they let a wrong number out. A training run is scoped the same way: the test comes first.
Tell me the task, what records you hold and where they have to stay. Two or three lines is enough to start, and there are no forms.
Email Chris: [email protected]Chris Apostolou · [email protected] · LinkedIn
RoboMacro is a macroeconomic data and research company. This page is operated by Chris Apostolou, RoboMacro, [email protected]. Results cited from published research belong to their authors, were measured on the tasks they describe, and are not a forecast of what a run on your data will achieve. Nothing here is an offer, a guarantee of performance, or legal advice. See our Privacy Policy.
