TensorRT is a compiler, and that single fact should shape the whole hire. You are not looking for somebody to change what a model computes — you are looking for somebody who can take a trained network and get materially more throughput out of the same NVIDIA hardware, while proving that the outputs did not shift.

It is specialist, it is measurable, and it is very often a fixed-length piece of work rather than a permanent role. Deciding which of those you have is the first thing to do.

Is this a project or a position?

Optimisation work has a characteristic shape: a large gain early, then diminishing returns. A good engineer might halve your latency in six weeks and then spend six months finding another fifteen per cent.

So the honest questions before advertising are whether you have one model to optimise or a continuous stream of them, and whether anybody will maintain the optimised build when the model is retrained. If it is one model and nobody owns the rebuild, hire a contractor and insist on documentation. If you retrain monthly and ship to several hardware targets, that is a real ongoing job and worth a permanent hire.

A surprising number of companies advertise the permanent version of this role when they wanted the project version, then find the engineer has nothing comparable to do in month four.

What the compiler actually does, and why it matters at interview

The mechanisms are worth understanding because they are what you should ask about. Layer fusion collapses several operations into one kernel so intermediate results never leave fast memory. Kernel auto-tuning benchmarks alternative implementations on your specific card and picks a winner. Precision calibration moves weights and activations to lower precision. Static shapes let the compiler make assumptions it cannot make about variable ones.

Each of those has a cost that a candidate should be able to name. Fusion makes the graph harder to debug. Auto-tuning ties the resulting engine to the card it was built on, so an engine compiled on one GPU generation may not load on another. Lower precision risks accuracy. Static shapes mean a variable-length input needs either padding or several engines.

An engineer who describes only the benefits has read the marketing. One who volunteers the trade-offs has shipped it.

The accuracy conversation is the one that separates candidates

Ask what they check after quantising, and how. This is the whole risk of the discipline: a model that is two and a half times faster and quietly worse on the cases you care about is a bad outcome that looks like a good one on a dashboard.

Strong answers involve a task-specific evaluation set rather than a published benchmark, a comparison against the original model on the same inputs, and particular attention to the tail — because degradation tends to concentrate in the rare, difficult examples that matter commercially. Calibration data should be representative of production traffic, and a candidate who mentions choosing that data carefully is telling you they have seen it go wrong.

The answer to be wary of is any version of “INT8 is basically lossless”. Sometimes it is. Establishing that it is for your model is the job.

Screening for the operational half

The optimisation is only useful if it survives contact with your deployment process, and this is where these engagements most often disappoint.

  • “Where does the engine get built?” Compilation is slow and hardware-specific, so it belongs in a build pipeline against the target card, not on somebody’s workstation. An engineer who has not automated it has left you a manual step nobody remembers in six months.
  • “What happens when the model is retrained?” The correct answer is that the optimisation is reproducible from a script, with the accuracy check as a gate. The wrong answer is a set of remembered flags.
  • “What is your fallback?” If the optimised engine fails to load on a given machine, does inference fall back to the unoptimised path or does the service fail? Anybody who has run this in production has an opinion.

The hardware-lock question you should raise, not avoid

This work binds you to one vendor more tightly than most. The engines are NVIDIA-specific and tuned to particular cards, and that has consequences for cloud portability, for cost negotiation and for anybody who might later want to run on different accelerators.

It is frequently the right trade — the performance is real and most teams are on NVIDIA anyway — but it should be a decision rather than a drift. If portability matters to you, the interchange-format route is the alternative and our ONNX engineer guide covers what you gain and give up. If your problem is serving language models specifically rather than optimising a graph, the vLLM engineer guide is the closer fit; the two disciplines overlap less than the job titles suggest.

Rates, and why the pool is smaller than the demand

Indicatively £85,000 to £135,000 permanent, and day rates from around £650, higher for anything involving custom kernels or unusual hardware. Around 110 UK employers name the tool across the pages we index.

The scarcity has a specific cause worth understanding. This work sits between machine learning and systems programming, and most engineers are comfortable on one side of that line. People who can read a profiler, reason about memory bandwidth and also understand what a quantisation error does to a model’s behaviour are genuinely uncommon, and they know it. If you find one, the conversation will be about the interest of the problem at least as much as the rate — which is worth preparing for, because a dull optimisation brief loses these candidates to people offering a harder one.