First, a disambiguation that costs teams real time: there are two unrelated things called Triton in this field. One is NVIDIA’s inference server, which hosts models and answers requests. The other is OpenAI’s language for writing GPU kernels. They share nothing but a name, and a CV saying “Triton” could mean either.
This guide is about the server. Establish which one a candidate means in the first five minutes, because hiring the kernel author to operate a serving fleet, or the reverse, wastes everybody’s quarter.
What the server is for, and the case that justifies it
It exists to host many models, written in different frameworks, behind one service. That is the whole proposition and it is worth being precise about, because if you have one model in one framework you almost certainly do not need it — a container with a web server in front of it will do, and will be simpler to operate.
The cases where it earns its complexity are specific. You are serving a dozen or more models and do not want a dozen deployments. Your models come from different frameworks and you would rather not maintain a bespoke service for each. You need several models loaded on one accelerator to use it properly. Or you have a pipeline — preprocess, model, postprocess, sometimes a second model — that you want executed server-side rather than across network hops.
Ask a candidate when they would not use it. Somebody who has operated it will happily tell you it is overkill for a single model, which is the answer of an engineer rather than an enthusiast.
The concepts the interview should cover
Model repository layout and versioning. The server is configured largely by directory structure and a config file per model. Ask how they managed that across environments, and whether model versions could be promoted without a restart. Anybody who has run it has an opinion about where that repository lives and how it gets there.
Dynamic batching. The main performance feature: requests arriving separately are combined so the accelerator does useful work. Ask what they set the queue delay to and how they chose it, because it is a direct trade of latency for throughput and the right answer depends entirely on the application. A candidate who has tuned it will talk about measuring both sides; one who has not will quote a default.
Concurrent model execution. How many instances of a model to run on one device, and what happens to memory when you get it wrong. This is where most capacity mistakes live.
Backends. Which they used — ONNX Runtime, TensorRT, PyTorch, Python — and why. The Python backend is convenient and slow, and knowing when that trade is acceptable is a useful signal.
The ensemble question, which is the real skill
Ask whether they have built a server-side pipeline, and what they put in it.
This is where the platform is genuinely valuable and genuinely easy to misuse. Moving preprocessing onto the server removes network round trips and keeps the logic beside the model, which is excellent. Moving too much business logic there produces something that is hard to test, hard to debug and owned by nobody — the same mistake as putting application logic in database triggers.
The engineers worth hiring have a line they draw: deterministic transformations that belong to the model go server-side, decisions that belong to the product stay in the application. Ask where their line is and why.
Operating it, which is most of the job
Once it is running, this is a serving fleet like any other, with some specifics worth screening for.
Ask what their dashboards showed. The useful metrics are queue time separate from compute time — that split tells you whether you need more capacity or faster inference, and conflating them is how teams buy hardware they did not need. Ask about model loading: cold starts on a large model are slow, and whether models are loaded eagerly or on demand is a real decision with a real memory cost.
Ask how they handled a bad model version reaching production. You want a story with a version rollback in it rather than a redeployment.
Rates, and a caution about the number
Indicatively £85,000 to £125,000 permanent, with day rates from around £600.
Our own count says roughly 160 UK employers name it, and I would not trust that figure. Triton is also a well-known British bathroom-fittings brand, so a share of those mentions are careers pages at companies that make showers, and the two meanings in this field make it worse. It is the clearest example on this site of why a keyword count needs a sanity check before anybody plans a budget around it.
The neighbouring hires, which are more often right
If you are serving language models specifically, the purpose-built route is usually better and the pool is larger — the vLLM engineer guide covers it, and the batching behaviour there is designed for generation rather than adapted to it.
If the problem is making one model faster rather than hosting many, that is compilation and the TensorRT engineer guide is the right one; the two are frequently used together, with the compiler producing the artefact the server hosts. And if what you actually need is somebody to own the whole serving estate rather than this component, the AI infrastructure engineer guide describes a broader and more available profile.