The distinction that matters here is one many adverts blur: this is API access to somebody else’s models under an enterprise agreement, not a platform for training your own. Nobody is fine-tuning a foundation model from scratch on it. They are building a product feature against a managed endpoint, with procurement, data residency and billing already settled by the wider cloud relationship.

Which means the hire is an application engineer, and the interesting questions are about judgement and cost rather than about machine learning.

Why companies end up here, and why it changes the brief

Very few teams choose this on technical merit alone. They arrive because the legal and procurement work is already done: the data processing agreement exists, the spend goes on an existing bill, the security team has already approved the account, and nothing needs to leave a network boundary somebody has signed off.

That is a genuinely good reason, and it has a consequence for hiring. The constraint on your engineer is not the model — it is the environment. They will spend more time on identity, network paths and guardrail configuration than on anything resembling data science, and a candidate who finds that beneath them will be unhappy.

So describe the job honestly in the advert. “Build AI features inside a regulated AWS estate” attracts the right person. “Work with cutting-edge foundation models” attracts somebody who wanted a research post.

Model choice as an engineering decision

The service offers several model families from different providers, and the useful candidate treats that as a decision to be measured rather than a preference to be held.

Ask how they chose. Strong answers describe running the same evaluation set across two or three candidates and comparing quality, latency and cost together — and they will have found, as everybody does, that the most capable model is rarely the right one for every call. Weak answers name whichever model was in the news when they started.

Follow with how they would handle a model being deprecated or a new version behaving differently. Managed model access moves underneath you, which is the trade for not running it yourself, and an engineer who has been through a version change will have a pinned-version story and a regression set.

The routing question, which is where the money is

Most production systems here end up using more than one model, because the cost difference between the cheapest and the most capable is large and most requests do not need the expensive one.

Ask how they would decide which model handles a given request. The thoughtful answers involve classifying difficulty cheaply first, escalating only when confidence is low, caching aggressively where prompts repeat, and measuring quality at each tier rather than assuming the cheap path is adequate. Somebody who has run this at volume will also mention that the routing logic itself becomes a thing to test, because a misrouted class of request degrades quietly.

This is the single highest-leverage skill in the role and it is almost never in the job description.

Guardrails, and the awkward conversation about false positives

The service provides content filtering and policy controls, and they are a genuine reason to be here rather than calling a provider directly. They also block things you did not want blocked.

Ask what they did when a legitimate request was refused. It is a real operational problem — a medical, legal or financial application will trip safety filters on entirely proper queries — and the answers that work involve measuring the false-positive rate, tuning the policy deliberately, and giving the user a sensible message rather than a failure. An engineer who has not met this has not run a serious application on it.

What to screen for beyond the AI part

  • Streaming and timeouts. Generation is slow by web standards. If the interface does not stream, users experience it as broken, and the surrounding infrastructure has to tolerate long-lived responses.
  • Quota and throttling. Per-model request and token limits are real and arrive at the worst moment. Retries with backoff, and a plan for sustained throttling rather than a spike.
  • Evaluation. The same question as for any model-backed feature: how did they know a change was an improvement? Nothing about a managed endpoint removes the need for a test set.
  • Cost visibility. Can they tell you what a feature costs per thousand requests, and attribute spend to a feature rather than to the account?

Rates, and the overlapping roles

Indicatively £75,000 to £110,000 permanent, day rates from around £550 — broadly the market for a strong backend engineer, with a modest premium for the AWS depth.

Two adjacent guides describe most of the people you will actually shortlist. If the work is principally product engineering with a managed model behind it, the AI engineer guide is the closer description and the pool is much larger. If the difficulty is grounding answers in your own documents, that is retrieval and the RAG engineer guide covers it.

Hire specifically for this service when the estate is the constraint: strict networking, a compliance regime, several accounts, and enough spend that routing and caching pay for themselves. Otherwise hire a good application engineer who knows AWS and let the service be the detail it usually is. If the requirement is training and hosting rather than consuming models, that is a different product entirely and the SageMaker engineer guide is the right one.