Deploying AI in a Bank: It Is Not Engineering

Deploying AI in a bank is not an engineering problem, because the blockers are ownership decisions rather than model quality. The model can pass every demo and still never reach production while nobody can name the person who owns the documents, the environment, or the review. Discovery is the engineering.

Why deploying AI in a bank stalls after the demo

Deploying AI in a bank usually stalls because the unanswered questions are ownership decisions, not engineering ones. A document exception assistant can demo perfectly and still stop at three questions: which environment hosts it, who approved the document use, and who reviews a wrong exception. Naming those owners in week one is what turns a working model into a system with a credible route to production.

Paden Gayle, a founding engineer who has worked at Accenture, Bloomberg and Google, opened his AI Engineer talk with a bank document exception assistant that reviewed client documents and drafted exceptions for an analyst. The demo showed the intended workflow and the value was obvious. Then security asked which environment would host it, data owners asked who agreed to that use of the documents, and someone in risk asked who reviews a wrong exception before it reaches the next system.

Each answer can change what the team is allowed to build. In Gayle's account the team had already invested in a design while several people responsible for that design had never seen it. The model was fine, the demo worked, and the system did not ship.

That gap is the article's central claim: discovery is the engineering. The people who own the decisions are part of the specification, and bringing them in early is cheaper than retrofitting an environment boundary, a data path or an action boundary after the build.

The four labels Gayle uses to organize the work are data, model risk, third party risk and audit. For each one he wants a person who can explain the decision, the evidence they need, and the next step. Teams writing AI deployment plans for regulated industries tend to skip that step because the demo creates the impression the hard part is done.

The four discovery labels: data, model risk, third party risk, audit

The four labels are data, model risk, third party risk and audit, and each one needs a named person rather than a department. Get those names on the record in week one and the rest of the project becomes a sequence of concrete, tractable approvals instead of a build that has to be unbuilt.

Data

Data ownership splits three ways: who owns the source, who can grant access, and who approves using it in the workflow. Those responsibilities may sit with different people, and being able to query a dataset does not settle how the application may use it.

Gayle's data platform work at Bloomberg made the split concrete: building LLM infrastructure there meant establishing which datasets could be used and who could approve that use. Carry the same discipline into each bank deployment, and into any deployment. Before designing retrieval, find the person accountable for each source, confirm the allowed use, and confirm the environment where the processing can happen. If any of the three answers changes, the design changes with it.

Model risk

Model risk asks which review applies to the intended use. Two practical questions help here: what a previous submission looked like, and who will own the system after launch, including the evidence that owner needs to maintain it.

Third party risk

Third party risk is where you draw the actual dependency chain, covering the model provider, host arrangements and your role in the operating service. The useful output is a list of relationships already reviewed and proposed changes that need another decision.

Audit

Audit covers reconstruction. Ask what someone would need to rebuild a consequential action: which input was used, which configuration produced the output, what the reviewer saw, and who approved the action. Those answers determine the records you must capture from the beginning.

A permissive data protection framework in a given jurisdiction still does not tell you who inside the bank approved the use, which is why the naming exercise matters more than the policy text.

Reading the org chart as a dependency map

The org chart is a dependency map, and reading the second row is what surfaces the unowned decisions. Gayle splits the diagram into two rows. The top row is the system sketch: documents go into the model and an analyst reviews the output. Most teams can draw that in a few minutes. The bottom row is the control row: the documents have a data owner, the model service needs a platform owner, the analyst workflow needs someone who can confirm the relevant risk requirements, and one name still carries a question mark.

That question mark is the dependency. The workflow has been designed without finding the person who can say whether its intended use is acceptable and what evidence will be required.

The method is to walk the diagram with a sponsor, confirm who owns the documents, ask whether they have heard about the use, and ask who can help reach the appropriate reviewer. Confirm the answers directly with those people rather than through the sponsor. A real deployment will carry more relationships than a sketch, and a single technical component may depend on several approvals. Some decisions span the entire system.

The value of the exercise is timing: dependencies become visible while the design is still easy to change, and an unnamed owner turns into a specific task with a person responsible for resolving it.

Discovery questions that produce names, decisions and next steps

The discovery call should end with names, decisions or next conversations, never with reassurance. Gayle brings a short list of questions to that first meeting:

Where does the data live and who can approve reading it?
Who has stopped a similar project before, and why?
Which existing controls could cover this workflow, and who owns them?
Where will this run in production, and who administers that environment?
Who signs the relevant risk review, and what have they signed before?
What obligation is the sponsor personally worried about, and who can explain it?

The failure mode is a vague answer. When somebody says security will be fine, the follow-up is to ask who will actually review the design and when you can meet them.

Close the loop the same week. Send each owner a short description of the proposed use and confirm what they need from you, so that by Friday the people whose decisions the deployment depends on know the project exists. That gives scoping a firmer starting point than any architecture diagram would.

Shadow, draft, shrink: scoping without a written specification

When a customer cannot supply a specification, shadow the work, draft a concrete workflow and shrink it with the control owners. Gayle describes a cash flow reconciliation system at Bloomberg as the example that made this approach concrete for him: he followed the flow and the people responsible for it, which gave him a fixed point to start from.

The same method applies to an onboarding assistant. Sit with an analyst, watch which systems they open, note where they make a judgment and when they ask for permission, and ask what happens when a document is missing. Writing down the actual sequence produces the beginning of the specification.

For the document exception assistant, the draft has three lines: review the documents, draft the exceptions, analyst submits. Then each line becomes specific. Which documents can the analyst access, and is that access approved for this processing? Should the model produce a highlighted passage, a draft form, or both? Who checks the draft, and who submits it?

The first version might output a highlighted exception and a draft form, with the analyst checking the source, editing the form and submitting. That boundary can be corrected by a person, which is what makes it reviewable. Gayle also proposes success criteria at this stage, such as less review time with no increase in misconceptions, and notes that the criteria are only a proposal until the people who own the workflow agree on how to measure them against current practice.

Take the draft to the sponsor and the control owners. They can Stripe out a document type, change the reviewer, or reject the output entirely, and every correction becomes a decision the build can absorb. Scope then becomes a series of approval questions rather than a document to negotiate.

The autonomy ladder: scoped autonomy in bank AI deployments

Autonomy works as a ladder, and the correct starting rung is the one below the authority level the workflow actually needs. Read the ladder from the bottom up: suggest, draft, acts with approval, and acts. Each step changes both the consequences of an error and the controls that must work.

Rung	What the system does	What the person does
Suggest	Returns a recommendation	Decides what to do with it
Draft	Prepares something for review	Checks it and submits
Acts with approval	Proposes each action	Authorizes it before it happens
Acts	Operates within pre-approved limits	Monitors and can stop it

The method is to start at the lowest rung that creates value, then collect evidence for the next decision. Are reviews overriding the drafts? Which failures keep recurring? Can people recover when something breaks? A model does not become more capable on its own, so the question is how much authority this particular workflow should hand it.

Gayle's hypothetical makes the boundary concrete. One version has an agent read client documents, decide which fall outside policy and file exceptions automatically, so the model's judgment becomes an operational action with no analyst checking each case. Keep the model and change the action boundary instead: the system highlights a likely exception, points at the source passage and prepares a draft form, and the analyst checks the evidence, edits the form and chooses whether to submit. That is model, human, submit.

Two details make the review real. The analyst needs the source evidence and the visible uncertainty to make a judgment, because a review button only helps if the person can actually check the work. And each automated step needs a control with a named owner, a discipline Gayle carried over from automating cash flow steps at Bloomberg: find the consequential action, name its owner, and decide what happens when the system is wrong. In the exception workflow, the submission is that action.

Human review does not guarantee approval and does not by itself determine a regulatory category. The intended use and the applicable reviews still have to be confirmed, which is the boundary the NIST AI Risk Management Framework addresses at a general level rather than per institution.

Testing each increment: from summarize to automatic send

Each increment of capability adds a specific approval question, and the smallest increment that carries useful value is the right place to start. Gayle grows the hypothetical system one step at a time:

Summarize a document the analyst already has. This still requires confirming that existing access covers the use.
Read from a second system. This adds a data path that someone has to approve.
Draft a response a customer will see. This raises who checks the content and which review applies.
Send a response automatically. This raises who owns the action, what happens when something wrong is sent, and how the system stops and recovers.

Those are four different engineering scopes, and the applicable reviews depend on the institution and the use case. Confirm them with the people who own those decisions rather than inferring them from the first version.

Then ask which increment creates useful value today. Drafting inside the analyst's current workflow often solves enough of the problem to ship first, provided the existing control genuinely fits. Build the smallest useful version with a credible path to approval, then use evidence about errors, overrides and recovery from real operation to justify the next decision about authority.

The evidence pack: five files and per-segment results

An evidence pack is how an evaluation becomes usable by the risk reviewers who are accountable for managing the system. It has five files:

Intended use, recording users, boundaries and prohibited actions.
Data lineage, recording each example source and its use approval.
Failure modes, recording detection and fallback.
Results by segment, recording coverage and errors.
Controls, recording reviews, thresholds, stopping rules and owners.

Generate results with each evaluation run, and preserve the raw record alongside it: the input, prompt, model version, configuration, output and reviewer action. Re-running the model later may produce a different answer, so the preserved record is what lets a reviewer reconstruct the historical decision. Calling the model again is retrieval from a separate store, not a reconstruction from a summary.

Segmentation is where averages stop being useful. Gayle would separate clean documents, scans, rare exceptions and new document types, then measure errors and control behavior inside each group and lead with the weakest result. A single average can hide a serious gap in one document type or user group. When a group has little evidence, say so and restrict that use until enough evidence exists.

The practice of recording intended use, limitations and evaluation results for a specific system is what the NIST AI RMF and the EU AI Act's high-risk documentation duties both point at, though neither substitutes for the institution's own signer.

Failure modes: is an unqualified yes to regulated deployment safe?

No. An unqualified yes to regulated deployment is not supportable. Per-segment results compare four document groups on what the reviewer can actually see, and that comparison matters more than a single accuracy figure.

Document group	Evidence concern	Restriction until resolved
Clean documents	Highest coverage, weakest signal of real difficulty	None beyond standard review
Scans	Extraction quality varies with image quality	Manual review of low-quality scans
Rare exceptions	Few examples per evaluation run	Limited scope, targeted offline cases
New document types	Little or no evidence available	Excluded from the workflow

Local execution, allowlist filtering and the absence of a required managed service can support an air-gapped architecture, but they do not by themselves establish security, tenant isolation, or suitability for healthcare, banking or any other regulated setting. The evidence pack shows which controls the library or workflow enables; the institution still supplies the compliance controls, the signer and the accountable owner.

For the hypothetical exception assistant, Gayle names three failure paths and their handling: if it invents an exception, the analyst checks the source passage and rejects the draft; if extraction produces unreadable text, the system stops and requests a clean file; if it misses exceptions, targeted offline cases expose that weakness, though offline cases do not detect every live miss, so affected work stays under manual review. Each failure needs a detection path, a fallback and an owner. Gayle's version of the question comes from his security and deployment work at Google: what happens when the system breaks, who notices, what stops, who takes responsibility, and how long until it is back up.

Naming the person who receives the alert and can stop the workflow is part of the plan, not an afterthought.

The Friday scorecard: five answers before committing to delivery

By Friday of week one, five questions should have answers. They take minutes to ask and they tell a team where the project actually stands before any delivery commitment is made.

Is the data owner engaged and aware of the project?
Are the production environment and its administrators named?
Can the sponsor describe the workflow without saying AI?
Can the smallest useful scope fit existing controls confirmed with their owners?
Do you know the risk signer, and what happened to the person who signed last time?

Use the answers as a planning heuristic, not a prediction. Four or five yeses suggest a credible path to ship. Three means unresolved decisions need owners and dates. Zero to two means resolving those decisions before committing to delivery, including shrinking or redefining scope. The score does not predict approval, and it does tell a team where to spend the next week.

The closing advice is short: draft the spec, then shrink it; give the system only the authority it needs; build the evidence pack in sprint one. The demo worked, the model was fine, and it still did not ship, so the next step is to go find the people.

FAQ

Why is deploying AI in a bank not an engineering problem?

The model can be accurate while the system remains unapprovable. The blocking questions concern which environment hosts the system, who authorized the document use, who reviews a wrong output and which existing controls the workflow has to satisfy. Those are ownership decisions, and each one can change what the team is allowed to build.

What are the four discovery labels in this method?

They are data, model risk, third party risk and audit. For each label, the goal is a named person who can explain the decision, the evidence they need and the next step. Data splits into source owner, access granter and use approver, which are often three different people.

What is the autonomy ladder?

It describes four levels of authority: suggest, draft, acts with approval and acts. Each step changes the consequences of an error and the controls that must work. Start at the lowest level that creates value, then use evidence from real operation to justify more authority.

What belongs in an evidence pack?

Five files: intended use, data lineage, failure modes, results by segment, and controls. Alongside them, preserve the input, prompt, model version, configuration, output and reviewer action for each evaluation run so a reviewer can reconstruct what actually happened.

Why preserve the original outputs instead of re-running the model?

Re-running the same prompt later can produce a different answer, so a fresh call is not a reconstruction of the historical decision. Preservation, not re-execution, is what makes an audit trail usable. The saved record shows which input, configuration and reviewer action produced the outcome.

Why report results by segment rather than as an average?

An aggregate can hide a serious gap in one document type or user group. Separating clean documents, scans, rare exceptions and new document types shows where coverage is weak. When a group has little evidence, state that and restrict the use until more evidence exists.

What is the Friday scorecard?

Five questions answered by the end of week one: is the data owner engaged, are the production environment and its administrators named, can the sponsor describe the workflow without saying AI, does the smallest useful scope fit confirmed controls, and do you know the risk signer and the outcome of the last similar signing. Four or five yeses suggest a credible path to ship.

Does human review automatically make an AI workflow approvable?

No. Human review does not guarantee approval and does not determine a regulatory category on its own. The applicable reviews still depend on the institution and the intended use, so they must be confirmed with the people who own those decisions rather than inferred from a review step in the workflow.

Where does AI governance guidance fit?

General frameworks describe intended use, limitations and evaluation records, but they do not name the signer for a specific deployment. The institution supplies the accountable owner and the controls, so treat external guidance as a checklist input rather than an approval.

Readers working in Portuguese may want the same planning discipline for a technical career. Gustavo Dev Doido publishes development training that follows this pattern: define a small scope, work inside real constraints and produce evidence of what was built. The Crazystack Typescript track applies that approach to typed application development in TypeScript, which matters for AI-assisted workflows where an untyped boundary between a model output and a downstream system is a failure path.

The Bootcamp do Dev Doido extends the same structure to a longer program with graded exercises. The connection to this article is narrow but useful: whether the deliverable is a bank exception assistant or a Typescript service, the work moves when the scope is small enough for someone to review and the evidence of what ran is preserved. You can find the material at crazystack.com.br.

Turning a talk like this into a written article

Gayle packs a lot into 15 minutes: four discovery labels, a four-step autonomy ladder, five evidence files and a five-question Friday checklist. That density works in a room where the slides carry the detail, and it loses something when the talk becomes a page of notes. The same is true for most expert knowledge recorded on video, because the spoken explanation and the written reference serve different readers.

If you have recorded talks, walkthroughs or interviews where the reasoning matters more than the recording, Skala Blog turns an existing YouTube video into a structured article through transcription and drafting. Paste the video URL, let it transcribe, and edit the result. The article you just read started that way: a 15-minute conference talk, a transcript, and the same arguments reorganized around the questions a team actually faces. If you have knowledge like that sitting in a video, the reasoning is worth more than the recording.

Skala Blog

Source video

Gayle is a founding deployed engineer and is writing a book on forward deployed engineering for O'Reilly. The source material for this article is his AI Engineer talk Deploying AI Inside a Bank Is Not an Engineering Problem.