Before the model: how we scope AI projects

Before the model: how we scope AI projects

Most AI projects don't fail on the model. They fail on data nobody released, rules nobody owns and risks nobody named. Here is the four-step method TTMS uses to scope an AI project before a single line of code is written.

Start with the job, not the technology

Ask a team what it wants from AI and you usually hear a solution: a chatbot, an assistant, "something with GPT". The useful question comes earlier: which function should the system perform, and what kind of AI sits behind it?

That second question matters more than it looks. The AI type decides almost everything that follows: which risks are realistic, which success factors carry the project, and what the existing data has to deliver. A search system over internal policies fails differently from a model that flags transactions for review. Treat them the same and you plan for the wrong problems.

So we work in four steps: function and type, then risks, then success factors, then data requirements.

Step 0: one function, one type – and first check whether a rule will do

We break the use case into functions and assign exactly one AI type to each. Before any of that, one test: does a fixed rule already solve it? Price lists, catalogues, policies, checklists and text templates on structured data don't need AI. Sometimes the most honest answer is: "This doesn't need AI – not yet."

TypeThe question it answersTypical riskWhat must exist first
Rule-basedIs there a fixed rule?A regulation or price change makes the rule outdatedWritten rules, a named rule owner, test cases per rule
ClassificationWhich category?Wrong assignment treats customers unequallyReleased case history with the correct outcome
ExtractionWhich fields are in the document?A wrong value ends up in a decision or contractAnnotated sample per form type
Search (retrieval)Which document fits?Outdated or restricted documents surfaceCurated corpus with an owner per document
GenerationWrite or summariseInvented content, convincingly wordedEvaluation set with target answer and source
ForecastWhat value comes next?A rule or process change makes the history uselessStable target definition, long consistent history
Anomaly detectionWhich cases stand out?False alarms burden customers and staffConfirmed results of past reviews
Speech / imageWhat is in the audio or image?Dialects and image qualityOwn ground-truth sample per dialect

Three rules of thumb follow. A "recommendation" is usually forecast plus rule plus human. The most sensitive data in the project sets the legal and infrastructure requirements for all of it. And any output a person reads needs an evaluation set before go-live. Our default level of automation: the system suggests, a person decides.

Step 1: name the risks – and how you'd notice them

We list six risks and rank them in a fixed order: first what would sink the pilot, then what would harm the people affected – customers, partners, employees – then the rest. Every risk is tied to a fact from the actual case, not to a generic AI fear.

Each risk gets two answers:

  • Prevent: one concrete measure before launch. For a search system that means a curated corpus and permission filters applied before the search, not after.
  • Fix: a trigger you can actually observe, an action and a named owner. Example: "Hit rate for a category drops below threshold → that category goes back to manual and is relabelled → owned by the business unit."

The test of a good risk list: turn risk number one around and you get success factor number one. And at most one risk is "consciously accepted" – with the reason written down.

Step 2: rank the success factors – no ties

Five success factors, strictly ranked. The ordering rule: first what blocks everything else, then what decides the benefit, last what is hardest to reverse.

In almost every project, number one is the same: the business unit releases its data together with the correct outcome. Without a labelled history or an approved evaluation set, nothing else can be measured – not accuracy, not benefit, not risk. After that typically come a clear human decision point, a measurable quality metric per category or field, and a correction loop that feeds back into the system.

Each factor gets a when (before launch, during the pilot, before scaling) and a who (the client, us, or jointly). Targets come only from the case itself; where the case is silent, we set them together with the client rather than inventing them.

Step 3: check the data in four dimensions

We compare what exists today with what the use case needs, row by row, and give every gap an owner. "Not mentioned" is a valid answer for the current state – and often the most revealing one.

DimensionWhat we check todayWhat must be in placeOwner of the gapWhy it pays off
LegalPersonal data? Especially sensitive? Classification?Legal basis and purpose, purpose limitation, DPIA decision before launch, retention rulesLegal / complianceNo rework, no forced stop
TechnicalFormats, languages, scans, volumes, ageEvaluation set with target values per language, metadata, permissions inherited from source systemsBusiness unit labels, we cleanMeasurable from day one
OrganisationalWho owns the data? Who releases? Who labels?Named data ownership, release process, correction loopThe client names the ownerNo waiting on approvals
InfrastructureWhere is the data? Which interfaces?Hosting according to data classification and cloud policy, interfaces, loggingInformation security and operationsNo rebuild during the pilot

The last row is the one people forget: what we don't need. Data minimisation is a design decision. Free text containing names or health details, or anything classified "confidential", usually stays out of the pilot entirely. Naming the bottleneck across all four dimensions is the most useful single sentence of the whole exercise.

Why this saves time instead of costing it

Four steps before any code can sound like overhead. In practice it is the opposite. Every gap found here is a gap that doesn't stop the pilot in month three. And the method is honest in both directions: it shows where AI genuinely helps – and where a well-maintained rule does the job better.

The model is the easy part. The judgment about function, risk and data is where experience counts.

Planning an AI project and want to know where the real bottleneck is? Talk to TTMS.

‍

AI & Delivery

AriseHealth logoOE logoOE logo2020INC logoThe Paak logoEphicient logoToogether logo