01. Project Intake and Decision Context

1. Purpose and when to read this#

"We want AI to help with X."

Some version of that sentence starts most projects, and there is nothing wrong with it. It is honest about the goal and honest about the vagueness. The problem is that nothing follows from it. It does not say what would count as help, who is harmed when the help is wrong, or what result would mean the idea was a dead end. Months later it also gives you no way to settle whether the thing you built worked.

This chapter is the conversion. Read it first on every new engagement, immediately after chapter 00 — before a model is shortlisted, before an evaluation is designed, before compute is rented or bought. What you have at the end that the opening sentence did not contain:

• a named decision — a question with a yes and a no, each implying something different to do next;
• a completed project profile, the intake record every later chapter reads its facts from;
• an assigned stakes tier, which fixes how much proof this project owes;
• an inventory of constraints, sorted into the ones that eliminate options and the ones you can trade against;
• a first-pass archetype — the project shape that sets your reading order;
• a committed first sprint, with a stated place where it stops.

Honesty note. Much of this chapter is playbook synthesis. The intake method here was assembled from general evaluation-practice literature and first-principles argument — it was not read off a large internal record of varied project intakes. Individual claims are marked (inference) where that is the case; where an external source corroborates a step, it is cited. So treat this chapter as a well-argued starting procedure, not as a validated instrument, and revise it in your own method decision record as it meets real projects.

2. Inputs required#

Three things. The first is the one that gets skipped.

• Whoever owns the business decision, reachable for the intake conversation (§5.1). Several of the fields below have no answer anywhere except in that person's head.
• Access to whatever already exists: an incumbent system, prior evaluations, production logs, support tickets, cost data.
• Chapter 00's principles and rigor dial. This chapter applies them at the intake boundary; it does not redefine them.

No completed template is required to start this chapter. It produces the first one.

3. Decisions this chapter supports#

Five decisions run through this chapter. Each is cheaper to make here than anywhere downstream of here.

• Whether a request is a decision-driving work item at all, or must be re-scoped before anything else happens.
• The project's stakes tier and therefore its mandatory artifact set (00 §6).
• Which candidate execution systems are admissible at all, before any are evaluated.
• A first-pass archetype and reading path — QUICKSTART is the normative router; this chapter previews it and hands off.
• What the first sprint commits to, and exactly where it must stop.

4. Normative principles#

5. Default procedure#

5.1 The intake question set#

Field names below are identical strings to templates/PROJECT_PROFILE.md's field headers, so the two tables stay grep-able as one questionnaire.

Twenty-one questions is a lot to ask in one sitting, so the third column says why each one earns its place here rather than later, and the fourth says which chapter is waiting on the answer. Other files in this playbook cite these fields by number — so the numbering is part of the interface, not presentation.

Fields 3, 4, 6, 19, and 21 are always mandatory, even for a one-person Tier-1 exploration: they are the inputs the never-skippable floor and the stakes tier depend on. The remaining fields SHOULD be answered as completely as the project's maturity allows. An honest "unknown — to be measured" is an acceptable value and is itself a profile entry, not a blank — usually it is also your next action.

Those five are also the tier conversation in miniature, which is why they cannot wait. Field 3 asks what a wrong answer costs and who pays it. Field 19 asks whether an auditor, a regulator, or a legal regime has a claim on the answer. Field 21 asks the decision owner to state out loud how wrong is acceptable. Read together against the tier signatures in 00 §6, they usually name the tier without an argument; when two of them point at different tiers, the higher one wins (§6). Do this at intake and the tier is what keeps a small project small — it is the mechanism that lets an exploratory piece of work skip artifacts a regulated deployment could never skip, deliberately rather than by drift.

5.2 Requirement → evaluation claim formulation#

Every requirement MUST be restated as an evaluation claim, with the decision it drives stated explicitly, before an evaluation is built for it or a candidate is chosen against it (00 P1):

Vague request        "We want AI to help our support team with X."
      │  name the decision
      ▼
Decision              "Should we deploy an AI system to do Y for population Z,
                       replacing/augmenting the current process, by date D?"
      │  name the claim the evidence must support
      ▼
Evaluation claim      "System S achieves quality bar Q on population Z (field 4),
                       verified by evaluation E, within the latency/cost bounds
                       of fields 5/14."
      │  hand off
      ▼
Chapter 03            E becomes a task ontology, a stratified corpus, and a
                       grading method — derived from real failure modes, not
                       assumed (03 §5, requirement → task-ontology derivation).

Each step down that ladder is narrower than the one above it. A decision carries a date and an alternative; a claim carries a threshold, a population, and the instrument that will measure it. If you cannot yet fill in Q or Z, intake is not finished — and the blank is telling you which conversation to have next.

The requirement → task ontology derivation itself (error-analysis-first, root-cause stratification, task ontology versioning) is chapter 03's subject; this chapter's job ends at handing off a decision and a claim that 03 can operationalize.

5.3 Constraints inventory: hard vs. soft, and kill criteria#

The split is about what a constraint does to your candidate set. A hard constraint removes candidates outright, however well they score. A soft one is something you buy and sell against — worse latency for lower cost, more spend for more quality. Sorting them at intake is cheap. Skipping the sort costs you the whole selection you then run without it.

Writing the kill condition down while you still have no results is the entire mechanism, and it is the part teams resist. Afterwards every number is negotiable: the pilot was unlucky, the prompt was never tuned, one more week would do it. Some of those objections will be true. What you cannot do afterwards is tell your own judgment apart from your own investment in the answer, because by then they feel identical from the inside. A threshold fixed while you are still indifferent to which side of it you land on is the only kind that survives landing on the wrong side.

This is the project-level analog of a consequence-bearing tolerance (04): it predates any experiment, but it obeys the same rule — the condition and its consequence are stated before the fact that would trigger it is known.

5.4 Archetype selection — preview#

Projects rhyme. Six shapes cover most of them, and the shape you are in decides which three or four chapters you open first — so this is the step that turns fifteen chapters into a short reading list.

Worked, non-authoritative profiles at three of these rows: SYNTH-01 (A), SYNTH-02 (B), SYNTH-08 (E). A full navigator pass, profile through first sprint, is worked end-to-end in WALKTHROUGH.

5.5 Cold-start parameterization#

1. Anchor on task consequence. Start from the stakes tier (00 §6), not from a convenient round number — a Tier-3 decision earns a tighter provisional band than a Tier-1 one, before any data exists.
2. Borrow external reference points as priors, explicitly labeled non-binding. A vendor default, a published comparable benchmark, or an adjacent project's calibrated value. State the source and name, in writing, the reason it might not transfer.
3. Run a calibration pilot sized for direction, not inference. Enough cases to see which side of a threshold you are on, not enough to claim a margin — this is screening, not inference (04); an MDE is not yet a meaningful statement at this sample size.
4. Pre-register how the first real data will revise the parameter. The revision rule — the formula, the split it reads, the sample size that triggers it — is fixed before the pilot runs, so the eventual change is legitimate under 04's amendment legitimacy rule rather than post-hoc threshold shopping.
5. Never let a placeholder threshold silently become policy. A value set under step 3 stays visibly provisional — recorded in the profile as, e.g., "cold-start; revision due at n = 300" — until step 4's procedure actually fires. Treating it as calibrated the moment it stops feeling arbitrary is the anti-pattern this procedure exists to prevent (§9).

Worked illustrative example (invented, round numbers; no project value travels here):

A team has no incumbent for a new document-classification task and must set a provisional "escalate to human review" confidence threshold before any labeled data exists. Step 1: the decision is Tier 2 (customer-visible, reversible), so the team allows a wider provisional band than a Tier-3 payment decision would get. Step 2: a comparable published system's threshold, 0.70, is recorded as a non-binding prior. Step 3: a 40-case pilot on the iterate split shows the region separating confident-correct from confident-wrong sits somewhere between 0.55 and 0.75 — enough to pick a provisional value of 0.65, not enough to defend a specific point. Step 4: the first real experiment contract pre-registers that the threshold will be re-derived from a stated formula once 300 iterate-split cases are labeled, before any qualify-split execution. Step 5: the profile records the value as "cold-start — revision due at n = 300," so no one downstream mistakes it for a calibrated production threshold.

5.6 The first-sprint contract#

Intake ends in a commitment, and the commitment is deliberately small: one sprint, with the place it stops written down in advance. Everything after that first sprint depends on what the first sprint finds, so committing to it now would be committing on information you do not have yet.

• Tier 2 and Tier 3: a frozen templates/EXPERIMENT_CONTRACT.md for the first real experiment (chapter 04) — question, decision, prediction, splits, MDE, and consequence-bearing tolerances all stated before data.
• Tier 1: a written lightweight plan — the completed profile, the chosen archetype, and the first three actions — is sufficient; a frozen contract is not yet required.

"Lightweight" scales the weight of each floor item, not which floor items exist: nothing in the never-skippable floor (00 §6) is waived. So the Tier-1 deliverable is not a separate content list to author from scratch. It is defined as satisfying all six floor items, mapped one-to-one onto what the completed profile, the locked archetype, and the first three actions already produce:

This mapping is what "written lightweight plan" means in this chapter and in QUICKSTART — QUICKSTART restates the identical six-item floor mapping at its own first-sprint endpoint; a Tier-1 plan satisfying this table satisfies both.

6. Project adaptation parameters#

Four things in this chapter are yours to set rather than to copy. The third column says what to check the setting against.

7. Decision gates and stopping conditions#

The three conditions below mean stop, not slow down. Each one says what to do instead.

8. Metrics and formulas#

N/A — intake produces no statistics of its own; there is nothing here to compute. The statistics canon begins at chapter 04; formula derivations live in references/STATISTICS_FORMULAS.md.

9. Failure modes and anti-patterns#

Three of the five below are one move wearing different clothes: something that should have been fixed in advance gets decided by a result instead.

• Solution-first intake. Choosing a model, vendor, or architecture before naming the decision it is supposed to inform. The candidate becomes the frame instead of the output of one.
• Deferred stakes tier. Assigning the stakes tier only after a first result is inconvenient, so that the "right" tier is whichever one the result already survives. The tier is set from consequence (§5.1 fields 3/19/21), not from outcome.
• Placeholder-to-policy drift. A cold-start value (§5.5) treated as calibrated the moment it stops feeling arbitrary, without ever executing the pre-registered revision procedure that would have made that legitimate. The value did not become more true; it became less examined.
• Kill criteria written after the fact. A "we would have stopped if X" told after a disappointing pilot is not a kill criterion — it is a story. Kill criteria that were not written down before the pilot do not count (§5.3).
• Treating the intake questionnaire as a one-time form. Constraints and budgets change; a profile that is never revisited quietly drifts out of sync with the project it describes, and downstream chapters keep citing a stale field.

10. Vendor recipes#

[ADAPT: EXT-EVAL-002] OpenAI's eval-design ordering (objective → dataset → metrics → run → iterate) is adapted here one level up the stack: apply "objective before dataset" to the project, not only to a single evaluation suite — name the decision (§5.2) before any evaluation work begins. As of verification, the methodology content is current; the vendor's specific hosted tooling around it is being retired and is not what is being adopted here.

[ADAPT: EXT-EVAL-001] Anthropic's eval-design documentation independently opens with explicit, multidimensional criteria before construction — the same ordering this chapter's §5.2 formalizes as requirement → decision → claim.

[REFERENCE: NV-AGENTICBLOGS-001] Vendor practitioner guidance corroborates evaluate-first as the operating order for the most successful teams observed, not merely a recommended one — cited in full in chapter 00 P1; referenced again here because it is the ordering this chapter's whole procedure exists to protect at the intake boundary.

[ADAPT: EXT-OPS-002] The ML Test Score rubric's category structure (28 tests across four sections; score = minimum across categories, not an average) is previewed at intake as a shape of the obligations a project will eventually need to demonstrate, scaled by the stakes tier assigned here — not run as a self-audit yet. The rubric's operational use as a periodic self-audit is chapter 12's subject.

11. Worked examples#

All of these are invented illustrations rather than case evidence. What travels from them is the reasoning, never the numbers.

• SCENARIO-11 — a cheap, pre-registered diagnostic determined whether a large experiment should run at all, before any of its cost was spent; illustrates naming the decision and its evidence bar ahead of spend (§5.2, §5.6).
• SCENARIO-05 — a demand ledger and a pre-committed purchase trigger, set before the hardware was wanted, prevented a purchase that a later measurement showed would not have paid for itself; illustrates kill-criteria discipline applied to a capex decision (§5.3).
• WALKTHROUGH — a full navigator pass, profile through archetype through first-sprint contract, on a synthetic non-trivial project.
• SYNTH-01, SYNTH-02, SYNTH-08 — profile-to- archetype routing worked across three stakes tiers.

12. Outputs and artifacts#

• A completed templates/PROJECT_PROFILE.md — the ~20-field intake, all always-mandatory fields answered.
• An assigned stakes tier and a recorded archetype (§5.4).
• A constraints inventory (hard vs. soft) and the intake-level kill-criteria check (§5.3), dated.
• Any cold-start parameters set under §5.5, each carrying its pre-registered revision rule.
• The first-sprint commitment: a frozen templates/EXPERIMENT_CONTRACT.md (Tier 2+) or a written lightweight plan (Tier 1) — with any later milestones stated as gated, not scheduled (§5.6).

13. Sources#

Gap dispositions in this chapter: G2 (rigor sizing, shared with 00): COVERED — the profile capture point is §5.1 field 21 and the always-mandatory field set of §7; the tier definitions and artifact mapping themselves are normative in 00 §6. G4 (cold-start parameterization): COVERED — the five-step procedure and worked example are §5.5.