Full SKILL.md

Hypothesis Stress Testing

Builds four to six rival falsifiable hypotheses about what is true now, scores them against sourced evidence in Heuer's Analysis of Competing Hypotheses matrix, and delivers a refutation-first analysis plan plus a surviving hypothesis with stated confidence, linchpin evidence and tripwires. Use whenever a team has settled early on one explanation, or the user says "stress-test my thinking", "am I fooling myself", "which explanation fits the data", "poke holes in this", "we think the cause is X", "is it really X", "what would prove us wrong", "how confident should we be". Not for how a chosen plan might fail (a pre-mortem), external futures or rival moves (war-gaming), or choosing among strategic options (strategic-choice).

Takes a day-one answer and its rivals, lays the evidence against all of them at once, and reports which survive and why. Heuer's eight steps and Rasiel and Friga's quick-and-dirty test are in references/frameworks.md; read it only when a step needs defending.

Routing

If the user's sentence contains "the plan", "the launch", "should we go ahead", it is not this skill. If it contains "the cause", "our thesis", "is it true that", it is.

Inputs

Ask for, or infer and label: the problem statement (decision-maker, question, success criteria, scope, accuracy); the issue tree or prioritised branches (build a minimal one in Stage 1 if absent); the current belief verbatim and who holds it (it becomes H1); every fact in hand with source and date, testimony tagged as testimony; analysis budget (people, weeks, data access, date for a tentative answer); anything already ruled out and why.

Ask one round of questions if the question or the evidence set is missing. Otherwise proceed, and put assumed inputs in one block after the surviving hypothesis, flagging which would change the set if wrong.

Workflow

  1. Day-one answer. The question as the decision-maker asks it; the team's best answer in one or two sentences with a number ("margin fell 230 bps mainly because national accounts, ~10 points lower margin, grew from 31% to 39% of revenue"). Record who believes it and how strongly; disagreement is free rival hypotheses. Done when the answer contains something evidence could contradict. Test: if wrong, what would we have seen that we have not looked for?

  2. Rival set. Four to six, each defensible by a reasonable colleague. Sources: prioritised branches rewritten as claims; perspective-taking (CFO, sales head, ops head, auditor, hostile competitor); the boring ones teams skip (measurement artefact, timing, one-off, null). At least one artefact or null. Each hypothesis has six elements (references/templates.md §2): subject, direction and magnitude, mechanism, time window, scope, killing observation. Done when each names a smart person who would bet on it and the observation that kills it.

  3. Evidence ledger (references/templates.md §3). Per hypothesis, what should be visible if true; then every relevant fact as a row: ID, source, date, reliability (audited, management estimate, single interview, team assumption). Include absences (the dog that did not bark) and the team's own assumptions. Testimony separate from measurement. Do not filter for the favoured hypothesis. Done when 8–15 rows exist with at least one absence and one assumption, and some rows came from looking for what rivals predict.

  4. Matrix. Hypotheses across, evidence down. One question per cell: if this hypothesis were true, how likely would I see this? Scale: CC strongly consistent (H predicts it); C consistent (fits, not required); N no bearing; I inconsistent in direction, magnitude, timing or scope; II strongly inconsistent. Work row by row across all hypotheses. Check magnitude, not just direction. One-sentence justification for every I and II. Rate each row's diagnosticity: High separates sharply; Medium separates some; Low reads C or N across. Test: cover the hypothesis names; can you tell the columns apart from the ratings?

  5. Refine. Demote Low rows to an appendix. Identical columns: merge, or add the separating evidence to the plan. Strongly contradicted hypotheses: split ("mix explains all" dead, "mix explains a third" alive). Add hypotheses the evidence suggested. Test: any reworded hypothesis is a sharpening, not a retreat to unfalsifiability.

  6. Eliminate. Rank by reliable evidence against, least first; consistent-cell counts are not the criterion. One audited contradiction outweighs three interview impressions. Say plainly if the day-one answer is losing and which row did the damage. Where the answer is a decomposition, the parts reconcile to the observed effect within stated tolerance. Test: which hypothesis is refuted, by which row, and would the decision-maker accept that row?

  7. Refutation-first plan (references/templates.md §7). For each live hypothesis: conditions that must hold; known or unknown; for each unknown the analysis, source, owner, due date and the specific result that kills it, stated in advance. Prefer analyses that discriminate between two live hypotheses at once. Sequence by information per day of cost. Done when every live hypothesis has a pre-stated refutation threshold and the plan fits the budget. Test: what result would make us abandon the leader? None means the analysis is decoration.

  8. Sensitivity. For the I and II cells of the top two hypotheses: what if this evidence is wrong, misread, incomplete? Name the vulnerability (an ERP segment tag, a controller's memory, one vendor file) and the cheapest confirming check; for numbers, how far they would have to move to flip the ranking. Record linchpin rows. Test: if E6 were wrong tomorrow, what would we conclude?

  9. Report. All hypotheses with standing, not just the winner; confidence as a band with rough probability and reasons (linchpin count, reliability, residual unexplained); linchpin evidence; at least two tripwires, observable, dated, owned ("if core margin does not recover 40 bps within two quarters of closing the update lag, H3 is weaker than concluded"). Test: would the decision-maker act differently at 60% vs 85%?

Output

Surviving hypothesis and confidence in the first 150 words. Then, each once, 1,500 to 3,500 words:

  1. Surviving-hypothesis report (one page: question, standing of every hypothesis, confidence with reasons, linchpins, tripwires)
  2. Hypothesis set in falsifiable form
  3. ACH matrix with diagnosticity column and I/II justifications
  4. Elimination table (and decomposition where the answer is shared)
  5. Refutation plan
  6. Sensitivity check
  7. Assumptions made in the absence of input, if any

No failure modes, mitigations or kill criteria for a plan; no competitor scenarios; those belong elsewhere. No method vocabulary, framework authors, skill names or the worked example's client in the deliverable. Full example: references/worked-example.md.

Quality bar

Run silently; fix, do not report.

  1. Every hypothesis has all six elements and a killing observation.
  2. At least one artefact or null hypothesis; no straw men.
  3. Ledger has an absence row and an assumption row; testimony is tagged.
  4. Every I and II cell has a justification; every row a diagnosticity rating.
  5. Ranking is by inconsistency, not consistency count.
  6. Decompositions reconcile to the total.
  7. Every planned analysis has a pre-stated refutation result.
  8. Linchpin rows named with vulnerability and check.
  9. Confidence statement changes what the decision-maker would do.
  10. Tripwires are observable, dated and owned.
  11. No leaked method vocabulary.

Anti-patterns

  • One serious hypothesis and two straw men.
  • Themes posing as hypotheses ("pricing discipline has deteriorated").
  • Working the matrix column by column.
  • Rating direction and ignoring magnitude.
  • Counting consistent cells as support.
  • Rewriting a losing hypothesis until nothing can touch it.
  • An analysis plan built to "understand the business" rather than to kill rivals.
  • Reporting only the winner.

Hand-offs

References

  1. Frameworks: hypothesis-driven problem solving and Analysis of Competing Hypotheses

    4,531 w
    references/frameworks.md11 sub-sections
  2. Templates and checklists: hypothesis stress-testing

    1,643 w
    references/templates.md15 sub-sections
  3. Worked example: Northgate Industrial Supply, gross margin decline

    3,818 w
    references/worked-example.md11 sub-sections