How to Evaluate Outputs (Simple Rubrics That Actually Work)

These prompts are only useful if you can consistently judge the quality of the output. Avoid binary “good/bad” thinking. Use a 1–5 scale with defined failure modes.

Each prompt has a primary evaluation metric. If you don’t enforce it, the prompt degrades.

You are not scoring style - you are scoring decision usefulness.

Foundation Check: Grounding (Hallucinated Logic)

Rule:

If the model introduces constraints, facts, or blockers not present in the input and does not label them as assumptions, → cap the score at 2

Allow:

• “Assuming budget is fixed…”

• “If stakeholder alignment is weak…”

Fail:

• “Given regulatory constraints…” (not provided)

• “Because the system cannot scale…” (not evidenced)

1. Objective Validity Test

What you are checking: Did the model genuinely challenge whether the project should exist?

Rubric (1–5):

5 - Board-level challenge

• Identifies at least one hidden assumption (e.g. “market demand is assumed, not evidenced”)

• Clearly separates real problem vs perceived problem

• Defines a specific “kill condition” (e.g. “If adoption <15% after 3 months, project is irrational”) - Does the output identify the Leading vs. Lagging indicators for that kill condition?

3 - Partial interrogation

• Covers the five perspectives but stays descriptive

• “Against” argument is weak or generic

• No clear threshold for cancellation

1 - Superficial validation

• Accepts the premise

• Provides generic pros/cons

• No meaningful challenge to existence

2. Pre-Mortem Architect

What you are checking: Are the failure scenarios specific and detectable early?

Rubric (1–5):

5 - Actionable failure design

• Three distinct root causes (technical / organisational / external)

• Each includes a weeks 1–4 leading indicator, e.g. “Sprint velocity drops by 30% in first 2 sprints”

• Explicitly links failure to a blinding assumption

3 - Plausible but vague

• Failure scenarios are realistic but overlap

• Early signals are generic (“team struggles”, “delays appear”)

1 - Narrative hindsight

• Reads like a post-mortem story

• No early warning signals

• No link to original assumptions

3. Red Team Brief

What you are checking: Did the model genuinely try to defeat the plan? In other words, Primary Metric: Adversarial Integrity → Did the model remain purely critical, or did it drift into solution mode?

Rubric (1–5):

5 - Full adversarial integrity

• No solutions offered (constraint respected)

• Objections attack core strategic logic

• Data gaps undermine decision confidence, not just completeness

• “Worse than doing nothing” scenario is credible

• Final question is board-level unanswerable

3 - Partial drift

• Strong critique, but includes soft suggestions or “fixes”

• Some objections are surface-level

• Final question is answerable with preparation

1 - Constraint failure

• Slips into solution mode

• Critique is diluted or polite

• No real pressure on the plan

4. Assumption Audit

What you are checking: Did it expose what the plan is quietly relying on? In other words, Primary Metric: Epistemic Honesty → Did the model distinguish facts from assumptions?

Rubric (1–5):

5 - High epistemic clarity

• Identifies load-bearing assumptions across categories

• Explicitly flags assumptions treated as facts

• Fragility ratings are justified (not arbitrary)

• Rewritten objective reflects uncertainty, e.g. “Test whether X is viable” vs “Deliver X”

3 - Partial honesty

• Lists assumptions but lacks prioritisation

• Fragility ratings are weak or generic

• Rewrite improves wording, not truthfulness

1 - Epistemic failure

• Accepts objective at face value

• Misses key assumptions

• No distinction between known vs assumed

Cross-Cutting Tool: Writer–Editor (Evaluation Layer)

Use Writer–Editor when:

• You have multiple outputs (or multiple model responses)

• You need forced prioritisation without hedging

Prompt (adapted for rubric use):

“Act as a Business Consultant. Rank these outputs from best to worst based on:

• adherence to the defined rubric

• decision usefulness

• internal consistency

No hedging. If two outputs are not meaningfully distinguishable, mark them as ‘non-separable’ rather than forcing a ranking. Score confidence in your ranking (1–5) and explain the limiting uncertainty.”

How to Use It (Practical)

Run 2–3 model outputs (or variations)

Apply the rubric (e.g. Adversarial Integrity)

Then use Writer–Editor to:

• rank outputs

• expose weak reasoning

• detect false precision

Disagreement = Signal (Use It Correctly)

If outputs rank differently or are “non-separable”:

• Ambiguity in inputs → run Assumption Audit

• Different model priors → compare reasoning, not conclusions

• Weak rubric enforcement → tighten criteria

Do not average answers. Resolve the cause.

Final Constraint (Non-Negotiable)

A high rubric score means: → “good relative to intent”

It does not mean: → “correct in reality”

Human expert judgement remains the final gate.

No rubric or model can validate real-world fit without it.

Practical Guidance (Don’t Skip This)

• Run these rubrics sparingly (e.g. monthly, post-failure, or pre-major decision). This is quality assurance, not daily workflow

• Do not rely on the model to self-certify reality. A high score ≠ real-world validity. These tools expose weaknesses; they do not replace domain expertise.

What “Good” Looks Like (Quick Example)

Weak output:

“Market demand may be uncertain; consider validating with users.”

Strong output:

“The plan assumes repeat usage without evidence. If retention is below 20% after 6 weeks, projected ROI collapses - making the project irrational.”

→ That’s the difference between analysis and decision support