Strengthen chapter review quality gates
- chapter_review.yml: mandatory PROSE EVIDENCE section (3-5 verbatim quotes) - CHARACTER VOICE AUDIT with per-constraint checks and violation quoting - Score anchors (95-100/85-94/70-84/<70) to break rubber-stamp clustering - MUST-FIX items require ORIGINAL quote + PROBLEM + FIX rewrite - Adjudication: prose_evidence(35) specificity(30) voice_audit(20) coverage(15) - chapter_roundtable.yml: require quoted evidence, add character_profiles check
This commit is contained in:
@@ -1,7 +1,8 @@
|
||||
name: chapter_review
|
||||
description: >
|
||||
Independent chapter review. Agent reads the chapter and produces a structured
|
||||
critique in their editorial domain with explicit must-fix vs optional categorization.
|
||||
critique with mandatory verbatim prose quotes, score-anchored verdict, and
|
||||
explicit character voice violation checks.
|
||||
debug: true
|
||||
rag_exclude: [business_plan, charter]
|
||||
model: power
|
||||
@@ -18,7 +19,7 @@ sections:
|
||||
|
||||
steps:
|
||||
- type: think
|
||||
max_tokens: 4000
|
||||
max_tokens: 5000
|
||||
hint: |
|
||||
TARGET AUDIENCE: {genre_audience}
|
||||
GENRE: {genre_name}
|
||||
@@ -29,54 +30,82 @@ steps:
|
||||
{chapter_text}
|
||||
---
|
||||
|
||||
Write a detailed editorial review from your perspective.
|
||||
Be specific — cite line numbers or quote passages where relevant.
|
||||
Structure your review using EXACTLY these six labeled sections:
|
||||
Write a detailed editorial review. You MUST quote the actual text to support every
|
||||
claim you make. Vague commentary ("the pacing drags") without a quoted passage is
|
||||
not acceptable and will cause this review to fail adjudication.
|
||||
|
||||
1. STRENGTHS TO PRESERVE
|
||||
List elements that are working well and must survive the editorial process unchanged.
|
||||
Be specific: quote or paraphrase the passage. Vague praise ("the writing is good")
|
||||
is not useful here.
|
||||
ALSO CHECK VOICE SIGNATURES:
|
||||
CHARACTER VOICE PROFILES:
|
||||
Structure your review using EXACTLY these seven labeled sections:
|
||||
|
||||
1. PROSE EVIDENCE
|
||||
Quote 3-5 passages verbatim from this chapter (use quotation marks and indicate
|
||||
approximate position: "early", "mid", "late").
|
||||
For each quote, give one sentence of inline commentary: what it does well or
|
||||
what it fails to do. This section is MANDATORY -- do not summarize or paraphrase.
|
||||
A review with zero verbatim quotes will automatically score below 70.
|
||||
|
||||
2. CHARACTER VOICE AUDIT
|
||||
{character_profiles}
|
||||
If the above block is non-empty: verify each named character in this chapter
|
||||
speaks to their profile. Can you identify each character's dialogue without
|
||||
speaker tags? Note YES or NO for each main character.
|
||||
If the block is empty: check RAG/project context for "## Voice Signatures"
|
||||
and note whether voice differentiation is present.
|
||||
If the above block is non-empty: for EACH named character who speaks in this
|
||||
chapter, check their voice against their profile. Quote one line of their
|
||||
dialogue and state YES or NO for each constraint:
|
||||
- Does the character use their signature vocabulary / verbal tics?
|
||||
- Do they avoid any explicitly forbidden speech patterns?
|
||||
- Is their emotional register consistent with their arc position?
|
||||
If a violation is found, quote the offending line and state exactly what rule
|
||||
is broken (e.g. "Dorian uses contraction 'don't' -- profile forbids contractions
|
||||
unless in extreme pain or physical exhaustion. This scene does not qualify.").
|
||||
If the block is empty: check RAG context for "## Voice Signatures" and report
|
||||
what you find.
|
||||
|
||||
2. MUST-FIX — CONTINUITY
|
||||
3. STRENGTHS TO PRESERVE
|
||||
List 2-4 elements working well that must survive the editorial process unchanged.
|
||||
Each item must include a verbatim quote or specific scene reference.
|
||||
Generic praise ("the atmosphere is strong") without evidence is not allowed here.
|
||||
|
||||
4. MUST-FIX -- CONTINUITY
|
||||
List factual errors, POV breaks, timeline inconsistencies, or world-rule violations.
|
||||
These are non-negotiable — they cannot be left as "optional." For each item, state:
|
||||
- What the error is
|
||||
- What the correction should be
|
||||
These are non-negotiable. For each item state:
|
||||
- ORIGINAL: quote the offending passage verbatim
|
||||
- PROBLEM: what rule or established fact it violates
|
||||
- FIX: the concrete correction (rewrite the sentence if necessary)
|
||||
|
||||
3. MUST-FIX — CLARITY
|
||||
5. MUST-FIX -- CLARITY
|
||||
List passages where meaning is genuinely obscured, transitions are dropped, or
|
||||
threads are left dangling in a way that blocks reader comprehension. For each item:
|
||||
- Quote or reference the passage
|
||||
- State the concrete fix
|
||||
- ORIGINAL: quote the offending passage verbatim
|
||||
- PROBLEM: why it blocks comprehension
|
||||
- FIX: the concrete correction
|
||||
|
||||
4. OPTIONAL SUGGESTIONS
|
||||
6. OPTIONAL SUGGESTIONS
|
||||
Craft improvements that would strengthen the chapter but are NOT required for it to
|
||||
pass. Label each suggestion clearly as optional. Do not inflate this section — only
|
||||
include suggestions with a clear upside and low risk of voice damage.
|
||||
⚠️ Do NOT suggest adding thematic contrast metaphors, making dialogue more elegant,
|
||||
or smoothing sentence variation — these are voice PRESERVATION issues, not fixes.
|
||||
pass. Label each suggestion clearly as optional. Include the relevant quote.
|
||||
Do not inflate this section -- only include suggestions with a clear upside and low
|
||||
risk of voice damage.
|
||||
Do NOT suggest adding thematic contrast metaphors, making dialogue more elegant,
|
||||
or smoothing sentence variation -- these are voice PRESERVATION issues, not fixes.
|
||||
|
||||
5. FORBIDDEN CHANGES / NON-GOALS
|
||||
List things that might appear to be problems but should NOT be changed — intentional
|
||||
voice choices, genre conventions, structural decisions that are working. This section
|
||||
protects the draft from over-editing.
|
||||
ALWAYS list here: verbal tics, repeated phrases, and "imperfect" speech that is
|
||||
7. FORBIDDEN CHANGES / NON-GOALS
|
||||
List things that might appear to be problems but should NOT be changed -- intentional
|
||||
voice choices, genre conventions, structural decisions that are working.
|
||||
ALWAYS include: verbal tics, repeated phrases, and "imperfect" speech that is
|
||||
clearly a character signature, not an error.
|
||||
|
||||
6. VERDICT
|
||||
8. VERDICT
|
||||
One of: PASS / REVISE
|
||||
SCORE ANCHORS -- be honest, scores that cluster at 90-96 without strong evidence
|
||||
signal a rubber-stamp review and will be rejected by the adjudicator:
|
||||
95-100: Zero MUST-FIX items. PROSE EVIDENCE quotes show above-average craft.
|
||||
Voice audit finds no violations.
|
||||
85-94: 1-2 minor MUST-FIX items. At least one PROSE EVIDENCE quote shows
|
||||
a clear weakness with explanation.
|
||||
70-84: Clear structural or voice problems. Every MUST-FIX item is quoted
|
||||
verbatim with a rewrite suggestion.
|
||||
Below 70: Multiple serious violations. Every failing passage is quoted and
|
||||
corrected. Automatic REVISE verdict.
|
||||
|
||||
State your score and a one-sentence justification citing your evidence.
|
||||
Use PASS if only optional improvements remain.
|
||||
Use REVISE if any MUST-FIX items are present.
|
||||
Reserve judgment on full rewrites — that decision belongs to the roundtable.
|
||||
|
||||
- type: document
|
||||
filename: "Chapter_{chapter_number}_review_{review_letter}"
|
||||
@@ -89,12 +118,16 @@ adjudication:
|
||||
pass_threshold: 60
|
||||
deliverable_type: editorial_review
|
||||
criteria:
|
||||
specificity:
|
||||
weight: 40
|
||||
description: "Review cites specific passages, scenes, or structural elements in all sections — not vague general commentary"
|
||||
actionability:
|
||||
prose_evidence:
|
||||
weight: 35
|
||||
description: "Each MUST-FIX item includes a concrete, implementable correction; OPTIONAL items are clearly labeled"
|
||||
description: "Review contains 3+ verbatim quoted passages with inline commentary. Zero quotes = automatic fail. Paraphrases do not count as quotes."
|
||||
specificity:
|
||||
weight: 30
|
||||
description: "Every MUST-FIX item includes the original passage quoted verbatim AND a concrete rewrite or correction. Generic commentary without quotes scores zero here."
|
||||
voice_audit:
|
||||
weight: 20
|
||||
description: "CHARACTER VOICE AUDIT section checks each speaking character against their profile constraints. Each violation is quoted and the broken rule named explicitly."
|
||||
coverage:
|
||||
weight: 25
|
||||
description: "All six sections are present; STRENGTHS and FORBIDDEN sections are populated, not left empty"
|
||||
weight: 15
|
||||
description: "All eight sections present and populated. STRENGTHS section includes evidence. FORBIDDEN section is not left empty."
|
||||
|
||||
|
||||
Reference in New Issue
Block a user