146 lines
4.8 KiB
YAML
146 lines
4.8 KiB
YAML
name: slop_gate
|
|
description: "AI-slop detection gate -- runs after chapter_production to catch AI fingerprinting. Fails the gate if aggregate AI score is too high; routes to human_polish_brief if flagged."
|
|
debug: false
|
|
model: power
|
|
system: agent_prompt
|
|
agent_prompt:
|
|
- "= identity.md"
|
|
sections:
|
|
- agent
|
|
- project
|
|
- rag
|
|
- message
|
|
- instructions
|
|
steps:
|
|
# Step 0: load the chapter content from staging
|
|
- type: tool
|
|
action: git_read_file
|
|
optional: false
|
|
output_key: chapter_text
|
|
params:
|
|
path: "staging/draft/{chapter.slug}.md"
|
|
|
|
# Step 1: run AI fingerprint detection via Tool.SlopDetector
|
|
- type: tool
|
|
capability: Tool_SlopDetector
|
|
optional: false
|
|
output_key: ai_detection_result
|
|
params:
|
|
action: "detect_ai_fingerprint"
|
|
text: "{chapter_text}"
|
|
provider_pool: ["local_heuristic"]
|
|
|
|
# Step 2: run style diversity analysis
|
|
- type: tool
|
|
capability: Tool_SlopDetector
|
|
optional: true
|
|
output_key: style_analysis_result
|
|
params:
|
|
action: "analyse_style_diversity"
|
|
text: "{chapter_text}"
|
|
|
|
# Step 3: load the author voice signature for comparison
|
|
- type: tool
|
|
action: git_read_file
|
|
optional: true
|
|
output_key: voice_signature
|
|
params:
|
|
path: "agents/{agent.name}/voice_signature.json"
|
|
|
|
# Step 4: evaluate the detection results
|
|
- type: think
|
|
max_tokens: 800
|
|
output_key: gate_evaluation
|
|
hint: |
|
|
You are {agent.name} evaluating a slop gate check for a chapter from {project.name}.
|
|
|
|
AI detection result: {ai_detection_result}
|
|
Style diversity analysis: {style_analysis_result}
|
|
Author voice signature: {voice_signature}
|
|
|
|
Rules:
|
|
- If aggregate_score >= 75: the chapter has strong AI fingerprinting. Flag it.
|
|
- If aggregate_score >= 50 and sentence_length_variance < 5.0: mixed signal. Flag it.
|
|
- If aggregate_score < 50 and opening_word_repetition_rate < 0.3: PASS.
|
|
- If voice_signature is available and pacing_profile or register is very different
|
|
from the signature, note the drift even if score is below threshold.
|
|
|
|
Output one of:
|
|
GATE_STATUS: PASS
|
|
GATE_STATUS: FLAG
|
|
|
|
Then provide 2-3 sentences explaining the decision.
|
|
If flagging, list 2-3 specific signals that triggered the flag (e.g., sentence
|
|
length variance, AI fingerprint score, opening word repetition).
|
|
|
|
# Step 5: extract the gate status
|
|
- type: think
|
|
max_tokens: 20
|
|
output_key: gate_status
|
|
hint: |
|
|
Read the GATE_STATUS line from the evaluation above.
|
|
Output ONLY the word: PASS or FLAG. Nothing else.
|
|
|
|
# Step 6: if PASS -- close and log
|
|
- type: branch
|
|
condition: "{gate_status} == PASS"
|
|
target: close_gate
|
|
|
|
# Step 7: if FLAG -- produce human polish brief and halt chapter for review
|
|
- type: think
|
|
max_tokens: 1200
|
|
output_key: polish_brief
|
|
hint: |
|
|
The slop gate has flagged this chapter. You are writing a Human Polish Brief --
|
|
a set of specific, actionable instructions for a human editor to revise this chapter.
|
|
|
|
Base your brief on:
|
|
- The AI detection signals identified in the evaluation
|
|
- The style diversity analysis (which sentence patterns are too uniform?)
|
|
- The voice signature drift (if any)
|
|
- The actual chapter text
|
|
|
|
Your brief must include:
|
|
1. WHAT TRIGGERED THE FLAG: the specific signals (be numerical where possible)
|
|
2. THE CORE PROBLEM: one sentence describing the main issue (e.g., "Every paragraph
|
|
begins with a pronoun and has nearly identical sentence length.")
|
|
3. REVISION TARGETS: 4-6 specific passages or patterns that need human rewriting.
|
|
Quote the actual text. Say what makes it feel AI-generated.
|
|
4. VOICE GUIDANCE: what the author's authentic voice should feel like for this
|
|
project and chapter. Reference the voice signature if available.
|
|
5. ACCEPTANCE CRITERIA: what the chapter needs to score on re-check to pass the gate.
|
|
|
|
Output ONLY the brief text. No preamble.
|
|
|
|
- type: document
|
|
source_step: polish_brief
|
|
filename: "slop_gate_flag_{task.id}"
|
|
dest_path: "staging/flags"
|
|
commit_msg: "slop_gate: flag {chapter.slug} for human polish -- task {task.id}"
|
|
|
|
- type: reply
|
|
target: channel
|
|
channel_name: "crimson_leaf_publishing:live-feed"
|
|
hint: |
|
|
Post: "Slop gate FLAGGED {chapter.slug} -- human polish required. Brief saved to
|
|
staging/flags/. AI score: {ai_detection_result.aggregate_score}."
|
|
|
|
- type: close
|
|
rag_update: false
|
|
|
|
# Gate pass path
|
|
- type: label
|
|
name: close_gate
|
|
|
|
- type: reply
|
|
target: channel
|
|
channel_name: "crimson_leaf_publishing:live-feed"
|
|
hint: |
|
|
Post: "Slop gate PASSED {chapter.slug} -- AI score within acceptable range."
|
|
|
|
- type: close
|
|
rag_update: false
|
|
|
|
adjudication:
|
|
enabled: false
|