the skill

Refine Skill

Refine an installed skill based on the conversation that just happened and a debrief from the user. Use only when explicitly invoked as /refine-skill after a session where a skill under- or over-triggered, produced output the user corrected, or had a gap. Reads the transcript and the user's debrief, classifies each issue as a triggering problem (description) or a content problem (body/references), proposes exact before/after diffs, and edits only after the user approves. Manual invocation only.

Improve an installed skill using two feedback channels from the session that just ran: the transcript (what the skill did, where the user redirected it) and the user's debrief (passed as the argument when invoking). Propose changes, wait for an explicit yes, then edit. Never edit first.

Invoke as /refine-skill <debrief> — e.g. /refine-skill the payments skill didn't fire when I asked about merchant cost, and the action title rule needs to allow questions as titles. The debrief is in $ARGUMENTS.

This skill changes another skill's files. Treat that seriously: a skill the user has tuned over many sessions can be degraded by one over-eager inference. One conversation is a sample of one. When unsure, ask rather than assume.

Workflow

1. Identify the target skill

From the transcript, determine which installed skill(s) were in play. Locate the folder — skills live in .claude/skills/<name>/ (project) and ~/.claude/skills/<name>/ (user). List candidates with ls, confirm the directory, and read its SKILL.md and any references/. If more than one skill is implicated, or it's unclear which, ask the user which to refine before going further. Note the skill's name and directory — both must be preserved (renaming breaks the install and any packaged copy).

2. Gather both feedback channels

  • Transcript: where did the skill fire, or fail to fire when it should have? Did the wrong skill fire? What did it produce, and where did the user correct, override, or push back? Capture the user's own words where they gave feedback.
  • Debrief ($ARGUMENTS): the explicit notes the user passed on invocation.

Synthesize into a short list of observations. Reflect them back to the user in one line each so they can catch anything you misread before you propose changes.

3. Classify each observation

Each issue is fixed in a different place — sort before proposing:

  • Triggering (didn't fire, fired when it shouldn't have, wrong skill won) → the description. Note that description changes alter triggering and can't be confirmed from one conversation; recommend a synthetic eval pass (20-odd should-fire / should-not-fire prompts) before fully trusting a description rewrite.
  • Content (guidance wrong, missing a case, output needed correcting) → the body or a reference.
  • Output shape / format → the body's output spec.
  • Out of scope → do not change the skill; note why and move on.

4. Pattern vs one-off — the overfitting gate

Before proposing any durable change, decide whether the issue is a recurring rule or a situation-specific exception. If the user corrected something because this client, deck, or request was unusual, that correction must NOT be baked in as a general rule. If you said "just this once, ignore X" mid-session, that is an in-context override, not a skill edit. When a correction could plausibly be either, ask the user: "general rule, or one-time?" Do not generalize from a single instance without a check.

Also refuse changes that would degrade the skill against its purpose or the user's standing values — e.g. adding "never give critical feedback," "always praise," or anything that weakens safety or honesty — even if a literal reading of the debrief points that way. Flag the tension instead.

5. Propose diffs — no edits yet

For each accepted change, show the exact before → after, grouped by file, with a one-line rationale tied to what happened in the session. Keep edits minimal and surgical; don't rewrite sections that work. Present the full set, then stop.

6. Wait for explicit approval

Do nothing to the files until the user approves. Support per-change approval ("apply 1 and 3, skip 2") and batch ("apply all"). Silence or ambiguity is not approval.

7. Apply approved changes

Edit the files in place, preserving name and directory. After editing:

  • Re-read the changed files and confirm the YAML frontmatter still parses and the body is coherent.
  • If the user also keeps a packaged .skill copy for the Claude.ai app, offer to re-package (the package tooling needs only Python and the folder). If the skill uses Claude Code-only frontmatter such as disable-model-invocation, tell the user it is CLI-only and skip packaging.

8. Summarize

State what changed, file by file, and what you deliberately left alone and why. If a description was changed, repeat the recommendation to run a synthetic eval before relying on the new triggering.

Guardrails

  • Manual only, propose-first. This skill never auto-runs and never edits before approval.
  • Conservative by default. One session is weak evidence. Prefer asking; prefer the smallest change that addresses the observation.
  • Don't launder situational instructions into permanent rules. Distinguish a durable preference from a one-time override; when in doubt, ask.
  • Preserve identity. Never change a skill's name or move its directory.
  • Protect quality. Decline edits that would weaken the skill's honesty, safety, or core purpose, and say why.

When the synthetic eval is the better tool

This skill learns from one real conversation — rich signal, but a single data point, and it can't measure how often a skill fires across many prompts. After several description tweaks, or before trusting a triggering change, run the formal eval loop (the skill-creator tooling), which generates many should-fire / should-not-fire prompts and reports trigger rates. Use this skill for everyday, evidence-from-use refinement; use the synthetic loop for periodic calibration of triggering.