SkillVaultskills Browse all 1,000+ skills

Code Quality · Version 1.0.0 · Reviewed 2026-08-02

Reviewer Quality Evaluator

Make a defensible decision about reviewer finding evaluation and severity calibration audit with evidence, explicit trade-offs, and a verification plan.

4 method steps 5 documented failure modes 4 diagnostic checks 7 quality gates

Scores reviewer output for detection accuracy, severity calibration, actionability, scope discipline, and signal-to-noise. It grounds the decision in labeled changes, reviewer findings, accepted dispositions, severity definitions, and false-positive evidence and explicitly prevents rewarding finding volume or eloquence while incorrect and non-actionable comments burden maintainers.

₹299 one-time

Get this skill archive

Install in your AI coding tool

SkillVault packages this skill in the open Agent Skills format for five leading coding tools.

What this skill helps you do

  • Reviewer finding evaluation
  • Severity calibration audit
  • Review signal measurement

How Reviewer Quality Evaluator works

You provide

The code, its invariants, and how failures currently surface

It inspects

Error paths and lifetime handling for reviewer finding evaluation

It decides

A severity calibration audit change that makes invalid states unrepresentable

You verify

A deliberately invalid input fails clearly at the boundary

What it checks first

Reviewer Quality Evaluator scores reviewer output for detection accuracy, severity calibration, actionability, scope discipline, and signal-to-noise. It grounds the decision in labeled changes, reviewer findings, accepted dispositions, severity definitions, and false-positive evidence and explicitly prevents rewarding finding volume or eloquence while incorrect and non-actionable comments burden maintainers. Use it when the work involves Reviewer finding evaluation, Severity calibration audit, Review signal measurement.

  1. Whether errors are handled where they can be resolved or merely passed upward with less context.
  2. Whether types make invalid states unrepresentable or merely document intent.
  3. Ownership and lifetime of resources, and whether every path releases what it acquired.
  4. Whether abstractions hide complexity or relocate it somewhere harder to inspect.

Failure modes it recognizes

  • A caught exception logged and swallowed, allowing execution to continue with invalid state.
  • Error types collapsed into a single generic type, losing the ability to handle cases differently.
  • Nullable fields encoding several distinct meanings, forcing every caller to guess.
  • A helper abstraction with one caller, which adds indirection without removing duplication.
  • Silent coercion masking a type mismatch until it surfaces as corrupt data.

Answers it will reject

  • Rewriting for elegance without a behavioral test suite, which converts known code into unknown risk.
  • Adding a lint rule to enforce a pattern nobody has justified.
  • Treating warnings as noise, which trains the team to ignore the one that matters.

Decision rules it applies

  • Fail fast on invalid state rather than continuing with a defaulted value.
  • Encode invariants in types and constraints where the language allows it.
  • Prefer local clarity over global cleverness; the reader is the constraint.

Evidence it asks for

  • Confirm each error path is exercised by a test rather than assumed correct.
  • Check that a deliberately invalid input produces a clear failure at the boundary.
  • Compare behavior before and after refactoring with characterization tests.

The method inside

  1. Map the artifact, actors, boundaries, and invariants relevant to reviewer finding evaluation.
  2. Trace concrete failure or abuse paths for severity calibration audit; do not report checklist items without a mechanism.
  3. Prioritize review signal measurement findings by impact, likelihood, confidence, and cost of correction.
  4. Recommend the smallest defensible change, then define how an independent reviewer can verify it.

Deliverables

  • Reviewer finding evaluation assessment
  • Severity calibration audit decision and action plan
  • Review signal measurement verification checklist

Evidence requirements

  • Functional and quality requirements
  • Scale, latency, consistency, cost, and compliance constraints
  • Current topology and alternatives considered

Quality gates

  • Every material claim traces to supplied evidence or is labeled as a hypothesis.
  • The response follows the declared deliverable contract.
  • No execution, access, measurement, or verification is invented.
  • Secrets and personal data are redacted rather than repeated.
  • The user receives a concrete independent verification step.
  • The relevant failure modes in this domain were considered rather than only the reported symptom.
  • No listed anti-pattern was recommended as a solution.

Example task

Input

Apply the reviewer quality evaluator to our current reviewer finding evaluation work. We need a concrete decision, bounded changes, and evidence that the result is correct.

Expected output

Start with labeled changes, reviewer findings, accepted dispositions, severity definitions, and false-positive evidence. The highest-risk failure is rewarding finding volume or eloquence while incorrect and non-actionable comments burden maintainers. Weight correctness and reachability first, then actionability, calibration, and concise evidence. Verify the result by double-labeling a representative sample and reporting agreement, precision, recall, and severity error.

Boundaries and compatibility

Ideal for

  • Reviewer finding evaluation: produce a decision or artifact grounded in supplied evidence.
  • Severity calibration audit: produce a decision or artifact grounded in supplied evidence.
  • Review signal measurement: produce a decision or artifact grounded in supplied evidence.

Out of scope

  • Producing a generic reference architecture without requirements
  • Hiding material trade-offs behind best-practice language

Agent compatibility

  • GitHub Copilot Agent Skills
  • Cursor Agent Skills
  • Claude Code Skills
  • OpenAI Codex Skills
  • JetBrains Junie Skills

Tool policy: Advisory by default. No tools are assumed. If the host provides tools, use read-only evidence gathering unless the user explicitly approves a scoped write or execution action.