SkillVaultskills Browse all 1,000+ skills

AI Engineering · Version 1.1.0 · Reviewed 2026-08-02

AI Skill Portfolio Auditor

Make AI behavior measurable and safer for skill inventory analysis and trigger collision detection with evidence, explicit trade-offs, and a verification plan.

4 method steps 5 documented failure modes 5 diagnostic checks 7 quality gates

Audits a skill collection for duplicated jobs, trigger collisions, missing coverage, inconsistent quality, and stale knowledge. It grounds the decision in skill frontmatter, descriptions, triggers, workflows, evaluation results, usage, versions, and ownership and explicitly prevents evaluating files independently and missing that two good skills compete for the same request or contradict each other.

₹199 one-time

Get this skill archive

Install in your AI coding tool

SkillVault packages this skill in the open Agent Skills format for five leading coding tools.

What this skill helps you do

  • Skill inventory analysis
  • Trigger collision detection
  • Portfolio coverage gap review

How AI Skill Portfolio Auditor works

You provide

The skill files, sibling catalog, host rules, observed failures, and evaluation cases

It inspects

Trigger overlap, authority, context cost, and enforcement paths for skill inventory analysis

It decides

A trigger collision detection change with regression cases and destination constraints

You verify

The installed portfolio routes a labeled intent corpus correctly and every declared reference resolves

What it checks first

AI Skill Portfolio Auditor audits a skill collection for duplicated jobs, trigger collisions, missing coverage, inconsistent quality, and stale knowledge. It grounds the decision in skill frontmatter, descriptions, triggers, workflows, evaluation results, usage, versions, and ownership and explicitly prevents evaluating files independently and missing that two good skills compete for the same request or contradict each other. Use it when the work involves Skill inventory analysis, Trigger collision detection, Portfolio coverage gap review.

  1. Trigger precision against neighboring skills, because an excellent method is useless when the wrong requests invoke it.
  2. Whether the skill encodes stable decision knowledge or merely restates a host operation that belongs in a tool or script.
  3. The complete authority chain from instruction to tool call, including confirmation requirements and failure propagation.
  4. Runtime context cost from always-loaded instructions, references, examples, and sibling overlap.
  5. Evaluation coverage for positive, negative, ambiguous, missing-evidence, and conflicting-skill requests.

Failure modes it recognizes

  • Two skills claim the same intent and route nondeterministically depending on superficial wording.
  • A role-style prompt describes expertise but provides no ordered method, refusal boundary, or verifiable deliverable.
  • Detailed references load on every invocation and crowd out the user evidence needed for the actual decision.
  • A skill assumes tools or permissions the host does not provide and converts unavailable execution into confident prose.
  • A narrow failure is patched with another exception until the skill contains contradictory routing and behavior rules.

Answers it will reject

  • Adding every observed failure example to the primary file instead of fixing the shared mechanism or evaluation gap.
  • Combining unrelated jobs into one large skill because they happen to use the same tool.
  • Claiming a skill is portable while hard-coding one host command, repository layout, or authentication model.
  • Judging quality from one successful demonstration without testing sibling routing and negative cases.

Decision rules it applies

  • Split skills when their triggers, evidence, authority, or deliverables differ; share references only when the decision method is genuinely common.
  • Keep routing, safety, and the minimal workflow in the primary file, loading detailed knowledge only when the task requires it.
  • Promote repeated and stable know-how into a skill; leave one-off execution details in a task plan or script.
  • A skill change is incomplete until the original failure becomes a regression case and previously passing cases remain protected.

Evidence it asks for

  • Run a labeled intent corpus through the complete installed skill set and record selected, missed, and competing routes.
  • Measure primary-file and loaded-reference token counts before and after structural changes.
  • Validate every referenced path, declared tool, and output schema from a clean installation.
  • Execute positive, negative, ambiguous, and tool-failure evaluation cases against the final packaged revision.

The method inside

  1. Map the artifact, actors, boundaries, and invariants relevant to skill inventory analysis.
  2. Trace concrete failure or abuse paths for trigger collision detection; do not report checklist items without a mechanism.
  3. Prioritize portfolio coverage gap review findings by impact, likelihood, confidence, and cost of correction.
  4. Recommend the smallest defensible change, then define how an independent reviewer can verify it.

Deliverables

  • Skill inventory analysis assessment
  • Trigger collision detection decision and action plan
  • Portfolio coverage gap review verification checklist

Evidence requirements

  • Prompts, model/version, tools, retrieval path, and examples
  • Evaluation dataset and failure cases
  • Latency, cost, privacy, and policy constraints

Quality gates

  • Every material claim traces to supplied evidence or is labeled as a hypothesis.
  • The response follows the declared deliverable contract.
  • No execution, access, measurement, or verification is invented.
  • Secrets and personal data are redacted rather than repeated.
  • The user receives a concrete independent verification step.
  • The relevant failure modes in this domain were considered rather than only the reported symptom.
  • No listed anti-pattern was recommended as a solution.

Example task

Input

Apply the ai skill portfolio auditor to our current skill inventory analysis work. We need a concrete decision, bounded changes, and evidence that the result is correct.

Expected output

Start with skill frontmatter, descriptions, triggers, workflows, evaluation results, usage, versions, and ownership. The highest-risk failure is evaluating files independently and missing that two good skills compete for the same request or contradict each other. Assess routing and coverage at portfolio level, then rank consolidation, specialization, and retirement actions. Verify the result by running a shared intent corpus through the full portfolio and measuring ambiguous, missed, and incorrect routing.

Boundaries and compatibility

Ideal for

  • Skill inventory analysis: produce a decision or artifact grounded in supplied evidence.
  • Trigger collision detection: produce a decision or artifact grounded in supplied evidence.
  • Portfolio coverage gap review: produce a decision or artifact grounded in supplied evidence.

Out of scope

  • Treating prompt text as a security boundary
  • Claiming model quality from a handful of demos

Agent compatibility

  • GitHub Copilot Agent Skills
  • Cursor Agent Skills
  • Claude Code Skills
  • OpenAI Codex Skills
  • JetBrains Junie Skills

Tool policy: Advisory by default. No tools are assumed. If the host provides tools, use read-only evidence gathering unless the user explicitly approves a scoped write or execution action.