Skip to content
HUIAHUB

anthropics/skills/skills/skill-creator

Skill creator

Creates new agent skills, edits and improves existing ones, runs evals and benchmarks on them, and tunes a skill's description so it triggers more reliably.

Agent skillAgent skills
View the skill on GitHub (opens in a new tab)

What Skill creator does

The skill walks an agent through building a skill as a loop. It first works out where the user is: some arrive with only an idea, others with a draft. For a new skill, the agent asks what it should do, when it should trigger, what output it should produce and whether test cases make sense, then writes a SKILL.md. The frontmatter description is the main trigger, so the instructions ask for one that names specific situations.

Next the agent writes two or three realistic test prompts to evals/evals.json. Where subagents exist, it launches paired runs for each prompt: one with the skill and one without it, or with the previous version. Assertions get drafted while those runs finish. A grader checks each run, scripts/aggregate_benchmark.py reports pass rate, time and tokens with mean and standard deviation, and eval-viewer/generate_review.py opens a review page where the user leaves feedback. The agent revises the skill from that feedback and runs the tests again.

An optional last step writes 20 trigger queries and runs scripts/run_loop.py, which uses claude -p to test and rewrite the description for up to five iterations, picking the winner on a held-out set. The result is a tested skill folder with benchmark files, and optionally a .skill package from scripts/package_skill.py.

Install

/plugin install example-skills@anthropic-agent-skills

Read the security check below before you run it. The repository's own instructions win if they differ.

Security check

What the skill's files do, from reading them. A flag that nobody has checked yet says "Not checked", never "No".

Runs scripts
Yes
Uses the network
Yes
Installs hooks
No
Asks for secrets
Yes

The folder ships Python scripts (10 by the scan's count) for benchmarks, reports, the review viewer and packaging, and the agent runs them. The scan flags network use in scripts/generate_report.py, whose HTML report loads Google Fonts. It flags scripts/improve_description.py for reading the environment: the script passes your environment to claude -p, so description tuning runs on your existing Claude login. scripts/run_eval.py writes temporary command files to .claude/commands/ and deletes them after each query. The scan found no hooks. Read the scripts and try the eval loop in a test repository first.

Scan details: Ships 10 script files: eval-viewer/generate_review.py, scripts/init.py, scripts/aggregate_benchmark.py, scripts/generate_report.py, scripts/improve_description.py, scripts/package_skill.py, …. skills/skill-creator/scripts/generate_report.py makes network requests. skills/skill-creator/scripts/improve_description.py reads a key, token or secret from the environment. Checked at commit 8a1541c4a3ff.

Use cases

  • Turn a workflow from the current conversation into a reusable skill.
  • Benchmark a skill against a no-skill baseline or its previous version.
  • Tune a skill's description so it fires on the right requests.
  • Package a finished skill as a .skill file.

Limitations

  • Paired runs, baselines and blind comparison need subagents; on Claude.ai they are skipped.
  • Description tuning needs the `claude -p` command line, so it does not run on Claude.ai.
  • The eval, report and packaging scripts need Python.
  • The repository README says its skills are demonstrations to test before relying on them.

Sources