SkillsBench systematically measures how well AI agent skills improve performance across diverse real-world tasks.
Agent Skills are structured packages of procedural knowledge that augment LLM agents at inference time. Despite rapid adoption, there is no standard way to measure whether they actually help. We present SkillsBench, a benchmark of 86 tasks across 11 domains paired with curated Skills and deterministic verifiers. Each task is evaluated under three conditions: no Skills, curated Skills, and self-generated Skills. We test 7 agent-model configurations over 7,308 trajectories. Curated Skills raise average pass rate by 16.2 percentage points(pp), but effects vary widely by domain (+4.5pp for Software Engineering to +51.9pp for Healthcare) and 16 of 84 tasks show negative deltas. Self-generated Skills provide no benefit on average, showing that models cannot reliably author the procedural knowledge they benefit from consuming. Focused Skills with 2--3 modules outperform comprehensive documentation, and smaller models with Skills can match larger models without them.
SkillsBench is the first benchmark systematically measuring whether AI agent skills actually improve task performance across domains.
The benchmark evaluates tasks under three conditions: no skills, expert-curated skills, and self-generated skills for rigorous impact isolation.
Rigorous quality control including automated validation, AI detection screening, and deterministic verifiers ensures reliable skill effectiveness measurement.
The benchmark covers 84 tasks across 11 domains, which may not represent all real-world AI agent use cases comprehensively.
The script ends abruptly before discussing actual benchmark results and findings about skill effectiveness patterns.
Meituan LongCat Team, Anchun Gui +160
We introduce LongCat-Flash-Thinking-2601, a 560-billion-parameter open-source Mixture-of-Experts (MoE) reasoning model with superior agentic reasoning capability. LongCat-Flash-Thinking-2601 achieves ...
Tianxin Wei, Ting-Wei Li +27
Reasoning is a fundamental cognitive process underlying inference, problem-solving, and decision-making. While large language models (LLMs) demonstrate strong reasoning capabilities in closed-world se...
Ailin Huang, Ang Li +213
We introduce Step 3.5 Flash, a sparse Mixture-of-Experts (MoE) model that bridges frontier-level agentic intelligence and computational efficiency. We focus on what matters most when building agents: ...
Veuns-Team, Changlong Gao +25
GUI agents have emerged as a powerful paradigm for automating interactions in digital environments, yet achieving both broad generality and consistently strong task performance remains challenging.In ...
Ji Zeng, Dayuan Fu +15
Recently, the frontier of Large Language Model (LLM) capabilities has shifted from single-turn code generation to agentic software engineering-a paradigm where models autonomously navigate, edit, and ...
