Papers
arxiv:2601.10338

Agent Skills in the Wild: An Empirical Study of Security Vulnerabilities at Scale

Published on Jan 15
· Submitted by
YI LIU
on Jan 16
Authors:
,
,
,
,
,
,
,

Abstract

Large-scale security analysis of AI agent skills reveals widespread vulnerabilities including prompt injection, data exfiltration, and privilege escalation risks.

AI-generated summary

The rise of AI agent frameworks has introduced agent skills, modular packages containing instructions and executable code that dynamically extend agent capabilities. While this architecture enables powerful customization, skills execute with implicit trust and minimal vetting, creating a significant yet uncharacterized attack surface. We conduct the first large-scale empirical security analysis of this emerging ecosystem, collecting 42,447 skills from two major marketplaces and systematically analyzing 31,132 using SkillScan, a multi-stage detection framework integrating static analysis with LLM-based semantic classification. Our findings reveal pervasive security risks: 26.1% of skills contain at least one vulnerability, spanning 14 distinct patterns across four categories: prompt injection, data exfiltration, privilege escalation, and supply chain risks. Data exfiltration (13.3%) and privilege escalation (11.8%) are most prevalent, while 5.2% of skills exhibit high-severity patterns strongly suggesting malicious intent. We find that skills bundling executable scripts are 2.12x more likely to contain vulnerabilities than instruction-only skills (OR=2.12, p<0.001). Our contributions include: (1) a grounded vulnerability taxonomy derived from 8,126 vulnerable skills, (2) a validated detection methodology achieving 86.7% precision and 82.5% recall, and (3) an open dataset and detection toolkit to support future research. These results demonstrate an urgent need for capability-based permission systems and mandatory security vetting before this attack vector is further exploited.

Community

Paper submitter

Really interesting and timely work—agent “skills” seem like a rapidly growing supply-chain attack surface, and it’s refreshing to see a large-scale empirical analysis rather than a purely conceptual treatment. The scale and methodology (tens of thousands of skills analyzed using a combined static and LLM-based pipeline) are particularly compelling, and the 14-pattern, four-category vulnerability taxonomy helps make the risk landscape much more concrete. The reported numbers are striking: roughly a quarter of skills containing vulnerabilities, with a non-trivial fraction exhibiting high-severity patterns suggestive of malicious behavior, which strongly motivates the need for better ecosystem defenses. It would be interesting to learn more about how ambiguous cases were handled when separating malicious intent from accidental insecurity, whether vulnerability prevalence differs across marketplaces with different governance models, and how developers might practically use the detection results for remediation. The call for capability-based permission systems is persuasive, and additional discussion of concrete enforcement mechanisms (e.g., sandboxing, least-privilege access, signing, or dependency controls) would further strengthen the practical impact.

Sign up or log in to comment

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2601.10338 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2601.10338 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2601.10338 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.