AI Skills Hide Malicious Code in Test Files—Here’s How It Works

A recent investigation by Gecko Security uncovered a critical blind spot in AI agent security: malicious payloads hidden inside test files that evade detection by standard skill scanners. The attack leverages common JavaScript and TypeScript testing frameworks like Jest and Vitest, which automatically execute test files during development and continuous integration—even when those files sit in directories meant for AI Skills.

When developers install an AI Skill using a command like npx skills add owner/repo-name, the entire Skill directory—including any bundled test files—is copied into the project. These test files, typically overlooked by AI scanners, run with full filesystem access, environment variables, and SSH keys. The payload executes silently, often before any test assertions even begin, making it nearly impossible to detect without specialized tools.

The Silent Execution Surface

AI Skills are typically vetted by scanners that inspect files like SKILL.md and referenced scripts, but they ignore test files by design. The attack exploits this gap by embedding malicious code in files such as reviewer.test.ts or conftest.py. These files are automatically discovered and executed by popular testing frameworks:

• Jest and Vitest: Both frameworks enable dot: true by default, meaning they crawl directories like .agents/skills/ and execute files starting with dots (e.g., .agents/.*).
• Mocha: Follows recursive discovery unless explicitly configured otherwise.
• Pytest: Runs conftest.py files during test collection, even if they reside in AI Skill directories.

The result? A payload hidden in a test file can exfiltrate sensitive data—environment variables, .env files, SSH keys, or cloud credentials—without triggering any scanner alerts. The exfiltration occurs during test setup, often before the test suite even runs, leaving no trace in the output.

Three Major Audits, One Common Limitation

Gecko Security’s findings arrived alongside two major security audits, both of which focused on vulnerabilities within the AI Skill execution surface. While these audits provided valuable insights, they missed the threat lurking outside it—namely, in test files that scanners don’t inspect.

1. SkillScan Academic Study (January 15): Analyzed 31,132 unique Anthropic Skills and found that 26.1% contained at least one vulnerability. Data exfiltration appeared in 13.3% of Skills, and privilege escalation in 11.8%. Skills bundling executable scripts were 2.12 times more likely to be vulnerable.
2. Snyk ToxicSkills (February 5): Scanned 3,984 Skills from major marketplaces and identified 13.4% with critical security issues. The audit also uncovered 76 confirmed malicious payloads, eight of which remained active on ClawHub at publication.
3. Cisco AI Agent Security Scanner (April 21): Integrated into IDEs like VS Code and Cursor, the scanner targets agent interaction layers but does not inspect bundled test files.

All three audits highlight a shared flaw: none address the developer toolchain layer, where test files execute with full local permissions.

Defending Against the Hidden Threat

The fix for this vulnerability is straightforward but requires a shift in security thinking. Developers and security teams should:

• Exclude AI Skill directories from test discovery: Update Jest, Vitest, Mocha, or pytest configurations to ignore directories like .agents/, .claude/, and .cursor/. For example, in Jest, add:

{
  "testPathIgnorePatterns": [".*\\.agents.*", ".*\\.claude.*", ".*\\.cursor.*"]
}

• Review test file structure in installed Skills: Audit any newly added Skills for unexpected test files, especially those in subdirectories like tests/ or __tests__/.
• Implement strict CI/CD policies: Ensure test runners in CI environments do not execute files from untrusted directories. Use explicit file path inclusions to avoid recursive discovery.
• Monitor for unusual network activity: Log outbound connections during test execution, as malicious payloads often exfiltrate data to external endpoints.

The rise of AI Skills has introduced new attack vectors, but the solution lies in expanding threat models to include the developer’s local environment. By recognizing that test files are a viable execution surface, teams can close this critical gap and prevent silent, high-impact breaches.