How to Vet Third-Party Agent Skills and MCP Servers Before Installing Them

When you add a third-party skill or server to an AI agent, you give it new instructions, capabilities, or both. A skill is a bundle of instructions and related files, sometimes including scripts or links to outside content. An MCP server is software that uses the Model Context Protocol to provide an AI application with tools or data, such as access to a repository or an external service.

Review both what the agent will read and what the installed software can access or execute. Check who published it, inspect its instructions and code, limit its permissions, and test it before sharing sensitive data or credentials. These steps reduce risk. A marketplace listing, scan, or approval alone does not establish that an integration is safe.

Review the instructions and the capabilities

A skill’s instructions can shape an agent’s decisions. Its scripts, installation commands, linked files, or content fetched while it runs can also affect what happens. Read the complete skill, including referenced materials, and look for directions to disclose secrets, hide actions, bypass user review, or perform unrelated tasks. For skills loaded from a shared repository, Anthropic warns that people who can edit the repository can change the instructions the agent receives.

An MCP server gives the agent actions it can request. It lists tools, with descriptions and schemas that explain what each action does and which inputs it accepts. The agent may use that information to decide which tools to call. Review each description and schema for instructions aimed at the agent, and check the server’s code, dependencies, startup command, environment variables, and network destinations where those details are available.

Researchers have demonstrated why tool descriptions deserve scrutiny. The MCPTox benchmark tested more than 45 real-world MCP servers and 20 AI agents using 1,348 malicious cases. The authors reported attack success rates above 60% for several tested models, with a maximum above 72%. In those tests, success meant that an agent followed a poisoned tool description and called a legitimate tool to carry out the described action. These results show a risk under controlled conditions; they do not measure how many servers are malicious or predict the outcome for a particular user. The MCPTox paper distinguishes this tool-description attack from indirect prompt injection, where untrusted instructions arrive in a tool’s results.

Snyk reports that its scan of 3,984 skills from ClawHub and skills.sh found 1,467 with at least one security issue and 76 confirmed malicious samples. The 1,467 figure is not a count of confirmed malicious skills, and the sample does not represent every marketplace or skill in use. Snyk’s study supports careful inspection, not assumptions about an individual skill based on its name or popularity.

Verify the publisher and inspect the full package

Check that the publisher, repository, package name, and installation instructions point to the same source. Look for an established release history and watch for lookalike names or unexpected redirects. If you find the package through a registry, confirm what the registry actually verifies. The official MCP Registry’s requirements describe publisher and package ownership checks; those checks help establish provenance, but do not amount to a comprehensive audit of code, dependencies, or runtime behavior. The registry requirements therefore provide a useful identity signal, not a safety verdict.

Inspect the files you will install instead of relying on the description or README. For a skill, follow every link and identify any content it fetches at runtime. For an MCP server, examine its advertised tools as well as its implementation. A tool called “search” might need read access to one repository; it does not automatically need permission to write across an account. Ask what each permission enables and remove access the task does not require.

Restrict access and test before connecting

Give the integration only the data and permissions it needs. Use a separate credential for each connection, with access limited to what that connection needs. Avoid storing secrets in skill files or plain-text configuration. If a repository tool only needs to read files, grant read access rather than write or administrative access. For remote servers, review the authentication method and requested scopes, and check how the server handles tokens.

Run local code in a constrained environment before giving it access to sensitive files or credentials. A local connection using stdio, a method that passes messages through the process’s input and output, does not make the process listen for network connections. It also does not limit what that process can read on the machine or where it can connect. OWASP’s MCP security guidance recommends least privilege and isolation; treat the local connection method and the process’s permissions as separate questions.

Use code scans, dependency scans, secret scans, and scans for suspicious instruction patterns as screening tools. A clean scan cannot prove that the integration is safe or predict every behavior, especially when it fetches content or depends on a remote service. During testing, watch what the integration reads and which network destinations it contacts. Track the tools it invokes as well. Require the user to review the full parameters and approve sensitive actions. These include deleting data, spending money, or sharing private information.

Pin the reviewed version and check for changes

Record the exact version or commit you inspected, then install that version rather than following an unpinned update. For MCP, OWASP recommends tracking changes to the complete tool definition, including descriptions, schemas, and annotations. A changed definition should trigger review. An unchanged definition does not prove that the server’s code or behavior stayed the same, so review server updates and remote-service changes separately.

Monitor tool calls for unexpected actions, destinations, or permission changes. Keep a way to disconnect the server, remove the skill, and revoke its credentials. Hosting incidents also belong in the risk assessment: GitGuardian reported that a path-traversal flaw in Smithery’s hosting infrastructure exposed overprivileged administrative credentials with access to more than 3,000 hosted MCP servers and associated secrets. The report described one incident and said the issue was patched after disclosure, with no evidence of exploitation. It shows why a hosted integration’s operator and credential handling matter alongside the package itself. GitGuardian’s disclosure does not establish that all hosted servers share that risk.

What I'm building

Delegate tasks. Get software.

Give Vroni a GitHub issue, bug report, spec, or rough idea. It reads the repo, plans the change, writes code, runs checks, and works toward a review-ready pull request.

Take a look at vroni.com