AI Security Assessment

How Signet's AI security assessments are produced, what they check, and — just as important — what they do not guarantee. An AI assessment is an automated pre-screen, never a signed attestation. Read the reputation & trust model for the signed-audit side.

1. What it is

Signet runs a local large language model over a package's source and records an AI-security verdict per released version. It answers one question cheaply and at scale — *"is this version worth a closer look before I depend on it?"* — so a consumer or a human auditor can triage where to spend real review effort.

• Local only. The model is llama.cpp serving qwen2.5-7b-instruct on the

registry's own host. Source is never sent to a hosted/third-party model.

• Deterministic. The model runs at temperature: 0, so the same source at

the same commit yields the same verdict — an assessment is reproducible, not a dice roll.

• A pre-screen, not an attestation. AI verdicts are labelled

source: "ai-pre-screen" everywhere they appear. They carry no signature and add no reputation to anyone. The signed feed at /v1/findings and the human/agent audit records remain the authoritative record.

2. What it checks

Each run applies these review techniques (recorded in the assessment's methodology.techniques, so the verdict is explainable):

• llm-static-source-review — general static review of the sampled source
• prompt-injection-detection — untrusted text reaching a model/instruction path
• insecure-tool-and-function-calling — unsafe tool/function invocation from model output
• unsafe-eval-exec-of-model-output — eval/exec/shell of model- or user-derived data
• ssrf-detection — server-side request forgery from untrusted URLs
• secret-exposure-detection — hard-coded credentials/keys
• classic-software-vulnerability-review — injection, path traversal, and the like

The emphasis is deliberately on AI-application security (prompt injection, tool calling, eval-of-model-output) — the classes generic scanners miss — while still catching classic vulnerabilities in the sampled slice.

3. How it works

1. Resolve source. The version's (repo, commit) is resolved; the assessor

fetches source at that exact commit — GitHub tree-API first, streaming the commit tarball from codeload as a fallback when the API quota is exhausted.

1. Sample a bounded slice. A local 7B model has a finite context, so the

assessor samples rather than reads everything: the README (capped so it can't crowd out code) plus the most security-relevant source files, ranked by name hints (agent, prompt, exec, eval, tool, server, app, main, index, cli, api). Vendored/test/examples/docs trees are excluded. Each file is byte-capped so one large file can't consume the whole budget — breadth over the full text of one file. Defaults: ~16 KB total across up to 10 files.

1. Review + record. The model returns a structured assessment; it is parsed

into a verdict plus normalized findings and stored, along with the full methodology (sampled files, bytes analyzed, techniques, model parameters). The portal renders that methodology on the package dossier — the trust surface.

Cost control (PD-10). The automatic sweep works the pending queue prioritized *unaudited-first* (packages no signed audit has covered on their own subtree lead; newest breaks ties) under a rolling-24h token budget (signet.ai.daily-token-budget, default 2M; 0 = unlimited). When spent, the sweep pauses and resumes as spend ages out; operator-triggered runs are exempt. Budget state is public in GET /v1/ai-verifications/stats (budget block).

Tampering re-reads. A release whose completed verdict predates a VERIFIED-grade mapping mutation (the registry artifact was swapped — the verdict is marked stale on every read surface) rejoins the pending queue and leads it outright, above unaudited-first: a fresh automated read on the swapped artifact is the highest-signal spend there is. It leaves the queue again once a post-mutation assessment completes.

4. Verdicts

A re-assessment supersedes its predecessor: only the latest completed run per (ecosystem, name, version) contributes to the findings feed and the fleet- coverage stats, so a fixed package stops showing stale findings.

5. Limitations — read this before trusting a verdict

An AI pre-screen is a triage signal, not a security guarantee. Concretely:

• It samples, it does not audit. Only a bounded slice of the most likely-

relevant files is reviewed. A vulnerability in an unsampled file, or one that needs cross-file dataflow to see, can be missed. clean means "clean in the reviewed slice," not "clean."

• False negatives and false positives both happen. A 7B model will miss real

issues and will sometimes flag benign code. Treat findings as "worth a human look," not "confirmed vulnerable"; treat clean as "nothing obvious here," not "safe to skip review."

• Not a replacement for a human audit. The value is *prioritization* — steer

scarce human review toward the packages a pre-screen flags. A signed human (or reviewed-agent) audit is what actually carries weight in the trust model and the skip-scan coverage decision.

• Bounded to a commit. A verdict is about one exact commit; a later commit is

a different subject and needs its own assessment.

Because of all this, AI verdicts never feed reputation, never sign anything, and never raise a package's skip-scan coverage on their own. They sit alongside the signed record as a fast, transparent, honestly-bounded first look.