Skip to content

Build · A curated list alongside the AgenTrust specifications

Awesome AI Agent Governance Awesome Discord

Community updates and contributor highlights: AgenTrust on LinkedIn.

A curated list of tools, frameworks, standards and resources for governing AI agents: keeping them safe, knowing who they are, seeing what they do, and showing that they followed the rules.

An AI agent is an AI system that takes actions on its own, such as sending email or updating a database, instead of only answering questions. Agents now reach email, CRMs, databases and financial systems. Content filters catch a lot of bad output, yet a company that lets agents act also needs a record it can check of what each agent did. This list collects the tools and practices that help make agents safe, checkable and trustworthy in real use. New to the terms? The AgenTrust plain-terms list explains the common ones in a line each.

AgenTrust is an independent open-source project. No company funds, owns or directs it; contributions are voluntary. Past sponsorship is credited in SPONSORS.md. Every entry is judged by the same published contribution rules.

Contents

Governance Frameworks

Platforms that sit between an agent and the systems it uses, check each action against your rules while the agent runs, and keep a record of what was allowed or refused.

  • AAES Govern - Self-hosted governance for supported agent actions routed through controlled credential paths, with permissions, human approvals, spending commitments, and offline integrity checks of exported records; pre-launch, with a separate Apache-2.0 verifier and documented coverage limits.
  • A-FORGE - AGPL-3.0 executor for agent file, service, and git changes, limited by per-action-class budgets, accepting only HMAC-signed requests, and writing a receipt per run.
  • Agent Governance Toolkit (AGT) - Production governance layer for autonomous agents with a policy enforcement kernel (<0.1ms p99), execution rings (Ring 0-3), cryptographic Merkle audit logs, and integrations across LangChain, CrewAI, AutoGen, Google ADK, and more. Python + .NET + Rust. Provides the software governance layer that integrates with hardware-attested enforcement via cMCP. ★4000+
  • AI Gateway HQ - Hosted AI gateway that applies identity, model, tool, data-handling, and budget rules before provider requests; supports Observe, Shadow, and Enforce rollout modes with reason-coded audit metadata and a free evaluation workspace.
  • Allowly - Hosted policy decisions and signed receipts for AI-agent actions; the caller enforces, and receipts verify offline with workspace Ed25519 keys using open-source Python (allowly-receipt-format) and TypeScript (@allowly/verifier) verifiers, with a CC BY 4.0 specification and interoperability map.
  • arifOS - AGPL-3.0 governance kernel for MCP agents that evaluates proposed actions against policy floors F1-F13, answers one of seven wire verdicts (SEAL, HOLD, SABAR, PARTIAL, PROVISIONAL, HOLD_888, VOID), and keeps a hash-chained decision ledger. pip install arifos.
  • Bifrost - Go-native, OpenAI-compatible gateway for runtime AI governance at the model and tool boundary, enforcing guardrails, rate limits, fine-grained access controls, and usage tracking while supporting MCP gateway flows with logs, metrics, and tracing.
  • Conarium - Self-hosted governance layer between AI assistants and a database: deterministic PII masking, allow/deny policy, a hash-chained audit log, and offline-verifiable Ed25519 receipts per access when a receipt sink is configured. MIT.
  • Coral Server - Agent coordination and trust server enabling safe multi-agent collaboration with structured communication protocols.
  • CIRVIX AgentControl - Apache-2.0 local runtime authorization for tool calls explicitly routed through its MCP gateway or returned SDK wrappers.
  • Cordum - Agent control plane providing governance, lifecycle management, and policy enforcement for autonomous agents.
  • CCS (Correctover Conformance Shape) - Cryptographic runtime-verification receipts for agent tool calls: Ed25519 (RFC 8032) over RFC 8785 JCS canonical JSON, with prev-receipt hash chaining. MIT-licensed, language-neutral conformance vectors (10 vectors covering valid, tampered, algorithm-substitution, key-substitution, and chain-linked cases) let any implementation verify receipt signing and verification without a dependency on the reference implementation.
  • Dome Systems - Hosted control plane for AI agents: per-agent identity and revocable credentials, Cedar authorization rules evaluated on every tool and model call (including rules on tool arguments), and an exportable audit trail of each decision. Free tier.
  • Enterprise Agentic AI - Runnable Python reference for bounded agent actions: separates requester and approver roles, binds approval to an action digest, enforces expiry, revocation, tenant and batch limits, and reconciles a committed CRM write after response loss.
  • ExecLayer - Deterministic execution governance kernel that evaluates AI-proposed actions against policy before execution, refuses out-of-bounds actions, and emits an Ed25519-signed receipt for every decision. Closed-source kernel; public live kernel at kernel.execlayer.io and DOI-registered architecture papers including the Governed Execution Artifact Standard (10.5281/zenodo.18749299).
  • Gate22 - MCP gateway with role-based access control, audit logging, and fine-grained permission management for tool access.
  • HOL Guard - Apache-2.0 local-first runtime control for AI coding agents that evaluates supported shell, file, MCP, plugin/skill, and package actions before side effects, applies allow/block/approval policy, and records local security receipts; optional cloud adds shared policy and evidence sync.
  • IBM mcp-context-forge - Enterprise MCP gateway with context-aware guardrails, request routing, and compliance controls.
  • Impri - Approval inbox for AI agents: agents propose actions via MCP or REST, a human approves/edits/rejects from web, Slack, Telegram or phone, then the agent executes. MIT-licensed open-core, self-hostable, free tier.
  • Invariant Guardrails - Rule-based guardrails engine with policy-as-code, trace analysis, and real-time intervention for agentic applications.
  • LedgerGuard - Apache-2.0 TypeScript runtime that sits between an AI agent and a PostgreSQL system of record: deterministic allow/deny/require-approval policy, approvals bound to a plan id and version that agents cannot grant themselves, typed allowlisted corrections instead of free SQL, and execute, verify and commit in one transaction with a persisted idempotency key. Pre-1.0, Postgres only, synthetic demo scenarios.
  • LiteLLM - Unified LLM gateway with spend tracking, rate limiting, guardrails, and access controls across 100+ LLM providers.
  • MARGINAL - Local-first runtime governor for AI coding agents that starts in Shadow Mode, records evidence for proven no-progress repetition, and enables narrow Codex tool enforcement only after repository-local evidence; Claude Code and OpenCode integrations are observe-only.
  • MREA - Multi-role enterprise agent governance framework with separate Architect, Auditor, and Implementer roles, enforcing human approval gates and independent audits to prevent self-validation bias in AI-assisted development.
  • Okto Nexus - Local-first MCP coordination hub for teams of AI coding agents, with single-winner task claims, human-in-the-loop approval on risky actions, and durable handoff history for crash recovery.
  • Okto Pulse - Local-first, spec-driven SDLC workbench enforcing independent validation and blocking evidence gates at the permission level, with a knowledge graph that catches drift from recorded decisions.
  • Operational Memory Gates - MIT specification and reference implementation of 21 mechanical pre-code gates for AI-assisted development, each documented as either a production incident made mechanical or a discipline made mechanical. Twenty refuse with a non-zero exit; one advises. The reference script is deliberately hardcoded to the author's stack, so the artifact is a checkable specification rather than a turnkey tool.
  • Proofpane - Runtime governance gateway for AI coding agents and MCP clients, enforcing policy gates and DLP redaction in the execution path with a hash-chained, offline-verifiable audit. Closed-source (proprietary daemon); public reference architecture under CC BY 4.0.
  • Provenza - Self-hosted AI governance platform with policy approvals, request auditing, prompt masking with optional named-entity recognition, and a registry of reported shadow-AI sightings. Apache-2.0.
  • Regulus - EU & UK compliance plane for Google ADK encoding 10 regulations (EU AI Act, GDPR, DORA, NIS2, EHDS, UK GDPR, FCA SYSC, PRA SS1/23, PRA SS2/21, NHS DSPT) and 6 governance frameworks as runtime ADK BasePlugin profiles; emits hash-chained audit envelopes with GRC adapters (ServiceNow IRM, OneTrust, MetricStream).
  • SEMAPRAX - Apache-2.0 experimental systems programming language (source) whose compiler gives declarations stable semantic identities, independently replays patch evidence before supported mutations, and models explicit capability and authority boundaries for agent-driven code changes. v0.2 pre-alpha; no built-in model, autonomous agent, transport, keys, or production authority.
  • ScopeBlind protect-mcp - Security gateway for MCP servers with Cedar policy enforcement (AWS Cedar via WASM), Ed25519-signed decision receipts, issuer-blind spending authority (VOPRF), and multi-agent swarm tracking. Merged into AGT.
  • Squelette - Repository-level governance for AI coding agents: work items declare the file paths they may touch and a pre-commit gate refuses anything outside them, human decisions are recorded in the repo, and a task cannot close without evidence verified against its artifacts.
  • sofagent - Open-source harness for governing AI coding agents: ontology-driven workflows plus 28 commit-time audit rules over git diffs (secrets, out-of-scope edits, blind modifications, prompt injection), HMAC-signed tamper-evident audit history, and MCP tools for governance aggregation. MIT.
  • Speakeasy - Enterprise AI control plane for governing access, policy, and auditability across agents, MCP servers, and Skills.
  • TrinityGuard - Multi-agent safety framework with three-layer defense for detecting and preventing unsafe agent behaviors.
  • Verax - MCP server that sits between an agent and its tools: a policy gate decides tool calls and writes signed decision records to a local ledger before execution; calls the policy will not decide alone are held for an operator on the same machine. verax verify reads the ledger back with nothing on the network. Apache-2.0.
  • WitnessOS - Runtime governance layer producing evidence-grade receipts for every agent action, with policy evaluation before execution and a tamper-evident audit chain.
  • YYLO - Command-line orchestrator that governs coding-agent repository changes: each task freezes the protected target SHA and runs in a dedicated branch/worktree behind typed task, validation, merge, and release-readiness boundaries; the merge queue owns risk-based review (low risk has no semantic reviewer, high risk two sequential reviewers on one frozen candidate) and stops as REVIEW_FINDINGS_EXHAUSTED instead of starting an unbounded review loop; runs retain declared receipt hashes and terminal manifests as receipt-backed repository changes. MIT, on npm as @yylo/cli; orchestrates Pi and Codex subagents.

End-to-End Governance: Software and Hardware

Tools that pair software rules with evidence from the hardware. The rules, the logs and the agent's identity run inside a trusted execution environment (TEE), a protected area of the chip whose memory stays encrypted while it runs, and the chip signs a report of exactly what was loaded (an attestation). A privileged operator, a tampered runtime or a supply chain attack cannot forge that report. The result is a compliance record that anyone can check offline, without having to trust the company running the agent.

The stack, in the order of the AgenTrust chain from model weights to evidence: Weight Custody Manifest hands out the key that decrypts a model's weights only to a machine whose attestation checks out. Agent Manifest signs all ten parts of an agent deployment into one identity document. AGT checks actions against Cedar policies in software, cMCP runs those checks inside a TEE for MCP tool calls (MCP is a common way agents connect to tools), and cA2A is a profile for checkable handoffs of work between agents over A2A. TRACE records the result as a signed receipt.

  • Agent Manifest - Signs all 10 parts of an agent deployment (system prompt, policy bundle, model identity, tool schemas, RAG corpus, memory baseline, decision trace, A2A delegation chain, build provenance, HITL approvals) into one record, so any later change to any part shows up. Can include a signed hardware report (attestation) from a TPM, AMD SEV-SNP or Intel TDX chip. Four conformance levels, with compliance mappings for the EU AI Act, DORA, GDPR, and HIPAA. 197 conformance tests. Python. Developer preview.
  • cMCP (Confidential MCP Gateway) - A gateway that sits between an agent and its MCP tools and checks each tool call against Cedar policy inside a TEE. On real hardware, the policy bundle is recorded in the chip's signed report before any code runs, and the signing key never leaves the protected area; software mode gives no hardware protection. Every tool call produces a signed receipt (a GatewayClaim) tied to that hardware record. Supports TPM, AMD SEV-SNP, and Intel TDX attestation. Developer preview.
  • cA2A (Confidential A2A) - A profile for the Agent2Agent (A2A) protocol, which agents use to hand work to each other. A2A's Signed Agent Card proves only who owns a domain. cA2A adds delegation where each agent can pass on the same or less authority than it received, provably; a hardware check of the other agent's runtime; a sealed channel between the two (tying the seal to a verified hardware measurement on a live call is on the roadmap); and a record of each handoff that can be checked offline. Reuses the delegation rules from Agent Manifest and the TEE and policy building blocks from cMCP. Python, developer preview; installable with pip install ca2a-runtime.
  • TRACE (Trust Runtime Attestation and Compliance Evidence) - Open specification (a Series of LF Projects, with an AAIF Sandbox proposal open) and Python SDK for signed receipts of what an agent run did. Each run produces a signed record of the model, the runtime, the policy version and the tool calls, which anyone can check offline without contacting the operator; proving which hardware it ran on needs a separately verified attestation. Built on IETF RATS (RFC 9334), EAT (RFC 9711), SCITT, SLSA, and SPIFFE. Spec v0.2.
  • Weight Custody Manifest (WCM) - Open, pre-1.0 specification and Python SDK that hands out the key to decrypt a model's weights only to a workload whose CPU (and, when required, GPU) attestation matches a signed manifest, with short-lived approval and a record of each release. The reference SDK verifies AMD SEV-SNP, Intel TDX, and NVIDIA H100 CC evidence captured on real hardware. Against an operator who physically owns the machine it offers accountability rather than cryptographic custody. Apache-2.0; pip install weight-custody-manifest; docs at wcm.agentrust-io.com.
  • SourceryKit - Verifies an agent's outbound requests and MCP handoffs against a source of truth using zero-knowledge proofs, so a call only goes out if the agent's claims check out. Hooks into the HTTP libraries, logs every outbound call, and blocks endpoints not on the trusted allow-list. Python, BSL 1.1 (source-available), with a hosted backend that runs the proof and source-of-truth check.

Policy as Code

Languages and engines for writing access rules as code and checking them. Useful for limiting what an agent may do, which tools it may call and which data it may touch.

  • agent-evidence-admission - Apache-2.0 Kubernetes admission policies for OPA, Kyverno, and Sigstore policy-controller that evaluate agent-execution evidence, with documented enforcement limits and conformance checks.
  • Casbin - Cross-language authorization library supporting ACL, RBAC, and ABAC models. Available in Go, Python, Java, and more.
  • Cedar - Amazon's policy language for fine-grained, type-safe access control. Used as the policy engine in the Agent Governance Toolkit. Fast, formally verified, and human-readable.
  • GOPAL - Apache-2.0 library of 85 Rego policies encoding AI-governance regulations (EU AI Act, NIST AI RMF, ICAO/FAA/EASA aviation, FERPA/COPPA, fair lending) as executable allow/deny checks for the OPA engine, versioned per framework with allow/deny tests in CI.
  • Open Policy Agent (OPA) - CNCF general-purpose policy engine. Decouples policy decisions from application logic using the Rego language. Widely deployed for Kubernetes and API authorization.
  • SpiceDB - Google Zanzibar-inspired database for fine-grained, relationship-based authorization. Useful for cross-agent and multi-tenant permission modeling.

LLM Safety & Guardrails

Filters that check what goes into and comes out of a language model (LLM): unsafe content, leaked personal data, and prompt injection, which is text written to hijack the model's instructions.

  • ai-evaluation - Open-source LLM evaluation framework with 50+ metrics, LLM-as-Judge, and guardrail scanners (jailbreak, PII, injection).
  • Arthur Shield - Firewall for LLMs that detects hallucinations, toxicity, PII leakage, and prompt injection in real time.
  • Guardrails AI - Framework for structural, type, and quality guarantees on LLM outputs. Guardrails Hub provides community validators.
  • Hyperion - Framework for evaluating and improving robustness of LLM-based agents against adversarial attacks.
  • Lakera Guard - Real-time API for detecting prompt injections, data leakage, toxic content, and other LLM security threats.
  • LLM Guard - Input and output scanners covering toxicity, PII, prompt injection, invisible text, and code detection.
  • Meta Llama Guard - Safety classifier models for filtering unsafe LLM inputs and outputs. Part of Meta's Purple Llama safety suite.
  • NVIDIA NeMo Guardrails - Open-source toolkit for adding programmable guardrails to LLM-based conversational systems using Colang.
  • Rebuff - Prompt injection detection using multi-layer defense: heuristics, LLM analysis, and canary tokens.
  • shim-cli - Local hooks for Claude Code, Codex, Copilot CLI and VS Code that detect secrets and personal data in agent traffic, mask them in Claude Code tool results before the model reads them, and keep a count-only session record.
  • Sunglasses - Open-source local scanner that checks text and files for prompt injection, credential leaks and data exfiltration. Runs as a CLI, a Python API or an MCP server.
  • Vigil - LLM security scanner for detecting prompt injections using embedding similarity, heuristics, and canary tokens.

Agent Frameworks with Governance Features

Toolkits for building agents that include places to plug in safety checks, human approval or policy rules.

  • AgentScope - Multi-agent platform with fault tolerance, agent-level monitoring, and configurable message validation.
  • AutoGen - Multi-agent conversation framework with human oversight, code execution sandboxing, and conversation policies.
  • CorvinOS - Self-hosted agentic OS connecting local and cloud models to Discord, Telegram, WhatsApp, Slack, and Email, with a fail-closed GDPR consent gate, hash-chained audit log, and bot-disclosure controls. Python, Apache-2.0.
  • CrewAI - Multi-agent orchestration with role-based agents, task delegation, and configurable guardrails.
  • Dify - LLM app development platform with content moderation, rate limiting, and annotation logging.
  • Google Agent Development Kit (ADK) - Google's agent framework with safety callbacks, evaluation tools, and multi-agent session management.
  • Google Sovereign Agent Mesh (SAM) - Apache-2.0 agent-mesh project with cryptographic node identities, authenticated peer connections and packets, discovery, authorization policies, and MCP sidecar routing. The repository states that it is not an officially supported Google product.
  • Haystack - End-to-end NLP framework with pipeline-based architecture supporting content filtering and validation.
  • LangChain - Composable framework with callbacks, tracing, and moderation chains for LLM applications.
  • LangGraph - Stateful, multi-actor agent framework with human-in-the-loop and persistence built in.
  • LightAgent - Python agent framework with narrowing-only security contexts, scoped capability gates, durable human approvals, and application-owned verification before publishing immutable DAG artifacts.
  • LlamaIndex - Data framework with observability callbacks, evaluation modules, and structured output guarantees.
  • Microsoft Agent Framework (MAF) - Microsoft's multi-agent orchestration framework for Python and .NET, with migration paths from Semantic Kernel and AutoGen. Governance-relevant pieces are human-in-the-loop workflow steps, checkpointing, and built-in OpenTelemetry tracing.
  • OpenAI Agents SDK - Official OpenAI SDK with built-in guardrails, input/output validation, and handoff controls.
  • PydanticAI - Agent framework with type-safe tool definitions, structured outputs, and dependency injection.
  • Semantic Kernel - Microsoft's AI orchestration SDK with plugin permission models, function filtering, and responsible AI hooks.
  • smolagents - Hugging Face's lightweight agent library with sandboxed code execution and security controls.

Agent Identity & Attestation

Ways to give an AI agent an identity that others can check with cryptography, and to trace where it came from. For agent identity backed by a signed hardware report, see End-to-End Governance: Software and Hardware.

  • Agent Card / AI Card - Specification for machine-readable agent capability and policy metadata, enabling discovery and trust decisions.
  • Agent Passport System - Apache-2.0 protocol for agent identity, scoped delegation with monotonic narrowing, runtime enforcement, and signed action receipts. TypeScript and Python SDKs; active IETF Internet-Draft (draft-pidlisnyi-aps).
  • Etch - Signed, offline-verifiable audit chain for AI agent decisions with portable per-agent identity across MCP client tools. Open-source reference verifier (world-model-mcp on PyPI, MIT): hybrid Ed25519 + FIPS 205 SLH-DSA-SHA2-128f signing over Merkle-chained epochs, chain-integrity + signature verification offline. Hosted service (etch.systems, BSL 1.1) extends with Sigstore Rekor + Bitcoin OpenTimestamps anchoring, key rotation, delegation grants and revocations, and memory-per-identity across MCP client tools. Portable identity envelope + JSON Schema + conformance vectors on Zenodo (DOI 10.5281/zenodo.22154537).
  • Kepil - AGPL-3.0 Python toolkit for agent-version passports, per-job mandates, and a gateway that checks submitted actions against permissions, spending limits, and expiry. Passport construction rejects the highest autonomy class. Uses the Python standard library.
  • scitt-cose - Payload-agnostic IETF SCITT + COSE verification substrate (Python): verifies Signed Statements and RFC 9162 SHA-256 Receipts with inclusion/consistency proofs, offline. Open source to run yourself, or hit the free community verify service at verify.agentactioncapsule.org to check a receipt without trusting the issuer. Apache-2.0.
  • SPIFFE/SVID - Secure Production Identity Framework for Everyone. Cryptographic workload identity applicable to agent-to-agent authentication.
  • W3C Decentralized Identifiers (DIDs) - W3C standard for decentralized, self-sovereign identifiers applicable to durable agent identity without centralized registries.

Agent Action Records

Formats and services that record what an agent actually did as signed receipts and public logs, which anyone can check later, offline if needed. The Agent Identity & Attestation section covers who the agent is; this one covers what it did.

  • Agent Action Capsule (AAC) - Open SCITT statement profile (IETF draft-mih-scitt-agent-action-capsule) for recording and verifying what an AI agent did: each action is sealed into a content-addressed, offline-verifiable capsule (JCS/RFC 8785) a third party can check without calling the operator. Apache-2.0; reference library, test vectors, and standard site at agentactioncapsule.org.
  • agent-evidence-vectors - Apache-2.0 conformance corpora and reference verifiers for proposed agent-execution evidence predicates, SCITT/COSE carriage, and related evidence-binding profiles.
  • agent-trace - MIT TypeScript verifier for Python-generated, hash-chained agent journals. Checks sequence, chain linkage, and record hashes, reporting the first failing record's sequence number. Developed in the ProofByte monorepo.
  • capsule-anchor - Software for anyone to run their own vendor-neutral SCITT Transparency Service (RFC 9162): submit a digest, get a COSE receipt anchoring it into an append-only, independently auditable log that anyone can verify offline. Run your own, or use the free community instance at witness.agentactioncapsule.org. Apache-2.0.
  • capsule-emit - The reference implementation of Agent Action Capsule: a one-call emit() producer that seals an action into a capsule, anchors it by default, and ships a ledger-view CLI with thin framework adapters. Apache-2.0.
  • Cedulon - Audit layer for agent-to-agent commerce: signed trade manifests, default-deny spend policy, COSE_Sign1 receipts, epoch checkpoints, and a completeness check of receipts against a rail extract. The published MCP server settles on a mock rail and holds no wallet; Cedulon is not a payment rail. Specified in IETF Internet-Drafts (draft-dogru-cedulon). Apache-2.0.
  • Mandare - Self-hosted LLM gateway that enforces human-signed per-agent spend mandates and writes every allow and refusal to a hash-chained, witnessable ledger. Apache-2.0 libraries, AGPL-3.0 servers.
  • Nobulex - Bilateral receipt primitive for tamper-evident agent audit trails: two Ed25519 signatures per action (pre- and post-execution), hash-chained via JCS canonicalization (RFC 8785). The receipt-signing approach is merged into AGT. MIT licensed.
  • PIC Standard (Provenance & Intent Contracts) - Open, local-first protocol for pre-execution action gating in AI agents: the agent declares intent, provenance, and evidence before a high-impact tool call; the verifier returns allow or block, failing closed on missing or invalid evidence, under a Trust Axiom that trust is verifier-derived, not producer-asserted. Apache-2.0. Pre-v1.0 draft.
  • proofbundle - Signed, portable receipts for AI evaluation and review results, verifiable offline with the public key alone: Ed25519 over canonical JSON (RFC 8785), RFC 6962 Merkle inclusion for samples, optional SD-JWT selective disclosure and in-toto statement export, a verifier with a fixed exit-code contract and a 130-case conformance corpus. Establishes evidence integrity, not whether the result is correct. MIT.
  • Quilzo - Self-hosted control plane for AI agents whose agent runs, app calls and erasures come as receipts: each record with an inclusion proof against a log head signed with Ed25519 and ML-DSA, checkable offline against published keys.

  • Signed Decision Receipts (IETF) - Internet-Draft defining a portable, cryptographically signed receipt format for machine-to-machine access control decisions. Ed25519 + JCS canonicalization. Independently verifiable offline.

  • YYLO Benchmark - Evaluation layer for task prompts that records each attempt for verification: every candidate runs in a private fresh-repository workspace behind an initial workspace receipt plus a deterministic post-execution manifest covering Git HEAD/tree/refs/index/status and every worktree file; recovery reuses hash-verified terminals, and retained state is rejected unless the complete chain from state, plan, and attempt through receipt, manifest, terminal, and evidence forms one exact linkage. Deterministic and LLM-judge evaluations append immutable records with evidence and evaluator provenance hashes; running a plan grants no production or external authority. MIT; CLI yylo-benchmark (also delegated unchanged by yy benchmark) and library on npm as @yylo/benchmark; part of the YYLO suite alongside the YYLO CLI orchestrator.

Observability & Monitoring

Tools for tracing, monitoring, testing and debugging what AI agents do.

  • AgentOps - Agent observability SDK with session replay, LLM cost tracking, compliance monitoring, and failure detection.
  • Aident Loadout - Hosted MCP server and CLI that runs agent tool calls through a credential vault (agents get results, not raw API keys), shows the cost of each action before it runs, and keeps an audit log of what ran, where it came from, whether it succeeded and what it cost. Commercial with a free pay-as-you-go tier; MIT-licensed client skill.
  • Arize / Phoenix - Open-source AI observability with LLM tracing, evaluation, retrieval analysis, and experiment tracking.
  • Braintrust - Evaluation and monitoring platform with logging, scoring, and experiment comparison.
  • Datadog LLM Observability - Enterprise monitoring with trace clustering, cost attribution, and quality scoring.
  • Future AGI - Open-source self-hostable end-to-end agent engineering platform with tracing, evals, guardrails, and gateway.
  • Helicone - Open-source LLM observability with request logging, cost tracking, caching, and rate limiting.
  • Jaeger - Open-source distributed tracing for monitoring agent workflows and debugging latency across services.
  • Langfuse - Open-source LLM engineering platform with tracing, prompt management, evaluations, and cost tracking.
  • LangSmith - LangChain's platform for debugging, testing, evaluating, and monitoring LLM applications.
  • MLflow - Open-source ML lifecycle platform with experiment tracking, model registry, and LLM evaluation tools.
  • OrcaPromptVault - Versioned archive of agent prompts, developer instructions, and tool schemas. Distinguishes locally captured material with reproduction commands from model-reported material whose accuracy is unverified. AGPL-3.0.
  • Prometheus + Grafana - Industry-standard metrics and visualization. Foundation for custom agent SLO dashboards.
  • Provena - Open-source Python library for tamper-evident audit trails of AI agent context inputs: hash-chained logging, provenance validation, and freshness checking, with EU AI Act and OWASP ASI06 compliance reporting.
  • traceAI - Open-source OpenTelemetry-native tracing for LLM and agent apps with 50+ framework integrations.
  • Weights & Biases - ML experiment tracking with LLM tracing, evaluation pipelines, and model monitoring.

Security Testing

Scanners, red-teaming tools (which run simulated attacks) and frameworks for testing how AI agents and LLMs hold up against attackers.

  • Commit - Supply chain trust scoring for npm, PyPI, Cargo, and Go packages. Scores maintainer concentration, OIDC publisher gaps, and release consistency to surface high-risk dependency patterns that vulnerability databases miss. CLI, GitHub Action, IDE hooks, REST API, and MCP server.
  • Counterfit - Azure's tool for assessing ML model security through adversarial attacks.
  • CyberSecEval - Meta's benchmark suite for LLM cybersecurity risks including insecure code generation and prompt extraction.
  • Garak - LLM vulnerability scanner from NVIDIA. Probes for hallucination, data leakage, prompt injection, toxicity, and more.
  • Hermes Jailbench - Deterministic jailbreak regression battery that runs repeatable attacks against an LLM endpoint and scores refusal, partial, and compliance outcomes across runs.
  • HouYi - Prompt injection attack framework for testing LLM-integrated application security boundaries.
  • mcp-evidence-validator - Reference implementation of SHA-256 evidence chains for MCP agents: verifies declared tool contracts against observed runtime behavior (declared vs observed). Apache-2.0, OpenSSF Best Practices badge.
  • PyRIT - Microsoft's Python Risk Identification Toolkit for red-teaming generative AI systems with automated attack strategies.
  • sentinel-scan-cli - Free, open-source CLI (and MCP server) that scans MCP manifests and LLM apps for tool poisoning, prompt injection, and rug-pulls, with OWASP LLM Top 10-mapped findings.
  • Snyk Agent Scan - Security scanner for AI agents and MCP servers. Detects vulnerabilities in tool configurations, permissions, and data flows.

Fairness & Bias Auditing

Toolkits for checking AI decision systems for unfair bias and measuring whether a fix worked.

  • Fair Code - Audits real-world-style AI decision systems (criminal justice, hiring, lending, insurance, welfare, hospital readmission, tenant screening) for algorithmic bias, pairing a biased baseline with a mitigated version and measured before/after fairness metrics.

Standards & Specifications

Protocols, specifications and regulations that matter for governing agents and for making agents from different vendors work together. - AIREP (AI Runtime Evidence Protocol) - Experimental, vendor-neutral runtime-evidence format for AI decisions, control delivery, execution, and observed effects, with signed/hash-linked artifacts and explicit missing or unevaluated states. Open specification; not a ratified standard.

  • AI Agent Trace Schema - Open JSON Schema normalizing AI-agent runtime events (tool calls, human interventions, errors, deployment changes) into a common format for compliance evidence, with converters for OpenTelemetry and LangSmith exports.
  • agent-evidence-vocabulary - Versioned CC0 vocabulary for adversarial-execution evidence claims, with crosswalk templates, governance rules, and validation tooling. Repository code and documentation are Apache-2.0.
  • Agent-to-Agent Protocol (A2A) - Google-led open protocol for inter-agent communication, task delegation, and capability discovery.
  • Universal Commerce Protocol (UCP) - Open protocol for interoperable agentic commerce, including checkout operations and response-carried request constraints that can narrow what a client is permitted to submit.
  • ACI/AIP/AJSON - Open standards for agent interoperability and governance: Agent Communication Interface (ACI), Agent Interaction Protocol (AIP), and Agent JSON (AJSON) machine-readable agent manifests for discovery, delegation, and governance.
  • CoSAI Risk Map v1 - Coalition for Secure AI's component-level risk taxonomy for agentic AI systems: 23 components, 35 controls, 36 risks (including 6 agentic-specific), 10 personas, 8 lifecycle stages. Cross-walks to MITRE ATLAS, NIST AI RMF, STRIDE, OWASP LLM Top 10, ISO 22989, and EU AI Act. Launched June 2026.
  • CSA Agentic Trust Framework - Cloud Security Alliance / MassiveScale framework defining 5 Core Elements (Identity, Behavior, Data Governance, Segmentation, Incident Response) with 25 requirements across a 4-tier maturity model (Intern, Junior, Senior, Principal). Public Review Draft v0.9.1 (April 2026). Two conformance tiers: ATF Compatible / ATF Certified.
  • CSA MCP Security Resource Center - Cloud Security Alliance community project for securing MCP servers and AI agents: hardening guides, audit database, and vulnerability database.
  • CSA Non-Human Identity & Agentic AI Governance v1 - Cloud Security Alliance whitepaper recommending a 6-field NHI registry (Identity, Owning Team, Business Purpose, Systems Accessed, Privilege Scope, Expiration/Review Date) for managing non-human and agentic identities. Published May 2026.
  • EU AI Act - EU regulation classifying AI systems by risk with requirements for transparency, human oversight, and governance.
  • Model Context Protocol (MCP) - Open protocol connecting LLMs to tools and data sources with standardized server/client architecture.
  • NIST AI 600-1 (GenAI Profile) - NIST profile of the AI Risk Management Framework focused on generative AI, enumerating 12 risk categories (CBRN, Confabulation, Dangerous Content, Data Privacy, Environmental Impacts, Harmful Bias, Human-AI Configuration, Information Integrity, Information Security, Intellectual Property, Obscene Content, Value Chain). 200+ suggested actions across Govern/Map/Measure/Manage functions. Published July 2024.
  • NIST AI Risk Management Framework - NIST framework for managing AI risk including governance and accountability.
  • NIST IR 8596 (Cyber AI Profile) - NIST profile crossing the Cybersecurity Framework 2.0 against AI in three focus areas (Secure AI System Components, Defend with AI, Thwart AI-Enabled Threats) across six CSF functions (Govern, Identify, Protect, Detect, Respond, Recover). Preliminary Draft (December 2025).
  • OpenTelemetry - Vendor-neutral observability standard for traces, metrics, and logs. Foundation for agent observability pipelines.
  • Oracle Agent Spec - Enterprise agent interoperability specification with governance and management capabilities.
  • OWASP Agentic Applications Top 10 (2026) - OWASP classification of the top 10 security risks in agentic AI: excessive agency, trust boundary failures, identity spoofing, and more.
  • OWASP AI Security Verification Standard (AISVS) - OWASP verification-controls standard with 14 chapters covering training data, input validation, model lifecycle, infrastructure, access control, supply chain, model behavior, memory/embeddings, autonomous orchestration (C9), MCP security (C10), adversarial robustness, privacy, monitoring, and human oversight. Releases v1.0 at OWASP Global AppSec EU Vienna on 24 June 2026.
  • OWASP Non-Human Identities (NHI) Top 10 (2025) - OWASP awareness list of the top 10 risks for non-human identities (Improper Offboarding, Secret Leakage, Vulnerable Third-Party NHI, Insecure Authentication, Overprivileged NHI, Insecure Cloud Deployment Configurations, Long-Lived Secrets, Environment Isolation, NHI Reuse, Human Use of NHI). Risk-ranked using OWASP Risk Rating Methodology. Published December 2024.
  • Singapore IMDA Model AI Governance Framework for Agentic AI - Singapore IMDA framework defining eight components of an agentic system (Model, Instructions, Memory, Planning & Reasoning, Tools, Protocols, Controls, Logging/Monitoring) and four governance dimensions (bounding risks, human accountability, technical controls, end-user responsibility). v1.5 published 20 May 2026
  • SPDX 3.0.1 AI Profile - SPDX specification for AI Bill of Materials (AIBOM) with AIPackage and DatasetPackage classes covering automation level, autonomy type, hyperparameters, model explainability, safety risk assessment, anonymization method, dataset size, and sensitive personal information flags. Stable since December 2024 (3.0.1); 3.1-RC1 pre-release adds EU AI Act-aligned dataset size properties (token count, item count, content duration).

Research Papers

Key academic papers on agent safety, governing groups of agents that work together, and trust in AI systems.

Industry Reports & Guidance

Practical guides, threat models (lists of how a system can be attacked) and industry analysis on governing agents.

  • AI Agent Incident Register - A numbered public corpus of AI agent incidents, each analysed for the legal duty engaged, who bears liability across the chain (deployer / shared / vendor), and the governance that would have prevented it. Includes a Liability Crosswalk mapping OWASP's agentic Top 10, the NIST AI RMF, Singapore's IMDA framework, and the EU AI Act to each other and to who carries liability. Free to read and cite; CC BY 4.0 machine-readable feed.

  • Singapore AI Governance Readiness Checklist - Free evidence-oriented planning checklist translating IMDA's 2026 agentic AI governance dimensions into 24 prompts for risk bounds, accountable ownership, lifecycle technical controls, and end-user responsibility. It is a planning aid, not a certification or compliance score.

  • Preventive vs. Reactive AI Agent Governance - Comparison of preventive control (gating an agent's write to a system of record behind human approval before it commits) versus reactive governance (detecting and remediating harmful actions after they have executed), and the trade-offs of each model.

  • Anthropic's Responsible Scaling Policy - Framework for responsible AI deployment with AI Safety Levels (ASL) and safety evaluation commitments.

  • Anthropic: Trustworthy Agents in Practice - Anthropic's practical guidance on deploying trustworthy agents: balancing autonomy with human oversight, prompt-injection resistance, and operational safeguards.
  • CSA AI Safety Initiative - Cloud Security Alliance publications on AI safety and security best practices.
  • Google Secure AI Framework (SAIF) - Conceptual framework for securing AI systems across the development and deployment lifecycle.
  • Microsoft Responsible AI - Hub for Microsoft's responsible AI practice: the six principles, the governance structure behind them, and transparency notes for shipped systems. Links out to the Responsible AI Standard itself, which is a separate document.
  • MITRE ATLAS - Adversarial Threat Landscape for AI Systems. Adversary tactics and techniques against AI/ML systems, structured like ATT&CK.
  • OWASP Agentic AI Threats and Mitigations - OWASP's threat catalog and mitigation strategies for agentic applications.
  • OWASP Top 10 for LLM Applications - Top 10 security risks for LLM applications including prompt injection, insecure output handling, and supply chain vulnerabilities.
  • EU AI Regulation Decoded - Practitioner reference mapping EU AI Act obligations to the specific evidence an auditor expects (by role, risk tier, and deadline), with common audit red flags. Includes a free interactive audit-readiness checklist and a CC BY 4.0 machine-readable obligation-to-evidence dataset.
  • Model Trust Gate - Adopter-side method for deciding whether to trust one AI model for one use, ending in a signed, expiring record with a JSON schema a policy engine can check.
  • Agent Attribution Practice (AAP) - MIT-licensed set of ten architecture decision records on how authorship, consequences and causal reconstruction of an autonomous agent's behavior are distributed (for example, capability absence, a named human approval gate for behavior-modifying writes, causal traceability), with four Business AI Quadrants for deciding whether a use case needs an autonomous agent at all.

Talks & Videos

Recorded talks and sessions on governing agents, enforcing rules while agents run, and running agents on hardware that can prove what it ran. Speaker and venue are listed so you can see who said what.

Conferences & Communities

Where research, practice and discussion on agent governance happen.

  • AAMAS - International Conference on Autonomous Agents and Multi-Agent Systems.
  • ACM FAccT - ACM Conference on Fairness, Accountability, and Transparency in sociotechnical systems.
  • AI Safety Camp - Research program for AI safety and alignment.
  • Alignment Forum - Community forum for AI alignment and safety research.
  • Confidential Computing Summit - Annual conference on confidential computing, TEE hardware, and hardware-attested workloads. Primary venue for TRACE, cMCP, and Agent Manifest launch (June 2026, San Francisco).
  • ICML - International Conference on Machine Learning, with safe and reliable ML tracks.
  • NeurIPS - Leading ML conference with AI safety, alignment, and trustworthy ML workshops.
  • OWASP GenAI - OWASP community for security guidance on generative AI and agentic applications.
  • r/agentic - Reddit community for autonomous AI agents, tools, and governance.

Contributing

Contributions are welcome. Read the contribution guidelines and open a pull request. Open-source governance tools, research papers and community resources are especially welcome.

Join the community on Discord.

Maintained by Imran Siddique, Project Lead, creator of the Agent Governance Toolkit, and contributor to OWASP ASI, CoSAI WS4, and the Agentic AI Foundation. The curator also maintains the agentrust-io repositories: Weight Custody Manifest, Agent Manifest, cMCP, cA2A, TRACE and its conformance suite and registry, and AgenTrust Telemetry.