transition2.aiStart a project

Enterprise AI Governance

Policy-as-Code for AI Systems

Turn AI governance rules into versioned, tested code that runs in the release pipeline and in front of every agent action.

Eric MacDougall ·

Answer first

Policy-as-code means writing governance rules as machine-checkable code, stored in version control, reviewed like any other change and tested before it ships. For AI systems it does two jobs: it blocks releases that lack required evidence, and it decides at runtime whether an agent may take a specific action. A rule that only lives in a PDF cannot do either.

Where policies run in an AI system

There are two enforcement points, and most teams need both.

Release time. The pipeline asks: does this change to a model, prompt, tool or data source carry the evidence its risk tier needs? Typical rules: an evaluation report exists and passes its thresholds, permission scopes have not widened without approval, a rollback path is defined, and the inventory entry is current.

Run time. A policy decision point sits in front of every tool call an agent makes and returns allow, deny or "needs approval". Inputs include the agent identity, the person it acts for, the tool, the arguments and context such as record owner, amount or destination. This takes the per-session, no-implicit-trust principle of NIST SP 800-207 (Zero Trust Architecture) down to each tool call.

Choosing an engine

You can write rules in your application language, but a dedicated engine keeps policy separate from business code and gives you tooling for testing and decision logs. Two widely used open-source options:

  • [Open Policy Agent (OPA)](https://www.openpolicyagent.org/) is a general-purpose policy engine with its own language, Rego. It is a graduated CNCF project and is common in Kubernetes and CI pipelines, which makes it a natural fit for release gates.
  • [Cedar](https://www.cedarpolicy.com/) is an open-source authorization policy language designed around principals, actions and resources, which maps directly onto "may this agent call this tool on this record".

Pick one engine per enforcement point and stick with it. Two half-adopted engines are harder to audit than one.

Step-by-step implementation

  1. Write the rule in plain language first, with its owner and the risk it addresses, for example "Tier 3 agents may not send external email without approval".
  2. Translate it into policy code with explicit inputs, and default to deny when an input is missing.
  3. Write tests for each rule: at least one case that must allow, one that must deny and one with missing or malformed input.
  4. Run the policy tests in CI so a policy change cannot merge with failing tests.
  5. Deploy the policy bundle with a version number, and log that version with every decision.
  6. Start new runtime rules in "log only" mode, compare their decisions with what actually happened, then switch to enforce.
  7. Review denied and approval-required decisions weekly to find rules that are wrong or too broad.

Example rules worth writing first

  • Block a release if the evaluation report for the changed component is missing or older than the change.
  • Block a release if any tool's permission scope widened and no approver is recorded.
  • Deny agent calls to tools not on that agent's allow list.
  • Require approval for payments, deletions, permission changes and messages to external domains.
  • Deny retrieval of documents the requesting user could not open directly.
  • Deny any call whose arguments contain secrets or credential patterns.

How to know the policies work

Policies fail quietly. Prove them with checks you can repeat:

  • Negative tests: each deny rule has a test that tries the forbidden action and expects a deny.
  • Coverage: count the share of tool calls that pass through the decision point. Anything below all of them is a gap to explain.
  • Drift: compare the deployed policy version with the version in main on a schedule.
  • Decision review: sample allow decisions as well as denies; a rule that never denies may not be wired in.

These measurements also give you evidence for the "Measure" and "Manage" functions of the NIST AI RMF.

Decision framing

Policy-as-code costs engineering time up front. It pays back when the same rules would otherwise be checked by hand in every review, and when an auditor or customer asks how a control is enforced. The honest pitch is faster, more consistent approvals and evidence that controls actually ran, not a guaranteed reduction in incidents.

Risks and limits

  • A policy can only decide on the inputs it receives. If the agent can call a tool by another path, the rule does not apply there.
  • Overly broad deny rules push people to work around the system. Watch the approval queue and exception count.
  • Policy code is code. It needs owners, review and its own change log.

Sources