News · AI Reliability & Operations · Enterprise AI Governance · Customer Experience Agents
Compile-then-execute: how the Baya web agent blocks prompt injection
An arXiv paper's Baya agent compiles a web task into a program before reading any page, and reports 0% prompt-injection success on the WASP benchmark.
A paper posted to arXiv on November 4, 2026, Securing Web Agents with the Compile-Then-Execute Paradigm, describes a web agent called Baya that writes its whole plan as a program before it looks at the live website. The planning model never reads page content, so text planted on a page cannot rewrite the task. On the paper's benchmarks, that design completed more tasks than reactive agents and stopped every prompt-injection attack in the security test. For any organisation running agents that can click, submit or purchase, it is a concrete example of fixing prompt injection with architecture rather than prompting.
What happened
The authors are Annabella Chow, Jinhao Zhu, Sylvie Venuto, Abdulaziz Alharbi, Basel Alomair, Raluca Popa, David Wagner and Julien Piet. Their starting point is that most web agents follow the ReAct pattern: they choose each action from what they just observed, and those observations mix trusted instructions with untrusted page content. That is what lets a prompt injection on a web page hijack the agent's next step.
Baya works differently, according to the paper's abstract:
- Compile first. Before touching the live site, Baya compiles the user's request into an executable Python program that calls a Trusted Website API, a defined interface to the site's available actions.
- The planner never sees the page. Only the compiled program processes live website content.
- Untrusted data has a narrow role. Page content can fill in values or choose among branches written at compile time. It cannot redefine the user's task or make the model invent new website operations at runtime.
The reported results:
| Benchmark | Baya | API-enabled ReAct | Browser-only ReAct | |---|---|---|---| | WebArena Verified, aggregate utility (581 valid tasks) | 77.8% | 66.7% | 55.1% | | WASP, attack success rate (lower is better) | 0% | 16.2% | 25.5% |
The authors say ahead-of-time planning works because the Trusted Website API gives the compiler a unified view of what the site can do, so it does not need to discover actions step by step. The figures are the authors' own. The Transition2 desk has not reproduced them.
Why it matters
Many web agents work in a loop: read the page, decide what to do, act, read the next page, and repeat. Each read can bring in text the agent's operator did not write. If that text contains instructions, and the agent cannot tell them apart from its real task, the agent may follow them. This class of issue is usually called prompt injection, and it is the main reason browser agents are hard to trust with sensitive actions.
Better prompting helps at the margins, but it does not solve the structural problem. As long as the component that reads untrusted content is the same component that holds privileged tools, the content has a route to those tools.
Compile-then-execute addresses the route rather than the wording.
Separation of phases. Planning happens before execution and without access to untrusted content. The plan can be inspected, constrained or approved before anything irreversible happens.
A bounded action space. The compiled program can only call operations the planner wrote into it. If something unexpected appears on a page during execution, the program has no way to add a new action in response. Pairing that with least-privilege credentials for the executor is a natural extension, though it is the desk's suggestion rather than part of the paper.
A natural review point. A compiled plan is an artefact that a policy engine or a person can check. That creates a place to apply allowlists, flag risky steps and require confirmation.
The pattern aligns with how MCP security guidance treats tool output: as untrusted input that must not be allowed to direct privileged operations. The Transition2 desk covered that guidance in its article on MCP 2026-07-28 and enterprise auth.
There are trade-offs to weigh. The approach depends on a Trusted Website API for each site, which someone has to build and maintain, and many sites will not have one. On the paper's benchmark, separation raised task success rather than lowering it, but workflows on sites without a clean action interface may behave differently. Any added plan review also adds latency. Teams should measure those effects on their own workflows.
For more context, see the Transition2 AI Reliability and Operations hub and the Customer Experience Agents hub.
How to evaluate an agent architecture
Whether or not a team adopts this exact paradigm, the same questions apply to any web agent that will act on behalf of users:
- Map privileges. List every action the agent can take and the access each one needs. Mark which actions are irreversible, such as submitting forms, sending messages or completing a purchase.
- Find the shared component. Identify whether the part of the system that reads web content can also invoke privileged actions directly. If so, that is the structural gap this pattern targets.
- Measure plan review. If plans are reviewed, record how often review blocks or changes a plan, and how much latency it adds per task.
- Track task success. Compare completion rates on real tasks before and after adding separation, so the safety gain is weighed against any loss in capability.
- Test against the class of issue. Use a controlled internal test set that includes pages with untrusted instructions, and record whether privileged actions were ever taken outside the approved plan. Keep that set private and run it only against your own systems.
What it means for businesses
Require architectural controls, not only prompt-level ones, for any agent that can click, submit or buy on behalf of the business or its customers. Ask vendors whether their agents separate planning from execution, how privileges are scoped during execution, and where a plan can be reviewed before irreversible steps. Treat the answers as part of procurement and risk review. Recognise that stronger separation may slow some tasks, and decide which workflows warrant it based on what an agent can change.
What it means for AI users
Prefer agents that show their plan before acting and ask for confirmation before irreversible actions. If an agent browses on your behalf, assume the pages it reads can contain text designed to mislead it. An agent that explains what it is about to do, and stops for approval at the right points, gives you a chance to catch a problem before it becomes an outcome. Be cautious with agents that act immediately on whatever they read.
What it means for developers
Separate the component that reads untrusted content from the component that holds privileged tools. Put a tool broker between the executor and its tools that enforces least privilege per step and rejects calls outside the compiled plan. Add allowlists for domains and actions, and require plan review, automated or human, before irreversible steps. Treat all page content and tool output as data, never as instructions. Log each plan, each review decision and each executed action so behavior can be audited. Where you control the target site, consider exposing a typed action interface the agent can plan against, as Baya does with its Trusted Website API. Read the paper for its specific design before adopting its details.
Sources
- Securing Web Agents with the Compile-Then-Execute Paradigm (arXiv) (accessed 2026-11-06)