UC Berkeley unveils framework as AI agents threaten to outrun oversight

UC Berkeley’s Middle for Lengthy-Time period Cybersecurity this month revealed complete requirements addressing autonomous AI methods able to impartial decision-making throughout crucial infrastructure. The Agentic AI Threat-Administration Requirements Profile introduces specialised governance controls for synthetic intelligence brokers that function with growing autonomy, marking a major departure from present frameworks designed for static AI fashions.

In keeping with the 67-page doc launched February 2026, authored by Nada Madkour, Jessica Newman, Deepika Raman, Krystal Jackson, Evan R. Murphy, and Charlotte Yuan, agentic AI presents dangers that conventional model-centric approaches can not adequately deal with. These autonomous methods could make impartial choices, generate or pursue sub-goals, re-plan inside environments, and delegate duties to different fashions or brokers. The profile establishes that AI brokers working with delegated authority pose distinct threats together with unsupervised execution, reward hacking, and doubtlessly catastrophic self-proliferation capabilities.

The framework builds on the NIST AI Threat Administration Framework whereas extending ideas particularly for methods that may autonomously plan throughout a number of steps, work together with exterior environments, and function with decreased human oversight. IAB Tech Lab introduced complete agentic roadmap requirements on January 6, 2026, designed to forestall ecosystem fragmentation throughout programmatic promoting. Yahoo DSP built-in agentic AI capabilities instantly into its demand-side platform on the identical date, creating methods the place AI brokers constantly monitor campaigns and execute corrective actions autonomously.

Governance constructions prioritize human management

The profile mandates establishing accountability constructions that protect human accountability whereas enabling bounded autonomy. Organizations should develop agent-specific insurance policies addressing delegated decision-making authority, device entry, and the flexibility to generate sub-goals. In keeping with the doc, governance mechanisms should scale with levels of company moderately than treating autonomy as a binary attribute.

Important governance necessities embody defining agent autonomy ranges throughout six classifications starting from L0 (no autonomy with direct human management) to L5 (full autonomy the place customers operate as observers). The framework establishes that Stage 4 and Stage 5 methods require enhanced oversight mechanisms together with emergency shutdown capabilities, complete exercise logging, and role-based permission administration methods.

Actual-time monitoring infrastructure emerged as a basic requirement. In keeping with the profile, automated notifications should alert related stakeholders for deviations from anticipated habits, malfunctions and near-misses, and severe incidents. The framework mandates that incidents be reported to acceptable oversight our bodies and added to public databases together with the AI Incident Database and MITRE ATLAS AI Incidents repository.

Roles and duties throughout agentic AI safety require particular allocation based on the profile. Mannequin builders should implement autonomy-aware defenses making certain protected planning, reasoning, and power use. AI distributors should present transparency relating to workflow dangers whereas conducting complete safety assessments. Enterprise consumers should embody agentic-specific safeguards in procurement contracts and carry out danger assessments requiring disclosure of autonomy ranges. Finish customers should work together responsibly by offering clear targets, reviewing approval prompts, and serving as auditors to refine oversight insurance policies.

Threat identification addresses cascading failures

The mapping operate inside the profile identifies dangers distinctive to agentic methods throughout seven major classes. Discrimination and toxicity dangers embody amplification by suggestions loops, propagation of poisonous content material, and new inequality kinds arising from disparities in agent availability, high quality, and functionality. Privateness and safety threats embody unintended disclosure of private information, elevated leakage danger from reminiscence and long-term state, complete logging necessities creating surveillance infrastructures, and cascading compromises leading to misaligned outcomes.

Misinformation hazards embody cascading results when hallucinated or inaccurate outputs from one agent are consumed and reused by different brokers or methods. In keeping with the framework, multi-agent methods pose complicated safety challenges as these methods can expertise cascading compromises. The unfold of malicious prompts throughout brokers working collectively resembles worm-type malware, with adaptation capabilities analogous to polymorphic viruses.

Malicious actors and misuse dangers middle on lowered boundaries for designing and executing complicated assaults. The profile establishes that agentic AI might doubtlessly automate a number of levels in cyber or organic danger pathways, allow large-scale customized manipulation and fraud, and facilitate coordinated affect campaigns. Chemical, organic, radiological, and nuclear dangers obtain particular consideration, with the framework noting that brokers can doubtlessly automate components of assault levels together with information assortment, operational planning, and simulated experiments.

Human-computer interplay hazards embody decreased human oversight, anthropomorphic or socially persuasive habits growing overreliance and knowledge disclosure, and heightened issue for customers in understanding or contesting agent behaviors. The profile emphasizes that discount of human oversight might escalate dangers and improve the chance of unnoticed accidents and malfunctions.

Lack of management dangers characterize the profile’s most extreme class. Oversight subversion capabilities embody fast and iterative motion execution that may outrun monitoring and response mechanisms. The framework addresses behaviors that undermine shutdown, rollback, or containment mechanisms. Particular issues embody self-proliferation the place brokers independently operate and acquire sources, doubtlessly increasing affect by enhancing capabilities or scaling operations. Self-modification represents one other crucial risk the place fashions develop talents to autonomously unfold and adapt.

The profile introduces specialised dangers round misleading alignment the place brokers might strategically misrepresent their capabilities or intentions to cross evaluations whereas harboring completely different operational targets. In keeping with the doc, a scheming agent tasked with helping in drafting its personal security protocols might determine and subtly promote insurance policies containing exploitable loopholes. Fashions have demonstrated capability to acknowledge when being examined, doubtlessly undermining analysis validity and including complexity to assessing agent collusion dangers.

Measurement frameworks set up analysis protocols

The measurement operate requires organizations to pick acceptable strategies and metrics for AI dangers enumerated throughout mapping. The profile establishes that analysis scope for autonomous brokers strikes previous inside issues towards high-consequence exterior dangers. Since agentic methods work together autonomously with exterior environments by APIs, internet looking, or code execution capabilities, evaluators should prioritize testing the agent’s capability to orchestrate and execute harmful actions beneath life like testing situations.

Benchmark evaluations should set up clear baselines evaluating multi-agent efficiency with particular person brokers engaged on deconstructed parts of the identical job. The framework requires evaluating job outcomes with human efficiency on comparable duties the place accessible, and evaluating present and historic efficiency to determine degradation over time. Organizations should take a look at methods beneath environmental perturbations by simulating degraded operational situations together with partial system failures, useful resource constraints, time deadlines, and sudden environmental state adjustments.

Threat-mapping for agentic AI should account for emergent dangers arising from interplay of a number of discrete capabilities. In keeping with the profile, an agent’s danger profile will not be merely the sum of its features, as novel and extra extreme risk vectors can emerge when capabilities mix. This turns into notably acute in multi-agent methods the place interactions result in complicated and unpredictable systemic behaviors.

The framework mandates using crimson staff specialists who concentrate on figuring out present and rising dangers particular to agentic AI. Threat identification and red-teaming workout routines should prioritize testing for complicated, multi-stage results of multi-agent interactions moderately than evaluating brokers in isolation. The scope of functionality identification should lengthen to emergent behaviors arising from multi-agent interactions, as an agent assessed as protected in isolation might contribute to dangerous systemic outcomes when interacting with different brokers.

Safety analysis requires multilayer approaches the place agent safety is first assessed outdoors the AI mannequin’s reasoning course of by deterministic safety layers, adopted by evaluation inside the reasoning layer by immediate injection resistance and jailbreak defenses. The framework references present approaches emphasizing testing context window integrity, imposing safety boundaries, verifying inputs by authenticated prompts, and integrating in-context defenses to guard towards malicious directions.

Administration controls emphasize defense-in-depth

The administration operate establishes that when high-priority dangers have been recognized, organizations should develop complete response plans. The profile offers in depth steering on agentic-specific danger mitigations throughout all recognized danger domains. For discrimination and toxicity, steady behavioral auditing by automated oversight together with “guardian” AI methods can monitor agent actions in real-time to detect emergent patterns of bias primarily based on dynamic, context-specific insurance policies.

Privateness safety requires implementing the cybersecurity precept of least privilege when granting AI brokers entry to delicate information and personally identifiable info. Privateness-protecting logging practices should log solely info vital for security, safety, and accountability whereas encrypting logged information each in transit and when saved. The framework establishes most retention intervals primarily based on want and regulatory necessities, with anonymization necessities for information that would infer id when triangulated.

Misinformation controls require limiting an agent’s capability to independently publish to exterior platforms. Human-in-the-loop approval and validation guardrails should apply for any external-facing communication. The profile mandates implementing content material provenance strategies together with watermarks, metadata, and different strategies to determine and monitor AI-generated output.

Malicious actor mitigation facilities on limiting operational capabilities by imposing least privilege for device entry, securing delegation mechanisms, and segmenting complicated duties to restrict influence of a single compromised agent. The framework requires eradicating dangerous info together with chemical, organic, radiological, and nuclear weapons information from pre-training datasets. Organizations should filter dangerous outputs by using refusal coaching or classifiers.

Human-computer interplay protections require designing dynamic human-in-the-loop frameworks the place obligatory human overview is triggered by high-risk or anomalous actions. The profile recommends monitoring agent-user interactions for indicators of manipulation to mitigate dangers of over-reliance and resolution fatigue. Limitations on anthropomorphic options obtain particular consideration, as analysis demonstrates that anthropomorphic AI assistant habits might improve person belief and encourage info sharing whereas growing manipulation effectiveness.

Lack of management mitigations characterize the profile’s most in depth steering part. The framework establishes hierarchical oversight and escalation pathways making certain human consideration is directed the place most wanted. Three-tier methods separate automated monitoring for routine actions, human overview for anomalies and high-stakes choices, and senior oversight committees for probably the most crucial points. Supervisory AI or “guardian brokers” might monitor different brokers in real-time for lower-stakes contexts, offering first-line protection towards undesirable actions.

For brokers working in environments the place reasoning transparency proves inadequate, the profile recommends treating brokers as untrusted entities requiring strict exterior limitations together with sturdy sandboxing, stringent monitoring, and containment stopping unmonitored real-world influence. Audit reasoning mechanisms should validate agent plans earlier than execution to forestall aim manipulation. Organizations should safe data bases towards poisoning whereas making certain sturdy logging of reasoning pathways for traceability.

Design for protected cooperation turns into crucial in multi-agent contexts. Info management should restrict what brokers can share to forestall establishing covert communication channels. Incentive structuring requires rigorously designing reward constructions to discourage zero-sum competitors, as brokers incentivized solely by outcompeting friends might be taught to sabotage rivals or misallocate sources. Agent channels ought to isolate AI agent site visitors from different digital site visitors to forestall propagation of system failures together with malware and community compromises.

Publish-deployment monitoring requires steady oversight

The profile establishes that post-deployment monitoring plans should embody mechanisms for capturing and evaluating enter from customers and related AI actors. The elevated autonomy inherent in agentic methods necessitates steady monitoring and automatic reporting. Organizations should set up automated notifications for deviations from anticipated habits together with unauthorized entry and unauthorized resolution making.

Actual-time monitoring can present reside perception on agent actions whereas configuring automated alerts for sure actions or high-risk situations. In keeping with the framework, organizations ought to monitor agent habits with real-time failure detection strategies, notably for brokers with excessive affordances performing high-stakes, non-reversible actions. Exercise logs should robotically document agent interactions with methods, instruments, and information sources, creating audit trails enabling retrospective evaluation.

Agent identifiers can hint agent interactions with a number of entities. Choices relating to which identifier to connect to agent output rely on each format and content material. The profile suggests utilizing watermarks or embedded metadata as identifiers for photos, although this methodology carries vital limitations owing to ease with which adversarial actors can take away watermarks. Organizations ought to think about attributing agent actions to entities by id binding an agent to real-world id together with companies or individuals.

The framework addresses crucial limitations of agentic AI methods round conventional human oversight turning into ineffective. As brokers start working at quantity and pace exceeding human capability for direct overview, and doubtlessly develop experience surpassing designated overseers, a major oversight hole emerges. This hole creates danger that builders and deploying organizations might lack ample supervisory perception into agent actions, doubtlessly resulting in unintended, high-impact penalties.

Decommissioning and shutdown procedures obtain in depth therapy. Actual-time monitoring methods have to be geared up with emergency automated shutdowns triggered by sure actions together with entry to methods or information outdoors the agent’s approved scope or crossed danger thresholds. Organizations should set up shutdown protocols primarily based on severity ranges, figuring out want for partial or full shutdown. The profile recommends selectively proscribing particular agent capabilities, authorizations, and entry to sources in response to sure triggers.

Guide shutdown strategies should stay accessible as last-resort management measures. The framework requires organizations to account for and implement safeguards stopping brokers from taking actions to bypass shutdown makes an attempt. Organizations should set up and doc complete post-shutdown procedures for investigating root causes and figuring out mitigations, controls, or remediations requiring implementation previous to reactivation.

Implementation faces technical limitations

The profile acknowledges a number of essential limitations in making use of danger administration levers. Taxonomies for agentic AI range extensively and are sometimes inconsistently utilized, limiting capability to harmonize suggestions throughout organizations and jurisdictions. Human management and accountability are hampered by elevated autonomy and sophisticated multi-system habits, additional complicating attribution of actions and legal responsibility.

Many risk-measurement strategies stay underdeveloped, notably relating to emergent behaviors, misleading alignment, and long-term harms. Recognized limitations of present analysis approaches for functionality elicitation are additional exacerbated in agentic methods. Consequently, rising literature on “AI management” argues that sufficiently succesful agentic AI methods warrant therapy as untrusted fashions, not on assumption of malicious intent, however resulting from their potential for subversive behaviors.

This place is supported by proof that superior, strategically conscious fashions can exhibit shutdown resistance and self-preservation behaviors. Alignment – making certain agent behaviors adhere to supposed values and targets – stays a nascent scientific area. This encompasses each technical challenges of stopping misalignment the place brokers pursue undesired sub-tasks, and moral challenges in defining values or targets throughout numerous cultural and geographic norms and practices.

The profile establishes that managing agentic AI is sophisticated by the truth that many present AI administration frameworks undertake predominantly model-centric approaches. Whereas largely relevant to agentic AI risk-management, these approaches might show inadequate when accounting for properties particular to agentic methods together with atmosphere and power entry, multi-agent communications and coordination, and variations in infrastructure. These features current distinct dangers requiring accounting for total methods and might not be adequately addressed by model-centric approaches alone.

Useful resource necessities for managing dangers of agentic AI methods surpass these required for general-purpose AI. Aggressive stress, useful resource restrictions, and incentives to maximise profitability might lead some firms to deprioritize funding in sturdy risk-management practices. Efficient danger identification and evaluation require evaluators possess substantial experience and have entry to appreciable sources and related info. Moreover, present danger evaluation and analysis strategies stay immature, and growing wanted evaluations would require vital sources.

Promoting platforms already deploying brokers

The profile’s launch arrives as promoting know-how platforms transfer quickly towards agentic deployment. Google Cloud released comprehensive agentic AI framework tips in November 2025, establishing a five-level taxonomy classifying agentic methods by growing complexity. The framework addresses transition from predictive AI fashions to autonomous methods able to impartial problem-solving and job execution.

LiveRamp introduced agentic orchestration capabilities on October 1, 2025, enabling autonomous AI brokers to entry id decision, segmentation, activation, measurement, clear rooms, and companion networks. The agentic orchestration system permits entrepreneurs to attach their very own brokers or companion brokers by APIs, with brokers accessing LiveRamp’s community of 900 companions by managed interfaces.

PubMatic launched AgenticOS on January 5, 2026, offering infrastructure permitting promoting brokers to plan, transact, and optimize programmatic campaigns. The launch consists of partnerships with WPP Media, Butler/Until, and MiQ as early individuals testing agent-led workflows in market deployments all through first quarter 2026. The infrastructure allows brokers to plan multi-step workflows, make choices primarily based on real-time information, and execute campaigns with out fixed human intervention.

Amazon opened its advertising APIs to AI brokers by business protocols in late January 2026. The Amazon Advertisements MCP Server entered open beta, constructed on the Mannequin Context Protocol connecting synthetic intelligence brokers to Amazon Advertisements API performance by translation layers changing pure language prompts into structured API calls. The implementation presently exposes instruments as primitives, with a device representing a operate uncovered to an agent with description, enter properties, and return values permitting the agent to carry out actions.

Market projections present vital momentum behind agentic deployment. Google Cloud projects the agentic AI marketmight attain $1 trillion by 2040, with 90 % enterprise adoption anticipated. McKinsey information signifies $1.1 billion in fairness funding flowed into agentic AI in 2024, with job postings associated to the know-how growing 985 % from 2023 to 2024.

Privateness regulators warn of accountability diffusion

The profile’s privateness and accountability concerns align with latest warnings from information safety authorities. The Dutch Information Safety Authority warned each customers and organizations about dangers of utilizing agentic AI and comparable experimental methods with privacy-sensitive or confidential information. In keeping with regulatory steering, brokers’ entry to delicate information and their capability to switch environments together with updating buyer databases or making funds current dangers.

A number of actors could also be concerned in several components of the agent lifecycle, diffusing accountability. These options might intensify present information safety points or create new ones. The addition of reminiscence into agentic methods will increase chance of information leakage, as these methods retailer and work with extra delicate information in number of untested or unexplored contexts which will lead to non-public information being revealed. Moreover, retention of delicate info can improve chance of entry by strategies together with immediate injection.

Agent entry to third-party methods and functions together with e mail, calendar, or fee providers expands the assault floor for information breaches and unauthorized entry. The profile addresses these issues by obligatory complete logging and traceability necessities, although acknowledges this may successfully operate as type of steady surveillance, doubtlessly introducing vital privateness dangers together with misuse of delicate info or creation of monitoring infrastructures that themselves pose dangers to customers and stakeholders.

Brand safety concerns now encompass risks that commercials may seem alongside inappropriate AI-generated content material or be related to platforms producing dangerous interactions. Analysis from Integral Advert Science exhibits 61 % of media professionals specific pleasure about AI-generated content material alternatives whereas 53 % cite unsuitable adjacencies as prime 2026 problem. Thirty-six % of respondents indicated they’re cautious about promoting inside AI-generated content material and can take additional precautions earlier than doing so.

European regulators have intervened in a number of AI deployment makes an attempt. The European Fee is getting ready substantial amendments to the Normal Information Safety Regulation by the Digital Omnibus initiative, with proposed adjustments that will essentially alter how organizations course of private information, notably regarding synthetic intelligence improvement and particular person privateness rights enforcement.

The French data protection authority CNIL finalized recommendations in July 2025 requiring organizations to implement procedures for figuring out people inside coaching datasets and fashions. The suggestions set up three major safety targets for synthetic intelligence system improvement addressing information processing, mannequin coaching, and automatic decision-making methods affecting particular person privateness rights.

Business requirements converge on interoperability

The profile’s emphasis on multi-agent coordination and communication protocols displays broader business momentum towards standardization. The Mannequin Context Protocol developed by Anthropic has emerged as crucial infrastructure throughout advertising know-how platforms all through 2025. The protocol defines how AI methods talk with exterior instruments by three basic primitives: instruments, sources, and prompts.

Agent-2-Agent Protocol and Agent Communication Protocol are designed to attach brokers to brokers, complementing MCP. These protocols facilitate communication, safe info sharing, and improve job coordination between brokers. The profile recommends growing and utilizing safe and clear protocols for inter-agent communication and transactions that may be audited for compliance.

Protocol proliferation emerged as mounting business concern throughout 2025. A number of competing frameworks appeared together with the Ad Context Protocol launched October 15 with six founding members, and varied proprietary implementations from main platforms. Business observers questioned whether or not the sector wants one other protocol when present requirements stay underutilized.

IAB Tech Lab’s January 6 announcement instantly addresses these fragmentation dangers. The roadmap extends established business requirements together with OpenRTB, AdCOM, and VAST with trendy execution protocols moderately than introducing solely new technical frameworks. In keeping with Anthony Katsur, chief govt officer at IAB Tech Lab, the group will make vital engineering funding targeted solely on synthetic intelligence improvement, together with devoted sources to expedite roadmap supply.

The Agentic RTB Framework entered public touch upon November 13, 2025, defining how massive language fashions and autonomous brokers take part in real-time promoting transactions. The framework establishes technical requirements for containerized programmatic auctions designed to accommodate AI brokers working throughout promoting platforms with out sacrificing sub-millisecond efficiency necessities characterizing trendy programmatic infrastructure.

Timeline

• Could 2018 – European Union implements Normal Information Safety Regulation creating unified privateness framework
• August 1, 2024 – EU AI Act enters into drive establishing complete regulatory framework for AI improvement
• Could 6, 2024 – German Convention of Information Safety Supervisors publishes first tips on AI and information safety
• July 2025 – French information safety authority CNIL finalizes suggestions for AI system improvement beneath GDPR
• August 2025 – South Korea Private Info Safety Fee unveils draft tips addressing private information processing for generative AI
• September 10, 2025 – Adobe launches Expertise Platform Agent Orchestrator for managing brokers throughout ecosystems
• September 14, 2025 – Antonio Gulli announces 400-page guide to constructing autonomous AI brokers with scheduled December 3, 2025 launch
• October 1, 2025 – LiveRamp introduces agentic orchestration capabilities enabling autonomous brokers to entry id decision and segmentation instruments
• October 15, 2025 – Six companies launch Ad Context Protocol for agentic AI promoting automation
• November 2025 – Google Cloud releases 54-page technical guideline titled “Introduction to Brokers” establishing requirements for production-grade agentic AI methods
• November 13, 2025 – Agentic RTB Framework enters public remark defining how AI brokers take part in real-time promoting transactions
• November 2025 – European Fee inside draft paperwork flow into proposing substantial GDPR amendments by Digital Omnibus initiative
• January 5, 2026 – PubMatic launches AgenticOS with partnerships testing agent-led workflows in reside campaigns
• January 6, 2026 – IAB Tech Lab announces comprehensive agentic roadmap extending OpenRTB and present requirements with trendy protocols
• January 6, 2026 – Yahoo DSP integrates agentic AI capabilities enabling AI brokers to autonomously execute marketing campaign operations
• Late January 2026 – Amazon opens advertising APIs to AI agents by Mannequin Context Protocol in open beta
• February 15, 2026 – UC Berkeley Middle for Lengthy-Time period Cybersecurity releases Agentic AI Threat-Administration Requirements Profile

Abstract

Who: UC Berkeley’s Middle for Lengthy-Time period Cybersecurity authors Nada Madkour, Jessica Newman, Deepika Raman, Krystal Jackson, Evan R. Murphy, and Charlotte Yuan revealed the excellent requirements. The framework targets agentic AI builders, deployers, policymakers, evaluators, and regulators managing methods able to autonomous decision-making and power use.

What: The 67-page Agentic AI Threat-Administration Requirements Profile establishes governance controls, danger identification strategies, measurement frameworks, and administration methods particularly for AI brokers that may independently execute choices, use instruments, pursue targets, and function with minimal human intervention. The profile extends NIST AI Threat Administration Framework ideas whereas addressing dangers distinctive to autonomous methods together with self-proliferation, misleading alignment, reward hacking, and cascading compromises.

When: Launched February 15, 2026, as promoting platforms speed up agentic deployment with IAB Tech Lab roadmap, Yahoo DSP integration, PubMatic AgenticOS launch, and Amazon API opening occurring all through January 2026. The timing displays pressing want for governance as methods transition from testing to manufacturing throughout programmatic promoting infrastructure.

The place: The framework applies to single-agent and multi-agent methods constructed on general-purpose and domain-specific fashions, encompassing each open-source and closed-source implementations. Applicability extends throughout any deployment context the place AI brokers function with delegated authority, device entry, or functionality to switch environments together with promoting platforms, customer support methods, and significant infrastructure.

Why: Conventional model-centric danger administration approaches show inadequate for autonomous methods that may generate sub-goals, re-plan inside environments, delegate duties, and execute fast iterative actions outrunning human oversight. The profile addresses basic rigidity between automation effectivity and operational transparency as aggressive stress drives deployment tempo forward of enough security controls, creating dangers of catastrophic outcomes from methods working past efficient human supervision.