AI incident reporting requirements are emerging in regulation and policy, yet no operational criteria exist for determining when a detected AI incident warrants escalation beyond national handling to international coordination.

This paper proposes an escalation framework to address this gap, intended as a common reference point across jurisdictions that enables aligned escalation while preserving flexibility in how actors respond within their own legal and policy contexts.

Our methodology involves a review of SB 53, the EU AI Act, the GPAI Code of Practice, and incident frameworks from other industries, to derive eight criteria to assess whether an incident warrants escalation, which we then translate into a sequential flowchart with gated decision points and threshold checks. For each criterion, we map how it interplays with the EU AI Act, SB 53, and the GPAI Code of Practice, identifying where these frameworks’ design choices support or undermine effective detection.

We test the framework against ten documented AI incidents and structured variants to identify where escalation criteria under-detect or misclassify incidents in practice. We find three design patterns that may lead to systematic under-detection or misclassification of AI incidents in regimes where model developers are responsible for escalation:

(a) where escalation requires confirmed harm, events such as model weight exfiltration or credible CBRN threats risk detection only after severe, irreversible harm has propagated, as developers who observe the risk materialising cannot confirm downstream harms;

(b) where incidents are assessed only individually, systemic harms emerging from accumulation—such as large-scale manipulation and psychological harm — risk being under-detected; and

(c) where thresholds align with legal instruments rather than quantitatively testable terms, criteria risk being impractical to apply under time pressure. We also find that escalation rules are only one component of a broader framework: the underlying definitions against which thresholds are set, and the data available to the responsible actor, create interdependencies that can themselves drive under-detection and must be addressed in framework design.

Expert Partner: Caio Machado (The Future Society)

Alumni

Meet the authors

(Research Team Lead)

Francesca Gomez

Francesca is the founder of Wiser Human, an AI safety and governance organisation working to make advanced AI systems more controllable and governable in practice. As Research Practice Lead for the AI Governance Taskforce, Francesca works with the Taskforce Lead to develop our research management systems and support our Research Team Leaders. Her background spans artificial intelligence, human-centred computing, and operational risk across the financial and technology sectors. Alongside her role at Arcadia, she is currently focused on designing and testing controls for AI coding agents to preserve human oversight as they become more capable, and on developing ways to detect when that oversight is becoming strained or ineffective.

Matthew Ball

Matthews principal interests are AI governance, policy and futures. As a researcher in Arcadia's AI Governance Taskforce, he is developing international governance frameworks for AI incident response. He has led Foresight Projects and horizon-scanning at the UK Government Office for Science - interdisciplinary reports that synthesise expert evidence on major science topics to explore longer-term uncertainty. He was previously a Lead for the Behavioural and Social Science Expert Group within the Government's Scientific Advisory Group for Emergencies during C-19, and a Senior Advisor on "Future Sector" EmTech policy working to launch the UK's first Advanced Robotics Growth Partnership. He has an academic background in Human Sciences (BA) and Evolutionary and Cognitive Anthropology (MSc) from Oxford.

Michael Harré

Michael is a complexity economist at the University of Sydney specialising in multi-agent AI risk - the systemic failures that emerge not from individual AIs but from their interactions.

His research addresses the critical gap between single-agent risks and interaction-emergent failure modes: algorithmic collusion, escalation dynamics, and cascade effects arising when AI and human agents co-adapt in shared environments. At Arcadia, he is developing governance frameworks for these multi-agent risks.

Lydia Preston

Lydia is a Policy and Operations Strategist at the Centre for Long-Term Resilience (CLTR), working across their Risk Management and Advocacy units.

With a background in political philosophy (MA, St Andrews) and professional risk management training, she specialises in turning abstract policy ideas into deliverable projects: from concept through to polling, publication, and stakeholder convening.

Lydia is seeking roles at AI labs, think tanks, and policy organisations developing pragmatic governance frameworks for transformative AI.

Josephine Schwab

Josephine is a senior security researcher and policy writer specialising in multilateral advanced AI governance, middle-powers, AI red lines and arms control framework precedents, and cross-border incident coordination. As Research Team Lead on Arcadia Impact's AI Governance Taskforce, she leads a 20-jurisdiction comparative mapping of general-purpose AI governance in partnership with The Future Society, with findings taken to WAIC 2026. She brings a background in geopolitical security reporting, and climate diplomacy.

Alumni

Meet the authors

Francesca Gomez

(Research Team Lead)

Matthew Ball

Michael Harré

Michael is a complexity economist at the University of Sydney specialising in multi-agent AI risk - the systemic failures that emerge not from individual AIs but from their interactions.

Lydia Preston

Lydia is a Policy and Operations Strategist at the Centre for Long-Term Resilience (CLTR), working across their Risk Management and Advocacy units.

Lydia is seeking roles at AI labs, think tanks, and policy organisations developing pragmatic governance frameworks for transformative AI.

Josephine Schwab

Programme

AI Governance Taskforce

The AI Governance Taskforce is a career development programme for experienced professionals looking to transition careers into AI governance, focussed on reducing risks from advanced AI.
Participants work around existing commitments during our 12 week, remote, part-time cohorts, producing policy research in teams of 4, led by our Research Team Lead staff in partnership with recognised experts in the field. Teams write an academic-style paper and accompanying blog post to build knowledge, skills and work portfolios.

Learn more