Why This Piece Needed Writing
The public conversation about artificial intelligence tends to oscillate between utopian enthusiasm and existential panic, and neither register is particularly useful for practitioners who have to make real decisions about deploying these systems. What I wanted to do with this piece, published in full over at The Dispatch, was slow that conversation down and ask the prior question: before we decide how to govern AI agents, do we actually agree on what kind of thing they are? The answer, it turns out, is no. And that disagreement has consequences.
The article draws on sources across computer science, moral philosophy, Catholic social teaching, international humanitarian law, and governance frameworks. That is an unusual combination, but I think it is the right one. The problem of autonomous AI systems is genuinely interdisciplinary, and treating it as purely a technical or purely a regulatory matter is how we end up with frameworks that look serious on paper but fail structurally in practice.
The Accountability Gap Is Not a Future Problem
The central argument of the piece is that the deployment of agentic AI systems, systems that act without waiting for human authorisation, has already outpaced the conceptual and legal infrastructure needed to assign responsibility when things go wrong. The UK National Cyber Security Centre has published work envisioning autonomous systems that detect, respond to, and contain threats at machine speed. The operational logic is sound: attackers do not wait for human approval cycles, and defenders cannot remain permanently slower. But the moment a system acts without asking, we have crossed a threshold that philosophy has a name for: moral agency. And moral agency carries with it the question of who is responsible.
The responsibility gap describes the structural condition that emerges when an AI system causes harm and no individual human actor is clearly culpable. The developer says the system behaved within its specified parameters. The deployer says they followed vendor guidance. The operator had no visibility into the decision logic. This is not a hypothetical edge case. It is the default condition of autonomous system deployment at scale, and it is precisely what both the NIST AI Risk Management Framework and the EU AI Act are attempting to address. Both are serious efforts. Both also share a structural limitation: they are built on risk management logic, which requires you to anticipate the failure modes you are managing. For genuinely novel agentic systems, that assumption does not hold. The Future Society’s analysis of the EU AI Act notes that it was not originally designed with agents in mind, meaning multi-purpose agents may default to high-risk classification while the technical standards being developed are likely to fall short of what is actually needed.
The Alignment Problem Is Political, Not Just Technical
One of the arguments I found most compelling in researching this piece came from Dr. Samuel Segun’s work on value alignment. The standard framing treats alignment as a technical challenge: how do you get a system to do what you intend? Segun’s critique is that this framing obscures a prior question about who “we” refers to. When AI systems are trained predominantly on feedback from Western annotators addressing Western concerns, those priorities get encoded as universal defaults. For cybersecurity specifically, this matters directly. An AI agent making autonomous defensive decisions carries embedded judgments about what constitutes a threat, what constitutes a proportionate response, and whose infrastructure is worth protecting. Those judgments are not labelled as such. They surface as outputs and are applied as if they were objective.
The primary mechanism for achieving alignment in large language models, Reinforcement Learning from Human Feedback, risks encoding majority preferences and reducing what Segun calls distributional pluralism. His proposed remedies include annotation-level transparency, architectural pluralism, and governance-level inclusion. The structural barriers to implementing those remedies, particularly for regions in the Global South that lack the resources to develop advanced AI systems independently, are significant and largely unaddressed in current governance frameworks.
The Humanitarian Dimension
The piece also covers the autonomous weapons debate, which cybersecurity practitioners sometimes treat as a separate conversation when it is actually the sharpest version of the same one. The Stop Killer Robots coalition, representing more than 250 member organisations across over 70 countries, defines autonomous weapons systems as those where targeting decisions are determined by sensor processing rather than human judgment. In November 2023, 164 states voted in favour of the first UN General Assembly resolution on killer robots. The UN Secretary General and the International Committee of the Red Cross jointly called for a binding legal instrument by 2026. That deadline has passed. There is no binding instrument. The philosophical debate has become a humanitarian emergency, and the emergency has outpaced the negotiation.
Pope Leo XIV’s encyclical Magnifica Humanitas, published in May 2026, enters this space with a seriousness that secular governance frameworks rarely match. It does not treat AI as a threat or a tool but as a question about what we owe each other when harm is done at a distance by a system no individual fully controls. That framing, I would argue, is more useful for practitioners than most of the risk management language currently on offer.
Read the Full Piece
This post covers the main threads, but the full article goes considerably deeper on each of them, including the 21 distinct proposed tests for machine consciousness, the specific provisions and limitations of the EU AI Act as applied to agents, and the governance implications of Kissinger’s observation that we may have created a dominating technology in search of a guiding philosophy. You can read the complete piece here.
Leave a Reply