Securing Defence Autonomy in the Age of AI-Native Systems

Written by Elisa Garbil – 24.07.2025

---

As AI-native systems redefine the contours of military capability, defence institutions face a fundamental challenge: how to harness the advantages of autonomy, connectivity, and speed without undermining democratic oversight, institutional control, and strategic clarity. While modern warfare increasingly depends on brilliant machines, which are autonomous, adaptive, and integrated, the risks introduced by these systems are equally profound. Ensuring that AI serves national security interests without compromising operational resilience or ethical values should be a core strategic imperative.

The European Defence Agency’s white paper, Trustworthiness for AI in Defence (TAID), alongside NATO’s modernisation initiatives, presents a multidimensional framework for managing the risks associated with autonomous defence systems. These frameworks emphasise sovereignty, resilience, explainability, and ethical compliance as pillars of responsible military AI.

Strategic Sovereignty Without Centralised Fragility

In a contested battlespace, centralisation becomes a liability. Defence AI systems must operate under degraded conditions, disconnected from cloud resources or denied access to satellite communications. Strategic sovereignty can no longer rely on centralised command and control architectures that collapse under cyberattack or jamming. The TAID paper stresses the need for resilient, distributed systems that continue functioning autonomously when isolated. Minimising operational dependencies and maximising mission continuity. This requires a shift in architectural design philosophy: from efficiency-centric to mission-survivable systems. 

Innovation Without Democratic Drift

The tempo of military innovation is increasingly driven by the private sector, where development cycles vastly outpace traditional procurement models. While this accelerates capability adoption, it also risks introducing systems whose design and purpose may misalign with democratic mandates or military doctrine. To bridge this gap, defence actors are urged to adopt comprehensive governance frameworks that integrate legal, ethical, and operational safeguards.

The TAID framework mandates a Governance, Risk, and Compliance (GRC) approach across the full AI lifecycle. These frameworks ensure that civilian-sourced technologies are systematically evaluated against military criteria such as robustness, security, traceability, and mission alignment. NATO’s six Principles of Responsible Use (PRU), which covers (1) lawfulness, (2) accountability, (3) reliability, (4) explainability, (5) governability, and (6) bias mitigation, further reinforces this alignment.

These standards create an important and vital traceable chain of accountability between AI providers, developers, integrators, and operators. Risk ownership is explicitly transferred at every lifecycle stage, ensuring that no capability enters the field without a clear legal and ethical anchor.

Brilliant Machines Without Black Boxes

As militaries field increasingly intelligent systems capable of decision-making and adaptation, a core concern is that such platforms could replicate the opaque logic and surveillance architectures used by adversarial states. The integrity of NATO and EU defence strategies depends not only on technological superiority, but on maintaining a moral and procedural distinction in how these systems are designed and deployed.

The EDA claims to have value-based engineering central to the TAID methodology. This requires that AI systems be designed with embedded transparency, fairness, and human-centered control structures. Technical features like explainability, traceability, and decision auditability are claimed not optional, and are claimed to be prerequisites for fielding autonomous systems in line with EU constitutional values and international humanitarian law.

Furthermore, human-machine teaming remains foundational. TAID defines operational roles along a continuum: human-in-the-loop, on-the-loop, and out-of-the-loop. This taxonomy enables tailored autonomy levels that match mission risk and ethical sensitivity, while ensuring that humans retain command responsibility, especially in kinetic operations.

Zero Trust by Design

Supply chain insecurity presents a growing risk as defence systems become more software-defined and hardware-integrated. Compromise at the firmware or component level can subvert entire platforms before deployment. To counter this, TAID prescribes a zero trust approach not only in software architecture, but in its organisational design and procurement culture.

This means validating not just what AI systems do, but how they were built, by whom, with what dependencies, and how they are updated post-deployment. The white paper outlines a risk assessment process that precedes the Critical Design Review, requiring explicit mapping of threats, mitigation strategies, and residual risk evaluations. Additionally, the acquisition lifecycle mandates integration of continuous validation mechanisms that test for hidden vulnerabilities and update AI behaviour in response to adversarial threats. Mission assurance now extends beyond technical function as it encompasses systemic trust, from component sourcing to runtime performance.

Control and Alignment in AI-Native Systems

AI-native systems do not merely automate traditional functions; they introduce new operational concepts. These systems are continuously learning, often connected to remote cloud environments, and capable of modifying their decision boundaries in response to changing data. Without institutional safeguards, such capabilities risk autonomy drift. Autonomy drift is where system behaviour gradually diverges from mission intent or legal bounds.

To address this, TAID calls for the formalisation of runtime assurance strategies, complementing traditional design-time validation. These include fail-safes, real-time monitoring hooks, bounded autonomy constraints, and ethical governors that can intervene when a system strays from policy or tactical parameters. Equally important is the development of new doctrines of controllability, defining the extent to which AI can act without immediate oversight and under what conditions human intervention must override.

This paradigm shift requires new skill sets within defence institutions: AI test pilots, ethical evaluators, system integrity officers, and continuous feedback loops to integrate lessons from field deployments into future designs.

Lessons from Deterrence and Strategic Stability

The Cold War introduced doctrines of deterrence, proportionality, and command-and-control resilience, principles still relevant in the age of autonomy. Where nuclear systems emphasised rigid control and catastrophic consequences, AI-enabled platforms introduce more diffuse but equally destabilising risks, such as algorithmic escalation, model spoofing, or automated miscalculation.

The logic of fail-deadly must be replaced with fail-safe-by-design. NATO’s strategic posture must evolve to incorporate confidence-building measures and transparent standards for the deployment of autonomous capabilities. Just as nuclear doctrines emphasised second-strike survivability, AI doctrines must emphasise predictability, accountability, and interoperability across allied forces.

Allied Interoperability Without Ethical Fragmentation

The alignment of autonomy standards across allied nations is not a technical challenge alone; it is a matter of trust and political cohesion. Disparate ethical baselines, regulatory maturity, or threat perceptions can lead to fragmentation, undermining coalition effectiveness. Thus, there should be a shared AI risk repository, cross-national standardisation management plans, and multidisciplinary teams to bridge ethical, legal, and operational divides. Such coordination ensures that systems developed in one jurisdiction are deployable and governable in another, without compromising strategic edge. Standards must balance compliance with competitive capability, enabling innovation while preventing arms-race dynamics or trust erosion among partners. The goal is not uniformity, but interoperable ethics: a shared operational grammar for responsible autonomy in multinational defence.

Conclusion

AI is transforming defence, but it does not rewrite the basic rules of trust, control, and legitimacy. The challenge is not whether to adopt intelligent systems, but how to do so in ways that preserve mission clarity, operational integrity, and democratic accountability.

The frameworks are already emerging, whether they are within NATO strategies, the EU AI Act, or in technical standards like those in the TAID white paper, still require the implementation of institutional will, cultural change, and disciplined engineering. In the age of brilliant machines, true strategic advantage lies not just in what AI can do, but in how responsibly it is used.