Training Against an Adversary That Learns: Adaptive Red Teaming for Defence Mission Preparation
Defence training and mission preparation face a recurring problem: blue-force courses of action are often assessed against an opposing force that follows a pre-written script. Greater fidelity in terrain, sensors, platforms, and effects does not solve this problem if red behaviour remains fixed. The resulting exercises may confirm the assumptions built into the scenario, conceal vulnerabilities that appear only against a different response, and train personnel to defeat the script rather than an adaptive opponent. This paper presents a reusable approach to adaptive red teaming in synthetic environments. Instead of representing red as one prescribed plan, multi-agent reinforcement learning and self-play generate a population of opposing decision-making policies from different stages of training. Each policy is retained with its training origin and tested against scripted baselines and other learned policies, producing a library of distinct red behaviours with which trainers can stress-test candidate plans. Sea Lock, a synthetic naval-blockade environment, is presented as the first use case rather than the limit of the approach. It models a contested mission in which heterogeneous forces combine manoeuvre, sensing, electronic warfare, resource allocation, and combat under partial observation. Experiments within this use case show why the choice of red opponent matters: aggregate training results can conceal material differences between saved policies, later policies are not necessarily stronger, and scripted and learned opponents can expose different weaknesses in the same force design. The interaction records also train a learned predictive model, or world model, illustrating how bounded short-horizon predictions can be generated from synthetic experience when real engagement data are scarce. These findings have direct implications for defence training and experimentation. Adversary behaviour should be treated as a continuously evaluated training capability rather than static scenario content; candidate courses of action should be tested against a portfolio of traceable opponents; and learned predictions should operate within explicit uncertainty and distribution limits while the simulator remains authoritative. The architecture separates scenario-specific rules, forces, and data from reusable adversary-learning, evaluation, audit, and interface components. It can therefore support additional scenario products—such as contested logistics, swarm and counter-autonomy, or air-defence experimentation—without pretending that one trained model transfers unchanged between domains. Depending on the user workflow, the capability can be delivered as an analyst workbench, an adaptive red-cell service, or a component integrated into an existing training federation. In each form, the human-supervised workflow remains the same: specify objectives, constraints, and a candidate course of action; let adaptive red-team policies probe it; inspect opponent-dependent vulnerabilities; then revise and retest the plan. Sea Lock demonstrates the first implementation, while the wider contribution is a family of scenario-based training products built around adversaries that respond rather than replaying what scenario designers expected.