Intelligent Games for UAV Systems: A Survey of Game-Theoretic and AI-Enabled Methods

A.AI APPLICATIONS IN PERFECT- AND IMPERFECT-INFORMATION GAMES

AI milestones in perfect-information games, from Deep Blue’s chess victory to AlphaGo’s success, highlight the powerful integration of gaming algorithms and deep neural networks. In 2016, AlphaGo utilized Monte Carlo Tree Search to explore game states. To predict opponent moves, it employed a policy network trained on high-level player records, while generating rapid move strategies based on local features. Furthermore, it used a value network optimized via RL to evaluate instantaneous board positions and calculate winning probabilities. The algorithm ultimately defeated the then-World Go Champion Lee Sedol 4–1, marking a significant milestone in AI’s success in perfect-information games. In the realm of imperfect-information games, in 2017, DeepStack [4] used Counterfactual Regret Minimization (CFR) to explore the game tree to a certain depth and then fed the state into a fully connected network to obtain the player’s regret value and derived strategies on demand. Since 2019, AI programs such as AlphaStar (StarCraft II AI) and Pluribus (Texas Hold’em AI) [4,5] have defeated professional human players in various human–machine competitions. These advances represented significant breakthroughs in imperfect-information games, with challenges in domains like StarCraft and Texas Hold’em driving cutting-edge research in AI.

B.INTEGRATION OF GAME THEORY, RL, AND DEEP LEARNING

The remarkable success of Intelligent Game technology primarily relies on the combination of game theory and RL paradigms. RL is a method for solving agent strategies, where agents interact with the environment to receive rewards or penalties, optimizing their policies to maximize long-term rewards. CFR and the virtual self-play algorithm FSP [2] are representative methods in the gaming and RL paradigms, serving as algorithmic foundations that bridge the two. Game theory provides the theoretical foundation for MARL [2]. The virtual self-play algorithm, built on large-scale distributed computing frameworks, has become a general RL framework for solving multi-agent system problems. It has been successfully applied to complex, large-scale adversarial scenarios such as Quake III Arena, StarCraft, Honor of Kings, and Texas Hold’em [3,8]. Deep CFR applies deep learning to the Monte Carlo CFR algorithm. This integration offers an effective approach to model the impact of states and actions on strategies. Consequently, it rapidly converges to a Nash equilibrium, which aids in designing agent strategies for both cooperative and competitive environments. Heinrich et al. proposed end-to-end virtual self-play methods in 2015 and 2016, namely XFP (Full-width Extensive-form Fictitious Play) [21] and NFSP (Neural Fictitious Self-Play) [5,6], which use DRL to approximate Nash equilibrium strategies. In recent years, with the rapid development of AI technologies, DRL in gaming has increasingly integrated with emerging fields, gaining prominence in areas such as autonomous driving, embodied intelligence, and generative AI.

C.APPLICATION OF LLMs

The use of game-theoretic tools to enhance the accuracy and internal consistency of models. By incorporating game mechanisms such as signaling games [4,17] and multi-agent debate frameworks into the interactive modeling of LLMs, task performance can be effectively improved. To make language model outputs more consistent and the models themselves more reliable, Jacob and colleagues designed a game in which two modes of the model are driven to find answers they can agree upon. This simple procedure, termed the “consensus game,” enables the LLM to compete with itself, utilizing game theory tools to enhance the model’s accuracy and internal consistency [5,18].

D.OPTIMIZATION IN INTELLIGENT GAME

From the perspective of intelligent optimization in gaming, it involves intelligent optimization algorithms. These algorithms primarily simulate the local or global dynamic behavior of agents, beginning with genetic algorithms introduced in the 1970s. This opened new pathways for solving complex optimization problems lacking precise mathematical models, accelerating the development speed, depth, and breadth of intelligent optimization algorithms. As research on intelligent optimization algorithms has evolved, six major categories have emerged: human-inspired intelligent optimization algorithms, evolutionary algorithms, swarm intelligence optimization algorithms, plant growth-inspired algorithms, natural phenomena-inspired optimization algorithms, and emergent computation [7,8]. These studies demonstrated significant advantages in solving NP-hard problems. Researchers widely applied intelligent optimization algorithms to complex real-world problems such as UAV path planning, power systems, and engineering design [17], providing broad avenues and powerful tools for addressing complex challenges in the UAV domain.

A.RESEARCH BACKGROUND AND BASIS

Game theory has important applications in the field of drones. Traditional game theory research primarily focused on two-player zero-sum and non-cooperative games. The emergence of AI has brought new challenges and opportunities to the study of game theory. The integration of game theory and AI forms Intelligent Game. It is a research area that has gained widespread attention, holds vast potential for development, and holds significant practical importance for in-depth exploration in the drone domain.

B.RESEARCH OBJECTIVES

This paper focuses on and explores the relationship between AI and gaming, seeking efficient and coordinated symbiotic models for AI and gaming, particularly by investigating practical application scenarios in the field of drones. Whether for military or civilian use, drone applications are becoming increasingly widespread, while Intelligent Game is developing rapidly. The paper discusses the hotspots, potential, new opportunities, and challenges brought about by the development of Intelligent Game in the drone field.

C.RESEARCH METHODS

This study employs a systematic literature review and comparative analysis to synthesize recent methodological trends. By deeply evaluating theoretical and applied research hotspots, we extract critical academic insights regarding the symbiotic models of AI and gaming within the drone domain, ultimately proposing key problems to be solved.

D.RESEARCH CONTENT

From the dual perspectives of theory and application with potential in the field of Intelligent Game for drones, an analysis of research hotspots is conducted. The paper delves into the research directions of Intelligent Game, centered around applicable scenarios in the drone domain. In Section IV, A, B, C, and D belong to the analysis of theoretical research hotspots with potential for Intelligent Game in drones, while E, F, G, H, and I belong to applied research hotspots with potential in the same field. The field of AI develops rapidly; the related content described in Section II dates from before 2021 and represents relatively mature achievements. This paper primarily expands the research in the following areas.

A.INTEGRATION OF GAME THEORY AND DRL

The interactions among agents can constitute a game, where each agent’s strategy depends on the strategies of other agents, and each agent’s payoff depends on the strategies of all agents. Agents can exchange information and coordinate through language, signals, actions, and other means or achieve individual or shared goals through competition or cooperation. Game theory can be used to analyze and predict the equilibrium of games among agents.

MARL studies how multiple agents in the same environment learn to achieve common goals or maximize their respective interests. Unlike traditional RL, agents in MARL must not only interact with the environment but also with other agents. Multi-agent DRL methods integrate technologies such as deep learning, RL, and multi-agent system theory, endowing agents with enhanced perception, reasoning, decision-making, and learning capabilities, and demonstrating significant potential in numerous application scenarios.

The integration of game theory and DRL exhibits four key trends: unified algorithms, high-dimensional equilibrium, multi-agent collaboration, and cross-domain engineering implementation [2,4,9]. The core revolves around a fusion paradigm that combines DRL with game-theoretic equilibrium solutions, addressing the efficiency and generalization bottlenecks of traditional methods in complex, dynamic, and large-scale scenarios. This approach forms a complete technical closed-loop encompassing “game modeling–deep representation–reinforcement learning–equilibrium convergence.”

This is applicable to scenarios such as UAV/USV pursuit-evasion, formation-based attacks and defenses, and guidance interception. For example, the incomplete-information pursuit-evasion game based on DRL + Policy Space Response Oracle (PSRO) enhances generalization capabilities in complex environments by designing reward functions for relative distance change rate and obstacle avoidance, thereby outputting stochastic optimal policies. Another example is the use of neural Hamilton–Jacobi–Bellman (HJB) approximation to optimize missile guidance laws, improving hit accuracy under conditions of highly maneuverable targets and nonlinear dynamics. The cooperative pursuit of multiple UAVs, classified as a zero-sum game under perfect- or imperfect-information, utilizes DRL to achieve dynamic decision-making at the second level, enabling real-time adaptation to changing battlefield situations [2,11,31].

• (1)Unified Game Learning FrameworkSuitable for both perfect and imperfect information, typical algorithms include [9,10] ReBeL, Student of Games (SoG), and Athénan. These frameworks maintain consistency through subgame resolving, reduce reliance on environmental priors with model-free self-play, and improve algorithm generalizability. ReBeL integrates policy iteration with equilibrium solving, achieving low exploitability in imperfect-information games like Texas Hold’em. SoG combines guided search with self-play learning and game-theoretic reasoning. It provides a unified solution for both perfect- and imperfect-information games. Given sufficient computation, it offers theoretical guarantees of convergence to an approximate Nash equilibrium, thereby reducing dependence on domain knowledge. DRL algorithms provide trainable agents with convergent learning algorithms, achieving stable and rational equilibrium in sequential decision-making processes [14,15]. Athénan optimizes the MCTS search of AlphaZero, outperforming traditional variants in a 22-class two-player zero-sum perfect-information game with shorter search times.
• (2)Efficient Fusion of DRL and Game EquilibriumSeveral typical research approaches and methods have been proposed [9,10]. PSRO + Generative Models: Generative neural networks are used to expand the policy space, combining joint optimization of policy-value networks with Nash/correlated equilibrium solving, enhancing adaptability to opponent strategy mutations and improving equilibrium discovery efficiency under imperfect information. Branching DRL + Prioritized Experience Replay: For high-dimensional state spaces, this approach prioritizes learning critical decision paths, mitigating the “curse of dimensionality.” Entropy-Guided Min-Max Decomposition: In two-team zero-sum games, entropy regularization balances exploration and exploitation, enhancing robustness.
• (3)Strengthening the Game-Theoretic Foundation of MARLResearch in this area primarily focuses on several directions [4,10]. Centralized Training with Decentralized Execution: Algorithms like Multi-Agent Proximal Policy Optimization (MAPPO) and Heterogeneous-Agent Proximal Policy Optimization (HAPPO) provide monotonic improvement guarantees in zero-sum or non-zero-sum games. Extending MCTS to Multi-Agent Games: This facilitates the integration of global value function learning with local policy execution, addressing issues of credit assignment and coordination consistency among multiple agents. Neural SDE Replacing Traditional Coupled PDEs: This reduces solving complexity from O(N²) to O(N), making it suitable for large-scale swarm control.

B.APPLICATIONS OF AI IN PERFECT-INFORMATION GAMES

C.APPLICATIONS OF AI IN IMPERFECT-INFORMATION GAMES

Imperfect-information games are commonly used to describe multi-party sequential strategic interaction processes. During the gaming process, participants cannot obtain complete or accurate information; the game state and the next action are often unknown, and the information available is usually incomplete and imperfect. The application of imperfect-information games in the drone domain primarily involves using probabilistic reasoning combined with MARL to solve for perfect Bayesian Nash equilibrium. This is well suited to real-world scenarios such as unknown target intent, limited communication, and interference/deception, supporting tasks like air combat confrontation, swarm coordination, and search and rescue [13].

Due to the presence of hidden information, imperfect-information games may involve deception, bluffing, and other behaviors, making them more reflective of real-world problems and thus highly applicable. However, due to their large state space and hidden information, participants cannot access all information about the game state, making it impossible to determine an optimal strategy solely by predicting the opponent’s actions. Consequently, participants must not only analyze all possible outcomes of the opponent’s decisions but also consider the opponent’s hidden information. This causes the state space to grow exponentially, increasing the challenge. Therefore, this remains a challenging yet popular research area in AI.

Methods combining virtual regret minimization game reasoning with RL are a significant research direction for imperfect-information games [14]. Currently, in scenarios involving imperfect-information processing, theories like Nash equilibrium are combined with DRL algorithms to solve multi-agent decision-making and optimization problems. The approaches can be broadly categorized into two types: the first is dedicated to algorithms that quickly converge the average policy to a Nash equilibrium strategy. The primary goal of this class of methods is to address the large state space problem in imperfect-information games. The second approach focuses on modeling opponent behavior. It aims to observe and collect opponent characteristics during the game to organize behavioral traits and hidden information. By estimating the opponent’s strategy, the agent can exploit strategic weaknesses to maximize its own payoff.

D.ACHIEVING MORE INTELLIGENT AND EFFICIENT DECISION-MAKING PROCESSES

As one of the theoretical foundations of AI, game theory can be used to analyze and solve various decision-making problems. The integration of game theory and AI enables more intelligent and efficient decision-making processes. In AI, game theory models can describe and address a range of decision-making problems. Core concepts in game theory, such as optimal strategies and equilibrium points, can help AI systems analyze and predict the outcomes of different decisions, guiding the system toward the best choice.

Game theory provides a decision-making model for resolving cooperation and competition problems among multiple participants. In real life, many decision-making problems involve conflicts of interest and common interests among multiple parties. In multi-party games, AI can analyze the strategic choices of each participant through model building, thereby predicting possible outcomes. Through game simulation, AI can quickly identify optimal strategies, helping participants make informed decisions. Common applications include resource allocation and cooperative negotiations. At the same time, the decisions of AI systems often involve gaming or collaboration with other participants, so the effectiveness and payoff of these decisions need evaluation and optimization. Game theory offers a quantitative method to assess the quality of decisions by analyzing the strategies and payoffs of participants, helping evaluate the effectiveness of different decision strategies, optimize system behavior, and thereby guide the improvement and optimization of AI.

Technologies related to AI-generated action (AIGA), such as behavior generation and model generation, provide feasible solutions for solving decision-making problems. Transformer, a representation learning model that uses attention mechanisms to perform sequence-to-sequence transformations, is leveraged to construct decision-making policy solutions for Intelligent Game problems. Representation learning methods based on transformers, sequence modeling approaches, and multimodal fusion learning methods have garnered sustained attention in Intelligent Game decision-making [15]. Current methods centered on Decision Transformers can be categorized into three main types: 1) direct utilization of LLMs (e.g., encyclopedic, video, and Internet knowledge), 2) representation and model learning based on framework transformations (e.g., representation learning and environment learning), and 3) conditional generation based on decision problem reconstruction (e.g., sequence modeling, behavior generation, and world model generation).

This field still faces challenges, such as the complexity of building and solving game models, computational efficiency issues, and information asymmetry [16]. The emergence of ChatGPT in late 2022 has triggered an “arms race” across various AI tracks. However, general language capabilities do not fully match the reasoning abilities required for decision-making. How to construct a “Decision Foundation Models” has become a frontier issue in the current fields of AI and intelligent decision-making.

E.GENERATIVE ADVERSARIAL NETWORKS

GANs essentially consist of two neural networks: a generator and a discriminator. These are two opposing models: a generative model and a discriminative model. In GANs, the generator’s task is to generate random images, while the discriminator’s responsibility is to classify these generated random images. The generative model creates new data samples, and the discriminative model determines whether these samples are authentic. This adversarial training approach has broad applications in the field of AI, providing a powerful tool for designing more intelligent and creative AI agents.

The training process of GANs can be viewed as a game: the generator and discriminator continuously optimize through confrontation, ultimately reaching a Nash equilibrium. This dynamic gaming relationship drives the improvement of generative model quality. GANs exemplify the application of game theory in AI, where the generator and discriminator engage in mutual gaming and optimization, achieving an equilibrium state where neither can further improve.

F.APPLICATION OF LLMs

LLMs are increasingly applied in agent systems, demonstrating significant advantages in handling open-ended tasks, modeling human intent, and reasoning. Their intersection and fusion with game theory enable establishing game benchmarks to evaluate LLM behavior, using game-theoretic algorithms to enhance model performance, describing model impact through game modeling, and analyzing the influence on equilibria in traditional game models.

To enhance agents’ capabilities in autonomous environmental perception, effective collaboration, and reasoning-based decision-making, LLMs are being introduced into multi-agent systems. By combining them with RL and role-based learning methods, a collaborative agent architecture is constructed that possesses the ability to understand complex games and perform high-level strategic reasoning. For partially observable problems, a distributed LLM-based multi-agent collaborative decision-making architecture is built, where each agent is equipped with a cognitive building block module to enhance autonomous perception and environmental modeling capabilities. The introduction of a role-learning module improves agents’ collaborative adaptability in complex games.

With the rise of LLMs like the GPT series, BERT, and Tongyi Qianwen [17], integrating them into the decision-making processes of agents and leveraging their reasoning and generative capabilities for strategy optimization provided new directions for multi-agent collaborative decision-making. To address decision-making difficulties arising from highly dynamic environments, multi-level reasoning modules were designed to guide agents in identifying key game nodes and generating forward-looking strategies, thereby improving decision quality.

Large Models Empowering Gaming: Large models were used for situation understanding, intent prediction, and strategy generation, enhancing cognitive domain gaming capabilities [18,19]. Representative methods included entropy-guided in-context learning (RA-ICL), adaptive chain-of-thought reasoning (CoT), and particularly LLM + game theory reasoning modules. These approaches enhance LLMs’ zero-shot reasoning ability in structured gaming tasks through retrieval augmentation and dynamic reasoning depth adjustment, reducing dependency on task-specific fine-tuning and achieving technological breakthroughs. They are applicable to gaming scenario comprehension, strategy explanation, and opponent modeling, such as in negotiation and strategy games. By leveraging LLMs and deep learning to accelerate the numerical solution and parameter calibration of replicator dynamics equations, model iteration cycles are shortened, improving responsiveness to real-time battlefield data for AI-driven evolutionary game solving.

G.DIFFERENTIAL GAMES

Differential game theory is a mathematical method for studying how multiple participants make optimal decisions in dynamic environments, particularly suited for describing adversarial, continuous-time decision-making processes, and it establishes a rigorous mathematical theoretical foundation. The description of a differential game problem includes agents’ equations of motion, terminal conditions, and performance indices. Participants’ decisions are described by differential equations, which characterize the change in participants’ long-term payoff functions over time. Each participant’s goal is to maximize their individual payoff by selecting optimal control variables for the differential equations. The equations of motion for a multi-agent system are a set of ordinary differential equations.

H.EVOLUTIONARY GAME THEORY

EGT is a branch of game theory inspired by Darwinian evolution, applying game theory to evolutionary contexts. It specifically focuses on how decision-makers, under conditions of bounded rationality, adjust their strategies through learning and adaptation, ultimately forming stable behavioral patterns. Core concepts of evolutionary games include bounded rationality, group strategy evolution, and Evolutionary Stable Strategies (ESS). In this type of game, a group of agents, through a repetitive selection process of evolution, continuously interacts with a variety of strategies, thereby creating a stable solution. The underlying idea is that many behaviors involve interactions among multiple agents within a group, and the success of an individual agent depends on how its chosen strategy interacts with the strategies of other agents. Unlike classical game theory, EGT focuses on how strategies evolve over time and which dynamic strategies are the most successful in the evolutionary process.

With the development of AI technology, this theory provides a powerful tool for analyzing decision-making processes in complex adversarial environments, showing broad application prospects in military simulation and training systems. Compared to other game-theoretic methods, EGT places greater emphasis on dynamic learning and adaptation processes, making it suitable for battlefield environments characterized by high uncertainty and incomplete information. It is well adapted to intelligent, unmanned, and asymmetric warfare scenarios. Through methods like replicator dynamics, multi-player public goods games, and complex network evolution, it provides a quantitative basis for group decision iteration, strategy diffusion, and adversarial equilibrium. However, its models are relatively complex and require substantial data support.

Applications of evolutionary games in the drone domain center on bounded rationality, distributed learning, and dynamic equilibrium. These approaches primarily address swarm task allocation, resource scheduling, attack–defense confrontation, and cooperative control. They deeply integrate with methods such as network potential games, evolutionary learning, ADP, and DRL. This integration effectively solves real-time computation and robustness problems in large-scale swarms. The applications are mainly reflected in the following areas [25–27]:

• (1)Dynamic Resource Allocation (Cooperation-Competition). Public goods game models are designed to balance individual payoff with global resource optimization, adaptable for dynamic scheduling of resources like communication bandwidth and battery power.
• (2)Swarm Attack-Defense and Pursuit-Evasion. Belonging to zero-sum evolutionary games, strategies are updated using replicator dynamics or logit learning. Under partial observability, opponent intent is predicted to enhance the success rate of encirclement and capture, as well as penetration and evasion capabilities, supporting “swarm vs. swarm” confrontations.
• (3)Fault-Tolerant Formation and Cooperative Obstacle Avoidance. Belonging to non-zero-sum evolutionary games, participants are leader-follower drones. ESS are used to optimize formation maintenance and fault compensation, maintaining formation stability during actuator failures.
• (4)Civilian Expansion. Mainly applied in agriculture and logistics. By constructing multi-party evolutionary game models, the strategic interactions among enterprises, service providers, and regulators are characterized to improve operational coordination and distribution efficiency, adapting to smart agriculture and urban logistics scenarios.

Game Model Construction: The payoff function encompasses factors like mission payoff, energy consumption, and fault tolerance efficiency and capture success rate. States include position, velocity, and heading. Constraints include airspace restrictions, dynamic capabilities, communication topology, and temporal limits.

I.MEAN FIELD GAMES

MFG theory, emerging in 2006, was a relatively new field within the family of game theory. When the number of agents in a system was very large, the possible ways of interaction among them could grow exponentially, making modeling exceptionally complex. The introduction of the Mean Field Scenario (MFS) provided an effective solution to this class of problems.

When the number of agents increased toward infinity, the interactions can be represented by MFG theory [27]. It studied the interaction and decision-making problems among populations in continuous time and continuous state spaces. In MFG, the decision-making of each agent was described by a differential equation or dynamic system model, and solving these equations or systems yields the optimal strategy. In MFG, since an agent’s decision is assumed to be influenced by the average behavior of the population rather than directly by other individual agents, it can effectively address the issue of a drastically increasing number of agents. The core idea of MFG is to use the average field formed by a homogeneous population of agents to approximate and characterize the complex interactions among a large number of individuals, thereby achieving effective simplification and dimensionality reduction for large-scale game problems.

MFGs were used to study differential games within large populations of rational players, solving the problem of traditional DRL methods being impractical in environments with infinitely many agents. When a system contains a vast number of agents, these agents not only have preferences regarding their own state (e.g., wealth and resources) but also care about the position of other agents within the overall distribution of the population. Typical progress in the integration of AI and MFG theory includes the following: