A Survey on Evolutionary Multitask Optimization

A.MATHEMATICAL DESCRIPTION OF EVOLUTIONARY MULTITASKING OPTIMIZATION PROBLEMS

Multitasking optimization (MTO) is an optimization method designed to concurrently address multiple related yet distinct tasks. The primary objective of MTO is to elevate overall performance and efficiency by leveraging shared information and the interdependencies among tasks. There exist K optimization tasks, where each task is characterized by an objective function denoted as fi:Xi→ℝ,i=1,…,K, where Xi is the search space. Each task may be subject to a set of equalities or inequalities, and a solution is considered feasible only when all these constraints are satisfied.

B.EVOLUTIONARY MULTITASKING OPTIMIZATION PROBLEMS

The EMTO problem involves the optimization challenge of addressing multiple interconnected tasks using the framework of EAs. Generally, there exists a correlation among multiple tasks, indicating that the optimization outcomes of one task might influence the optimization results of others. The optimal solution x for EMTO problems is the optimal solution obtained by simultaneously optimizing all N tasks. In the context of the EMTO problem, the search space associated with each task is converted into a single, unified search space Ω for the purpose of representation.

In the EMTO problem, each population member in a multitasking environment can be compared. Therefore, a series of definitions is provided in. We assume that each individual in population P is pj, where j∈{1,2,…,|P|}. It is important to note that every individual can be encoded into a unified search space Y, from which it can also be decoded from the unified space Y to the search space of any specific task. Therefore, the solution of individual pj decoding to the search space Xi of the i-th task can be represented as Xij. In the multitask optimization environment, the individual pj attributes are described as follows:

• •Factorial cost: The factorial cost Ψji of individual pj in the task Ti is defined as Ψji=λ·δji+fji, where fji represents the objective function value, δji and λ represents the total number of constraint violations and penalty factors, respectively.
• •Factorial rank: Factorial rank rji represents the rank of individual pj in task Ti, which is the sequential number of individual pj in the population sorted in ascending order of its factorial cost.
• •Skill factor: The skill factor τj of individual pj is defined as τj=argmini{rji},i∈{1,…,N}, indicating that individual pj is most adept in task Ti among all other tasks.
• •Scalar fitness: The scalar fitness ϕj of individual pj is defined as ϕj=1/mini∈{1,…,N}rji.
• •Multifactorial optimality: If and only if individual pj is the global optimal solution among all tasks T, then individual pj is called a multifactorial optimum.

Individuals in the population can be compared for performance based on these definitions. For instance, consider two individuals, p₁ and p₂. If they have the same skill factor (τ₁ = τ₂), meaning they are evaluated on the same task, their performance is compared directly based on their factorial costs for that task. In a minimization context, p₂ is considered better than p₁ if its factorial cost is lower (Ψ₂ < Ψ₁). If the individuals have different skill factors (τ₁ ≠ τ₂), a direct comparison is not meaningful, and their relative ranks are determined by their scalar fitness ϕ.

C.INTRODUCTION TO MFEA ALGORITHM AND CODE

The MFEA is an EA specifically designed to tackle multitask optimization challenges. In the domain of MTO, the primary goal is to simultaneously improve multiple tasks, each potentially characterized by distinct objective functions, constraints, and search spaces. The core principle of MFEA is to harness the potential interdependencies between tasks to boost algorithmic performance. When tasks demonstrate significant interrelations, the optimal solution for one task can be effectively applied to others, thereby substantially enhancing the algorithm’s efficiency. A key advantage of the MFEA is its capability to address multiple continuous or discrete optimization problems concurrently within a multitask setting, facilitated by a unified solution representation strategy. This strategy involves encoding each variable of the candidate solutions as random key values ranging from 0 to 1, thus representing each task within a cohesive search space in the multitask environment. The foundational framework of MFEA is illustrated in Algorithm 1.

Algorithm 1. Basic structure of the MFEA.

During the initialization phase, a set of N randomly generated individuals within the unified search space is regarded as the initial population P (line 1). For each individual in population P, it is necessary to separately evaluate factorial cost and calculate skill factors τ (lines 2–3). Population evolution involves the use of assortative mating and vertical cultural transmission to create offspring, as detailed on lines 4–6. Finally, the parent population is joined with the offspring population, and the best individuals are selected to continue to the next generation (lines 7–10).

Assortative mating is an optimization method based on the simulated binary crossover (SBX) operator, aiming to retain individual characteristics with high similarity and introduce a certain degree of variability to promote the evolution of the population. Assortative mating, therefore, implies a significant likelihood of mating between parents with comparable skill factors, whereas parents with dissimilar skill factors are less likely to produce offspring. In MFEA, different skill factors represent different tasks and individual cultural backgrounds. When two parent individuals possess identical skill factors, the chromosome crossover operation is conducted within a shared cultural context. This can help maintain consistency within the cultural background and encourage offspring individuals to inherit this cultural trait. When two parent individuals exhibit different skill factors, cross-cultural chromosome crossover or chromosome mutation operations are executed. This can promote gradual optimization and improvement in offspring individuals during the evolutionary process. In MFEA, the setting of the parameter rmp is crucial as it is used not only to balance exploitation and exploration within the search space but also to control the extent and scale of knowledge transfer between tasks. The pseudocode for assortative mating is given in Algorithm 2.

Algorithm 2. Assortative mating.

In vertical cultural transmission, skill factors are allocated to offspring individuals through the strategy of selective imitation. The offspring individual generated by the selective imitation strategy randomly imitates any parent individual and inherits its skill factor. Subsequently, the evaluation process for the offspring individual is limited to the task indicated by the inherited skill factor, without needing to cover all tasks. For a K-factor multitask optimization problem, using the selective imitation strategy can reduce the computational burden of the function evaluation process by K times, greatly decreasing the amount of calculations required. To be specific, let C_k be the computational cost of evaluating an individual on task k. Without selective imitation, evaluating a new offspring would require a total cost of Σ(C_k) for k = 1 to K. With selective imitation, an offspring inherits a single skill factor and is evaluated only on that specific task, reducing the evaluation cost for that individual to C_j for some j ∈ {1, …, K}. Therefore, the overall computational complexity of the function evaluation phase per generation is reduced from O(N * Σ(C_k)) to approximately O(N * C_avg), where C_avg is the average evaluation cost across all tasks. This efficiency gain is particularly significant in expensive optimization scenarios where function evaluations are the main computational bottleneck. The pseudocode of selective imitation is shown in Algorithm 3.

Algorithm 3. Vertical cultural transmission via selective imitation.

A.CHROMOSOME ENCODING AND DECODING SCHEME

Chromosome encoding and decoding schemes are fundamental aspects of EAs, which are utilized to solve optimization problems. The encoding process represents the solution to a problem in a chromosome, which is a string of bits, numbers, or characters. The decoding process translates the chromosome back into a solution. Therefore, the process of encoding and decoding chromosomes is crucial for the operation of EAs. The solution’s fitness is gauged through a fitness function, and the EA repeatedly employs selection, crossover, and mutation processes on the chromosome population to evolve toward an optimal solution.

The unified representation scheme not only standardizes the search space to be more uniform and continuous but also simplifies the application of optimization techniques, such as genetic algorithms. By employing a unified representation scheme, it is easier to compare and cross over information between different individuals, thus more effectively searching for the optimal solution. Thus, MFEA utilizes a uniform representation scheme within the integrated search space. Notably, each variable of an individual is represented by a random key that ranges from 0 to 1. Consider a chromosome y, with its i-th random key taking values yi∈[0,1], if the i-th variable of individual is bounded in the range of [Li,Ui], then the decoding process is calculated as follows

For random keys, decoding can be achieved in both continuous optimization and discrete optimization scenarios. However, when using the random key to represent permutation-based combinatorial optimization problems (PCOPs), there are issues such as low decoding efficiency and easy loss of information during the decoding process. Therefore, Yuan et al. introduced a permutation-based unified representation and applied it to PCOPs. Binh and colleagues [3] as well as Trung and associates [4] independently utilized specific encoding and decoding techniques within a shared search space to address the clustered shortest-path tree (CluSPT) issue and the minimum routing cost clustering tree (CluMRCT) issue. Recently, in addressing the CluSPT issue. In [5], a solution utilizing edge-sets representation was also developed for the CluSPT problem.

Recently, Binh and colleagues [6] proposed a novel solution representation and an associated decoding strategy within the framework of MFEA. To address the challenge of building the optimal data aggregation tree in wireless sensor networks, Tam et al. not only innovatively proposed an encoding and decoding strategy but also introduced a novel multiple minimum energy cost aggregation trees solution. In addition, to address the two major challenges of community detection and active module recognition, they designed a unified genetic representation and equipped it with corresponding problem-specific decoding schemes. Under this framework, each individual is described as an integer vector; here, each integer represents the community label to which the corresponding node belongs, offering a clear and efficient representation for this problem.

B.INTRO-POPULATION REPRODUCTION

Intra-population reproduction is a fundamental search operator within the realm of multitask evolutionary computation (MTEC), with crossover and mutation being the prevalent techniques derived from genetic algorithms. Traditional crossover and mutation strategies comprise SBX, one-point crossover, differential evolution (DE) crossover [7], partially mapped crossover, two-point crossover [6], guided differential evolutionary crossover [8], ordered crossover, one-point mutation [6], Gaussian mutation [9], mutation utilizing the Powell search method [8], swap mutation, DE mutation [7], polynomial mutation, swap-change mutation [10], and uniform variation [7]. In addition to the genetic algorithm and evolutionary strategy, other basic algorithms commonly used in the MTEC paradigm include DE, particle swarm optimization, genetic programming, the self-organized migrating algorithm, the artificial bee colony, the bat algorithm [11], the fireworks algorithm, and brain storm optimization.

In addition, inspired by multifactor genetics, A. Gupta et al. proposed an evolutionary multitask algorithm. In the evolutionary multitask algorithm, this inspiration means that by simulating genetic and cultural exchanges during the biological evolution process, the algorithm can handle multiple optimization tasks within a unified framework. This approach not only effectively addresses individual optimization challenges but also tackles multiple optimization issues within a single evolutionary population, thereby attempting multitask optimization.

C.INTER-POPULATION REPRODUCTION

Inter-population reproduction is essential for enabling the transfer of knowledge across different tasks in the realm of multitask optimization challenges. This process enables the sharing and utilization of knowledge across various tasks, significantly enhancing optimization outcomes. The precise timing and methods for reproduction across populations can differ based on the algorithm’s architecture and the characteristics of the optimization problems. For example, in reference [12], the authors proposed a framework that involves dividing the initial population into different task groups based on skill membership values. This approach enables the identification of individuals with high problem-solving abilities for specific tasks, which can then be utilized to guide the evolutionary process. By concentrating on the exchange of knowledge among these task groups, the algorithm seeks to boost the collective performance across all tasks.

The process of reproduction between populations can be engineered to take place at regular intervals of generations, guaranteeing that the transfer of knowledge is a consistent part of the evolutionary search process. This regularity aids in preserving the diversity of the population and prevents the algorithm from converging prematurely on suboptimal solutions. In EMT and SGDE, the interval for reproduction between populations was established at 10 and 20 generations, respectively. The timing and implementation of inter-population reproduction in multitask optimization problems are critical for effectively transferring knowledge across tasks.

To assess the similarity between tasks, a strategy based on similarity is employed to directly gauge the distance or disparity between their comprehensive distributions. Specifically, Chen et al. [13] employed Kullback–Leibler divergence (KL-D) to gauge the similarity between the population distributions of tasks.On the other hand, in reference [14], KL-D was employed to gauge the similarity across different tasks’ dimensions. However, when the support sets of two distributions do not overlap or partially overlap, KL-D cannot accurately reflect the similarity between them. Consequently, a number of researchers proposed the Wasserstein metric as a means to dynamically ascertain the similarity between tasks. The Manhattan distance between representative solutions in different tasks can also be leveraged for evaluating task similarity. Zhou et al. [15] conducted a comparative analysis of the optimal solution distances for tasks, the rank correlation between tasks using uniformly sampled solutions [16], and the correlation determined through fitness landscape analysis. Cai et al. evaluated similarity by the relative distance between task-specific populations in the search space. Liang et al. calculated the task similarity of the maximum mean difference between two task-specific populations in high-dimensional space.

The implicit transfer process in the classical EMTO method does not account for the negative transfer of individuals from different domains during the evolutionary process. Thus, a collection of explicit transfer strategies has been created to explicitly shift individuals from the source domain to the target domain, replacing the unified representation in MFEA. Utilizing explicit distribution data, the distribution-based strategy can be implemented in individual transformation processes to counteract negative transfer that arises from differences in optima locations or population distribution variances. For example, Liang and colleagues achieved the transformation of high-quality individuals across different domains by computing two mapping vectors, which enabled the restoration of differences in population distribution. In [17], Wu et al. achieved the transformation of high-quality individuals across domains by subtracting the sample mean of the source domain and adding the sample mean of the target domain. Unlike distribution-based strategies, matching-based strategies are designed to primarily address the issue of negative transfer resulting from differences in landscape similarities. The LDA-MFEA algorithm uses a mapping matrix to map individuals from the source domain one-to-one to the target domain, directly solving the transfer problem from the perspective of hierarchical correlation.

Information sharing among tasks is essential for the efficiency of evolutionary multitasking. However, drawing upon data from other tasks, including those related to parasitism, does not invariably aid the current task. This necessitates the application of random mating probability (rmp) to regulate the level of knowledge transfer across various tasks. Consequently, Liaw et al. [18] managed to facilitate information exchange for adaptive control tasks through the use of equation (3).

Conventional evolutionary multitask optimization algorithms usually distribute identical computational resources to every task. However, because tasks have varying levels of computational complexity, using an equal allocation method can result in a significant waste of computational resources and can greatly diminish algorithm performance, particularly when computational resources are constrained. To tackle this problem, Gong and colleagues [20] introduced a dynamic resource allocation strategy capable of assigning varying amounts of computing resources to different tasks in an adaptive manner, enhancing the efficiency of EAs designed for multitask optimization. Yao et al. [21] approached the multi-objective optimization tasks uniquely by breaking them down into a sequence of single-objective subtasks. They dedicated increased computational resources to those subtasks that exhibited rapid evolutionary progress throughout the evolutionary process. For the dynamic allocation of computational resources in addressing multi-objective EMTO challenges, Wei et al. [22] presented a generalized resource allocation approach, which is rooted in the theoretical underpinnings of conventional resource allocation and the distinctive aspects of multi-objective optimization.

A.MULTITASK OPTIMIZATION FOR MANY-TASK OPTIMIZATION PROBLEMS

Evolutionary many-task optimization (EMaTO) is a research paradigm that focuses on simultaneously addressing three or more optimization tasks through knowledge transfer within the framework of an EA. The success of this approach is essential for the efficacy of EMaTO. The current EMaTO algorithm for multitask optimization problems (MaTOPs) focuses on two primary concerns.The first issue is how to select tasks similar to the target task as source tasks to acquire useful knowledge, as the transfer of useful knowledge between similar tasks can be beneficial. The second matter at hand is how to efficiently convey knowledge to the designated task. More precisely, the EMaTO algorithm needs to address how to efficiently convey pertinent knowledge to the target task and leverage this knowledge to support the target task. Thus far, numerous algorithms have been proposed to address these issues. For instance, Bao et al. [23] introduced the AEMaTO-DC algorithm, which employs a maximum mean discrepancy metric to choose pertinent tasks and utilizes a density-based clustering approach to ascertain the strength of interaction between tasks. In 2023, Zhou and colleagues [24] applied maximum mean discrepancy for the adaptive selection of tasks and utilized a multi-armed bandit approach to regulate the intensity of knowledge transfer between tasks.

B.MULTITASK OPTIMIZATION FOR DYNAMIC OPTIMIZATION

The evolutionary computation method has successfully solved many static optimization problems. However, there are still certain difficulties in solving a dynamic optimization problem (DOP). A DOP refers to a problem where the environment (such as optimal values, environment, conditions, and range) will change over time. When dealing with DOP, two important aspects are mainly considered. One is how to jump out of the previous optimal area after environmental changes. Another approach is how to swiftly locate new optimal solutions in the wake of environmental changes. Thus, Luo et al. [25] appropriately segmented the entire population into several subpopulations during the initialization phase to thoroughly explore distinct subareas of the search space, thereby enhancing the algorithm’s diversity. Wu et al. [26] suggested a region-based subpopulation initialization (RSI) technique to enrich population diversity. This technique establishes several subpopulations in a balanced manner within a new environment, facilitating searches across various regions of the search space. Concerning the second aspect, Li et al. introduced an optimal particle calibration strategy capable of rapidly identifying the best solution in a dynamic environment. In [27], Liu and colleagues applied a dual-archive-based particle swarm optimization to identify the optimal solution subsequent to environmental alterations.

C.MULTITASK OPTIMIZATION FOR LARGE-SCALE OPTIMIZATION

A large-scale optimization problem (LSOP) involves scenarios where the number or dimensions of decision variables are significantly large. Solving such problems mainly focuses on two key points: first, how to reduce the complexity of LSOPs to improve the efficiency of the solution; second, how to enhance the effectiveness of search algorithms in high-dimensional decision spaces to more accurately find the optimal solution. In 2022, Li et al. introduced the double difference grouping method, which has been proven to efficiently decompose LSOPs, providing new perspectives and solutions for researchers in related fields. Zhong and his colleagues viewed the decomposition problem as a combinatorial optimization challenge and designed a link measurement function to guide the optimization process. To achieve balance in multiple dimensions and identify global optimal solutions, Wang et al. proposed a gene-targeting differential evolution technique, which showed significant optimization effects for LSOPs.

D.MULTITASK OPTIMIZATION FOR MULTI-FORM OPTIMIZATION

Multi-form optimization presents a cutting-edge strategy that integrates multiple alternative approaches to better meet the requirements of a single core objective task. In practical operation, differentiated expressions can trigger unique search methods. This undoubtedly increases the difficulty of selecting the optimal expression for specific problems in limited computing resource environments. In this context, the emergence of multi-form optimization provides an efficient solution for integrating different expressions: it merges diverse expressions into a unified multitask optimization algorithm framework, cleverly avoiding the selection dilemma of a single expression. At the practical level, we construct various mathematical models to address different optimization problems. Taking the research results of Da [28] and his team as an example, they innovatively applied the concept of evolutionary multitask learning (EMTL) to effectively solve the specific single-objective optimization challenges brought by multi-objective optimization expressions. Additionally, Wu et al. [29] successfully constructed a complex polymorphous optimization problem by designing two correlated registration tasks with significant differences in functional landscapes, effectively addressing evolutionary multitasking challenges in point cloud registration. These studies not only demonstrate the flexibility of problem-solving strategies but also highlight the importance of tailored solutions for specific problems.

A.CONTINUOUS OPTIMIZATION PROBLEM

EAs demand extensive numerical simulations or costly physical experiments to appraise potential solutions when tackling optimization problems that are computationally demanding. Zhang et al. regarded assigning individuals to different tasks as a computationally expensive problem. They utilized phenotypic characterization techniques to conduct in-depth and precise measurements of the behavior of scheduling rules. Based on these behavioral measurement data, they tailored a proxy model for each specific task, which provided strong data support and a model foundation for subsequent evolutionary multitask optimization work. In the context of bilevel optimization problems, Gupta et al. proposed a multitask bilevel evolutionary algorithm to enhance the performance of bilevel optimization. For each generation of upper-level optimization, this algorithm needs to account for nested lower-level problems to assist in the exploitation of inherent commonalities between them. MFEA can not only be used for SOO problems but can also be extended to the field of MOO. Wei et al. introduced an innovative multi-objective, multitask optimization algorithm known as the inverse model-based multi-objective evolutionary algorithm, which incorporates the principles of inverse model mapping and an objective transformation approach. It adopts an innovative adaptive transformation strategy that can automatically adjust the scaling factor of Euclidean distance between the two populations. In addition, it incorporates an efficient variable correlation analysis strategy to train inverse models, thereby more effectively assisting the optimization process of target tasks.

B.DISCRETE OPTIMIZATION PROBLEM

In the MTEC framework, a series of NP-hard combinatorial optimization problems have been successfully solved, including the traveling knapsack problem, sudoku puzzles, the travel salesman problem (TSP), the quadratic assignment problem (QAP), the linear ordering problem (LOP), the job-scheduling problem (JSP), vehicle routing problems (VRPs), and the deceptive trap function.

Inspired by the innovative practices in the field of vehicle routing planning under the new models of “crowdsourcing delivery” and “sharing economy,” Feng et al. creatively proposed a novel variant of the vehicle routing problem – the vehicle routing problem with heterogeneous capacities, time windows, and occasional drivers (VRPHTO). This complex problem considers multiple variables in practical operations, providing a more realistic solution for logistics planning. In order to meet the urgent demand for multitasking in cloud computing environments, they further developed the EMTO algorithm. This algorithm aims to process multiple VRPHTO challenges in parallel, which not only greatly improves the efficiency of problem-solving but also optimizes the overall operational performance of the system, thus bringing revolutionary progress to the modern logistics industry. MFEA not only solved the CluSTP problem using genetic operators but also addressed a new problem derived from the CluSTP. Petrovan et al. innovatively proposed two genetic algorithms aimed at more effectively addressing the complex issue of clustering shortest path trees. Thang et al. innovatively proposed a hybrid multitask algorithm, the multifactorial firefly algorithm, which integrates the firefly algorithm and MFEA, it combines the respective strengths of both algorithms and provides a fresh approach for optimization problems. This algorithm can simultaneously solve multiple CluMRCT problems. Rauniyar et al. presented an innovative EA targeting the pollution routing problem (PRP) – the multi-objective EA based on NSGA-II. This algorithm skillfully leverages the search advantages of NSGA-II, aiming to more efficiently address the complexities inherent in pollution routing planning. This study deeply explored the two conflicting goals in PRP: one is to minimize fuel consumption to achieve more efficient energy utilization, and the other is to strive to shorten the total driving distance to reduce transportation costs and improve efficiency. Chandra et al. innovatively proposed an advanced method called EMTL. The core of this approach lies in optimizing modular network topologies to address complex n-bit parity check problems. Their research demonstrates that the combination of specific neural network architectures, training algorithms, and neuro-evolution techniques can significantly enhance the effectiveness and superiority of solutions.

A.MINIMAX OPTIMIZATION PROBLEM

Minimax optimization problems (MMOPs) refer to a type of optimization problem that aims to minimize the maximum value among multiple decision variables, with the core concept being to reduce potential losses in the most unfavorable situations. When optimizing such problems, algorithms not only consider the decision space included in general optimization tasks but also the scenario space. For solving MMOP, traditional optimization methods require a large number of function evaluations, which is time consuming and inefficient. In order to reduce evaluation costs, various methods have been proposed, many of which are based on co-evolution strategies to achieve this goal. However, this method can easily lose effectiveness due to asymmetry issues. Therefore, Cramer et al. proposed a method based on sampling scenarios. Through a carefully designed sampling strategy, this method can effectively improve the efficiency of solving asymmetric problems. In addition, some current research works consider the search process of different solutions in the worst-case scenario as independent optimization processes, without considering the effective sharing of information between solutions. Wang et al. proposed the SA-MM-MFEA algorithm by combining evolutionary multitask optimization and surrogate model theory. This algorithm treats the search process of each solution in the worst case as independent tasks in a multitasking environment. In this way, it can comprehensively and efficiently utilize knowledge sharing among different solutions, thereby improving the overall optimization performance. However, the process of building a proxy model may involve high computational costs, and it is not an easy task to create an accurate proxy model. To solve the MMOP problem, Qiu et al. proposed a scheme that can play a basic enhancing role in the auxiliary differential evolution algorithm. Although this algorithm has more advantages compared to some of the most advanced algorithms, it only considers the optimal individuals while ignoring other promising individuals. Based on these observations, researchers systematically developed a two-phase differential evolution algorithm (TPDE) [31], which consists of two distinct phases. In the first phase, individuals with superior objective function values are selected to generate offspring, thereby facilitating the optimization process. In the second phase, the algorithm shifts focus to performing a global optimality search in the solution space, aiming to identify promising candidate solutions and enhance the overall search efficiency.

B.CLOUD SERVER COMBINATION PROBLEM

In recent years, with the rapid advancement of cloud computing technology, the complexity of cloud service systems has gradually increased, and the demand for concurrent use of cloud services by multiple users has also continued to grow. Faced with this situation, the traditional service model has become inadequate and unable to effectively respond to the growing practical needs. To address this challenge, many modern enterprises operating on public clouds have begun to adopt a new strategy: by flexibly combining diverse atomic services and atomic services with different quality of service characteristics, they can quickly build large-scale distributed application scenarios. However, even so, in order to further enhance the processing capability and optimization level of cloud computing service composition (CCSC) solvers, the industry has proposed a CCSC solution based on the evolutionary multitasking algorithm (MEA-CCSC). This solution can optimize two or more CCSC tasks in parallel, significantly improving the overall service throughput and providing new possibilities for service optimization in modern cloud computing environments.

C.BI-LEVEL PROGRAMMING PROBLEM

Bilevel programming problems are considered a complex class of optimization problem, encompassing two levels of optimization. In this context, the decision variables of one optimization problem (the upper-level problem) are regarded as the parameters of another optimization problem (the lower-level problem). This form of problem is widely used in fields such as economics, engineering design, and management science, especially in simulating market competition, resource allocation, production planning, and other scenarios. However, solving bilevel programming problems using traditional methods is often time consuming and difficult. Gupta et al. [32] initially incorporated evolutionary multitasking into the optimization procedure of a bilevel programming problem, allowing for the concurrent handling of numerous subordinate optimization tasks. This method enhances the utilization of inherent similarities between them. Motivated by [33], the multitask bilevel DE (MT-BLDE) algorithm was introduced. This algorithm uses multitasking and multiple surrogate models to simultaneously solve many (possibly similar) low-level problems.

D.SPARSE RECONSTRUCTION PROBLEM

The sparse reconstruction problem refers to the problem of recovering the original data from limited observation data. In fields such as signal processing, image processing, and machine learning, it is common to encounter situations where there is a large amount of data, yet sparsity exists, meaning only a small number of elements in the original data are nonzero. When confronted with the issue of sparse reconstruction, numerous conventional approaches concentrate on addressing the sparse reconstruction issue of a single measurement vector; however, there is no equivalent solution for the sparse reconstruction issue of multiple measurement vectors. When dealing with practical applications, it is frequently required to handle several sparse reconstruction tasks at the same time, and these distinct tasks commonly exhibit similar sparse characteristics. For the purpose of exploiting analogous sparse features shared among distinct tasks, a pioneering multitasking sparse reconstruction framework has been developed. This framework facilitates the simultaneous optimization of multiple sparse reconstruction tasks through the utilization of a unified population.

In the design of collective dynamics control systems, time-series reconstruction of complex networks is a key issue. At present, the prevalent approach in academic circles involves converting intricate network reconstruction problems (NRP) into sparse reconstruction problems and applying convex optimization algorithms to address them. However, existing algorithms predominantly concentrate on the learning process of individual networks and have not yet attempted to leverage similar structural features across networks for transfer learning. When dealing with practical applications, networks that share similar feature patterns with the target network are frequently encountered. Consequently, leveraging this analogous information can significantly boost the precision and productivity of network reconstruction. Rooted in the motivation mentioned earlier, the MFEA-Net algorithm introduced in reference seeks to apply the EMTO algorithm to tackle the sparse reconstruction challenge that stems from NRP. Additionally, it aims to perform the learning and reconstruction tasks of two networks simultaneously within the same evolutionary population.

E.NEURAL NETWORK OPTIMIZATION PROBLEM

Chandra et al. integrated MFEA into the solving process of neural network optimization problems and proposed a neural network EMTL algorithm. This algorithm fully utilizes the relevant information between different network modules to obtain optimized and more complex neural network capabilities, making knowledge transfer between different network modules possible. Chandra et al. [34] introduced the concept of multitask learning to achieve modularity in knowledge representation within neural networks, utilizing modular network architectures. The suggested technique underwent experimental validation on feedforward networks, employing a range of n-bit parity problems with differing complexities, confirming that the technique preserves its performance quality. In addressing dynamic time series prediction challenges, a co-EMTL approach has been introduced [35], which fosters a collaborative effect between co-EAs and multitask learning for dynamic time series issues.

F.COMBINATORIAL OPTIMIZATION PROBLEM

Permutation-based combinatorial optimization problems (PCOPs) are a type of combinatorial optimization problem that represents solutions in a sequential manner. In real life, it is often necessary to optimize multiple tasks simultaneously. However, traditional evolutionary computing methods can only optimize one task simultaneously. Due to its ability to handle multiple tasks simultaneously, Yuan et al. utilized the advantages of evolutionary multitasking MTOoptimization to address multiple PCOPs. Yuan et al. considered the characteristics of PCOPs and addressed the TSP, QAP, LOP, and JSP. They made enhancements to the fundamental structure of MFEA, tailoring it to better solve PCOPs. As an illustration of combinatorial optimization challenges, Feng et al. [36] introduced a novel explicit evolutionary multitask optimization algorithm designed to address the vehicle routing problem. The capacitated vehicle routing problem is a cornerstone issue within the realm of combinatorial optimization.

G.INDUSTRIAL ENGINEERING OPTIMIZATION PROBLEM

The process of beneficiation entails the removal of valuable substances from crude ore to create a refined concentrate. This represents a significant industrial optimization challenge that comprises a multitude of discrete operational phases. During this production process, performance parameters like product quality and production efficiency at each step are known as operational indices. Due to various reasons, operational indices optimization of the beneficiation process (OIOB) is challenging. In the beneficiation process, to achieve optimal outcomes, it often requires simultaneous consideration and optimization of multiple operational indicators. This not only increases complexity to the problem but also transforms it into a multi-objective optimization problem. Additionally, the interdependent yet non-autonomous relationships among various unit processes lead to pronounced nonlinear characteristics between production and operational indicators. Furthermore, the emergence of random biochemical reactions throughout the production process further complicates the modeling process, intensifying the challenge of OIOB. Yang et al. [37] introduced a multitasking, multi-objective evolutionary approach to tackle the OIOB problem, leveraging evolutionary multitask optimization algorithms to effectively resolve it. In the proposed evolutionary multitask optimization algorithm for achieving operational indicator optimization, the two-stage selective mating strategy in TMO-MFEA is employed to enhance the diversity and convergence of multi-objective and multi-factor optimization (MO-MFO).