Vertical federated learning: a structured literature review

Taking into consideration the objective of the review, the following set of research questions were formulated as the initial stage of this SLR.

After the formulation of the research questions, a plan was made to design the search strategy for the SLR. The search strategy includes the initial task of selecting literature databases. Kitchenham et al. listed several high-quality databases for searching research resources [67]. Since VFL is a trending research topic, there are many articles which are pre-prints. We chose to include both published (conference and journal papers) and pre-prints as a part of the SLR. The selected databases were Google Scholar, Web of Science (WoS), IEEE Xplore and arXiv. We experimented with different search terms on the chosen databases (Table 2). Finally, it was decided that the following search term would yield the most relevant results:

The results obtained using the defined search strategy were filtered based on a set of criteria. Only the articles which met the following criteria were considered further for first-round screening:After the initial screening, the chosen articles were checked for redundancy, which resulted in 174 unique articles. The unique articles were then investigated by going through the full text. Additionally, we used “snowballing” [117] technique to identify relevant articles. This was done by identifying relevant articles from the reference section of the previously selected articles. The use of the “snowballing” technique led to the finding of 9 additional relevant articles which were included in this review. These papers might have not appeared in the search due to database constraints, varied indexing terms, or inadequate keyword usage. The final articles which are included in this SLR were selected based on the fact that they answered the following questionsIn addition, survey papers were also considered where vertical federated learning had been addressed.

We extracted data from each of the included papers and organized it in a manner such that we could provide an analysis of the reviewed literature and use them for synthesis in the next section. The data which were extracted from the articles were title & year of publication, source of publication, research question/problem solved, proposed method, availability of theoretical analysis, dataset evaluated on and model evaluated with.

As a last step of the literature review, we performed an analysis of 183 articles relevant to VFL and clustered them based on the problem they addressed and solved. A detailed review of these articles has been provided in the further section.

In a VFL setting, two types of parties are involved [170]: the active/guest party and the passive/host party. The active/guest party holds both features and ground truth/labels, while the passive/host parties possess only features of the same instances. The main idea behind VFL is that the active party aims to collaborate with passive parties to leverage additional features in order to improve the model’s performance, all while preserving the privacy of the data involved. This collaboration allows for a richer set of data to be used in training the model, potentially leading to more accurate and robust predictions.

Moreover, the architecture of VFL can be broadly categorized into server-client and decentralized. In a server-client VFL architecture, there is a trusted coordinator that facilitates the learning process, as proposed by Hardy et al. [46]. This coordinator is responsible for computing training loss and generating homomorphic encryption key pairs to ensure privacy. An example of a centralized VFL framework includes a setup with one trusted coordinator and two parties, where each party represents a single client. On the other hand, decentralized VFL architectures, as proposed by Yang et al. [149], He et al. [48], and Sun et al. [119], do not rely on a trusted coordinator. Instead, the active party takes the role of the co-ordinator and communication occurs between the active and passive parties, thereby reducing the system’s complexity. This architecture has been further extended to include multiple collaborating parties or clients, as seen in works like Cheng et al. [23] and Zhao et al. [166]. In the literature, distinctions between server-client and decentralized VFL architectures have been outlined. However, several key differences have been identified through observations (Table 3).

Each architecture has its own advantages and disadvantages, and the choice depends on the specific requirements of the federated learning scenario.

In this section, we discuss the general VFL protocol which consists of two phases: training and inference.

Algorithm 1 demonstrates the training procedure for a general VFL model. A vertically federated logistic regression algorithm has been proposed in Zhu et al. [170], where participants train their local models using stochastic gradient descent (SGD) for computing intermediate results. Apart from logistic regression, VFL has also been proposed for other machine learning algorithms such as linear regression [41, 66], decision trees [65], random forests [84] and neural networks [167]. Liu et al. [84] proposed employing a combination of the CART tree algorithm, bagging techniques, and a third party to co-ordinator to design a vertical federated random forest algorithm. Wu et al. [137] developed a novel privacy-preserving solution for VFL, Pivot that enables secure decision tree training and prediction without relying on a trusted third party. They also address privacy leakages in plaintext decision tree models and extend their solution to ensemble models like random forest and GBDT. The framework PyVertical [108] makes the use of split neural networks for secure training of models with vertically partitioned data. It also incorporates the PSI protocol for secure alignment of samples before the training.

There has been some existing research conducted focusing on implementing the idea of VFL in extended scenarios. For instance, current VFL systems are developed on the assumption that the labels are possessed by only one client, i.e., the active party. However, there might be practical cases where multiple collaborating clients possess labels which arises the need to apply VFL in a modified manner. The Multi-VFL proposed by  [97] makes use of split learning in a scenario where there are multiple data and label owners. Here, forward propagation is performed by the data owners on their corresponding partial models until the cut layer and then, their activations are sent to the label owners. These activations are concatenated by the label owners in order to complete their forward propagation. Subsequently, the losses are computed and back propagation is performed to compute the gradients. The gradients are then sent back to the data owners who are supposed to use them for completing their back propagation. Moreover, Zhu et al. [169] proposed a secure vertical FL framework PIVODL, to train gradient boosting decision trees (GBDTs) with data labels distributed on multiple devices. PIVODL presents a setting of training XGBoost decision tree models in VFL where each of the participating client holds parts of data labels that cannot be disclosed to others during the training process. A similar approach is also observed in Wang et al. [129] in order to deal with VFL when labels are distributed among multiple parties.

Efficient communication and asynchronous updates are critical in VFL to achieve efficient and scalable distributed machine learning. As VFL involves multiple parties collaborating to train a model while keeping their data private, effective communication protocols are necessary to ensure that parties can exchange information and update model parameters accurately and efficiently. Asynchronous updates enable parties to update model parameters independently, reducing the waiting time for other parties to complete their updates. This approach can improve the efficiency of the training process and allow parties to work independently, making VFL more scalable. This section discusses some of the existing approaches found in the studies to deal with communication efficiency and asynchronism in VFL settings.

Some learning approaches that has been used to address specific challenges in vertical federated learning have been discussed in this section.

Federated learning ensures privacy of data while federation among clients since training of models occurs locally and data never leaves their local sites. In vertical federated learning process, the federated model is trained by sharing of gradients or intermediate results among clients. However, some studies conclude that, there are still possibilities of sensitive private data being leaked through the local gradients in Aono et al. [5], and participants’ data can be inferred through a generative adversarial network during the prediction stage in VFL [90].

The concept of fairness in FL also includes collaboration fairness which implies the incentive mechanism in a FL setting. Since FL is based on collaboration, rewarding mechanism is crucial for the allocating rewards to current and potential participants of FL. To achieve that, FL needs a fair evaluation mechanism to give agents reasonable rewards. Moreover, to analyze business value of VFL in real-world application determining a proper incentive mechanism for the participating clients is crucial since not doing do may result in lack of motivation from the clients to collaborate. The Shapley value (SV) [109] is a provably fair contribution valuation metric originated from cooperative game theory. Song et al. [115] introduced computing SV in HFL setting in order to evaluate the contribution of participants. The conventional approach for computing the SV in federated learning involves retraining and evaluating models on all possible coalitions of participants. This process is straightforward in HFL because the global model is formed by aggregating the local models from each participating client. The aggregation process is additive, meaning that each client’s contribution can be seamlessly added to or removed from the global model. Therefore, by simply aggregating local models from various subsets of participants, the SV can be computed effectively for each participant. On the other hand, in VFL, clients hold different features for the same set of samples, and the global model is formed by concatenating these feature-based local models rather than aggregating them. Each client’s model contributes a different aspect of the feature space, and the combination of these models is not straightforwardly additive. Instead, the global model relies on a composite structure that integrates these diverse feature sets. This global model is never shared with a server or co-ordinator. Thus, calculating SV by direct concatenation of local models from only a subset of clients is not applicable here. Since the conventional approach of computing SV suffers from the exponential time required for retraining models with all possible coalitions of participants, many studies [115, 127, 130] solved this problem by introducing approximation methods. Song et al. [115] proposed two gradient-based methods to reconstruct models during federated learning in order to avoid extra training. The first updates the initial global model with gradients from different rounds and assesses contributions based on the performance of these reconstructed models. The second updates the global model from the previous round with current round gradients and then aggregates the contribution indexes from multiple rounds using weighted sums. However, this approach requires testing model performance exponentially, leading to high computational costs. Wang et al. [130] measured the contribution of participants by calculating federated Shapley value, FedSV for each participant. This approach eliminated the need to retrain models; rather, the Shapley values were computed from local updates during each iteration and then aggregated after completing all iterations to obtain the final federated Shapley value. Again, this approach is not applicable to VFL due to the non-additive nature of local models.

Some works [34, 127] have tailored different techniques for data valuation in VFL settings. Wang et al. [127] proposed using situational importance (SI) to compute the difference between the actual contributions of feature values and their expected contributions in a model. This approach, particularly suited for additive models like linear regression, leverages the direct influence of individual features on model predictions. However, in case of VFL, where data is siloed and sharing across parties is restricted to preserve privacy, knowledge of these expected values for computing SI is not feasible. Moreover, this approach is only applicable for synchronous settings. Fan et al. [34] further extended this idea to both synchronous and asynchronous vertical federated algorithms. Wang et al. [128] proposed estimating SV for participants based on locally computed results that are securely aggregated without revealing individual data entries. This process has been enhanced by a re-weighting mechanism that dynamically adjusts participant weights based on their contributions to improve the model’s convergence speed and accuracy. However, for VFL, homomorphic encryption has been used as a privacy protocol which can lead to high computational overhead in case of large number of participants. Zhang et al. [165] extended the idea of contribution measurement of participants in VFL by incorporating the data pricing model based on Stackelberg with the hosts as the leader and the guest as the follower. This approach provided managerial guidance on revenue distribution for FL platform owners based on their contributions.

From the literature, it is evident that while the SV is extensively used for contribution measurement and data valuation in FL, most works are not specifically tailored for application in VFL or lack sufficient experimental validation in VFL settings. Alternatively, exploring other concepts from cooperative game theory might offer viable solutions for contribution measurement and incentive mechanisms in VFL, potentially reducing computational complexity while maintaining fairness in reward distribution.

An area of study that has gained a great deal of attention recently is explainable artificial intelligence (XAI). Models must be explainable in high-stakes applications like healthcare or finance when there is a strong need to justify decisions made. The same applies in case of VFL as well. However, when exploring through the literature, only a handful of research was found focusing on the explainability aspect of VFL. Kang et al. [60] proposed a method to enhance VFL prediction models by grouping original features into explainable groups and learning meaningful high-order features for improved interpretability and transferability. Moreover, Chen et al. [21] proposed a credible federated counterfactual explanation (CFCE) approach, utilizing a federated counterfactual loss to analyze the importance of features distributed across multiple parties in VFL models. However, proper privacy analysis for these methods would be required. Future research could focus on balancing the need for clear model interpretation with strict privacy requirements. Developing methods that offer meaningful insights without compromising the privacy of the distributed data involved in VFL poses a significant challenge and is a crucial area for exploration.

There have been significant efforts in designing federated learning frameworks for industrial usage but most of them are still in their development stage. A few of the existing FL frameworks support VFL either completely or partially.

FATE [86], developed by WeBank and released in 2019, is a business-ready FL framework for both horizontal and vertical settings. It integrates privacy-preserving technologies such as Homomorphic Encryption and Secure Multi-Party Computation. Despite its robust capabilities, including a variety of pre-built modules for different machine learning algorithms like gradient-boosted decision trees and tools such as FATEBoard for live visualization and FATEServing for model deployment, FATE’s complexity and limited flexibility in terms of customization pose challenges. It supports major deep learning libraries like PyTorch and TensorFlow, enhancing its applicability in numerous applications.

FederatedScope [142] developed by Alibaba Group employs an event-driven architecture that enhances flexibility in the development and execution of horizontal and vertical FL processes, accommodating both synchronous and asynchronous training strategies. It supports multiple machine learning libraries such as PyTorch and TensorFlow via sophisticated message translation mechanisms. It also integrates privacy-enhancing technologies including Differential Privacy, Secure Multi-Party Computation, and Homomorphic Encryption. HE is implemented using the Paillier scheme specifically for linear models within vertical FL settings, enhancing its capability to handle sensitive data securely.

NVFLARE [110] is an open-source, business-ready FL framework developed by Nvidia, specifically designed with healthcare applications in mind for both horizontal and vertical settings. Released in 2021, it supports a wide range of machine learning models, including neural networks and tree-based models. It features robust project management capabilities, supports parallel job execution, and offers advanced privacy features such as differential privacy and homomorphic encryption. While it provides substantial flexibility and comprehensive functionalities, getting used to its architecture may require some effort. The framework includes orchestration through Docker-compose and a user-friendly dashboard for effective monitoring and management of FL projects.

Flower [10] is a framework-agnostic FL platform that supports both horizontal and vertical federated learning, developed by the German company Adap. It facilitates easy migration of a wide range of ML models, including neural networks, into the federated setting. It offers robust documentation, example projects, and experimental support for Differential Privacy. Despite its flexibility and scalability, Flower’s current version lacks advanced privacy features and deployment functionalities.

FedML [47] is a flexible, distributed FL framework designed to facilitate the easy implementation of new algorithms across various network topologies and real-world hardware platforms. It separates distributed communication and model training into distinct modules, with the former based on MPI and supporting advanced privacy protocols and the latter built on PyTorch. FedML is notable for its client-oriented programming interface that promotes flexible distributed computing, crucial for scenarios constrained by GPU memory and training times. Additionally, it supports practical FL applications on devices like smartphones and Raspberry Pis, emphasizing real-world usability and scalability.

PySyft [172] developed by OpenMined primarily supports PyTorch and TensorFlow, and ensures privacy through Differential Privacy and Secure Multi-Party Computation. PySyft’s integration with PyGrid facilitates network management across operating systems using containers. However, its support for vertical federated learning requires the use of extra library PyVertical [108], introducing dual dependencies that may complicate implementation. Despite its variety of features, documentation inconsistencies pose challenges, although ongoing development and a proactive community signal future enhancements.

PaddleFL [8] is designed to facilitate the easy replication and comparison of FL algorithms across various applications like computer vision and NLP. It supports both horizontal and vertical federated learning, integrating advanced strategies such as Multi-task and Transfer Learning. PaddleFL performs well in large-scale deployments with its efficient use of Kubernetes for job scheduling. Additionally, it offers Secure Multi-party Computation for privacy-preserving training and predictions, making it suitable for secure data sharing across different organizational setups.

FedTree [77] is designed for tree-based models like GBDT, adapted for both horizontal and vertical federated learning environments. It employs a unique histogram-based sharing approach. Key features include secure multi-party computation protocols and privacy mechanisms like homomorphic encryption and differential privacy to protect data during training. FedTree is structured to optimize communication and computation processes, supporting effective federated learning deployment across diverse data environments.

CrypTen [68], developed by Facebook AI research, is a privacy-preserving machine learning framework built using PyTorch, enabling researchers and developers to train models on encrypted data. It primarily utilizes Secure Multi-Party Computation for encryption, integrating closely with PyTorch to streamline the transition from traditional to secure environments. This approach not only enhances usability but also ensures that computations remain efficient and compatible with modern machine learning operations and GPU support.

As observed from Table 6, most FL frameworks adhere to a server-client architecture, which may limit scalability and flexibility compared to FedML and FederatedScope, which also support decentralized architectures. In terms of model support, frameworks like FedTree focus on specific models such as tree-based models, offering specialized tools but lacking the versatility seen in FederatedScope, Flower or Crypten that support a wider range of models including GNNs and neural networks. Handling asynchronous updates, a critical feature for large-scale deployments, are notably absent in most of the frameworks except NVFLARE and FederatedScope, potentially reducing their effectiveness in environments with highly variable network conditions. Privacy protocols vary widely: FederatedScope offers comprehensive privacy solutions including HE, SMPC and DP, providing robust data protection. However, the inclusion of multiple complex protocols can increase computational overhead and complexity. On the other hand, frameworks like Flower and Crypten may offer fewer privacy options, which might simplify operations but at the cost of reduced data security. Therefore, the choice of an FL framework greatly depends on specific use cases and requirements. Each framework has its own unique strengths and weaknesses, making them suitable for different scenarios. For instance, NVFLARE focusing on healthcare with its emphasis on DP may be ideal for sensitive medical data, while the flexibility of FedML and Flower might be better suited for academic research environments exploring novel federated learning algorithms. Selecting the right framework requires careful consideration of the intended application, required model complexity, necessary privacy safeguards, and network architecture.

Federated learning is a cutting-edge modeling mechanism which is capable of training a unified machine learning model using decentralized data while maintaining privacy and security. Thus, it has a promising application in financial, healthcare and many other industries where direct aggregation of data for training models is not feasible due to concerns like data security, privacy protection and intellectual property. Most applications are focused on horizontal federated learning and it is quite difficult to observe VFL being used in real-world applications yet. However, in our literature review, we found a limited number of studies proposing and as well as implementing VFL in applications.

Sun et al. [123] considered a scenario involving two hospitals; Inner and Outer hospital possessing records of daily performance and clinical test of patients. Due to patients privacy regulations, it was not allowed to share raw data among the hospitals even though they had records of the same patients. Hence, the Inner- and Outer-hospital information had been bridged via vertical Federated Learning for perioperative complications prognostic prediction. Cha et al. [19] proposed using a communication-efficient VFL on the Schwannoma dataset for predicting hearing impairment after surgery by representing the sensitive medical data into compressed latent representations for privacy. Tang et al. [124] combined the techniques homomorphic encryption and secret sharing to implement intention-hiding vertical federated learning (IHVFL) which predicted breast cancers efficiently and with better accuracy. Islam et al. [54] applied VFL to four healthcare applications; predicting in-hospital mortality, forecasting duration of patient stay, classifying phenotype and, detecting decompensation using the MIMIC-III dataset. Zhang et al. [160] used VFL with a secure feature selection technique to reduce communication overheads for training on real-world medical datasets. Some works [118, 135] have used privacy-preserving protocols for analysis of healthcare data in a vertically partitioned scenario and mentioned the future implementation of VFL in these use cases. Worm et al. [135] implemented the secure multi-party computation protocol to analyze cancer patient data in a vertically partitioned setting to predict the chances of survival of patients after diagnosis.

Vertical federated learning when applied to financial institutions is also able to boost profits by collaborating data among them maintaining privacy [33, 81]. Liang et al. [81] implemented a Hetero Secure Boost Tree (HSBT) algorithm for training VFL model which enabled collaboration between telecom and finance companies improving the success rate of marketing of a commercial bank. Efe [33] proposed a Multi-Institutional Credit Scoring (MICS) for VFL to enable cooperation among different telecom and banks for better credit scoring of consumers in a privacy-preserving manner. Cheng et al. [24] discussed a use case FedRiskCtrl which involved designing a risk control model for small and micro-enterprise loans where invoice agencies collaborated with a Webank in a VFL setting. Li et al. [74] proposed a privacy-preserving sampling strategy for VFL in case of imbalanced financial data by having participants provide partial neighbor information for each sample during the intersection stage and, filtering out controversial negative samples. Abadi et al. [1] discussed using VFL financial fraud detection through the collaboration of third party financial service providers and banks.

VFL has also been useful in the field of e-commerce as observed in Zhang and Jiang [155] where a method based on clustering and latent factor model under the vertical federated recommendation system was implemented. Taking into account the diversity of a large number of different users in each participant and the complexity of the matrix factorization of the user-item matrix, the users were clustered to reduce the dimension of the matrix and improve the accuracy of user recommendations. Similarly, efficient online advertising was achieved through the application of VFL [78]. Mai and Pang [92] developed a vertically federated graph neural network (VFGNN)-based recommender system that has competitive prediction accuracy with existing privacy-preserving GNN frameworks along with privacy protection for user interaction information. Cao et al. [16] studied a contextual bandit learning problem [2] for recommendation in the vertical federated setting. In this approach, an encryption scheme named orthogonal matrix-based mask mechanism was applied to bandit algorithms for privacy-preserving online recommendation. proposed a differential privacy (DP)-based federated cross-domain social recommendation (FCSR) algorithm to protect user connections in a cross-domain social recommendation system. It aims to facilitate collaboration between information-oriented and social-oriented websites by securely integrating different types of user data.

In [168] a VFL scheme has been developed for the purpose of human activity recognition (HAR) across a variety of different devices from multiple individual users by integrating shareable features from heterogeneous data across different devices into a full feature space. Liu et al. [84] proposed an Aligning, Integrating and Mapping Network (aimNet) which enhances image representations by combining data from various vision-and-language tasks. It efficiently used data across different tasks through VFL for more powerful image representations which allows for data privacy preservation. Zhang et al. [163] introduced a vertical federated learning-based cooperative sensing (VFL-CS) scheme for cognitive radio networks to avoid malicious exploitation of sensitive sensing results by keeping the data at the local points. A special use case of VFL was observed in the aviation sector [45] where a flight delay prediction model based on federated learning was designed by integrating horizontal and vertical federated frameworks.

While VFL has shown promise in sectors such as healthcare, finance, and e-commerce, its applications are still limited and in the nascent stages. To expand its real-world applications, further development is needed in areas such as interoperability, scalability, and standardization of protocols.

In machine learning, model drift is a critical challenge that impacts the performance of models over time. It can be categorized into two types: concept drift and data drift. Concept drift occurs when the statistical properties of the target variable change, while data drift refers to alterations in the distribution of input data used for training. In VFL, concept drift can affect the model by altering the underlying patterns or relationships the model is trying to learn from different data sources. Since VFL models rely on data from various entities, a change in the underlying concept in one entity can disrupt the overall learning process, leading to inaccurate predictions. Data drift, on the other hand, impacts VFL by changing the statistical properties or distribution of the input data. This can lead to a mismatch between the current data and what the model has learned, causing a decline in model performance as it struggles to make accurate predictions based on the new data characteristics. While considerable research [15, 57, 100] has been conducted on drift handling, notably in HFL, there is a growing need to extend these strategies to VFL. Liang and Chen [80] presented a method, DVFL, designed to adapt to dynamic data distribution changes in VFL through knowledge distillation. However, it still does not address the issue of concept drift. Effective management of drift would involve detection, mitigation, and adaptation of drift as this is crucial for designing fault-tolerant VFL systems. One promising solution lies in incorporating Continual learning [70], which involves the ability of a model to learn continuously, adapt to new data, and retain previously acquired knowledge. By integrating continual learning into VFL, models can become more adaptable to concept and data drift. Moreover, conventional clustering methods have been employed to detect and manage concept drift in machine learning problems. For instance, the Fuzzy Kernel C-Means clustering model [116] has been utilized to handle concept drift in data streams. However, these approaches often lack privacy preservation, which is essential in federated learning settings. Integrating privacy-preserving clustering methods [50, 150] for addressing drift in VFL could be particularly useful, enhancing both drift detection and adaptation securely.

In the context of ML, fairness can be classified into model fairness and collaborative fairness. Model fairness tackles biases in the data, ensuring that models do not unfairly favor or discriminate against any group. In VFL, if the data from one or more sources is biased, the federated model may develop skewed predictions, favoring certain groups over others. There is a good chance of this happening in VFL due to the diverse and decentralized nature of data sources. To address this issue in VFL, approaches like incorporating regularization techniques [96] and fairness-aware model evaluation [71] can be utilized. Fairness in machine learning is often assessed using bias detection metrics like Equalized Odds and Disparate Impact [103]. In VFL settings, techniques such as SMPC and HE can allow participants to collaboratively compute fairness metrics without exposing their private data. Collaborative fairness, on the other hand, focuses on equitable participation, ensuring that all participants in the VFL process are fairly rewarded for their contributions, which is crucial for a balanced and cooperative environment in federated learning setups. Not ensuring collaborative fairness might lead to unequal participation, where certain contributors are undervalued or overburdened. This imbalance can reduce the overall trust in the VFL system from the participant’s end (Fig. 5).

Explainability is significantly important in vertical federated learning (VFL), as it establishes trust, collaboration, and accountability among participating entities, all while ensuring compliance with strict privacy regulations. However, achieving explainability in VFL is a challenging task due to the inherent decentralization of data, the complexity of federated models, and the prime importance of data privacy. Limited communication between parties, disparate data distributions, and the absence of standardized data structures enhance the challenge. To address these issues, research efforts could focus on developing innovative techniques that balance model transparency with data privacy, such as federated explainability methods [72, 162] and privacy-preserving explanations [12] for vertically partitioned data. To further enhance explainability in VFL, research could explore cross-party frameworks that enable shared interpretability while maintaining privacy. By integrating adversarial methods [3] to test and strengthen explanation robustness, and employing federated causal inference [143] to uncover causal relationships in distributed data, VFL systems can achieve a balance between transparency and privacy.

Incentives have an important role in VFL to motivate multiple data parties to come together to train machine learning models while keeping their data decentralized. The right incentives not only encourage participation but also ensure that each party contributes optimally toward the common goal. In the literature, incentive mechanisms in VFL focus on evaluating participants’ contributions by using methods such as Shapley values, which fairly assess each participant’s value to the collaboration. However, the design of an effective incentive mechanism in VFL is multifaceted and includes several key steps: Participant Selection, Contribution Measurement, Incentive Allocation, and Explainable Decision (Fig. 6).

The first step, Participant Selection in VFL [55] should involve using privacy-preserving techniques to assess data quality of potential participants beforehand, ensuring the selection of those most likely to add value to the federated model. This step helps avoid collaboration with less beneficial participants, aiming for optimal results. Cui et al. [29] proposed an approach for collaboration equilibrium in FL, which facilitates the formation of optimal collaboration coalitions among clients. By using a benefit graph and Pareto optimization. This identifies collaborators who can maximize utility and model performance for each client, ensuring that no client could benefit more by forming different coalitions. In VFL, the utility of collaboration is not solely dependent on data volume or completeness but on the complementarity of the feature sets that each participant brings. This means that determining the maximum achievable utility (MAU) and identifying optimal collaborator sets (OCS) in VFL would require not only understanding individual data contributions but also how these contributions interact in multidimensional and often nonlinear ways. The iterative optimization and the use of benefit graphs in the collaboration equilibrium rely heavily on being able to clearly see how contributions and benefits are distributed among participants. In VFL, this visibility is limited because data must often be encrypted or anonymized. To the best of our knowledge, only two works till now proposed participant selection methods specifically designed for VFL. Jiang et al. [55] proposed using vertically federated Mutual information(MI) estimator to measure mutual information between the labels and features of the participants. A group testing method was further incorporated search for the subset of participants that maximizes this MI. However, this approach requires prior knowledge of how many participants to select which might not be always feasible. To solve this problem, [49] proposed a Joint Mutual Estimator (JMI) for the same purpose as before along with a greedy search algorithm that maximizes the gain of VFL model given by a set of selected participants. But, the scalability of this approach poses doubt, as the greedy search algorithm used is tested only on a limited number of clients (up to 8) and may become computationally expensive with larger participant pools. An important issue not fully addressed is what happens if several participants provide the same or similar data. This could lead to inefficiency. A better approach would be to prioritize participants who offer unique and diverse data, which could make the learning process more effective.

Contribution Measurement in the next step determines the relative importance or impact of each participant’s contribution to the performance of the federated model. The cooperative game-theoretic approach, Shapley value has been used commonly for this purpose in existing studies. However, computing Shapley values for participants is computationally expensive. Furthermore, approximation methods intended to reduce this computational burden are not always accurate, leading to potential inaccuracies in measuring contributions. Therefore, exploring other approaches that balance accuracy with computational feasibility still remains a challenge. Thirdly, Incentive Allocation refers to the distribution of rewards, compensation, or other forms of incentives among participants, based on the contributions they have made, as assessed in the Contribution Measurement stage. This is beneficial for encouraging participation, ensuring fairness, and rewarding participants in a way that reflects their contributions to the collaboration. Possible strategies for this purpose can include incorporating game theoretic approaches that deal with rewarding players such as the Nash Bargaining Solution [125] or Nucleolus [93] in VFL. Khan et al. [64] formulated the incentive allocation problem in VFL as a bankruptcy game, where the performance gain achieved by the federated model serves as the “estate,” and the claims are derived from the unique contributions of each party to this performance improvement. These claims are satisfied using the Talmud division rule, as Aumann and Maschler [7] proved that this rule inherently computes the Nucleolus of a bankruptcy game without requiring the explicit evaluation of all coalition values, making it both fair and computationally efficient. The final step, Explainable Decision, is vital for the overall integrity and acceptance of the incentive mechanism by the participants. It involves transparently communicating the rationale behind participant selection, contribution assessments, and incentive allocations. Therefore, designing fair and effective incentive mechanism VFL represents a valuable direction for future research.

Datasets are important for evaluating and benchmarking VFL algorithms or methods, ensuring their effectiveness and applicability to real-world scenarios. Table 7 lists a variety of datasets that are commonly used in VFL studies, spanning different domains. However, in most VFL studies, datasets are often partitioned arbitrarily, lacking standard splitting benchmarks. This approach can lead to unrealistic scenarios, biased samples, and a limited evaluation scope. Such arbitrary partitioning may not accurately represent real-world data distributions and the unique privacy and security challenges in VFL. Moreover, there is a scarcity of datasets supporting multimodality, which hinders research into applying multimodal approaches in VFL.

A recent study [140] addressed this issue and proposed a framework that introduces novel feature-splitting methods for generating synthetic datasets and a new real-world VFL dataset, particularly for image-image scenarios. It also included methods to quantify the importance and correlation in real-world VFL datasets, aligning them with synthetic ones, and conducted comprehensive benchmarks of VFL algorithms across diverse scenarios. Additionally, to address the issue of dataset unavailability, established research communities, task forces, or conferences could take a proactive role by organizing competitions in collaboration with companies across various industries, focusing on realistic VFL challenges. These competitions would encourage the creation of high-quality multimodal datasets that mirror real-world privacy and security constraints. After the competitions, the datasets and benchmarks could be made publicly available which will advance standardized evaluations in VFL problems.