Estimating Fungibility Between Skills by Combining Skill Similarities Obtained from Multiple Data Sources

People-driven organizations tend to rely on an employee-centric organizational structure. The representation can enable the lookup of employee position and associations within the hierarchy. Employee skills are directly or indirectly encoded in many different information sources ranging from their CVs to skills-sets and projects associated with them within the organization. As a result of this, a full understanding of the skills available in an organization is typically unavailable. Direct implications of this are that the resource (people) available for a particular skill is not known and that a measure of “adjacency” between people and skills does not exist.

Agile organizations need to be responsive to changing market scenarios. Demands from products and services constantly change; this results in changing skill-set requirements. Management in an organization needs to have resource information available at multiple levels of abstraction to facilitate capacity planning and decision making, both for immediate needs and for looking into the future. For instance, it is often easier to up-skill an existing employee with closely related skills than go through the process of hiring a new employee. These goals require a notion of fungibility between employees and in particular, their skill sets; fungibility in such a context may be defined as substitution with minimal up-skilling. Similarly, a measure of fungibility between skills allows organizations to improve demand forecasts for those skills considering skill usage data from prior engagements. This is due to the fact that the number of skills across an organization may be fairly large and many skills may occur relatively infrequently in historical engagement data. Aggregating skills into skill clusters using fungibility can enable more accurate demand forecasts, at the level of skill clusters, and provide a balance between forecast accuracy and usability.

This paper focuses on the problem of estimating fungibility between skills for use in capacity planning and, potentially, in the development of a skill-centric representation of an organization. In this work, fungibility is defined and estimated as a composite similarity measure between skills, considering every available data source on those skills. The paper presents a case study of a real-world business problem; it describes the context, data sources available, challenges in using the data to address the problem, design choices and methods employed, the outcome and its application to a real-world business application that has been deployed to the significant benefit of one largest IT organizations in the world. The approach presented in this paper is generic in that it is not organization or data source or industry specific.

We discuss some relevant work in Sect. 2. In Sects. 3 and 4, we first describe the two information sources used in this work—(1) skill descriptions and (2) people skill transitions, and then discuss our method for estimating skill similarity from these sources. We subsequently cast the estimation of fungibility (Sect. 5) as a problem of combining skill similarity matrices obtained using these individual data sources into a composite similarity measure. The matrix integration can be done in a supervised or unsupervised manner based on the available data. This approach to fungibility matrix generation is currently deployed by a large IT organization toward forecasting demand across groups of skills. To this end, Sect. 6 motivates and describes our demand forecasting algorithm, which incorporates notions of fungibility between skills, past/future engagements and uncertainty in forecasts. In Sect. 7, we present experimental results showing that the integrated results (fungibility) obtained from multiple sources are better than any individual source alone; we also describe its deployment in the context of demand forecasting by the large IT organization and show the improvement in demand forecasts over current practice. Finally, we discuss future work and offer concluding remarks in Sects. 8 and 9, respectively .

The recent-past has experienced a spurt in skill-based analytics for human capital management. Many of these works leverage recent advances in text analytics and information retrieval to provide decision support systems based on employee skills. Richter et al. [20] describe one such decision support tool that provides globally optimal workforce assignment recommendations for a given supply demand scenario; optimality is defined by a utility function incorporating different business metrics. Judy [9] describes a method to estimate a transferability measure for workers moving between occupations using occupational attributes based on publicly available structured databases of occupations; this requires manual selection of important dimensions/attributes involved (for each occupation as well as for combining dimensions for pairs of occupations). Bauer et al. [2] similarly address a related problem of developing skill associations for groups of employees through the use of co-clustering. Instead of a fixed skill-taxonomy representation, skills are extracted from enterprise and social data, and used to identify employees who have expertise in desired skills. Connors et al. [6] describe a system for providing substitutable employees against a demand. They use a structured database of employees and their attributes (e.g., skill or skill level), a demand (e.g., a project or a stage within it) that clearly specifies acceptable substitutions on relevant attributes (penalties for different substitutions) and develop a system to rank potential substitute employees, to give to a decision maker. No notion of fungibility between skills is developed; it is a ranking with multiple attributes problem. Finally, Ramamurthy et al. [18] present an approach for developing an adjacency model for skills as a weighted combination of other skills by using only historical employee skills data.

Other than [18], these approaches are generally concerned with identifying employees who have certain (in-demand) skills so that they can be considered for deployment to meet such demand; they do not attempt to develop a quantitative notion of adjacency or fungibility of skills. While Ramamurthy et al. [18] do address this problem more directly than others, their approach still does not yield a pairwise skill fungibility estimate. More importantly, it is unable to infer a signal for skills that are new, or have been recently introduced, and thus have few or no employees associated with that skill. These issues are addressed in this paper through the use of multiple skill data sources, including unstructured skill descriptions, skill similarity matrix generation from each data source and skill similarity matrix integration into a skill fungibility estimate.

The problem of integrating similarities between data (matrices), with a view of clustering entities, is becoming increasingly prevalent across domains in a data-driven world. Abdi et al in [1] present a principal component analysis-based approach to combining distance matrices into a single “compromise matrix” that represents a best aggregate of the original matrices. Lanckriet et al in [10] propose a kernel-based approach to data fusion of heterogeneous descriptions of the same set of entities. Similarity between entities is captured through a kernel representation. Multiple kernel representations are combined using a convex optimization formulation solved using semidefinite programming techniques. In the context of recommender systems, Wang et al [23] unify user-based and item-based collaborative filtering approaches to overcome sparse ratings data being available; similarity fusion is done using a generative probabilistic framework that incorporates multiple conditional probabilities. The work [11] proposed the fusion of similarity matrices into a composite similarity matrix for clustering entities. The clustering problem is formulated as a non-negative matrix factorization problem, solved using iterative minimization process to minimize a cross-entropy measure. The fusion problem finds weights for a linear combination of the similarity matrices through a modified iterative minimization process which alternates between finding the weights and the matrix factorization given the other term. In [22], the authors use similarity network fusion to combine patient similarities obtained from multiple DNA/RNA expression types. Patient similarity matrices (on each gene expression) are represented as graphs which then inform each other through a graph message passing algorithm to converge into a fused patient similarity network. Recent papers that survey the state of the art in “multi-view” clustering (and more generally learning) include [5, 21, 24, 26]. These provide a taxonomy of recent approaches and broadly cover approaches based on co-training, multi-kernel learning, graph clustering, sub-space learning (including non-negative matrix factorization and component analysis methods) and multi-task (in a joint prediction sense) multi-view clustering. The approaches explored in this work to address the similarity matrix integration problem draw motivation from the cited works; the specific methods reported in this paper were motivated by the nature of the data and the availability of only very sparse a priori data on fungible skills.

This paper uses people skill data and skill representations of one of the largest IT organizations in the world to develop its thesis. The organization is people and skill driven. It has an employee-centric organization; however, it also has an established skills taxonomy, one particular level of which is actively used by management in decision making. This taxonomy is briefly described in Table 1 with an example.

The JRSS level of this hierarchy is particularly important to decision making—it is the functional unit of skill measurement for the organization in consideration, i.e., skill-capacity decisions are taken at this level. JRs, JRSSs and skills have descriptions that provide more elaborate details on them. Every JRSS therefore has an associated JR, JR description, JRSS description, associated skills and their skill descriptions. Whereas the existence of a skills taxonomy lends additional structure and provides information that may be useful in other contexts, it is not a requirement for the approach presented in this paper; the approach only requires the existence of a functional unit of skill measurement (hereafter, referred to simply as skill) and descriptions associated with skills; the more detailed the description, the richer the data source. As an unstructured data source (descriptions), particular challenges posed by it include sparsely populated descriptions and the existence of multiple forms of the same words (e.g., abbreviated and long forms of the same word). Skill names and descriptions (unstructured data) are one of two data sources used in developing the approach in Sects. 4, 5 and in the experiments presented in Sect. 7.

In any organization, from a skills perspective, people generally grow with time, i.e., they acquire new skills. Employees may have joined with a certain set of skills. With experience, the employee may transition to new roles or take on new projects, thereby acquiring new skills. By viewing historical data of each employee, it is possible to identify skill transitions that have occurred. Such skill transitions may represent fungibility between the skills. Potential issues with the availability of this data include (a) incomplete employee information—not all employees of all business units have up-to-date records, (b) imbalance—due to constant change in workforce in response to changing skill requirements, certain skills have only very few employees associated with them, whereas others have many and (c) skills information of employees is dispersed in many different databases. For this paper, the data are available from a single database and are assumed to be largely complete. Skill transitions captured from historical employee skill records (structured data) is the second data source used in developing the approach in Sects. 4, 5 and in the experiments presented in Sect. 7. The two data sources mentioned are specific to the organization that is the subject of this case study. In other organizations, there may be other databases or data sources available. The approaches presented can be adapted suitably to factor them.

This section discusses approaches to estimating skill similarities given skill descriptions and people skill data. The basic idea is to encode the data source (skill descriptions or people skill data) into features that enable the computation of a similarity measure between skills. Based on the structure of the data, established methods from the literature such as word2vec [16] or TF–IDF [14, 15] have been used; these choices are not requirements and alternative feature options may be used.

This paper defines and estimates fungibility (or substitutability) between skills¹ as the composite similarity measure obtained by combining skill similarities obtained from multiple information sources. There is a direct connection between skill fungibility and up-skill-ability or skill substitutability in practical business applications—higher fungibility increases the likelihood of a successful up-skill or substitution with minimal cost. There are two levels to this argument—(1) skills and (2) people with those skills. At the level of skills, this paper is of the view that given enough data sources, fungibility provides the most comprehensive estimate of up-skill-ability or skill substitutability. The current paper operates at this level; two informative data sources are used but the approach is not limited to these and can incorporate other data sources toward obtaining a better fungibility estimate. Most organizations tend to plan skill requirements (vs cost) at the skill level. At the level of people (with the skills in question), skill fungibility alone does not fully determine up-skill-ability or skill substitutability as subjective or soft skill or other un-recorded factors come into play that are considered by the decision maker when taking a decision on a particular substitution or up-skilling. Our paper does not address this problem but aids the decision maker with one critical input—fungibility between skills.

Two sources of skills information are considered in this paper—skill descriptions and people skill transitions. Pairwise skill similarity matrices obtained from these sources are thus integrated into a single measure of fungibility or substitutability between skills, also a pairwise similarity matrix between skills. If the skill similarity matrix from skill descriptions is denoted by \(S_{d}\) and that from people skill transition data are denoted by \(S_{p}\), then the fungibility matrix denoted by S is a function F of these input matrices.This function F may be a linear summation of the inputs or may be a more complex function. The method that determines the specific function is strongly influenced by the data that is available, in the specific context. For instance, where abundant labeled exemplars (fungible skill pairs) are available, the function could be a classifier. When data and particularly labeled exemplars are scarce, a linear summation of similarity matrices may be a very reasonable option. This work is an example of such a case, where labeled data are scarce (see Sect. 7) and hence, the function is taken to be a linear summation of the two input matrices, that maximizes an objective; where labeled exemplars are available, the objective is to produce a fungibility matrix S that maximizes the occurrence of those exemplars and where no labeled exemplars are available, the objective is to produce a fungibility matrix S that maximizes information captured from the input matrices. These scenarios and methods are discussed in the sub-sections that follow.

Future extensions to this work may include the inclusion of additional skill data sources or metrics such as number of years, type of experience (on-site, off-site) to get a better estimate of fungibility between skills. Some of these may produce additional similarity matrices between skills; some may be better suited as constraints within a specific application problem. Based on deployment feedback and the availability of further fungible skill-pair exemplars, more complex functions F and methods to learn them may also be explored. The resulting fungibility matrix can in turn provide the basis for any subsequent clustering of skills to develop skill-centric representations of an organization and for applications like short-term capacity planning, strategic planning, shortlisting of candidates for re-skilling and other skill-based analyses. Combining similarity matrices can be done in a supervised or unsupervised manner depending on the data that exist. A visual depiction of this process is shown in Fig. 3.

The notion of fungibility, as described by the fungibility matrix obtained from Sect. 5, has been successfully applied to the business application of forecasting demand across groups of skills. This section serves to lay out the manner in which the fungibility matrix between skills is used in the context of demand forecasting and presents our forecasting algorithm. Details on the deployment of the fungibility computation and the demand-forecast application in a large IT organization are presented in Sect. 7.

Basic demand forecasting methods essentially rely on some form of averaging over historical demands of various skills. Advances in this area have focused on improving the forecasting algorithm or outcomes through the use of hybrid multiplicative/additive models [13], the use of skill-categories (keywords) for forecasting [13], the incorporation of different factors like the status of ongoing engagements, potential signings [4]. The work [6] required explicit specification of acceptable substitutes (and associated penalties) from the demand side.

While these advances are all relevant, this paper extends the state of the art through the incorporation of multiple novel features within the proposed forecasting approach. These include (1) the use of skill clusters obtained using the previously described (see Sect. 5) automated method to estimating fungibility between skills, given multiple skill information sources, (2) the use of both historical engagements and those in the pipeline and (3) the factoring of uncertainty, to produce forecasts with confidence bounds, through a simulation of successful outcomes of engagements which are in the pipeline.

Given the expertise taxonomy in Table 1 and given that JRSSs are the functional unit of skill measurement for the IT organization being studied in this paper, demand forecasts at the level of individual JRSSs are generally most useful to a business decision maker. However, the lack of historical data often results in inaccurate outcomes. Methods such as forecasting skill-categories as demonstrated in [13] or highly abstract classes of skills (e.g., at the JR level) may provide more accurate forecasts due to availability of more data but are typically not useful for a business decision maker due to the abstract nature of the forecasts; these abstractions do not capture a function-level similarity between skills that is critical to enabling meaningful substitutions within a capacity planning and optimization scenario. This work proposes a trade-off between accuracy and usefulness of the forecasts by grouping JRSSs into fungible clusters and estimating forecasts at the level of these clusters. The algorithm used to produce these forecasts is presented below; JRSSs are replaced with the word “skill” following the taxonomy-agnostic notation used throughout this paper. A more detailed description of proposed demand forecasting approach, that includes the rationale for each step and implementation details, is presented in [7].

A set of 1351 skills of a large IT organization were considered. As mentioned in Sect. 3 and Table 1, each skill had associated meta-data, capabilities and all of their descriptions associated with it. These were subject to preprocessing to handle word-variations (short forms), typos, singular/plural word forms, uninformative words, symbols and punctuation marks. Preprocessed words were concatenated into lists of words with the skill word as an additional identifier/tag word for each such list. These data were used as input to learn word2vec feature vectors for the skill tag words. The representation basically learns to predict co-occurring words to the skill word in question. Given feature vectors for each skill, pairwise similarities between them can be computed using the cosine similarity between the corresponding feature vectors. The result is a similarity matrix obtained from skill descriptions.

Employee records of over 400 thousand employees were mined to extract their current skills and all skills associated with them until the current time. A percentile plot of employee counts per skill is shown in Fig. 4. The figure shows 0–90th percentile to clearly show the distribution across a majority of skills. About 265 skills had no recorded employee counts and about 304, below 20 employees each. Among the 1351 skills in consideration, the maximum employees for any one skill ranged up to 21,132—corresponding to the 100th percentile.

A TF–IDF feature vector representation for each skill was learnt in terms of all skills in question; the TF and IDF values for each token were decided by the employee skill records. As before, given feature vectors for each skill, pairwise similarities between them were computed using the cosine similarity between corresponding feature vectors. The result is a similarity matrix obtained from employee skill records.

Similarity matrix integration was then performed using the PCA and LINSUM approaches. The LINSUM integration was initially carried out using a preliminary SME feedback of 8 groups of skills obtained through earlier discussions; the 8 clusters covered about 30 skills of the total of 1351. A sample of these clusters is shown in Fig. 5. A single skill cluster (obtained separately from each similarity matrix) depicted in Table 2 showed a similarity between skill clusters obtained from each information source used. Note the similarity in the clusters generated in Table 2 and the first cluster provided by the SME in Fig. 5.

The resultant fungibility matrix was then used to cluster the skills into about 462 groups of fungible skills (the number was chosen empirically based on the groups produced) which were presented to the business; these were very well received. At this stage, the quality of the skill clusters generated using both approaches was similar; thus, the decision was made to deploy the PCA-based clusters as the business felt using the unsupervised approach made more sense, given the limited amount of feedback available initially.

Upon presentation of these groups of skills to the SMEs, additional feedback collection was also started. The feedback expected was in the form of comments against each skill, listed by group. SME feedback obtained included blank lines (no comments) and explicit comments grouping specific skills together. Blank lines (i.e., no comment) may be assumed as approval. However, given that SMEs had to parse 1351 skills and group numbers, blank lines may also reflect the case of partial feedback for the most obvious of skill groupings that were apparent to the SME. For the purpose of this paper, only skills accompanied with explicit feedback on their groupings (i.e., fungible) with other skills are considered for evaluation. Their comments were manually parsed and a set of 19 groups of skills (about 71 in total) were obtained. With this additional data, we could once again evaluate LINSUM against PCA as well as the other methods. The aforementioned skill groups constituted the exemplars against which different methods were evaluated. The LINSUM deployment process is shown in Fig. 6.

Similarity matrix integration was once again performed using the different methods presented in Sect. 5. The rows and columns corresponding to the exemplar skills were extracted from the fungibility matrix. Using this fungibility (sub-)matrix, a clustering procedure was performed and the clustering outcome was evaluated with respect to the SME input. These evaluation steps are similar to step 4 of the supervised approach to similarity matrix integration presented in Sect. 5.3. The resulting fungibility matrices obtained agreed with SME inputs to the extent of the precision, recall and F-measure scores listed in Table 3. The table also shows the metrics obtained using only people skill transition data or skill description data. It shows outcomes of the people data support approach—using both a penalty-based support measure (Equation 3) and the switching model (Equation 2 or zero subject to the threshold), for three different confidence thresholds defined in terms of the employee counts per skill.

The similarity matrices from skill descriptions and people skill transition data are, respectively, shown in Figs. 7 and 8. The three best integrated fungibility matrices obtained using the LINSUM, PDS and PCA approaches are, respectively, shown in Figs. 9, 10 and 11. The LINSUM outcome is composed approximately of about 72% people skill transition similarity information and 28% skill description information and hence appears rather different from the other two outcomes. This particular weighting was chosen as it maximized F-measure, maximized the sum of precision and recall and balanced information used from individual matrices. The inverse combination of about 26% people skill transition-based similarity and 74% skill description similarities performed almost identically well—the outcome of this case would appear similar to the PDS and PCA approaches. Provision of further exemplars of fungible skills could further narrow down the weight combinations giving best outcomes in the supervised LINSUM case. As such, we intend to replace PCA with LINSUM in subsequent deployments as more feedback becomes available. Each of the three outcomes shown produced outcomes superior to that obtained using the individual similarity matrices by themselves; this demonstrates capture of synergy between the different skill similarity information sources.

Fungibility between skills was used to cluster them into skill clusters. These skill clusters are being used by the IT organization to forecast demand, given skills usage information from prior engagements and those in the pipeline. Forecasting demand at the level of skill clusters (instead of individual skills) achieves good trade-off between accuracy and usability. This is due to the fact that the number of skills is fairly large, but many skills are rare in historical engagement data and forecasting their demand directly is not accurate. However, aggregating them using fungibility, into skill clusters, results in higher prediction accuracy.

Figures 12 and 13 show the validation test of the demand forecast. Forward validation revealed that the approach presented was superior to the status quo for the large IT organization being studied in this paper. An existing system had workforce-management provide their anticipated demand for projects in the pipeline. This system was compared to the demand forecast approach presented in this paper by examining their accuracy relative to actual demand for a six-month period—this is shown in Fig. 12. In comparison with actual demand, the demand forecast from the proposed approach, based on fungibility between skills, was closer than the existing management system for 93% of the skill clusters. Additionally, a view of forecast demand compared to actual demand at the skill cluster level is available as shown in Fig. 13. A more accurate demand forecast allowed this large IT organization to increase IT labor gross profit by 2.3%. This resulted in a multi-million dollar impact due to better planning of high-demand skills and avoiding high premiums on short-notice labor resource shortages. More details on the proposed demand forecasting approach are discussed in [7].

It should be emphasized that an objective measure of fungibility between skills does not exist in the organization whose data have been used for this study; we believe this is the norm across organizations. Given the current state of no clear baseline and sparse SME inputs to work or validate models with, the similarity matrices from individual data sources already provide a measure of substitutability between skills. This paper further defines fungibility as a composite similarity score between skills, considering all available data sources in the organization; the improved outcomes from the composite measure demonstrate the potential for synergistic use of multiple skill data sources, that an organization might have, toward estimating an objective measure of fungibility between skills.

While the paper describes and evaluates both supervised and unsupervised approaches to integrating skill similarities, the emphasis of the paper is in the definition and estimation of fungibility between skills obtained by integrating skill similarities obtained from different information sources; the integration is performed using a supervised or unsupervised approach as afforded by the application scenario. Other integration methods exist and may be used in place of those presented in this paper. Quality or uncertainty measures for individual data sources may also be incorporated if there is explicit prior knowledge or a reference to compare with and a validation mechanism. The fungibility measure may be used within skill-based recommender systems; its definition in terms of multiple information sources of skill similarity lends it a level of robustness to “cold-start” problems (ill-defined or incomplete information source) that may occur in enterprises with less rigorous skill taxonomy or recording mechanisms.

Numerous avenues for extending this work exist; these may consider additional data sources or evaluate alternative representation and integration methodologies. Some specific examples include the inclusion of Wikipedia (or similar resource) as an additional source of skill description information, particularly in relation to domain-specific keywords, the use of alternative vector-space representations to encode skills (e.g., paragraph vectors in [12]) and the detailed comparison of multiple methods of integrating skill similarity matrices.

A measure of fungibility between skills was defined to enable effective capacity planning, analytics and optimization for modern organizations that have to cope with ever-changing skill requirements. Fungibility was estimated as a combination of skill similarities obtained from different information sources including (but not limited to) people skill transition data and skill descriptions. Supervised and unsupervised similarity matrix integration methods were presented and experiments (with this measure) demonstrated improved outcomes compared to using any single skill similarity source alone. Fungibility matrix estimated using the unsupervised PCA-based matrix integration approach has been deployed by a large IT organization for clustering skills for use in demand forecasting. To this end, a demand forecasting approach for fungible clusters of skills that factors historical engagements, potential engagements and uncertainty over their successful acceptance, was also presented. Demand forecasts obtained were shown to be very competitive with respect to actual demand and significantly more accurate than the current approach employed by the IT organization. Feedback on the current deployment of the fungibility estimation approach has resulted in additional improvements in the performance of the supervised similarity matrix integration (LINSUM) approach which is, thus, expected to replace the unsupervised approach in subsequent deployments.