Resolving Workforce Skills Gaps with AI-Powered Insights

Digital transformation is a continuous journey, with new technologies emerging on an ongoing basis. Yet for organizations to harness these technologies, their workforce needs to develop an increasingly expanding variety of skills. Many organizations struggle here: leaders responding to a 2022 MIT CISR survey[foot]MIT CISR 2022 Decision Rights for the Digital Era Survey (N=342).[/foot] estimated that on average 38 percent of their organization’s workforce required fundamental retraining or replacement within three years to address workforce skills gaps.[foot]Workforce skills gaps are the discrepancy between the collective skills proficiency that an organization requires to achieve its strategic objectives and the current skills proficiency of its workforce.[/foot]

To make evidence-based decisions on how to best resolve such skills gaps, however, organizations first need to move beyond estimates. What’s required is precise insight into their workforce’s current skills and how proficiency in these skills differs from that needed for future success. Functional competency models often fall short in this regard, as they need to be validated for each job and thus can’t keep pace with rapid technological change. Relying on employee or manager feedback from interviews and surveys may lead to inaccuracies because of inherent biases. And active assessment of an entire workforce across a wide range of skills is both impractical and costly.

Artificial intelligence (AI) offers a new and scalable alternative to such approaches by enabling skills inference, which we define as the process of analyzing employee data to quantify skills proficiency. This allows for detailed insight into workforce skills gaps, which can, for instance, be broken down by line of business and geography. In this briefing we explore the AI-powered skills inference process, and illustrate how resulting insights can help resolve workforce skills gaps by drawing on lessons learned from a case study of global healthcare company Johnson & Johnson (J&J).

Digital Talent Transformation at J&J

J&J’s mission is to profoundly impact health for humanity. As the trajectory of health and wellbeing is increasingly determined by emerging technology and a growth in data and computing power, J&J’s Technology group has become a cornerstone for the organization’s future success. It drives technological innovation at J&J and modernizes the organization’s tech ecosystem. More importantly, however, the Technology group is enabling J&J to evolve as a digital organization by helping to develop the digital acumen of its global workforce of over 130,000 employees.[foot]This case study of J&J draws from N. van der Meulen, O. Tona, I. A. Someh, B. H. Wixom, and D. E. Leidner, “Developing a Digital-First Workforce: AI-Driven Skills Enablement at Johnson & Johnson,” MIT CISR Working Paper No. 461, November 2023, https://cisr.mit.edu/publication/MIT_CISRwp461_JohnsonandJohnsonAIDrivenSkills_VanderMeulenTonaSomehWixomLeidner.[/foot]

In early 2020, J&J’s Technology group began its journey of building the organization’s digital acumen with the help of AI-powered skills inference, starting with its own workforce of 4,000 technologists. By the time the group introduced skills inference to other parts of J&J in 2021, it had successfully put in place a three-step process, as illustrated in figure 1. First, it created a skills taxonomy, defining what skills would be required across the organization to reimagine business processes and develop future digital offerings. Second, the group gathered skills evidence by selecting and preparing employee data sources to analyze. And third, it conducted a passive skills assessment, for which it trained a machine learning model to measure the skills proficiencies of each employee. The result was workforce insights that guided employees’ personal development and enhanced leaders’ strategic workforce planning, both of which reduced skills gaps at J&J.[foot]J&J used skills insights only as a guide for employees’ own development; the insights did not factor into employees’ performance reviews. The organization used deidentified insights at an aggregate level to support strategic workforce planning.[/foot]

Figure 1: The Three Steps of the Skills Inference Process

Skills inference involves (1) defining a taxonomy of skills required to realize your organization’s purpose and strategic objectives, (2) gathering employee data as evidence of these skills, and (3) conducting an assessment of this evidence to quantify employees’ skill proficiency.

Defining a Skills Taxonomy

The journey of J&J’s Technology group began with figuring out what future skills J&J would need. Guided by industry benchmarks, its Digital Talent team—a team dedicated to driving the organization’s transformation with the best and most diverse talent—examined strategic plans throughout the organization to create a J&J-specific skills taxonomy. This taxonomy comprised a list of forty-one skills the team referred to as “future ready” (e.g., master data management, robotic process automation) grouped into eleven capabilities (e.g., “Scientific & Digital Health Technology”) that would be required to realize the organization’s purpose and strategic objectives.

To ensure the taxonomy’s accuracy and gain broad support for it, the Digital Talent team asked over one hundred senior leaders from across the company to validate the list. Each indicated whether the taxonomy reflected the needs of their area of business—both at that moment and in the long term—and offered their perception of the current and required state of the listed skills. Given the diversity of J&J’s operations, required proficiency levels naturally varied by functional area. For instance, employees in Innovative Medicine might need stronger capabilities in data engineering and analytics, whereas those in MedTech might need to be especially skilled in software engineering.

Additional tailoring of the taxonomy’s skill and proficiency definitions to J&J’s unique context and terminology fell to subject matter experts (SMEs), employees known for their expertise and thought leadership in a particular future-ready skill. These SMEs crafted current, precise, and yet broadly applicable definitions that novices and experts alike could understand. The result was a skills taxonomy that clearly communicated to every employee what future-ready skills were considered top priorities for J&J’s leadership. Moreover, the taxonomy specified how skills would manifest in employees’ data.

Gathering Skills Evidence

As a machine learning algorithm can only learn from provided data, the quality of chosen data sources is key to the overall skills inference process. As such, the Digital Talent team collaborated with HR data experts to identify data sources that were used across most of the organization yet also provided enough semantic data to calculate skill proficiency, ideally providing evidence for 60 to 70 percent of each employee’s skills. Four of J&J’s data sources fit these criteria: the organization’s HR information system, recruiting database, and learning management system, and one of its project management platforms.

To improve data quality, leaders encouraged employees to update the data fields in those systems that the algorithm would use to infer their skill proficiency. For instance, J&J’s HR information system allowed employees to showcase their experiences and accomplishments by sharing information about their job history, education, certifications, recognitions, goals, personal interests, and volunteering activities. If these fields were missing or incomplete, the algorithm could not infer from them. The Digital Talent team therefore also illustrated how employees could update their data most effectively by providing examples of rich statements that the algorithm could pick up on in gradations of good, better, and best quality.

For employees to be willing to provide additional data for the AI to infer from, however, they first had to trust the skills inference initiative and the intended purpose of the AI model. In accordance with J&J’s commitment to the transparent use of AI, the Digital Talent team and senior leaders communicated early and often with employees—both electronically and in person. They explained how the skills inference process could help employees identify their current skills proficiency and discover new development opportunities. They also gave them the option to opt out at any time. In addition, the Digital Talent team enhanced employee trust and engagement by establishing strict norms of acceptable data use with the help of HR data experts, multiple oversight functions at J&J, and external partners. These norms safeguarded compliance, but also respected employee privacy and reduced the risk of bias by maximizing accountability, explainability, fairness, privacy, and transparency regarding the skills inference process.

Conducting a Skills Assessment

To measure employees’ skills proficiency, the Digital Talent team relied on a proven machine learning model augmented by human input. An experienced solution provider supported the team and provided the model, which used natural language processing to generate proficiency scores for each of the forty-one future-ready skills in J&J’s taxonomy.[foot]Looking ahead, the Digital Talent team is now exploring whether generative AI can further enhance the skills inference process at J&J.[/foot] These scores ranged from zero (no skill detected) to five (thought leadership).

To improve the accuracy and reliability of the inferred scores, the Digital Talent team asked employees to self-assess their skills proficiency and managers to evaluate that of their direct reports. To avoid bias, the solution showed the proficiency scores inferred by the AI only after the participants submitted their perceptions of proficiency levels. With this input, the model generated an “agreement score” that quantified the consistency between the perceived levels and inferred scores. The goal was not to achieve perfect agreement, but rather directional accuracy: the Digital Talent team considered the inferred scores usable if they deviated by at most one point (out of five) from the perceived proficiency levels.

Limit Use Cases for Workforce Insights

Skill proficiency data has many potential uses. Yet, for employees to trust the skills inference process and not opt out of it or otherwise skew the data, it is important to limit what this data is used for. J&J therefore only used skills inference to provide more personalized career development journeys for employees, and (at an aggregate level) to support leaders’ strategic workforce planning efforts. Other use cases were not permitted by J&J’s Privacy function.

With a detailed understanding of their skills proficiency, employees could chart personalized career paths, supported by learning and development opportunities uniquely tailored to their skill proficiency levels. After the first round of skills inference, J&J saw a 20 percent uptick in participants’ voluntary learning activities. These activities not only enhanced practical expertise but also fostered habits of continuous learning and increased knowledge sharing within the organization. In subsequent years, these habits of continuous learning have only become more ingrained, as demonstrated by strong adoption of J&J Learn, the organization’s global, AI-powered learning and development ecosystem that offers training programs, growth assignments, and mentoring opportunities. By March 2024, over 90 percent of employees in J&J’s Technology group had accessed J&J Learn.

Leaders used an executive dashboard to gauge aggregated employee skills proficiency, with insights broken down by geographic region and line of business. This dashboard, displayed as a heat map, resulted in more informed hiring processes, enhanced retention efforts, and improved talent movement and advancement across J&J. The executive committee used a scorecard to track key performance indicators related to these outcomes, while each operating company and supporting function devised its own metrics based on its strategic plans and the capabilities it had to develop.

Skills Inference: People + Technology = Workforce Insights

Resolving skills gaps no longer falls to human resources or learning and development functions alone. Instead, it has become a strategic imperative, reshaping organizational capabilities based on workforce insights. AI emerged as a powerful tool in this endeavor, enabling skills inference at a scale previously unimaginable. However, the success of this process hinges on more than just advanced technologies. It continuously requires collective effort, trust, and support of stakeholders—including employees—across many organizational levels and functions. Just as digital transformation is an ongoing journey, so too is the need to regularly (re)define the skills taxonomy, gather new skills evidence, and conduct skills assessments as required skills evolve.

For those looking to embark on their own skills inference journey today, we suggest you first focus on these inherently human success factors. Begin by generating employee support with a bounded use case that embraces your workforce’s potential and signals a commitment to developing employee skills. Then gather broad input from experts to declare your desired workforce capabilities in the form of a skills taxonomy that is aligned with your purpose and strategic objectives. That way, your employees can focus their development efforts while simultaneously providing more informed input for an eventual AI to process. Start by laying this groundwork today, so that your organization may reap the rewards of AI-powered workforce insights in the future.

© 2024 MIT Center for Information Systems Research, van der Meulen, Tona, and Leidner. MIT CISR Research Briefings are published monthly to update the center’s member organizations on current research projects.