HOW TO MANAGE THE TWO FACES OF GENERATIVE AI—TOOLS AND SOLUTIONS—FOR BUSINESS SUCCESS

The MIT CISR research briefing, titled “Managing the Two Faces of Generative AI,” examines data, analytics, and technology executives’ early GenAI experiments and the challenges and management principles for implementations of GenAI tools and GenAI solutions.

The findings draw from a series of three consecutive virtual roundtable discussions and 23 semi-structured interviews with a total of 93 data and technology executives from 50 global organizations that generate over $1 billion in annual revenues. All participants were from the MIT CISR Data Research Advisory Board (aka the Data Board), a community of data and analytics leaders who participate in and inform MIT CISR research. The authors of the briefing are Nick van der Meulen, a research scientist at MIT CISR, and Barbara H. Wixom, a principal research scientist at MIT CISR and founder of the Data Board.

Recommendations for Leaders to Overcome Challenges with GenAI Tools

GenAI tools include conversational AI systems (e.g., OpenAI’s ChatGPT) and digital assistants embedded in existing productivity software (e.g., Adobe’s Acrobat AI Assistant) that primarily enhance users’ personal productivity, aiding workers in tasks such as summarizing documents, brainstorming ideas, and writing first drafts of emails. One executive in the study referred to the benefit of such uses as “productivity shaves,” saving users a few minutes of effort with each task. But minutes quickly cumulate into hours. Recommendations include:

• Reduce the allure and risks of BYOAI (Bring Your Own AI) with clear usage guardrails and guidelines. Providing enterprise-sanctioned access to a select number of GenAI tools creates a safe space for employees to experiment while diminishing the appeal of BYOAI, which often increases the risks of data loss, intellectual property leakage, copyright violation, and security breaches.

A new MIT Sloan Management Review article titled, “Bring Your Own AI: How to Balance Risks and Innovation,” written by Wixom and van der Meulen, expands on this recommendation and shows why bans on BYOAI don’t work.

• Educate employees by investing in ubiquitous training. Prioritize establishing effective AI direction and evaluation practices, which involves teaching employees to effectively instruct and interrogate GenAI tools and the underlying models, along with using the tools ethically and responsibly. -
• Control costs by standardizing on a select set of vendors. Providing users with licenses to tools from multiple vendors can quickly become expensive once free trials and early adoption incentives expire. Instead, form a cross-functional team of potential GenAI tool users to help the IT organization determine which tools hold the most potential for your organization.

Recommendations for Leaders to Overcome Challenges with GenAI Solutions

GenAI solutions are based on business case-driven development initiatives that address strategic business objectives and create monetary value for specific groups of organizational stakeholders — ideally at scale. For example, a GenAI solution for a call center might use an LLM to process the content and tone of conversations and provide real-time coaching to agents. Such solutions generate financial returns through increased efficiencies or revenue growth, such as improved agent productivity and customer retention in the call center context.

“We summarize three key approaches to developing GenAI solutions as ‘buy, boost, and build,’” said van der Meulen. “With the buy approach, companies quickly adopt GenAI solutions provided and maintained by vendors, ideal for immediate needs without model customization. The boost approach enhances vendor-provided models with proprietary data to fine-tune performance for specific contexts, offering more tailored results. Finally, the build approach allows organizations to fully develop, run, and maintain their own GenAI models, providing complete control and differentiation but requiring significant investment in data management and AI capabilities.”

Recommendations to succeed with GenAI solutions include:

• Avoid the risk of “shadow GenAI” development by establishing a formal, transparent GenAI innovation process. Develop clear governance structures, early and consistent stakeholder engagement, and a focus on scalable solutions. This helps dissuade groups of stakeholders from independently pursuing unsanctioned GenAI solutions when employees’ growing interest for new GenAI solutions is not addressed.
• Realize financial value by formulating guidelines for GenAI development decisions. Make sure to differentiate among the three GenAI development approaches—buy, boost, build — to help teams make informed decisions on trade-offs in transparency, context-awareness, and cost.
• Prevent risks such as exploitation of GenAI model behavior, data leaks, and inaccurate outputs by creating a GenAI vendor partnership strategy. View GenAI vendor partnerships as ongoing relationships that rely on mutual understanding and long-term collaboration, not one-time transactions. Vendors benefit from direct feedback on organizations’ willingness to pay and insights into how they will use their offerings to create value, while organizations gain from vendors’ transparency, advice, and custom support. “For companies early in their GenAI journey, the best approach is to adopt a few GenAI tools from trusted vendors, supported by hands-on training and close oversight to manage risk and costs,” said Wixom. “Those further along should focus on developing GenAI solutions that most contribute to strategic business objectives—buy or boost—to move fast, and build to create differentiated, competitive advantage.”