How Microsoft is turning AI skeptics into AI power users

Since generative AI burst onto the scene in 2023, it quickly swept from personal to workplace use. Microsoft’s global, industry-spanning survey, the 2024 Work Trend Index Annual Report, found that 75% of knowledge workers are already using AI at work, and that usage has doubled in the past six months. But 78% of that is BYOAI, or workers bringing their own AI tools.

“The challenge for every organization and leader right now — or opportunity — is how to channel that individual enthusiasm and experimentation into business value,” said Colette Stallbaumer, WorkLab Cofounder and Copilot GM at Microsoft, joining VB CEO Matt Marshall onstage at VB Transform.

They spoke about how AI can be effectively integrated at work in a way that drives value for the business and for the employees who use it. It’s all about activating at every level of the organization, from the CEO to all line of business leaders across every function, Stallbaumer explained. As you hunt down the business problems that can most effectively be tackled with AI, it’s about embracing experimentation, identifying AI champions and channeling enthusiasm. Since the launch of Microsoft Copilot, they’ve chased that enthusiasm by continually working to keep customers engaged in an active feedback loop, learning what’s working and how to make the product better.

Adding critical new capabilities to Copilot

Customer feedback has prompted the company to add a number of new capabilities to Copilot and Copilot Lab, including prompts specific to a workplace, function-specific prompts and the ability to share them and reuse prompts. They’ve also stepped up aid in the prompt-writing process, adding both an auto-complete feature and a rewrite feature to get the best possible response. Customers are also now able to schedule prompts to run at certain times of the day, every day.

“When we introduced Copilot, we said, this is the most powerful productivity tool on the planet, and all you need are your own words,” says Stallbaumer. “It turns out that was not entirely true. We’ve learned along the way that one of the hardest things for people to do is change behavior. And in the case of Copilot and these generative AI tools, we are asking people to both create new habits, and break old ones.”

Measuring the impact of gen AI

How do you quantify enthusiasm, or employee productivity and satisfaction? Nailing down metrics, KPIs and benchmarks has generally proven to be a tricky undertaking, across industries. Microsoft has shifted its own approach over the last year, and since the introduction of Copilot. The original focus was on time savings and productivity across all the universal tasks that happen in knowledge work, such as email, writing, meetings and searching for information. What they found is that there was a growing body of evidence, both Microsoft-sourced as well as customer and third-party academic research, all saying a very similar thing: These tools do save knowledge workers anywhere from 20 to 40%. But at the same time, customers were saying they wanted more — they wanted to use these tools to fundamentally change the shape of the business, with metrics centered on real business value.

“In 2024, we’ve pivoted to that goal, which starts with understanding the business problems that AI is best positioned to solve,” Stallbaumer says. “We are working on it now, across every function — like sales, finance, HR, marketing — looking at all of the processes, hundreds of processes, and then [determining] the KPIs within those processes where AI can actually make a difference. That’s how we’re thinking about measurement now.”

For example, Microsoft has one of the largest customer service organizations in the world. They found that when applying Copilot to specific areas of an agent’s workflow, they can resolve cases 12% faster, resulting in happier customers, and requiring about 13% less intervention or peer support to resolve cases.

For the finance team, dealing with accounting and treasury means reconciling many hundreds of accounts a week, consuming a huge number of hours. With Copilot they found that tasks that used to take hours of work can be cut down to about 10 minutes.

“It’s about breaking down knowledge work,” she says. “Every job is a series of tasks, and when you can help people start to think about it that way, you start to think, ‘is this a human-powered task or an AI powered-one, and can I delegate this to AI and then build on that work?’”

The future of work, AI and AI-powered work

“We see knowledge work fundamentally changing, and how people spend their time changing,” Stallbaumer says. “We’re going to see it evolve from where people are rarely doers [and instead are] supervisors of both their own work and also AI-generated output. Things that they can set and forget, if you will, but also where AI will come back to the human and ask for further instructions when needed.”

For a salesperson that could mean that backend data entry is taken entirely off their plate in the future, so they can instead focus on the customer relationship. For a finance person, it would enable them to focus on strategically growing the business rather than spending so much of their time on accounts reconciliation.

To get to that future, it’s important to prioritize training, in order to change long-ingrained, pre-AI habits. But Microsoft research shows that even for those using AI, only 39% have received AI training, and only 25% of organizations plan to offer training in the year ahead.

“So that’s our message to customers, that training is important if you want to create that flywheel of usage,” she adds.

Turning skeptics into power users

As their own best customer, they’re currently doing an internal experiment to understand the most effective training interventions, whether its nudges from a manager, or peer-to-peer. The Work Trend Index Annual Report found that organizations across the world have a spectrum of users from the skeptics to the power users, and that matches what they see internally.

The report’s research found that the power users who embrace experimentation are also the type of people who will try again if they don’t get the response they expected, as they do not give up easily. And 90% of those users say AI helps them feel more productive, enjoy their work more and manage their workload so they can focus more easily. They’re also 39% more likely to have heard from their line leader about the importance of using AI to transform their function.

“There’s a great quote from William Gibson: ‘The future’s already here, it’s just not evenly distributed,’” Stallbaumer said. “We see that inside Microsoft: functions that are already applying AI to transform in significant ways. It’s going to be a journey.”