Incorporating AI into 30,000 employees’ most critical and beloved tool
Our team built an AI-powered assistant embedded directly in PowerPoint that helped consultants draw on the firm’s collective knowledge while producing slides faster and with stronger insight.
If we can tap into BCG’s collective knowledge and IP to deliver AI-accelerated help, can we improve productivity and the quality of content created by 30,000 consultants?
CONTEXT
As enterprises began adopting AI tools in 2023, many productivity tasks could be supported with off‑the‑shelf solutions. But the speed and specific needs of management consulting presented a unique opportunity.
Slides are not just presentation artifacts—they are how consultants think, structure analysis, and communicate strategy to clients.
For this firm, improving productivity meant improving how consultants work inside PowerPoint itself. Rather than introducing a separate AI tool, our opportunity was to integrate AI directly into the authoring environment consultants already relied on.
WHAT WE MEASURED?
Adoption of feature usage
Quality of users’ output
Perceived time savings
The beta version launched to 300 beta users in October 2023. Over the first 3 months, as we iterated and CSAT scores trended up, word-of-mouth access requests grew quickly. We scaled access to nearly 3,000 users, while maintaining good usage metrics and CSAT scores. Global roll out to over 30,000 employees was scheduled for Q2 2024.
OUTCOME
Easy proof of concept, hard real product
Getting from a proof-of-concept that “worked” to a valuable tool was the main challenge we faced. To integrate GenAI into a product that is used for hours each day—if not the entire day—is a high bar to clear for user adoption. Custom workflows, shortcuts, and button layouts are the norm for PowerPoint power users in consulting. Breaking through that meant that value and usability needed to be sky high—and work well for both first-time users and power users as they integrate it into their workflow.
Tech showed what was possible, design & research set the direction
As we got traction early on, the excitement from leadership was growing. This also meant that imaginations were in overdrive. A mix of incremental feature improvements and fully automated agent-based solutions were dreamed up every time a senior stakeholder was in the room with our team. And the quickly evolving tech capabilities made all of these seem just barely out of reach—and closer by the day.
I led the team in developing a framework to organize our user journeys around Finding, Creating, and Refining—the primary tasks a consultant is doing in PowerPoint. This was valuable in both managing up to stakeholders as we communicated our priorities, and to focus our ongoing user research efforts around these pillars of consultant workflow.
New ways of working to adapt to evolving tech capabilities
Most of the technology we were using was novel to the entire team—each month in 2023 was a big step forward in LLM capabilities.
Because of the emergent capabilities of these models combined with the relative immaturity of our product, we quickly started to adopt several new ways of working. These varied from our typical product design and development, where things are often more predictable.
HOW’D WE DO IT
-
Things move extremely fast, which broke our “design-test-build” pattern. As ideas came from users, leaders, engineers, and everyone else, even our small team had to separate experiments from develop tasks. Many items were explored for a day or a week before being tossed or incorporated into roadmap. Feasbility has never been harder to pin down than when working with generative AI in an authoring environment like PowerPoint.
-
In parallel to the blurred line between discovery, exploration, and delivery, we changed how we would typically run a design critique to take advantage of the broader team’s explorations. We renamed it a Product Critique, as we opened the agenda for the everyone to bring their work in progress for discussion. This meant everything: developing slide templates and best practices, the latest prompt engineering tests, and sandboxed engineering work that explored new ways of rendering content in PowerPoint. These are some of the best collaborative sessions of my career—where everyone truly is a designer.
-
For this type of multi-modal interaction design and content generation, evaluating the quality of output is quite a unique challenge. One bottleneck is that only developers can define the prompts, test the output, and tweak the instructions in a tight loop. To break this down, I spearheaded the effort to create a non-technical tool for anyone to tweak and test new prompts for our features. That way, the strategy consultants on our team could make a ton of progress in improving our prompt design without taking away our engineering team’s development time.
-
A new way of thinking about testing is crucial to developing and scaling GenAI tools. In this case, balancing trust, capability, and usability was our focus. These three pillars meant that we needed more user involvement throughout the process—and more involvement of non-designers to capture the opportunities and evaluate what was successful. An engineer is not as good at judging slide output as a consultant, and a designer might not understand the possibilities as well as the data scientists when we run into a shortcoming.