AI in the Driver’s Seat? Research Examines Human-AI Decision-Making Dynamics

Artificial Intelligence (AI) is no longer a futuristic idea—it’s woven into our workplaces, classrooms, and daily lives. Rather than viewing AI as a competitor, researchers at the University of Washington are exploring ways to make AI a collaborative partner, a friend rather than a foe. The big question is: can humans and AI work together to solve complex problems more efficiently and effectively?

Understanding AI’s role in decision-making is more critical than ever for businesses, organizations, and individuals. AI can process vast amounts of data, detect patterns, and provide insights at speeds beyond human capability. Yet, as with any tool, its effectiveness depends on how we use it. Rather than deferring entirely to AI, engaging with AI effectively is the key challenge.

Foster School of Business Assistant Professor Léonard Boussioux and his research partners from Harvard Business School and MIT Solve examine this dynamic in a recent working paper. Their field experiment involved experts and non-expert evaluators who assessed real-world solutions submitted to MIT Solve’s 2024 Global Health Equity Challenge.

The evaluators assessed proposals with and without AI assistance, allowing researchers to analyze how different human evaluators collaborated with AI. Through this experimental design, Boussioux and colleagues uncovered key lessons about the strengths and limitations of AI-assisted decision-making.

Their findings point to a crucial takeaway: the most effective use of AI isn’t in replacing human judgment but in enhancing it. Based on the research findings, companies and people who develop AI interaction expertise will be better positioned to navigate an increasingly AI-integrated world. By learning how to collaborate with AI effectively, individuals and organizations can unlock new levels of efficiency, accuracy, and innovation in their decision-making processes.

AI cannot replace human judgment

To better understand the study’s implications, we spoke with Boussioux about the challenges of AI-assisted decision-making and what businesses can learn from his research.

How helpful was AI in evaluating ideas?

Léonard Boussioux: Our study showed that AI can significantly help non-experts reach conclusions similar to those of experts. With AI assistance, individuals who knew nothing about the subject could evaluate proposals as effectively as experts.

However, we did notice a difference in how experts interacted with AI. Experts tended to scrutinize the AI-generated suggestions more critically, seemingly looking for proof to validate or challenge AI’s suggestions. Most of the time, this deep analysis led them to agree with AI. It also meant that AI didn’t speed up the evaluation process, which was the point of incorporating AI into the review process.

So, the question is: Do we want AI to automate decision-making fully, or should it serve as a tool that prompts deeper human reflection?

What were some key differences between human evaluations and AI observations?

Léonard Boussioux: AI is great at objective assessments. It can filter and sort information easily based on metrics, which helps streamline specific tasks. However, when subjective judgment is required, human intuition is still superior.

Our research supports the idea that AI should act as an augmentation tool, providing reasoning humans can incorporate into their decision-making rather than replacing it. By using AI for objective assessments, we free up human capacity for more nuanced decisions.

What are the key takeaways from your research?

Léonard Boussioux: AI is rapidly becoming a core part of our professional and personal lives. Rather than resisting this shift, we must rethink our decision-making processes and learn how to integrate AI effectively. AI interaction expertise—engaging critically with AI, asking the right follow-up questions, and verifying AI-generated insights—is a crucial skill in today’s workforce. It is also a skill that we can learn.

Our research revealed a fascinating “human oversight paradox” at work when people collaborate with AI. We discovered that AI-generated explanations significantly increased people’s tendency to follow algorithmic recommendations. This effect was especially strong when AI recommended rejecting solutions.

Here’s the interesting twist: knowing how to navigate AI effectively helps people make informed decisions even without subject-matter expertise. But the best outcomes still come from a collaborative approach where experts critically evaluate AI suggestions rather than simply accepting them at face value.

While AI assistance (with or without explanations) led to better screening outcomes overall, companies shouldn’t encourage automatic deference to AI recommendations. Instead, organizations should invest in helping employees develop AI interaction expertise—the ability to thoughtfully engage with AI as a partner rather than treating it as an infallible authority. This balance ensures we leverage AI’s incredible information processing capabilities while preserving the irreplaceable value of human judgment.

What are some benefits of using AI in decision-making?

Léonard Boussioux: AI can assist with both small and large-scale problems. For example, tasks like creating a PowerPoint presentation can be enhanced by AI tools such as Microsoft Copilot, Canva, or Gamma.app, making the process easier.

On a larger scale, AI can contribute to solving complex global challenges. In our study, AI helped screen proposals for the Global Health Equity Challenge. Evaluating proposals requires a lot of time and bandwidth. Incorporating AI into the review process filters out weaker ideas, allowing human evaluators to focus on the most promising solutions. 

AI also provides consistency that humans struggle to maintain—giving equal attention to each evaluation. Humans might experience cognitive burden or fatigue after reviewing multiple submissions, patients, or samples. We found AI can sometimes catch important details that humans overlook, especially in lengthy or complex proposals.

With AI removing the lower quality proposals, humans can focus their energy on the exciting part: solving the big problems. This approach allows people to focus their mental energy on tasks where their expertise is most valuable. 

What are some challenges in incorporating AI into decision-making?

Léonard Boussioux: One of the biggest challenges is the rapid evolution of AI tools. Many people struggle to keep up with AI’s shifting capabilities, leading to hesitation in integrating AI into their workflows. Additionally, there’s a risk of over-reliance. When AI becomes too convenient, some users may defer to its suggestions without proper examination, leading to poor decisions.

Another major challenge is finding the right balance between AI and human intelligence. Our research suggests that human-AI collaboration is not always more effective than either one working alone. The challenge lies in combining human expertise with AI to enhance overall performance.

The trickiest part is unlocking and harnessing the full potential of human-AI collaboration. The ideal process is one in which humans make the final decision, but AI can guide them through the thinking process. 

The Future of AI & Human Collaboration

Boussioux’s study was conducted in collaboration with researchers from Harvard Business School, the Digital, Data, and Design Institute at Harvard, and MIT Solve. In addition, three Foster School of Business students, Camila (Chia-Yu) Lin, Pei-Hsin (Amy) Wang, and Ian (Ying Hao) Chen, played a critical role in this research by building the backbone platform for the field experiment. As students in Foster’s Master of Science in Information Systems program, their previous experience at the 2024 AI EarthHack positioned them for this research opportunity. The student’s contributions underscore a broader trend highlighted by the study: the growing need for AI interaction expertise across all types of businesses. 

Congratulations to Professor Boussioux and his research partners on winning the First Prize at the recent Wharton People Analytics White Paper Competition.

Read the research paper here: 

“The Narrative AI Advantage? A Field Experiment on Generative AI-Augmented Evaluations of Early Stage Innovations”—Jacqueline N. Lane, Leonard Boussioux, Charles Ayoubi, Ying Hao Chen, Camila Lin, Rebecca Spens, Pooja Wagh, Pei-Hsin Wang. Available in the Harvard Business School Technology & Operations Management Unit Working Paper Series (2024).