Teams vs. Crowds: A Field Test of the Relative Contribution of Incentives, Member Ability, and Collaboration to Crowd-Based Problem Solving Performance

Christoph Riedl, Anita Williams Woolley
Academy of Management Discoveries
Vol. 3, No. 4
December 13, 2016


Organizations are  increasingly turning to crowdsourcing to solve difficult problems. This is  often driven by the desire to find the best subject matter experts, strongly  incentivize them, and engage them with as little coordination cost as  possible. A growing number of authors, however, are calling for increased  collaboration in crowdsourcing settings, hoping to draw upon the advantages  of teamwork observed in traditional settings. The question is how to  effectively incorporate team-based collaboration in a setting that has  traditionally been individual-based. We report on a large-field experiment of  team collaboration on an online platform, in which incentives and team  membership were randomly assigned, to evaluate the influence of exogenous  inputs (member skills and incentives) and emergent collaboration processes on  performance of crowd-based teams. Building on advances in machine learning  and complex systems theory, we leverage new measurement techniques to examine  the content and timing of team collaboration. We find that temporal  â€œburstiness” of team activity and the diversity of information exchanged  among team members are strong predictors of performance, even when inputs  such as incentives and member skills are controlled. We discuss implications  for research on crowdsourcing and team collaboration. This well-written paper  focuses on the phenomenon of crowdsourcing and asks the question: How might  groups of individuals collaborate most effectively in a crowdsourcing setting  to produce high quality solutions to problems? The paper describes a rigorous  field study with random assignment of individuals to groups that seeks to  examine the conditions that could facilitate a team’s performance on a  problem-solving task in a crowd-based setting. The “discovery” is that  the temporal “burstiness” of the team members’ contributions, which  suggests some effort to coordinate attention to the problem, plays a highly  significant role in influencing the quality of solutions that teams produce.  As one of the reviewers noted “this paper is a perfect ‘fit’ for the  Academy of Management Discoveries.” I wholeheartedly agree—it focuses on  an important yet poorly understood phenomenon and reports on the results of a  rigorous field study that provides potentially important insights into  developing our understanding of that phenomenon. I highly recommend that all  Academy members read this paper.