Collective Intelligence and Collaboration in a Networked Society
Final Report Abstract
Both, public and private organizations are increasingly trying to involve large groups of users/citizens into collective decision-making and problem-solving activities. This approach is used to address issues of fundamental importance such as developing innovative products and services, or finding effective solutions to counter global warming. This research area is termed “collective intelligence” (CI). Collective creation and decision making promises huge benefits, including higher acceptance rates for innovations/new policies and better solutions and decisions through the integration of diverse contributors. The aim of this research project was to gain a better understanding of how collective intelligence initiatives can be designed to support a move from a “collection only”-based approach toward more collaboration and to gain a better understanding of collective decisionmaking. This research project analyzed this research question in the context of TopCoder, a leading online platform for collective intelligence with a large user base of currently 400,000 registered participants. We designed, conducted, and analyzed two large-scale field experiments on this prominent collective intelligence community in which we had the exceptional opportunity to exert relevant influence on the design of the field experimentation as well as on the design of online platforms used during the crowdsourcing competition. In particular, it was possible to design the field experiments such that teams of competitors (teams of five in the first experiment, teams of two in the second) competed against each other rather than individuals. In experiment one, 52 teams of five randomly assigned participants (260 individual competitors in total) competed against each other, trying to solve a difficult optimization problem over 10 days. Random team member assignment offers an exceptional opportunity to study causal effects of individual characteristics and team composition on team outcomes such as the exerted effort level and team performance. We find substantial evidence for the existence of peer effects, whereby individuals exert more effort when other team members work harder. We find little evidence that diversity in terms of skills or demographics negatively affects the emergence of collaboration. In experiment two, 232 teams of two (463 participants), were matched with each other through two randomly assigned treatment mechanisms. In treatment one, pairs freely formed by themselves through individual-level communication with each other. In treatment two, pairs were automatically assigned using information about their preferred teammate. We find that the decentralized mechanism can dominate but is subject to frictions and not scalable and that an optimized clearing-house could do better in some instances. TopCoder actively relies on our findings in their design of upcoming challenges to be run on their platform.
