Turkopticon: interrupting worker invisibility in amazon mechanical turk

  title={Turkopticon: interrupting worker invisibility in amazon mechanical turk},
  author={Lilly C. Irani and M. Six Silberman},
  journal={Proceedings of the SIGCHI Conference on Human Factors in Computing Systems},
As HCI researchers have explored the possibilities of human computation, they have paid less attention to ethics and values of crowdwork. This paper offers an analysis of Amazon Mechanical Turk, a popular human computation system, as a site of technically mediated worker-employer relations. We argue that human computation currently relies on worker invisibility. We then present Turkopticon, an activist system that allows workers to publicize and evaluate their relationships with employers. As a… 
Being a turker
An ethnomethodological analysis of publicly available content on Turker Nation, a general forum for Amazon Mechanical Turk (AMT) users, provides novel depth and detail on how theTurker Nation members operate as economic actors, working out which Requesters and jobs are worthwhile to them.
TurkBench: Rendering the Market for Turkers
This paper presents emph{TurkBench}, a tool meant to provide workers with personalized market visualization and session management, and discusses the design philosophy of the tool, briefly discuss four Turkers' reaction to a demo, and outline future work.
Turk-Life in India
Previous studies on Amazon Mechanical Turk (AMT), the most well-known marketplace for microtasks, show that the largest population of workers on AMT is U.S. based, while the second largest is based
Human-centered computing and the future of work: lessons from Mechanical Turk and Turkopticon, 2008-2015
Author(s): Silberman, M. Six | Advisor(s): Tomlinson, Bill; Nardi, Bonnie A | Abstract: Online labor markets such as Amazon Mechanical Turk (AMT), Uber, and TaskRabbit are contributing to rapid
Human computing via online labor markets. The perils and promises of crowdsourcing in data-rich ecosystems
The need for "human computing" in data-rich ecosystems, notably as a consequence of data variety and typically for name disambiguation, is focused on, and ways to manage it via online platforms for paid crowdsourcing are explored.
Turkish Judge: A Peer Evaluation Framework for Mechanical Turk Appeals
An automated appeals system called Turkish Judge is presented which employs crowd workers as judges to adjudicate whether work was fairly rejected when their peers initiate an appeal on Amazon Mechanical Turk.
Mechanical Turk is Not Anonymous
While Amazon’s Mechanical Turk (AMT) online workforce has been characterized by many people as being anonymous, we expose an aspect of AMT’s system design that can be exploited to reveal a surprising
REGROW: Reimagining Global Crowdsourcing for Better Human-AI Collaboration
Crowdworkers silently enable much of today’s AI-based products, with several online platforms offering a myriad of data labelling and content moderation tasks through convenient labour marketplaces.
Turker Tales: Integrating Tangential Play into Crowd Work
Turker Tales, a Google Chrome extension that uses tangential play to encourage crowd workers to write, share, and view short tales as a side activity to their main job on Amazon Mechanical Turk, is presented.
Who's the boss?: requester transparency and motivation in a microtask marketplace
It was found that providing information about the work requester led to increased effort, with further differences in effects depending on the location of the task participants.


Who are the crowdworkers?: shifting demographics in mechanical turk
How the worker population has changed over time is described, shifting from a primarily moderate-income, U.S. based workforce towards an increasingly international group with a significant population of young, well-educated Indian workers.
Demographics of Mechanical Turk
We present the results of a survey that collected information about the demographics of participants on Amazon Mechanical Turk, together with information about their level of activity and motivation
Working the Crowd: Employment and Labor Law in the Crowdsourcing Industry
This Article confronts the thorny questions that arise in attempting to apply traditional employment and labor law to “crowdsourcing,” an emerging online labor model unlike any that has existed to
Profanity use in online communities
Analysis of comments from a social news site is provided to provide evidence that current profanity detection systems are performing poorly and evaluate the cases on which they fail, and to address community differences regarding creation/tolerance of profanity.
Feminist HCI: taking stock and outlining an agenda for design
The state of the art of feminism in HCI is summarized and ways to build on existing successes to more robustly integrate feminism into interaction design research and practice are proposed.
Layers of Silence, Arenas of Voice: The Ecology of Visible and Invisible Work
This article provides a framework for analyzing invisible work in CSCW systems and sample across a variety of kinds of work to enrich the understanding of how invisibility and visibility operate.
Slash(dot) and burn: distributed moderation in a large online conversation space
Analysis of the site Slashdot.org suggests that the answer is a qualified yes, but that important challenges remain for designers of such systems.
FEATURELearning from activists: lessons for designers
The technical community in general, and interaction designers and computer-human interaction specialists in particular, can learn from contestational designers that offer new models of engaging with technology, people, and social issues that challenge implicit assumptions about how design operates.
Cheap and Fast – But is it Good? Evaluating Non-Expert Annotations for Natural Language Tasks
This work explores the use of Amazon's Mechanical Turk system, a significantly cheaper and faster method for collecting annotations from a broad base of paid non-expert contributors over the Web, and proposes a technique for bias correction that significantly improves annotation quality on two tasks.
The Intellectual Challenge of CSCW: The Gap Between Social Requirements and Technical Feasibility
It is argued that there is an inherent gap between the social requirements of CSCW and its technical mechanisms and that the challenge of the social-technical gap creates an opportunity to refocus CSCw.