In the ever-evolving landscape of technology, the blending of human and digital brain power has given birth to a new phenomenon known as “artificial artificial artificial intelligence.” Researchers at the Swiss-based university EPFL have shed light on this emerging trend, expressing serious concerns about its implications.
The concept of mechanical turks, first popularized by Jeff Bezos at the turn of the century, involved employing low-paid remote workers to tackle small portions of complex computer projects. This human perspective on perplexing tasks became known as “artificial artificial intelligence,” combining the capabilities of humans and computers.
Currently, approximately a quarter of a million people are employed through Amazon’s Mechanical Turk marketplace, which is just one of many sources providing such services. However, the EPFL researchers have discovered a disconcerting shift in the behavior of these turks. They are now relying on AI-generated content to complete their tasks, leading to the term “artificial artificial artificial intelligence.”
While the term may elicit a few smiles, the implications are far from amusing. Veniamin Veselovsky, one of the researchers involved, emphasizes the potential damage to the utility of crowdsourced data. According to Veselovsky, if workers increasingly depend on AI generators, it could severely diminish the value of human input and undermine the reliability of AI-based operations.
Large language models, despite their impressive processing capabilities, still benefit from human input for certain tasks. Humans outperform computers in labeling data, describing images, and even responding to CAPTCHA screens. While it may be tempting to rely on crowdsourcing for validation or comparison purposes, the EPFL researchers caution against the contaminated data pool resulting from workers employing large language models to boost their productivity and income on crowdsourcing platforms.
The term “turk” originates from an 18th-century chess-playing “robot” that astonished players across Europe, including prominent figures such as Napoleon and Benjamin Franklin. Unbeknownst to the players, a hidden human chess expert controlled the machine’s movements.
Crowdsourcing with modern-day turks has flourished into a billion-dollar industry. However, this industry faces a new threat in the form of large language models. A recent study revealed that a ChatGPT 3.5 turbo model outperformed crowd workers at a fraction of the cost for classification assignments. As a result, workers are likely to face increased pressure to produce more output at a faster rate, potentially leading to greater reliance on AI resources.
Based on a limited study of workers at MTurk, Amazon’s crowdsourcing operation, the EPFL researchers estimated that 33% to 46% of worker assignments were completed with the assistance of large language models. This statistic serves as a warning sign, highlighting the increasing popularity of large language models and the need for platforms, researchers, and workers themselves to find innovative solutions to preserve the essence of human data in an age dominated by artificial intelligence.
Veniamin Veselovsky urges stakeholders to treat these findings as a “canary in the coal mine,” signaling the importance of preserving the human element in data-driven processes. As large language models continue to gain prominence, and multimodal models supporting text, image, and video inputs rise in popularity, it is crucial to ensure that the integrity of human data remains intact.
As society navigates the uncharted territories of AI and human collaboration, it is essential to strike a balance that harnesses the strengths of both worlds while safeguarding against the potential pitfalls that lie ahead.
