Microsoft executive described AI scraping as ‘the largest theft of labour in human history,’ unredacted filings reveal

Newly unredacted information from a copyright lawsuit filed by The New York Times against OpenAI and Microsoft has brought to light internal admissions from the tech giants. A senior Microsoft executive reportedly characterised AI scraping as "the largest theft of labor in human history," while OpenAI's leadership acknowledged that their AI models posed an "existential threat" to publications and journalists whose content was used for training.

The unsealed documents detail allegations that the companies acquired and utilised content by bypassing paywalls without detection, built extensive training datasets through mass scraping, and intentionally removed copyright notices from the data used for training.

Internal Microsoft data indicated a substantial decline, with its Copilot "answer engine" causing click-through rates for The New York Times' domain to drop by as much as 93% when compared to traditional Bing search results. The filings also revealed that OpenAI's mid-training datasets alone contained over 91,692 copies of works published by The New York Times, Daily News, and Center for Investigative Reporting.

Furthermore, a dataset derived from Common Crawl included more than 2 million documents specifically from nytimes.com. Documents also showed that Project Mango data, shared between Microsoft and OpenAI, allegedly contained copies of at least 160,903 unique works from news publishers.

The ongoing copyright lawsuit pits The New York Times against OpenAI and Microsoft. Key figures whose statements and internal communications are cited in the unredacted filings include Microsoft CEO Satya Nadella, who testified on the necessity of licensing paywalled content for AI training. Brent Hecht, Microsoft's director of Applied Science, authored an internal presentation detailing the decline in click-through rates and described the scraping as "astonishing theft."

OpenAI's head of ChatGPT, Nick Turley, and President Greg Brockman also made internal remarks concerning the substitutive nature and quality of their AI models. Steven Lieberman, counsel for the New York Daily News, issued a statement regarding the evidence revealed.

These newly disclosed internal documents present a significant challenge to the "fair use" defence often invoked by AI companies in intellectual property disputes. The admissions, particularly concerning the potential for AI products to substitute original works and the alleged deliberate circumvention of paywalls, could have a profound impact on the legal framework governing generative AI.

The case underscores the critical debate surrounding the legality of using copyrighted material to train AI models and the potential economic ramifications for content creators and publishers in the digital landscape.