One million public Bluesky posts scraped for AI training

Bluesky is already facing its first major AI scrape, despite the stance of its owners that it will never train generative AI on user data. Reported by 404Media on Nov. 26, one million public Bluesky posts — complete with identifying user information — were crawled and then uploaded to AI company Hugging Face. The dataset was created by machine learning librarian Daniel van Strien, intended to be used in the development of language models and natural language processing, as well as general analysis of social media trends, content moderation, and posting patterns. It contains users’ decentralized identifiers (DIDs) and even has a search function to find content from specific users.According to the dataset’s description, the set “contains 1 million public posts collected from Bluesky Social’s firehose API (Application Programming Interface), intended for machine learning research and experimentation with social media data. Each post contains text content, metadata, and information about media attachments and reply relationships.” SEE ALSO: Leaving X for bluer pastures? What to know about Bluesky’s owners and policies. Bluesky users didn’t opt-in to such uses of their content, but neither is it expressly prohibited by Bluesky. The platform’s firehose API is an “aggregated, chronological stream of all the public data updates as they happen in the network, including posts, likes, follows, handle changes, and more.” Bluesky’s API — coupled with the public and decentralized Authenticated Transfer (AT) Protocol the site is built on — means Bluesky content is open and available to the third party developers the platform…One million public Bluesky posts scraped for AI training