Reddit is now blocking big search engines and their AI web crawlers from bringing up relevant posts – unless they pay up, and Google already has
A small price to pay for subreddits
When you purchase through links on our site, we may earn an affiliate commission.Here’s how it works.
If you don’t useGoogleas your search engine, then you won’t see Reddit posts in your search results – or at least not recent content from Reddit, anyway.
This is the state of play in the search world now, following a fresh move by Reddit, as explained by spokesperson Tim Rathschmidt in a statement provided toThe Verge.
Ratchschmidt said: “We have been in discussions with multiple search engines. We have been unable to reach agreements with all of them, since some are unable or unwilling to make enforceable promises regarding their use of Reddit content, including their use for AI.”
So, only Google will be able to surface recent Reddit posts, according to the report - though rival search engines likeBingwill still show results for older content on the site.
Like many people, I often use ‘site:reddit.com’ in searches when I want more personal, human answers, for example when I’m looking for help navigating a tough spot in a videogame or skincare recommendations – or I even just add the word ‘Reddit’ at the end of my search query to bring up the best results. Right now, however, if you try this on any other search engine than Google you’re probably going to wind up pretty disappointed and staring at outdated Reddit posts.
Earlier in the yearGoogle struck a deal with Reddit worth $60millionto allow its hungry AI bots to feast on Reddit’s sweet, sweet content. Said deal followed a Reddit blackout last year (a protest against API changes) that leftGoogle seriously in the lurch without access to a swathe of subreddits.
We should note that Ratchschmidt also clarified that this latest move with search results is “not at all related to our recent partnership with Google.”
Get the best Black Friday deals direct to your inbox, plus news, reviews, and more.
Sign up to be the first to know about unmissable Black Friday deals on top tech, plus get all your favorite TechRadar content.
You scratch my back, I’ll scratch yours…
So, it seems like Reddit is playing a strong hand here, in this game of data scraping – and blocking AIs from hoovering up its posts and presenting them as results for queries. As The Verge further observes, last month, Reddit updated its robots.txt file to stop web crawlers from doing exactly this – hammering home the fact that Reddit data is not for free.
A spokesperson forMicrosoft, Caitlin Roulston, previously acknowledged that: “Microsoft respects the robots.txt standard, and we honor the directions provided by websites that do not want content on their pages to be used with our generative AI models.”
Overall, these are gutsy moves on Reddit’s part, attempting to draw firm lines regarding AI being ‘entitled’ to data for free.
Right now, if you try to use the handy Reddit tag on search engines like Bing, you’ll still get Reddit posts related to your query – but they’ll be several years old. If you need more recent advice, you’ll have to be searching with Google, at least for the time being – until other search engines are ready to stump up some cash for the privilege.
You might also like…
Muskaan is TechRadar’s UK-based Computing writer. She has always been a passionate writer and has had her creative work published in several literary journals and magazines. Her debut into the writing world was a poem published in The Times of Zambia, on the subject of sunflowers and the insignificance of human existence in comparison.
Growing up in Zambia, Muskaan was fascinated with technology, especially computers, and she’s joined TechRadar to write about the latest GPUs, laptops and recently anything AI related. If you’ve got questions, moral concerns or just an interest in anything ChatGPT or general AI, you’re in the right place.
Muskaan also somehow managed to install a game on her work MacBook’s Touch Bar, without the IT department finding out (yet).
Anthropic’s new Claude 3.5 Haiku AI model is 4 times more expensive than its predecessor
Your doctor may have an AI assistant taking notes during your next Zoom call
A critical Palo Alto Networks bug is being hit by cyberattacks, so patch now
