Over the past several months, we have repeatedly reported on Google’s decision to voluntarily remove pirate sites from its search results.
Over the past several months, we have repeatedly reported on Google’s decision to Google will do so if local court orders require ISPs to block access to the sites. These measures are geo-targeted at the applicable countries, which include the UK, France, and the Netherlands.
Thus far Google hasn’t publicly commented on its motivation but several rightsholders have confirmed the actions. Interestingly, however, Google is not the only search engine where pirate sites have gone ‘missing’. In fact, it’s quite a persistent problem on other search engines as well.
Bing’s Missing Pirate Sites
This week we conducted various tests on Bing’s search engine across multiple countries. This revealed that several of the top pirate sites only have their home pages indexed, while others are completely unfindable.
For example, when we use the site: command to find all Pirate Bay pages indexed by Bing there’s just one visible result. A message at the bottom clarifies that “some results have been removed” but that’s an understatement, as there are supposed to be millions.
For comparison, the same search on Google claims to return ~24,000 URLs from The Pirate Bay, provided that the site hasn’t been stripped completely due to the aforementioned voluntary removal process.
The problem is also apparent when searching for “Pirate Bay Linux”. On Bing this doesn’t return any useful results from The Pirate Bay, while Google lists a collection of relevant pages from the notorious torrent site.
This isn’t to argue that Google is better than Bing, we merely want to show the differences. Depending on the search term and your location, Microsoft’s search engine may provide better results for other search terms.
Bing’s meager search results are not limited to The Pirate Bay. The same applies to various other torrent sites such as Fitgirl-Repacks.site, and TorrentGalaxy.to. In addition, other pirate portals including 9anime.to yield the same result.
Entire Domains Disappear
The domains mentioned above still have their home pages listed, but the same can’t be said for many others.
When we search for 1337x.to there are no results (for that domain) at all, even though Bing does pull up the correct Wikipedia page in an information panel. Similarly, results are missing for Fmovies.to, Rarbg.to, and other popular sites as well.
For the record, these results are the same in the various geographical locations we tested.
And there’s more to this mystery, as other prominent pirate sites remain indexed as usual. This includes the most popular torrent site YTS.mx which, aside from a few DMCA takedowns, has numerous pages in Bing’s search results.
‘No Editorial Decision’
So what is going on here? Is this an anti-piracy measure? Or are these domains being punished for another reason?
When we reached out to Microsoft, the company couldn’t immediately give us an explanation. However, the team stressed that the removals are not the result of an editorial decision.
“We can share that we did not make an editorial decision to remove these domains. We have passed this along to our technical team,” a spokesperson informed us.
Whatever the reason, the removals are not without consequence. Several other search engines that rely on Bing data are missing the same results. This includes the privacy-centered search engine DuckDuckGo.
Colateral Damage
Last year we reported on an issue with disappearing pirate sites at DuckDuckgo. This was also related to Bing. While some links were put back manually by DuckDuckGo, these issues persist.
When we search for “rarbg” on DuckDuckGo, Rarbg.to is correctly identified as the official site and the top result.
This result is only triggered by certain keywords. When we search for “rarbg official site,” RarBG is nowhere to be found. Instead, we get the same results as we do in Bing.
That brings us to another side-effect of the disappearing sites. With the disappearance of the official domains, copycats are now making their way to the top results. These are generally more likely to show malicious advertisements or cause other trouble.
This copycat problem equally applies to Google, where demoted or removed sites are regularly replaced by dubious sites.
