Google is secretly blacklisting certain sites to prevent them appearing in search results – despite publicly denying doing so – according to an investigation.
The Wall Street Journal report claims that the search giant has been blacklisting certain spam sites since the early 2000s, as well as those featuring child abuse or copyright infringement, to prevent them appearing in search results.
The newspaper also reported that conservative publications have also been blacklisted in Google News, and said that it had seen documents to support this claim, fuelling cries of political bias.
Right-wing websites The Gateway Pundit and The United West included on a list of hundreds of websites that wouldn’t appear in Google News or featured products – although they could appear in organic search results, it said.
The WSJ compared Duckduckgo (right) and Google (left) autocomplete results in Joe Biden and other terms. They found when researching for Joe Biden ‘creepy’ was suggested every time by the former but never by Google
A Google spokesperson said the company does ‘not manually determine the order of any search result.’
However, she said sites that don’t adhere to Google News ‘inclusion policies’ are ‘not eligible to appear on news surfaces or in information boxes in Search.’
Google has said repeatedly it doesn’t make decisions based on politics, and has previously said in congressional testimony that it doesn’t use blacklists. And former employees claim there is no political bias involved in decisions.
In a 2018 hearing whether the company had ever blacklisted a ‘company, group, individual or outlet…for political reasons,’ Karan Bhatia, Google’s vice president of public policy, responded: ‘No, ma’am, we don’t use blacklists/whitelists to influence our search results,’ according to the transcript.
Responding to the WSJ report, a Google spokesperson told Mail Online: ‘This article contains a number of old, incomplete anecdotes, many of which not only predated our current processes and policies but also give a very inaccurate impression of how we approach building and improving Search.’
Larry Page, pictured, is the co-founder of Google and CEO of parent company Alphabet. Google has said repeatedly it doesn’t make decisions based on politics, and has previously said in congressional testimony that it doesn’t use blacklists
According to a the draft policy document from August 2018 seen by the WSJ, the purpose of the blacklist is ‘to bar the sites from surfacing in any Search feature or news product sites’.
The policy instructs engineers, known as ‘maintainers’, to focus on sites that actively aim to mislead – such as ‘a publisher misrepresenting their ownership or web properties’ and having ‘deceptive content’ – rather than those that have inaccurate content.
Any changes or additions to the blacklist must be made by at least two people – one to make the change and another to approve it – according to the person familiar with the matter.
The WSJ report also claims that Google made algorithmic changes to its search results that favour big businesses over smaller ones, and in at least one case made changes on behalf of eBay, which is a major advertiser.
‘Listening to feedback from the public is a critical part of making Search better, and we continue to welcome the feedback’, a Google spokesperson told Mail Online
The company also boosts some major websites, such as Amazon and Facebook in search results, according to people familiar with the matter.
Google’s engineers have also created blacklists to weed out more-incendiary suggestions in auto-complete, the feature that predicts search terms as the user types a query – particularly those relating to controversial subjects such as abortion or immigration.
The WSJ compared Duckduckgo and Google autocomplete results in Joe Biden, and found that ‘creepy’ was suggested every time by the former but never by Google.
In response to the report, a Google spokesperson told Mail Online: ‘We have been very public and transparent around the topics covered in this article, such as our Search rater guidelines, our policies for special features in Search like Autocomplete and valid legal removals, our work to combat misinformation through Project Owl, and the fact that the changes we make to Search are aimed at benefiting users, not commercial relationships.
‘This article contains a number of old, incomplete anecdotes, many of which not only predated our current processes and policies but also give a very inaccurate impression of how we approach building and improving Search.
‘We take a responsible and principled approach to making changes, including a rigorous evaluation process before launching any change — something we started implementing more than a decade ago.
‘Listening to feedback from the public is a critical part of making Search better, and we continue to welcome the feedback.’ — Google spokesperson