When Bing was launched, it was considered a threat to Google. Though Bing is yet to compete with Google in terms of user loyalty and trust; it seems it has a long way to go before it even challenges Google.

The mere fact that it disobeys the webmaster’s instructions is enough for webmasters to dump Bing. And once webmasters overlook a search engine, the users too, don’t stick with that search engine. Currently, Bing is disappointing webmasters because of its loophole in treating the content within noindex and nofollow tags.

Ideally, any content that is marked noindex or nofollow, either through robots.txt file or via meta robots tag should not be indexed by the search engines. Major search engines are following this practice and allow the webmasters to hide certain pages from the public. But Bing is out rightly disobeying this practice.

Recently, a conversation on Bing Forum highlighted this issue. Here’s how the conversation went:

Webmaster:

I have a site containing pages that I don’t want their content being indexed by search engines so all of those pages have
<META NAME=”ROBOTS” CONTENT=”NOINDEX, NOFOLLOW” />
in their header and it has been like this for a while now.

Both Google and Yahoo respected the tag and those pages are not being indexed by them but today I checked and found all those pages even recent ones are indexed and cached on Bing.

Program Manager, Bing Webmaster Center:

This is a known issue we are working quickly to resolve.  If you have pages you would like permanently removed from our index, please send me a mail to bwmc@microsoft.com with your domain name and “MSNBot ignoring robots tags” in the subject line. Please also include the URLs in the body of the message. You may use an * wildcard for any directories such as:

http:example.com/ wrongdirectory/*

Normally, I would request that you fill out a content removal request, however, since this is a problem on our side, I’ll do the leg work for you.

So why does Bing disobey noindex / nofollow?

As evident from what the Bing representative said, “it’s a known issue in Bing.” Microsoft does say that it’s working to fix this problem but when? Having signed a search deal with Yahoo, is Microsoft waiting for Yahoo to fix their search engine bugs? Common MS, that way you will never be able to challenge Google’s search monopoly.

Has anybody experienced similar problems, with Bing indexing your noindex / nofollow content?