Bing Not Indexing Site

The site has been well received by google, you have top placements from the big G and you must admit that google loves you. On the other hand, Bing! is not a fan, it doesn't see any value in your site and considers it spam.

Sounds familiar?

Knock Knock

Check that Bing can see your site. Although google is allowed through your website security, it may be a different story for Bing. The Microsoft search bot is quite abusive and it can get itself blocked at times. Use these Bing Webmaster tools to check you are not seeing as many bing errors as crawl requests. You will see some, if not quite a few bing errors. The important factor is more successful crawler hits than fails.

bing crawl errors vs requests
23K requests and 1.7K errors - Requests Wins.

Jump through hoops

None of it makes any sense, If the biggest search engine on the planet loves your content then why is Bing so difficult to please?

Have you reached out to the bing support? You should do this to ensure they can't give you a reason. They will send you links to follow about providing great content, making your site easy to navigate and reduce the number of ads above the fold. Well, google is happy with your site so it won't be due to the basics at fault.

Hopefully your interaction with bing! support will be fruitful and give you something to try. Exhaust their suggestions and reply politely to their emails. I got this far and things didn't improve.

Blacklisted

Bing had blacklisted the site and this had a knock on effect with the other search engines that piggy-back from Microsoft's Bing. Not a huge factor in itself but it is annoying that bing is just wrong which can be proved if google is correct.

Go all in

With confirmation that the crawler can access your site successfully and microsoft staff fully exhausted their advice and every suggestion enabled the next step was to go all in. The only way I could convince bing that the site was worthy of inclusion was to make sure the crawler could see every single page I had written.

Invoke Max crawl rate.

bing crawler on max crawl rate

Index Now!

If your site is using cloudflare there is the option to enable 'IndexNow! - Crawler Hints' in the bing webmaster tool. It generates a fair bit of dross, but it's free and helps send good signals to bing from cloudflare about your site.

crawler hints index now

Time

After months of waiting. The bing started to show results. During this time the crawler was all over the content with its rate set to the max. Cloudflare crawler hints was working away doing good things. The graph show when the search engine decided it would be a good idea to index the content that google has acknowledged for years.

bing indexing zero day

Up until the red arrow, bing indexed pages (green line) are zero. After the red arrow pages are steadily being added to the search engine.

A drop-off happens, but it recovers.

Something changed to enable bing to add the site. To the left of the red arrow show the blue line rather flat and steady until the max crawl rate was set. More crawls can be seen by the peaks of blue. These blue peaks are bing learning about the site, seeing how great it is and considering the addition to the index. When the decision is made to add the site you can see the green line pick up and take off.

Mission Accomplished

Bing search is now showing results that come from the site. Just as it should have always been, but for whatever reason the engine was stuck in a (sad face) mode. Giving the crawler a kick and allowing it plenty of time seemed to do the trick. While this is only a guess as to why things changed, they definitely did change and bing is now happily serving results from the site it was originally ignoring.


Posted

in

by

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *