Google-Safety BOT

Why does the Google-safety bot appear to visit your site?

Is the Google-Safety bot considered to be a good or bad bot and what does it mean when the safety bot visits?

User Agents

User Agents (UA) can be easily spoofed, so just because the logs show a visit from Google-safety doesn't mean that it is guaranteed to be from Google. You must check the IP that the bot visits from is a google bot IP address. Simple enough to check, but this can be easily overlooked.

Google-Safety

The Google-Safety user agent handles abuse-specific crawling, such as malware discovery for publicly posted links on Google properties. This user agent ignores robots.txt rules.

Two key things mentioned by Google are:

• The safety bot ignores robots.txt
• Malware discovery

ignoring the robots.txt is a big thing, you could expect to see the safety bot in places you have excluded the other Google bots from visiting. Malware can't hide behind robots.txt, which is a good thing.

Malware discovery sounds like a good thing to do, that way the SERPS (Search Engine Results Page) can warn a user about malware on a site before they visit (or choose not to visit) the link.

Another Reason

There are other, undocumented reasons that Google-safety bot can visit your site and one of those we have seen was when a link is added to a YouTube video description. The bot will check the link in the video description, presumably for the malware part of the bot's job.

A way that Google can clamp down on malware links spreading through YouTube while ignoring robots.txt directives.

So don't be alarmed by the Google-Safety bot visiting. It doesn't necessarily mean that the big G considers your site to be unsafe or to be a source of malware. It may be good that YouTube uploaders recognise your site and Google comes to visit (even if it is a malware-sniffing, robots-ducking safety bot).