An error like “404 File not found or 503 Services Unavailable, or 403 forbidden” and “Connection to the server timed out” is a common experience for internet users. Since Googlebots visit your websites more often than real users do, it is likely that they too encounter the same problem. Whenever such error is encountered by Googlebot, Google sends you messages in the Webmaster Tools Message Center. This helps webmaster indentify and fix the problem.
To further assist webmasters Google has announced a new feature Webmaster “Tools Crawl Errors feature” that will further simplify the task of webmasters. The Webmaster Tools Crawl Errors feature categorizes errors as URL errors and Site errors.
When ‘Site error’ alerts are received it indicates that there is a systemic issue and there may not be any issue with specific web pages. So, what causes this? Here are a few pointers
- The web server is firewalled off
- Configuration issues with DNS server or DNS server is down
- Over loaded web server or web server is down
- Websites robots.txt is inaccessible
- Web server refuses connection to Googlebot.
These are some of the common errors that Google encounters and they happen all over the globe. However, they should not happen to well operating sites. Google on its part will send you the alert as a message in the Message Center. In case your site shows an error rate of 100% in one or more of the above mentioned categories then it means that your site is not configured properly or it is down. However, in case your error alert tells you that the error alert is less than 100% in one of these categories then it would well mean a temporary issue. In both cases webmasters need to investigate.
As for URL errors, if a large site has a significant number of error alerts then it could be termed normal. However, for a small site this could be a serious problem. Google sends alerts only when the error alerts is abnormally high. The alert when sent will clearly describe the category of the web error and will have a link to the section of Webmaster Tools so that webmasters can identify which pages are causing the problem and take corrective measures or troubleshoot the issue.
These alerts are important for the smooth functioning of your website and therefore webmasters should check the Webmaster Tools section regularly. In case of time constraints webmasters can turn on the ‘Message forwarding’ option and Google will send the error alert to the email address of their choice.