Site   Web

October 14, 2008

Google Webmaster Tools Now Reveals Crawl Error Sources

Big news from Google’s Webmaster Central Blog today.

For years, webmasters have been asking Google for a way to identify the sources of 404 Not Found errors that are listed in the “Crawl Errors” feature of Webmaster Tools.  This week, they’ve added a “Linked From” column that lists the number of pages that link to a specific “Not found” URL.  Clicking on an item in the “Linked From” column opens a separate dialog box which lists each page that linked to this URL (both internal and external) along with the date it was discovered. You can even download all your crawl error sources to an Excel file. You can read the official post about the feature here.

Why is this helpful? Well, you can use the source information to fix the broken links in your site, place redirects to a more appropriate URL on your site and/or contact the webmasters linking to missing pages or using malformed links and ask them to fix the links. For those of you who aren’t familiar with 404 errors, Matt Cutts has a more detailed post on the new feature.

If your webserver doesn’t comprehend 404s or fetch error pages very well, Google has also introduced a widget for Apache or IIS that consists of 14 lines of JavaScript that you can paste into your custom 404 page template to helps your visitors find what they’re looking. It provides suggestions based on the incorrect URL.

The new Crawl Error Sources feature has got to be the easiest way to obtain inbound links to your site in a short space of time and best of all, the method is Google approved!

9 Responses to “Google Webmaster Tools Now Reveals Crawl Error Sources

    […] Go­o­gle W­eb­m­aster To­o­ls No­w­ Reveals Craw­l… « Easily Create Printable Pages with […]

    avatar Warren Duff says:

    Hi Kalena

    Good article – it is always great to get more information from Google.

    However, you can also use your LOG files combined with some common sense. By looking at all the 404 errors in your logs, you can determine if it is bad request due to an external link or an internal link – that is where the common sense comes into it.

    The discipline of looking at your logs is something that all webmasters should do.

    I do agree that this feature means those webmasters who don’t have access to their logs, can now do the same thing.

    Of course, neither method is going to pick up badly formed links at the domain level, because they are not going to be picked up as either a link to that domain, or as a request onto the server.



    avatar Robert says:

    Thanks for highlighting this Kalena. Matt’s post was quite informative and already I’m fixing a few bugs (or simply adding in 301’s).

    Warren, while the logs are a good way of measuring most things this is Google telling you exactly what they are seeing. I’m mostly pleased as I optimise for Google only at this time (hey in South Africa people don’t even know what MSN is!) and this tells me exactly what they see, what they get and so on. Great news for me.

    […] errors when spidering your site. Now with the sitemaps and webmaster tools utility they’ll show you where the link was woohoo…. I’m glad to see that I’ve been running into a few 404 errors from time […]

    @Warren – thanks for your comment. I agree that savvy webmasters may know how to find 404 sources via their log files, but this Google feature makes the process a breeze for newbies and webmasters of very large sites. The downloadable Excel file means there is really no excuse now for broken links.

    @Robert – glad you enjoyed the post, please keep reading SPN!

    avatar Diamonds says:

    Good to know that they finally put a column Linked From, I had difficulty finding where these links came from.

    avatar Sandie Metzger says:

    I downloaded this excel file, but as a newbie I don’t know where to past it in my website. Can someone help me?

    Its always great news when Google decides to share more of its information with us webmasters 🙂 301 is a great way to recover broken links quickly, and now its just that much easier to accomplish.

    avatar Downloads says:

    Yes, yes, yes! I have waited more than 1 year for this … Nothing more frustrating than seeing all those “URLs-not-found” msgs in WMT and *not* being able to identify and get the external links fixed!!


Submit a Comment

Your email address will not be published. Required fields are marked *

Please leave these two fields as-is:

Protected by Invisible Defender. Showed 403 to 6,663,108 bad guys.