Sebastian Kirsch: Blog

Tuesday, 25 January 2005

Blogs and search engines

Filed under: — Sebastian Kirsch @ 11:50

Judging from personal experience, responsible behaviour towards search engines is not every site owner’s first priority. But if a sizeable part of your visitors come from Google, you should pay attention to the way a search engine sees your site, and make sure that the visitors who come from a search engine get the optimal results from your site.

How your site is indexed can be controlled in a number of ways; they are described on the robots.txt web site.

For blogs, the most important thing is that the front page and the archive pages should not be indexed; only pages for the individual posts should be indexed. Why is that? Two reasons: What if Google indexes your front page, and serves it as a result two weeks later, when the indexed content has already dropped off the front page? Users will go to the front page, will not find what they are looking for, and go on to the next result. They will not search the archives of your blog in the hope of finding something relevant. And if Google indexes an archive page, it may present this page as a search result because of keywords that are in two different posts. Users will find their keywords, but the individual posts (each with one of the keywords) will not be relevant, thus they will move on to the next result. Sure, one more hit for your web site – but in the end, users will disregard your web site, because “there’s never something relevant on that site.”

So how do you accomplish this? Put the tag <meta name="robots" content="noindex, follow"> on your front page and archive pages, and <meta name="robots" content="index, follow"> on the pages with the individual posts. This will cause the search engines to not add the front page to their index, but still follow all links on the index page to the individual posts.

There is a thread about how to do this with WordPress in their support forum (WordPress is the blogging software I use.) I use the following code in index.php:

        <?php if ($single) { ?>
                <meta name="robots" content="index,follow" />
        <?php } else { ?>
                <meta name="robots" content="noindex,follow" />
        <?php } ?>

Another thing was detailed in a Slashdot article some weeks ago as a way to curb comment spam: Flag all links in comments with a tag “rel=nofollow", so that a search engine won’t follow them, and so that they won’t contribute to a site’s search engine rankings. That way, comment spam will be pointless. I hope there will be a WordPress plugin for this soon.

Bloggers are often accused of polluting the search engine results, and of misleading the search engine’s ranking algorithms because they link so heavily to each other. (Popular search engines like Google use the number of links that point to a web site to determine its popularity.) It’s our choice to do our share and play nicely with the search engines!

Comments

No comments yet.

RSS feed for comments on this post.

Leave a comment

Sorry, the comment form is closed at this time.


Copyright © 1999--2004 Sebastian Marius Kirsch webmaster@sebastian-kirsch.org , all rights reserved.