Setting up robots.txt on Blogger

Blogger now allows custom robots.txt, this is very useful because we can set the visibility of our articles on search engines, we can determine whether the article will be indexed by search engines or not.

By default, every blog that uses the Blogger platform will have a robots.txt as follows:

User-agent: Mediapartners-Google
Disallow: 

User-agent: *
Disallow: /search
Allow: /

Sitemap: http://www.example.com/feeds/posts/default?orderby=updated

And has the following explanations:

Mediapartners-Google is a robot from Google Adsense, leave it as is because if you mistakenly change that than the ads served will not fit with your content.

The next line is for all the robots and marked with an asterisk (*). On the default configuration, it is clear that the label of our blog is not indexed Disallow: /search.

Keep in mind that a slash (/) is as your homepage, so for example if you want the label to get indexed, do not just fill up with a slash like this Disallow: / because that would be you do not allow the robot tracing your blog, but it should like the example below:

User-agent: Mediapartners-Google
Disallow: 

User-agent: *
Disallow:
Allow: /

Sitemap: http://www.example.com/feeds/posts/default?orderby=updated

With the configuration as above then all of the articles and the label will be indexed. And to block a robot for particular page (I take the example of my FAQ page) you can simply write as follows:

User-agent: Mediapartners-Google
Disallow: 

User-agent: *
Disallow: /p/faq.html
Allow: /

Sitemap: http://www.example.com/feeds/posts/default?orderby=updated

Update: To resolve the pagination problems on blogspot after we remove the Disallow: /search than we can use the following configuration to block the pagination page:

User-agent: Mediapartners-Google
Disallow: 

User-agent: *
Disallow: /search?updated-min=
Disallow: /search?updated-max=
Disallow: /search/label/*?updated-min=
Disallow: /search/label/*?updated-max=
Allow: /

Sitemap: http://www.example.com/feeds/posts/default?orderby=updated

After the changes, make sure everything is fit like what we want by visiting www.example.com/robots.txt. Replace the Example.com with your domain name.

Warning! Use with caution. Incorrect use of these features can result in your blog being ignored by search engines.

Note: Last updated on January 5, 2013.

Blogger new features: Search preferences

Starting today Blogger added some new features, the "Search preferences" that aims to maximize our blogs in search engines. If you care about the ranking of your blog, then this new feature is very exciting news.

Here are the new features are:

Maybe you realize that there is new option in the settings tab, the "Search preferences".

By clicking on it will bring up the three categories, the Meta tags, Errors and redirections, Crawlers and indexing.

For Meta tags category is serve to add a description tags on our blog, by enabling the Meta tags option then everytime we make new article will appear a new option called "Search Description" that allows us to add meta description to any post.

The next category is Errors and redirections, with this feature you can set the message that appears when visitors visit a page that is no longer exists on our blog, the "Page Not Found 404" error, and more helpful is that now we can redirect visitors to a particular page as we wish.

The last feature is Crawlers and indexing, here we can create our own robots.txt and header tags for our blog. It seems that now we can set the label of our blog to got indexed in the search result.

And there is one additional feature on the link tool, where there is a new option, the ability to add rel = "nofollow" on every link.

Blogger is now becoming increasingly intense. Lots of improvements, and all these new features become my favorite.

Cheap or even free SSL certificate

SSL certificate is essential if you want more security for your website, but unfortunately the price of a valid and cheap SSL certificate is rarely to be found. The average price of SSL certificate ranges from $100 - $200, a price that usually can only get the lower-class certificate.

A few days ago, I made an observations to some of SSL Certificate providers out there, where I want the SSL certificate to meet my needs as follows:

  • Multiple Domains (UCC)
  • Wild Card Capability
  • S/MIME Client + Auth / SMTP

And after several days of searching, I finally found a SSL certificate provider that provides features like I was looking for with a very cheap price at only $59.90 for two years. Not only that, they also provide a free SSL certificate, valid for 1 year and can be renewed again.

Company that provide this SSL certificate is StartCom Ltd, which is an Israel-based company and founded by Eddy Nigg. Get your cheap or free SSL certificate here.

See also: Comparison of SSL certificates for web servers.

Blogger Dynamic Views and SEO

Previously I was a little worried when switching my blog theme to Blogger Dynamic Views theme, it’s because I read many articles if the Dynamic Views theme is not SEO friendly. But I decided to install it regardless of people’s opinions.

After trying it for over a week, I did not find any bad effects on traffic, even an increase in visitors and page views, indicating that changing the theme from regular theme to Dynamic Views theme is the perfect choice.

Update: Blogger finally adding Search preferences feature for SEO purposes.