From the monthly archives:

July 2010

SEO and Robot.txt File for Wordpress

by Charles Grimm on July 15, 2010

Sample-wp-robots.txt-file

Do robots like your source code?

I was searching the other day for a robots.txt file  example and it took me a few minutes to find some fragmented samples in forums. So here is a quick post for you wordpress bloggers who are managing your own Search Engine Optimization..SEO.

Joomla fortunately, automatically sets up a robot.txt file when you do your install through cpanel.

With wordpress for some reason, you need to do it manually. I find this a bit bizarre especially since WP espouses to be very search engine friendly right out of the gate. Anyways here is a a basic robot.txt sample.

Disallow: /cgi-bin
Disallow: /wp-admin
Disallow: /wp-includes
Disallow: /wp-content/plugins
Disallow: /wp-content/cache
Disallow: /wp-content/themes
Disallow: /trackback
Disallow: /feed
Disallow: /comments
Disallow: /category/*/*
Disallow: */trackback
Disallow: */feed
Disallow: */comments
Disallow: /*?*
Disallow: /*?
Allow: /wp-content/uploads

# Google Image
User-agent: Googlebot-Image
Disallow:
Allow: /*

# Google AdSense
User-agent: Mediapartners-Google*
Disallow:
Allow: /*

# digg mirror
User-agent: duggmirror
Disallow: /

Sitemap: http://www.example.com/sitemap.xml

Copy and paste this into a file named robots.txt;

You can use notepad, word or any text program will work. Then, upload the this to the root level of your domain. Another way is to just add a plugin to your wordpress blog. There are several robot.txt plugins to choose from, some will add a meta robots tag to your wordpress pages.

Here is another more comprehensive wordpress robots.txt example.

For more tips on robot.txt and search engine optimization you can post a question in the comments section.

Note, if you are a building blog, wordpress has the best user interface and is great for fast preformatted content development. However if you are building a website with several different types of content that need function, I would check  out Joomla 1.5 and it’s joomla seo extensions.

What CMS platform do you prefer for building the best search engine optimization into your website, Joomla, Drupal or Wordpress?

{ 0 comments }