This post is a follow-up of last week post about SEO: On Page SEO: Search Engine Unfriendly Technologies. I invite you to read this previous post as well. If you would like to have a whole picture of my SEO guide, you can check the series from the very first post: Comprehensive introduction to SEO.

Here is an example Robots.txt:
This improves drastically the speed of indexation of your newest pages, which is very useful when you launch a new website.
You can find all the useful information about Sitemaps on http://sitemaps.org which also provides a file validator to check the coding of your sitemap file.
You can generate this file manually, but it is better to have it generated dynamically when your website is updated. It is recommended to put the freshest urls at the top of your Sitemap file.
A Sitemap shouldn’t include more than 50 000 Urls and weight more than 50Mo. In the case you have more Urls, you should have additional Sitemap files regrouped in a Sitemap Index.
The Sitemap.xml file should be declared in the Robots.txt (located in the root folder).
Added to the classical Sitemap, you can add specific Sitemaps for some types of content.
Robots.txt File
The Robots.txt file is used only by Search Engines. It is located in the root folder of your website. It has 2 goals: Give the Sitemap Url to the Spiders, tell the Search Engine Robots which part of your website they shouldn’t index.Here is an example Robots.txt:
1) User-Agent: *
2) Disallow: /cgi/
3) Disallow: /admin/
4) Disallow: /content/this-article.html
5) Sitemap: http://www.mysite.com/sitemap.xml
Sitemap XML
Sitemap is an internet protocol which is widely accepted by Search Engines. The aim of the Sitemap is to feed the Search Engine Robots in an automated way the Urls of the new pages to index.This improves drastically the speed of indexation of your newest pages, which is very useful when you launch a new website.
You can find all the useful information about Sitemaps on http://sitemaps.org which also provides a file validator to check the coding of your sitemap file.
You can generate this file manually, but it is better to have it generated dynamically when your website is updated. It is recommended to put the freshest urls at the top of your Sitemap file.
A Sitemap shouldn’t include more than 50 000 Urls and weight more than 50Mo. In the case you have more Urls, you should have additional Sitemap files regrouped in a Sitemap Index.
The Sitemap.xml file should be declared in the Robots.txt (located in the root folder).
Added to the classical Sitemap, you can add specific Sitemaps for some types of content.
This post is great. Thank you for this post. I like the people who share knowledge with others.
ReplyDeleteSoftware Development Company India
ReplyDeleteHello,
we provide affordable and result-oriented SEO services, please give a chance to serve you.
Thanks
Admin: E07.net
Flymedia Technology is the best SEO company to contact if you want to get an amazing service of SEO. We have the best SEO experts who never fail to show the best results to our clients.
ReplyDeleteThat’s a excellent perspective, nonetheless isn’t make every sence whatsoever dealing with which mather. Just about any method with thanks in addition to pondered try and promote your own article straight into delicius nevertheless it is very much a problem in your information sites is it possible i highly recommend you recheck it. gives thanks again. web2 sites
ReplyDelete