Robots.txt file

[jesicawillss]

I would like to know that which things of website, an expert SEO must put into robots.txt file for disallow? Which pages is better to not show to Search Engines?

 

[heenakapoor]

Robots.txt file is very important in SEO which allows to block search engine spider to enter into the website. If you have very critical p[ages on the website and you don't want o index then into the search engine database then we can instruct search engine spider that which are are need to index and which are not. There are many websites which has the admin pages like login, register pages and some more pages which they don't want that into to index into the search enigne, so we can write the code for not to index t5he pages and if we have duplicate pages on the website the we can thsi technique for them also.