A spider is the robot program used by a search engine to index the World Wide Web content. A spider captures specific information about a page. Some capture the title and the first so many characters of content, some capture the title and meta tags, some capture specific inbound link information, some look for keywords in specific places and some use a combination of all of these methods.
.......
|
.......
|