The SEO world is full of new terms, especially when one is first starting out. It is important to start understanding what the definitions are. As the definitions are understood it is easier to follow what any one particular expert is saying and how that information can be used to further a websites prominence in the search engines.

There are two words that I would like to cover today. They were prompted by an article written by SEO-speedwagon. The two terms are Index and Crawling.

Crawling: “is the process of an engine requesting — and successfully downloading — a unique URL.”

What this means: Google, Yahoo and other search engines have the ability to surf the internet. To surf the internet they use a small program called a spider or robot. What these spiders or robots do is read different pages that they come across. They follow links to find the page that they read and that is one reason why links are so important. When they arrive at a page the spider starts at the top and reads all the content on that page. If a spider follows a link to a specific page and receives an error then the spider cannot crawl that particular page.

Index: “is the result of successful crawling.”

What this means: Once a spider or robots has read everything on a particular page it adds all of that information to what is known as its index. A search engine index is very similar to that used in a book. It is a quick reference guide as to where the information is and what it may be about. Once a page has been successfully indexed and made available when a search is performed the page will appear in the Search Engine Results Pages.

Problems that may be encountered when a spider is crawling pages are, it may be blocked by a robots.txt file or the server could be switched off or coding on the page may stop the search engines from reading the page correctly. There is normally a delay in time between a successful crawl and the page being displayed in the results of a search engine. This delay depends on the site and can be as little as a few minutes, for some sites that the spiders trust and access regularly for example a news site. For a new site without any page rank it could be anything up to a week before the page is available in the search results of Google and other search engines.