Freelance Web Designer, Leeds. Need a website? Call Abi (0113) 279 7731 / (0781) 805 8158
LEEDS: (0113) 279 7731 MOB: (0781) 805 8158
web design leeds yorkshire

How Do Search Engines List a Website?

search engines list

When you get a new website you will be eager for people to find it, but you maybe wondering how search engines list a website given this will be one of the places you hope you are found – in the organic search results. How do the search engines decide where to list you, under which search terms and in fact whether to list your site at all? The answer is a search engine spiders such as Googlebot.

How does a Search Engine Discover a website?

What is Googlebot?

  • Googlebot is software that Google uses to gather information about every document/image made publicly available on the web. Googlebot is commonly known as a search engine spider, and the most well-known of them all. All search engines use this type of software (albeit their own versions) to crawl the web for new and updated websites. They all programme this software slightly differently depending on what factors they consider important in deciding where and how to list a website in their search engine. Each search engine bot will use a different algorithm to rank websites.

What is Crawling?

  • Crawling is the process where the search engine spiders go around the internet, finding new and updated information. Search engine spiders find what to crawl using links from one site to another, or internally within a website. Getting links to your site and having a good menu system on your site will obviously be an advantage with the above in mind.

What is Indexing?

  • Search engine robots such as Googlebot, crawl the web for new information and then indexes all that information processing the information against its preferred algorithm (rules). Once processed, search engines will add the website to search pages (index), but only if it is identified as quality content. In simple terms (far more to it than this) during the indexing process, the search bot’s analyse the words on a page and where they have been placed in the content, (the higher on the page the more importance). Information such as title tags and ALT attributes are also analyzed during indexing. As a result for a single search term such as “blue car” you may rank number 1 on the first page, or number 10 on the 100th page of a search engine. Like I mentioned, there is far more to search engine algorithms than the above but the explanation is the essence of how they decide where to list your site on their search engine.

What are SiteMaps?

  • We know now that search engine spiders (bots) crawl the web to find new information to index, but how do you make sure your website is discovered by search engine bots in the first place so your website appears in a search engines lists? And more so, how do you make sure every page of your website is discovered and indexed or NOT indexed if you don’t want it to be? Well, search engine bots begin with web pages captured during previous crawl and also look at site maps provided by webmasters (you or your developer). As it browses pages crawled previously, it will detect links upon those pages to add to the list of pages to be crawled specified by your sitemap. Hence, new content  is discovered through sitemaps and links on your website.

How to Get on the Search Engines List Quickly

Here are the basics of how websites are crawled and indexed, with ways to get the search engines to index your site quickly.

  • Create a Sitemap – A sitemap is an XML document placed in the folder on a server where your website files are stored. The sitemap basically lists each page on your website. It tells search engines when new pages have been added (so long as you update it) and how often to check back for changes on specific pages. If your website is built with a CMS like WordPress, you can install plugins which will automate the process for you keeping your sitemap up-to-date easily and submitting it to the search engines every time it is updated. Google XML Sitemap plugin is an example of a plugin you can use on WordPress. You can also use tools such as the XML Sitemaps Generator if you have a static website.

xml sitemap example

google webmaster tools

  • Submit to Google Webmaster Tools – If you don’t already have one, simply create a free Google Account, then sign up for Webmaster Tools. You can also submit your sitemap to Bing and using their webmaster tools too! This will alert the search engines to your new website, allowing them to crawl it easily and quickly. Once again it’s a benefit of using a CMS like WordPress to run your website since the submission process can be automated through plugins mentioned above. Undoubtedly it will get your website on the search engines list far more quickly.
  • Submit Website to Search Engines – It only takes a moment, and it certainly doesn’t hurt the potential of having your site crawled more quickly. Sign into Webmaster tools mentioned above on Bing, Yahoo and Google look for the submit site url options.
  • Google Analytics – Adding analytics will make google notice your site but also give you some useful statistics to show you who is looking at your website.
  • Social Media – Crawlers get to your site via links. Twitter, Facebook, Google+, LinkedIn etc all provide opportunity to get a quick link pointing to your website and add potential for crawling. So add your URL to any social pages you have both business and personal, at the very least it’s a simple way of making people aware of your new business.
  • Bookmark It – Use social bookmarking sites like Delicious and StumbleUpon.
  • Create Offsite Content – Get some more links to your new website by creating content such as submitting guest posts to blogs  and quality article directories. Please note this is about quality content from quality sites! Dont just submit to spam directories this WILL hurt your rankings.
  • Blog / News Sections – If possible add a news section or blog to your website, frequently updated sites get noticed by google more quickly and it also helps SEO as your site will be deemed active.

Ultimately a website without visitors is pretty pointless, so let search engines know you exist, at the very least adding links to suitable social media profiles, relevant directories, submitting the url to search engines and creating a sitemap should form part of your initial marketing to get your website on the search engines list.

Similar Posts:

    None Found

Leave a Reply