That’s the central question which needs to be asked when people say that their site is not being indexed by Google. Of course, other factors like website architecture, crawling, indexing, etc. do play an important role but the ease with which crawlers browse your site is the most important factor.

There are numerous sites which make use of Javascripts or Splash or other technologies that are good for the user. But what about the search engines? They don’t understand sound or color or images. Unfortunately, they are literate only to read the text and not the special effects that webmasters crave for.

So how do you get a website crawled (and indexed) by Google?

Apart from manual submission to search engines and other factors, one method which can be controlled by the webmasters is the URL structure. URLs are like bridges, which help your readers as well as the crawlers to move from one page to another. Therefore it is imperative to have a simple and clean structure of URLs so that the crawlers spend less time in navigating and more time in indexing your content.

Here is a presentation by Google in which they talk about the best practices for crawling and indexing of a website. It also talks about what not to do with your website URLs. The basic guidelines they suggest to reduce inefficient crawling are:

1. Avoid maverick coding practices
2. Remove user-specific details from URLs
3. Optimize dynamic URLs
4. Rein in infinite spaces
5. Disallow actions Googlebot can’t perform

Go through the presentation and check out whether you are following any of those practices or methods that the Google bot does not understand!

for more information visit  Golden Street Media.