Using dynamically generated content has become the norm these days and left me wondering, “can this content be crawled by spiders as easily as static content?” Dynamic sites give control of the site content to the customer but at what cost? This is something to consider if the site traffic heavily depends on search engines.

Most static sites use URLs that look something like this:
http://www.somewhere.com/category/pagename.htm

A URL like the one above is ranked higher than say a URL like this (a dynamic one):
http://www.somewhere.com/main.aspx?page=pagename

URLs with ?, &, =, etc. risk being cutoff after one of these symbols.
http://www.somewhere.com/main.aspx?page=pagename1
http://www.somewhere.com/main.aspx?page=pagename2

Both of these URLs above would become:
http://www.somewhere.com/main.aspx

“Now, you don’t have unique pages, and consequently, the duplicate URLs won’t be indexed.”

The description and keywords that are attached to these pages are ignored. To combat this problem a site like this needs a way to use friendly URLs that will be unique and point to the proper dynamic URL. This is explained in greater detail on webconfs.com but is something worth thinking about when designing a new site.

Blogged with Flock