The Difference Between Dynamic URLs and Static URLs



Search engine spiders don't like dynamic web pages when parameters are attached to the end of the URL after question mark ? character.

What web pages are dynamic or static

URLs that contain "?" are called dynamic URLs. It's dynamic because the page is normally generated by using data from a database. The part after "?" is database row identifier which is used to retrieved the specified data from one or more database tables. Dynamic web page gives us the advantage of reusing a web page for different data when we issue a different ID to it.

If you have knowledge about HTML, you know the web pages you create normally have .htm, or .html, file extension. These pages are static because HTML code don't change on the fly when requested and they are not processed by web servers for logic. They can be viewed without using a web server.

A web page is said to be dynamic if it is created by using server-side scripting languages such as php, asp, jsp, perl, cgi and so on. These languages are like normal programming languages such as C++, Java, etc. The major difference is scripting languages can't be compiled beforehand. They can only be processed by web servers on the fly when the page is requested by a visitor. Dynamic pages can't be viewed without a web server.

When a dynamic page is requested, the web server first looks at the page's source code and if any server-side scripting code exist, it will process them and generate static HTML result. When processing of the full page has been completed, web server sends only pure HTML code to the web visitor's browser.

Using scripting languages to create web pages gives you the power to do nearly anything you want. If the dynamic page has no "?" character in its URL, search engine spiders treat the page the same as a normal HTML static page.

Query string parameters

When "?" character is used, the page's full URL changes when values after "?" change. The portion after "?" is called the page's query string parameter(s), or simply query parameter(s). Every time when parameter(s) changes, the resulted page will be different.

A page URL can contain more than one "?" character. When this happens, search engine spiders will have difficult time to index the resulted page. If the page has only one "?" character, major search engine spiders can crawl that page well. For example, Google can index and store a page's URL as http://www.examplesite.com/product.asp?id=12345. But if the same page's URL is

http://www.examplesite.com/product.asp?id=12345&category=23&page=3

Most search engines will not be able to index it well even though Googlebot and Yahoo! Slurp may be able to crawl it.

(Note: Googlebot is Google's web-crawling robot. Yahoo! Slurp is Yahoo's web-crawling robot. Search engine robots collect documents from the web to build a searchable index.)

Yahoo help says

Yahoo! does index dynamic pages, but for page discovery, our crawler mostly follows static links. We recommend you avoid using dynamically generated links except in directories that are not intended to be crawled/indexed (e.g., those should have a /robots.txt exclusion).

Google's Webmaster Guidelines:

If you decide to use dynamic pages (i.e. the URL contains a "?" character), be aware that not every search engine spider crawls dynamic pages as well as static pages. It helps to keep the parameters short and the number of them small.

Let's analyze what Google has stated above.

• The URL contains a "?" character

  This means the definition of dynamic pages are those containing "?" characters in URL.

• Keep the parameters short

  This means the number of characters in each individual parameter should be short. There is no quantitative measurement given by Google but we can check some web forums to see examples.

  http://www.cre8asiteforums.com/viewtopic.php?t=8386

  This page was crawled by Google. The length of its query parameter is 4 characters. There are many other examples on the internet that have more characters and were crawled successfully. The maximum number of characters that can be accepted by Google is unknown.

• Keep the number of them small

  This means we should keep the number of parameters in each URL as small as possible. The above Cre8ASiteForums example has one parameter.

At least now we can say Googlebot can crawl dynamic pages that have one query parameter and the number of characters in the parameter can be 4.

How to convert your dynamic URLs to static - Rewrite your URLs

We know search engines don't like dynamic web pages even though Google or Yahoo can crawl them properly (if the number of parameters in the URL is low). Other search engines may have difficulties to crawl. So the best way is to find a way to convert dynamic URLs to static.

Converting every dynamic page URL by hand is time-consuming and practically impossible. If you use Apache web server, there is a Apache mod_rewrite module to help you rewrite the requested URL to a static one with no "?" character embedded on the fly. Other web servers should have similar functions to do the job.

Here is what rewrite can achieve:

Before URL rewrite: http://www.examplesite.com/product.asp?id=12345&category=23&page=3

After URL rewrite: http://www.examplesite.com/product12345-23-3.asp

After rewrite, the complexity of the URL has been greatly reduced. It's good looking to human readers as well.


Copyright© GeeksEngine.com

Related Articles: 1. Absolute Path and Relative Path Explained 2. Robots Meta HTML Tag Syntax Explained 3. What If You Don't Want Your Pages To Be Crawled and Cached by Search Engines 4. What Robots.txt is And Search Engine Robots Explained 5. The Right Domain Name Drives More Website Traffic Other Recent Articles from the Webmaster Help category: 1. How to load IP addresses for countries into your MySQL database 2. How to set up your website connection details in FileZilla 3. How To Find Out Everything You Want To Know About A Website 4. Robots Meta HTML Tag Syntax Explained 5. What If You Don't Want Your Pages To Be Crawled and Cached by Search Engines 6. What Robots.txt is And Search Engine Robots Explained 7. How to Tweak HTML Table To Speed Up Page Load Time