Google is the world's largest search engine. It crawls millions of webpages everyday. But one problem it faces is duplicate URLs on a website linking to the same content. For instance, the URLs example.com/blog/category/news and example.com/blog/index.php?category=news can both point to the same page, hence creating duplicates, and reducing site efficiency for Google's crawler. In the interest of good SEO, it is important that you optimize your URLs to avoid duplication. To this end, Google Webmaster Tools now gives you a Configure URL Parameters feature, which will help you recommend actions for the GoogleBot crawler to take, so as to increase your site's crawl efficiency. It will also lessen the resource burden on Google that goes into the crawling of your website. In this post, we will show you how to set easy settings for URL Parameters.
Update: Read: Optimize URL Parameters For Blogger
Update: Read: Optimize URL Parameters For Blogger
What are URL parameters and how do they increase URL count?
URL parameters are parameters, or values, passed into a URL for searching or refining. For example, if you search for something on this blog, you will see a URL like this; http://www.mybloggertricks.com/p/search.html?MBT=search_query, where MBT=search_query is the URL parameter. They usually come in a key/value pair, with MBT being the key, and search_query being the value. You can manually search this blog by replacing search_query with anything you want to search for.
You might have seen eCommerce websites, where a product can be found via multiple ways. For example, example.com/tablets/apple/ipad.php and example.com/tablets/index.php?make=apple&product=ipad will both point to an iPad. Notice that multiple URL parameters have been used, separated by an ampersand (&). There can be more parameters, such as color=black, 3g=no, and so on.
Here is a video from Google summarizing what URL parameters are, and how to configure them in Google Webmaster Tools.
To configure URL parameter actions for your site, log onto your Google Webmaster Tools account, and select URL Parameters under Configuration on the left sidebar. You will either see an option to add a parameter, or you will see a message telling you that Google has no problems with your site.
Caution! If your website is okay, then you should leave it at that! If you accidentally set a site-wide block, then it could result in your pages getting de-indexed from Google. So beware of what you do from now on.
You might have seen eCommerce websites, where a product can be found via multiple ways. For example, example.com/tablets/apple/ipad.php and example.com/tablets/index.php?make=apple&product=ipad will both point to an iPad. Notice that multiple URL parameters have been used, separated by an ampersand (&). There can be more parameters, such as color=black, 3g=no, and so on.
Here is a video from Google summarizing what URL parameters are, and how to configure them in Google Webmaster Tools.
Types of URL parameters
Before you dive into configuring URL parameters for your blog, take a look at the different types of URL parameters. This will help you in the configuration process. There are two basic types of URL parameters.- Parameters that don't change your content - These parameters have no effect of your actual content, and are simply for tracking purposes to let site owners know where a certain user has come from, or which affiliate id has referred him. E.g. example.com/index.php?affiliateid=12345, and so on.
- Parameters that change your content - These parameters will change the content on your page. Take a look at our search bar. The URL parameter MBT=search_query effects what is displayed on the site. If I write MBT=SEO, it will show me a list of posts that has the keyword SEO in it. It I change the word SEO, then it will show me something else, hence it has changed the content. As per Google's URL parameter configuration tool, there are 5 different types of URL parameters that change a page's content.
- Sorts - These are used to sort a list of items according to a certain value, e.g. sort=price_ascending
- Narrows - These refine a search and narrow it down to lesser content from a larger set. For example, color=black
- Specifies - These determine what content will be displayed on a page. For example, make=apple&product=ipad. Our example with the search feature on this blog falls under this category
- Paginates - Displays a specific page in your website. E.g. page=3.
- Translates - Displays the content in a specific language. E.g. lang=en
Configuring URL parameters in Webmaster Tools
Before starting off, it is important to note that you can only configure URLs that have parameters in the key=value format. URLs that have a different structure are 'ineligible'. For e.g. example.com/page/3 is not eligible, but example.com/index.php?page=3 is.To configure URL parameter actions for your site, log onto your Google Webmaster Tools account, and select URL Parameters under Configuration on the left sidebar. You will either see an option to add a parameter, or you will see a message telling you that Google has no problems with your site.
Caution! If your website is okay, then you should leave it at that! If you accidentally set a site-wide block, then it could result in your pages getting de-indexed from Google. So beware of what you do from now on.
Parameters that do not change content
The first thing you should do is, think of all the URL parameters that you know don't change your content. These can include affiliate ids (affiliateid=12345), sources (source=facebook), tracking id (track-id=abc123), and so on. Add each of these parameters (whichever may apply), and then mark them as "Does not affect content".Parameters that change content
For parameters that change content, select the appropriate option. You will see more options pop up. You can tell Google how that parameter changes the content by selecting from among the five types of parameters we discussed about that change content. You also have the "Other" option for parameters that do not fit into the given five behaviors. You cal also tell GoogleBot how to handle that particular parameter. Here are some of the response types.- Let GoogleBot decide - This is the standard option if you don't know what you're dealing with, or if you can't decide how a certain parameter should be handled. You will be safe with this option selected :0
- Crawl Every URL - Every URL with this parameter will be crawled, as Google will identify each unique value. In our MBT search example, MBT=SEO and MBT=blogging are two different URLs, and should be treated accordingly.
- Crawl Only URLs with Value - Only the URLs with the specified values in their parameters will be crawled. For example, if I add only the value SEO in the MBT parameter configuration, Google will only crawl URLs with MBT=SEO in them, but not MBT=blogging. This is only useful for sorting purposes, where you can only ask Google to crawl your content that's ordered in an ascending order, and not descending.
- Crawl No URLs - Beware! If used wrong, this could have catastrophic results. URLs with the specified parameters will not be crawled at all! If I select this option for the MBT parameter, Google won't be able to access our search results.
What option to use with what parameter types?
Sorts need to be handled with care. If a parameter is never used in a URL by default, then you can select "Crawl No URLs". Otherwise, Sorts generally fall under the "Crawl Only URLs With Value" category. Sorts offer the same content, with just the order changed, which doesn't make a bit of a difference to Google. So between sort=ascending and sort=descending, you should go with either one, but not both.
Narrows are tricky as well. Mostly, they select data from a larger set, making the narrowed data somewhat redundant. For example, lets say example.com/search?product=shoes&color=black shows some 50 black shoes at a store. But example.com/search?product=shoes&color=black&price-from=10 may show just 5 shoes that are above $10. This smaller data-set isn't of much use to Google. So you can select the Crawl No URL option. But again, be sure to double-check your options, because wrong implementation can lead to disaster =/
The rest of the parameter types, Specifies, Paginates and Translates generally fall under the "Crawl Every URL" option.
If you are not sure about something, just select "Let GoogleBot decide". Don't carry out experiments of your own! If you have questions, feel free to ask them here, and maybe we could help solve them. Cheers :)