Some browsers such as Firefox implement various types of prefetching. The basic idea is that the browser will start to preload hyperlinked pages in the background to so that once a user clicks on a link the web page will already be ready to be displayed.

Speculative prefetching like this is obviously wasteful from a DNS and network bandwidth perspective because there will be plenty of links on a page that are prefetched but that the user will not click on. On the other hand, prefetching in the browser gives a better user experience with reduced wait time.

Unfortunately for an ISP, prefetching is outside of their control. Prefetching is something that is turned on and off at the browser level. A web site owner can potentially limit prefetching by analyze its HTML code and making sure that it doesn’t have too many links on its most popular pages and also by adding special tags to links that shouldn’t be prefetched.

This blog post provides additional details about prefetching and what impact it has on DNS. In Firefox, the settings for prefetching can be changed by typing about:config in the address bar and then scrolling down to the network.dns.disablePrefetch setting. This setting is disabled by default (note the double negative) meaning that DNS prefetching is turned on. Chrome and other browsers have similar settings.

I ran a small experiment just to see if there is a large difference between turning prefetching off or on. Below is a table showing the results of this experiment. As you can see, there is quite a substantial difference in the number of queries generated. www.wikipedia.org and www.ebay.com in particular seem to generate a lot of DNS queries with prefetching turned on. This is because these pages have links to hundreds of different sub sites pages in other languages such as de.wikipedia.org (Germany) sv.wikipedia.org (Sweden), etc.  The google.com page, on the other hand, does not generate any additional queries with prefetch turned on.

	without prefetch	with prefetch	increase factor
www.wikipedia.org	3	155	51.7
www.yahoo.com	10	12	1.2
www.fox.com	45	64	1.4
www.cnn.com	11	66	6.0
www.youtube.com	12	14	1.2
www.msn.com	28	85	3.0
www.google.com	1	1	1.0
www.ebay.com	31	147	4.7
www.amazon.com	8	65	8.1
average	16.6	67.7	4.1

From this experiment using a short list of domains it is clear that browsers with DNS prefetch enabled generate a very substantial number of additional queries to the DNS system.

Due to the 3-tiered architecture of DNS, the increased load and additional cost for browser prefetching in is borne by the Service Provider. The web site owner and the end user do not incur any significant extra cost. Companies like Wikipedia and Ebay should really look into how they have their sites coded. From what I understand, there are simple HTML codes you can add to instruct the browser not to do prefetching. To give you an example: every browser with prefetching turned on in the world will prefetch the DNS record for the Icelandic (330,000 native speakers) version of the Wikipedia site. This seems wasteful to me as it adds unnecessary extra queries to DNS systems at service providers around the world.