Speed up next-page navigations with prefetching

What is prefetching?

Loading web pages could be made faster by fetching the next page (or set of resources for the next page) a user is likely to visit ahead of time. We call this prefetching. This can make subsequent navigations appear to load instantly in some cases.

Modern prefetching can be achieved in a number of ways. Most commonly this is done using the browser’s <link rel=prefetch> Resource Hint, which works declaratively via HTML..

<!-- HTML -->
<link rel="prefetch" href="/pages/next-page.html">
<link rel="prefetch" href="/js/chat-widget.js">

or a HTTP Header:

Link: </js/chat-widget.js>; rel=prefetch

Prefetching is supported in Webpack too (see prefetch/preload in Webpack).

<!-- Without prefetching -->
import("ChatWidget");

<!-- With prefetching -->
import(/* webpackPrefetch: true */ "ChatWidget");

Webpack’s runtime will inject the correct prefetch statements in the page once the parent chunk has finished loading. In the above case, <link rel="prefetch" href="chat-widget.js">.

Per CanIUse, <link rel=prefetch> has decent cross-browser support.

If a resource in Chrome is prefetched with <link rel=prefetch>, it will be fetched at a low priority and kept around for 5 minutes. This lasts until the resource has been consumed, at which point the normal cache-control rules for the resource will apply.

Unsure what pages on your site may be worth considering prefetching? Check out our experimental helper tool for suggested “top next pages” (based on your analytics):

“Prefetching” can also mostly be accomplished using Service Workers [see prefetching during registration]/fetch() or XHR. Note however, not all of these mechanisms have the same “low priority” fetch behavior as <link rel=prefetch>.

Who is using prefetching today?

Several brands you’ve probably heard of use prefetching in production:

• Netflix prefetch JavaScript bundles needed for subsequent navigations ahead of time
• IndieGogo prefetch Stripe’s JavaScript library for credit-card processing on future pages
• Craigslist prefetch their JS bundle for search results pages
• [Heineken] (https://www.heineken.com/us/agegateway?returnurl=%2f) prefetch JS and CSS bundles pages after the date-of-birth verification page may need.

What are the challenges of prefetching?

When a user is on a constrained network connection or a limited data-plan, every last byte counts. While prefetching a number of pages ahead of time can make sense for users on a great WiFi connection, it requires care for those that are not. You don’t want to waste anyone’s data plan.

You can use the navigator.connection.effectiveType API (part of NetInfo) to only prefetch pages when a user is on a connection that is effectively 4G.

One thing to also be careful about with prefetch is that if requests aren’t finished by the time a navigation is initiated, the in-flight prefetch request is cancelled. This can mean wasted bytes so just be sure a user is likely to actually see value from these fetched when using this capability.

Case study

When Japanese publisher Nikkei were confident users would navigate to specific pages, they didn’t wait for a navigation to happen.

They leveraged <link rel=prefetch> to prefetch the next page before users tapped on links. Below we can see the impact of waiting for network + server overhead to deliver a page (taking a total of ~880ms). Compare this to prefetching, with almost no request overhead and an immediate page navigation (~37ms).

More on Nikkei can be found in this recent case study.

What is predictive prefetching?

Sites using prefetching typically have a good idea of what top-level pages or resources a user is going to visit and can manually add <link rel=prefetch> tags for them. This manual process can however become harder to achieve for large sites at scale. Once you go several levels deep, it can be difficult to know what the “best” next page is going to be nor how to wire this up without it being a manual process.

Predictive prefetching uses additional data (e.g analytics) to predict what documents users are likely to visit given any URL on a site. This data can then be used to <link rel=prefetch> those pages, improving subsequent page load times. Predictions can be optionally improved using machine learning to ensure only pages with the highest changes of being accessed are fetched ahead of time.

Although there’s nothing here specific to it, Google Analytics has a Reporting API with just enough information to build out predictive prefetching for sites. Minko Gechev, Katie Hempenius and I took advantage of this recently to build a project called Guess.js, which has solutions for 1) predictively prefetching relevant pages and 2) predictively bundling and prefetching the right JavaScript resources that may be needed for those pages (via webpack).

Where can I learn more about predictive prefetching?

Rick Viscomi chatted with Katie Hempenius and I about this topic in depth in case interested in learning more:

We’re going to continue exploring opportunities to evolve prefetching on the web. Most recently, we’ve explored privacy-preserving preserving prefetching using Signed Exchanges, a subset of the emerging web packaging specification.

PS: Keep an eye out for quicklink - a <1KB library I’m releasing that aims to enable faster next-page navigations by prefetching in-viewport links during idle time :)

With thanks to Yoav Weiss for his helpful input on how <link rel=prefetch> works behind the scenes in Chrome

I cross-posted this article to dev.to for folks interested in reading it there.