permalink

19

Backbone.Paginator – New Pagination Components For Backbone.js

Pagination is a ubiquitous problem we often find ourselves needing to solve on the web. Perhaps most predominantly when working with back-end APIs and JavaScript-heavy clients which consume them.

On this topic, today I'd like to announcethe pre-release of a new set of pagination components for Backbone.js, which should hopefully come in useful if you're working on applications which need to tackle this problem. They're part of an extension called Backbone.Paginator.

When working with a structural framework like Backbone.js, the three types of pagination we are most likely to run into are:

Requests to a service layer (API) - e.g query for results containing the term 'Brendan' –  if 5,000 results are available only display 20 results per page (leaving us with 250 possible result pages that can be navigated to).

This problem actually has quite a great deal more to it, such as maintaining persistence of other URL parameters (e.g sort, query, order) which can change based on a user's search configuration in a UI. One also had to think of a clean way of hooking views up to this pagination so you can easily navigate between pages (e.g First, Last, Next, Previous, 1,2,3), manage the number of results displayed per page and so on.

Further client-side pagination of data returned - e.g we've been returned a JSON esponse containing 100 results. Rather than displaying all 100 to the user, we only display 20 of these results within a navigatable UI in the browser.

Similar to the request problem, client-pagination has its own challenges like navigation once again (Next, Previous, 1,2,3), sorting, order, switching the number of results to display per page and so on.

Infinite results – with services such as Facebook, the concept of numeric pagination is instead replaced with a 'Load More' or 'View More' button. Triggering this normally fetches the next 'page' of N results but rather than replacing the previous set of results loaded entirely, we simply append to them instead.

A request pager which simply appends results in a view rather than replacing on each new fetch is effectively an 'infinite' pager. 

Let's now take a look at exactly what we're getting out of the box:

Backbone.Paginator is a set of opinionated components for paginating collections of data using Backbone.js. It aims to provide both solutions for assisting with pagination of requests to a server (e.g an API) as well as pagination of single-loads of data, where we may wish to further paginate a collection of N results into M pages within a view.

Paginator’s pieces

Backbone.Paginator supports two main pagination components:

  • Backbone.Paginator.requestPager: For pagination of requests between a client and a server-side API
  • Backbone.Paginator.clientPager: For pagination of data returned from a server which you would like to further paginate within the UI (e.g 60 results are returned, paginate into 3 pages of 20)

Downloads And Source Code

You can either download the raw source code for the project, fork the repository or use one of these links:

Live Examples

Live previews of both pagination components using the Netflix API can be found below. Download the tarball or fork the repository to experiment with these examples further.

Demo 1: Backbone.Paginator.requestPager()

Demo 2: Backbone.Paginator.clientPager()

Demo 3: Infinite Pagination (Backbone.Paginator.requestPager())

Paginator.requestPager

In this section we’re going to walkthrough actually using the requestPager.

1. Create a new Paginated collection

First, we define a new Paginated collection using Backbone.Paginator.requestPager() as follows:

    var PaginatedCollection = Backbone.Paginator.requestPager.extend({

2: Set the model and base URL for the collection as normal

Within our collection, we then (as normal) specify the model to be used with this collection followed by the URL (or base URL) for the service providing our data (e.g the Netflix API).

    model: model,
        url: 'http://odata.netflix.com/v2/Catalog/Titles?&',

3. Map the attributes supported by your API (URL)

Next, we’re going to map the request (URL) parameters supported by your API or backend data service back to attributes that are internally used by Backbone.Paginator.

For example: the NetFlix API refers to it’s parameter for stating how many results to skip ahead by as $skip and it’s number of items to return per page as $top (amongst others). We determine these by looking at a sample URL pointing at the service:

http://odata.netflix.com/v2/Catalog/Titles?&callback=callback&$top=30&$skip=30&orderBy=ReleaseYear&$inlinecount=allpages&$format=json&$callback=callback&$filter=substringof%28%27the%27,%20Name%29%20eq%20true&_=1332702202090

We then simply map these parameters to the relevant Paginator equivalents shown on the left hand side of the next snippets to get everything working:

        // @param-name for the query field in the
        // request (e.g query/keywords/search)
        queryAttribute: '$filter',
        // @param-name for number of items to return per request/page
        perPageAttribute: '$top',
        // @param-name for how many results the request should skip ahead to
        skipAttribute: '$skip',
        // @param-name for the direction to sort in
        sortAttribute: '$sort',
        // @param-name for field to sort by
        orderAttribute: 'orderBy',
        // @param-name for the format of the request
        formatAttribute: '$format',
        // @param-name for a custom attribute
        customAttribute1: '$inlinecount',
        // @param-name for another custom attribute
        customAttribute2: '$callback',

Note: you can define support for new custom attributes in Backbone.Paginator if needed (e.g customAttribute1) for those that may be unique to your service.

4. Configure the default pagination, query and sort details for the paginator

Now, let’s configure the default values in our collection for these parameters so that as a user navigates through the paginated UI, requests are able to continue querying with the correct field to sort on, the right number of items to return per request etc.

e.g: If we want to request the:

  • 1st page of results
  • for the search query ‘superman’
  • in JSON format
  • sorted by release year
  • in ascending order
  • where only 30 results are returned per request

This would look as follows:

        // current page to query from the service
        page: 5,
        // The lowest page index your API allows to be accessed
        firstPage: 0, //some begin with 1
        // how many results to query from the service (i.e how many to return
        // per request)
        perPage: 30,
        // maximum number of pages that can be queried from
        // the server (only here as a default in case your
        // service doesn't return the total pages available)
        totalPages: 10,
        // what field should the results be sorted on?
        sortField: 'ReleaseYear',
        // what direction should the results be sorted in?
        sortDirection: 'asc',
        // what would you like to query (search) from the service?
        // as Netflix reqires additional parameters around the query
        // we simply fill these around our search term
        query: "substringof('" + escape('the') + "',Name)",
        // what format would you like to request results in?
        format: 'json',
        // what other custom parameters for the request do
        // you require
        // for your application?
        customParam1: 'allpages',
        customParam2: 'callback',

As the particular API we’re using requires callback and allpages parameters to also be passed, we simply define the values for these as custom parameters which can be mapped back to requestPager as needed.

5. Finally, configure Collection.parse() and we’re done

The last thing we need to do is configure our collection’s parse() method. We want to ensure we’re returning the correct part of our JSON response containing the data our collection will be populated with, which below is response.d.results (for the Netflix API).

You might also notice that we’re setting this.totalPages to the total page count returned by the API. This allows us to define the maximum number of (result) pages available for the current/last request so that we can clearly display this in the UI. It also allows us to infuence whether clicking say, a ‘next’ button should proceed with a request or not.

    parse: function (response) {
            // Be sure to change this based on how your results
            // are structured (e.g d.results is Netflix specific)
            var tags = response.d.results;
            //Normally this.totalPages would equal response.d.__count
            //but as this particular NetFlix request only returns a
            //total count of items for the search, we divide.
            this.totalPages = Math.floor(response.d.__count / this.perPage);
            return tags;
        }
    });
});

Convenience methods:

For your convenience, the following methods are made available for use in your views to interact with the requestPager:

  • Collection.goTo(n) – go to a specific page
  • Collection.requestNextPage() – go to the next page
  • Collection.requestPreviousPage() – go to the previous page
  • Collection.howManyPer(n) – set the number of items to display per page

Paginator.clientPager

The clientPager works similar to the requestPager, except that our configuration values influence the pagination of data already returned at a UI-level. Whilst not shown (yet) there is also a lot more UI logic that ties in with the clientPager. An example of this can be seen in ‘views/clientPagination.js’.

1. Create a new paginated collection with a model and URL

As with requestPager, let’s first create a new Paginated Backbone.Paginator.clientPager collection, with a model and base URL:

    var PaginatedCollection = Backbone.Paginator.clientPager.extend({
        model: model,
        url: 'http://odata.netflix.com/v2/Catalog/Titles?&',

2. Map the attributes supported by your API (URL)

We’re similarly going to map request parameter names for your API to those supported in the paginator:

        perPageAttribute: '$top',
        skipAttribute: '$skip',
        orderAttribute: 'orderBy',
        customAttribute1: '$inlinecount',
        queryAttribute: '$filter',
        formatAttribute: '$format',
        customAttribute2: '$callback',

3. Configure how to paginate data at a UI-level

We then get to configuration for the paginated data in the UI. perPage specifies how many results to return from the server whilst displayPerPage configures how many of the items in returned results to display per ‘page’ in the UI. e.g If we request 100 results and only display 20 per page, we have 5 sub-pages of results that can be navigated through in the UI.

        // M: how many results to query from the service
        perPage: 40,
        // N: how many results to display per 'page' within the UI
        // Effectively M/N = the number of pages the data will be split into.
        displayPerPage: 20,

4. Configure the rest of the request parameter default values

We can then configure default values for the rest of our request parameters:

        // current page to query from the service
        page: 1,
        // a default. This should be overridden in the collection's parse()
        // sort direction
        sortDirection: 'asc',
        // sort field
        sortField: 'ReleaseYear',
        //or year(Instant/AvailableFrom)
        // query
        query: "substringof('" + escape('the') + "',Name)",
        // request format
        format: 'json',
        // custom parameters for the request that may be specific to your
        // application
        customParam1: 'allpages',
        customParam2: 'callback',

5. Finally, configure Collection.parse() and we’re done

And finally we have our parse() method, which in this case isn’t concerned with the total number of result pages available on the server as we have our own total count of pages for the paginated data in the UI.

parse: function (response) {
            var tags = response.d.results;
            return tags;
        }
    });

Convenience methods:

As mentioned, your views can hook into a number of convenience methods to navigate around UI-paginated data. For clientPager these include:

  • Collection.goTo(n) – go to a specific page
  • Collection.previousPage() – go to the previous page
  • Collection.nextPage() – go to the next page
  • Collection.howManyPer(n) – set how many items to display per page
  • Collection.pager(sortBy, sortDirection) – update sort on the current view

Views/Templates

Although the collection layer is perhaps the most important part of Backbone.Paginator, it would be of little use without views interacting with it. The project zipball comes with three complete examples of using the components with the Netflix API, but here's a sample view and template from the requestPager() example for those interested in learning more:

First, we have a view for a pagination bar in our UI that allows us to navigate around our paginated collection:

(function ( views ) {
	views.PaginatedView = Backbone.View.extend({
		events: {
			'click a.servernext': 'nextResultPage',
			'click a.serverprevious': 'previousResultPage',
			'click a.orderUpdate': 'updateSortBy',
			'click a.serverlast': 'gotoLast',
			'click a.page': 'gotoPage',
			'click a.serverfirst': 'gotoFirst',
			'click a.serverpage': 'gotoPage',
			'click .serverhowmany a': 'changeCount'
		},
		tagName: 'aside',
		template: _.template($('#tmpServerPagination').html()),
		initialize: function () {
			this.collection.on('reset', this.render, this);
			this.collection.on('change', this.render, this);
			this.$el.appendTo('#pagination');
		},
		render: function () {
			var html = this.template(this.collection.info());
			this.$el.html(html);
		},
		updateSortBy: function (e) {
			e.preventDefault();
			var currentSort = $('#sortByField').val();
			this.collection.updateOrder(currentSort);
		},
		nextResultPage: function (e) {
			e.preventDefault();
			this.collection.requestNextPage();
		},
		previousResultPage: function (e) {
			e.preventDefault();
			this.collection.requestPreviousPage();
		},
		gotoFirst: function (e) {
			e.preventDefault();
			this.collection.goTo(this.collection.information.firstPage);
		},
		gotoLast: function (e) {
			e.preventDefault();
			this.collection.goTo(this.collection.information.lastPage);
		},
		gotoPage: function (e) {
			e.preventDefault();
			var page = $(e.target).text();
			this.collection.goTo(page);
		},
		changeCount: function (e) {
			e.preventDefault();
			var per = $(e.target).text();
			this.collection.howManyPer(per);
		}
	});
})( app.views );

which we use with a template like this to generate the necessary pagination links (more are shown in the full example):

<span class="divider">/</span>
		<% if (page > firstPage) { %>
			<a href="#" class="serverprevious">Previous</a>
		<% }else{ %>
			<span>Previous</span>
		<% }%>
		<% if (page < totalPages) { %>
			<a href="#" class="servernext">Next</a>
		<% } %>
		<% if (firstPage != page) { %>
			<a href="#" class="serverfirst">First</a>
		<% } %>
		<% if (lastPage != page) { %>
			<a href="#" class="serverlast">Last</a>
		<% } %>
		<span class="divider">/</span>
		<span class="cell serverhowmany">
			Show
			<a href="#" class="selected">3</a>
			|
			<a href="#" class="">9</a>
			|
			<a href="#" class="">12</a>
			per page
		</span>
		<span class="divider">/</span>
		<span class="cell first records">
			Page: <span class="current"><%= page %></span>
			of
			<span class="total"><%= totalPages %></span>
						shown
		</span>
<span class="divider">/</span>
	<span class="cell sort">
		<a href="#" class="orderUpdate btn small">Sort by:</a>
	</span>
	<select id="sortByField">
		<option value="cid">Select a field to sort on</option>
	 	<option value="ReleaseYear">Release year</option>
	 	<option value="ShortName">Alphabetical</option>
	</select>
</span>

Contributing

I'm more than happy to discuss others thoughts on these components and how they can be improved. If there's a particular bug or feature you would like to submit for consideration, please feel free to send it upstream on the issue tracker.

In lieu of a formal styleguide, take care to maintain the existing coding style. Add unit tests for any new or changed functionality. Lint and test your code using grunt.

Coming Soon

Please note that as mentioned, this project is currently in a pre-release phase. I plan on adding comprehensive unit tests shortly and (depending on feedback) the Paginator API may be subject to change. Watch this space!

Release History

  • 0.15 – rewrite to simplify the project API, unify components under the same collection hood. Addition of new examples.
  • 0.14 – rewrite of all components
  • 0.13 – initial release of client and request pagers
  • 0.11 – initial work on version to work with requests to a server
  • 0.1 – basic pagination of a single response from the server

License

Copyright © 2012 Addy Osmani. Licensed under the MIT license.

19 Comments

    • The requestPager() allows you to do this easily. Both requestPager() and clientPager() support goto() methods, allowing your Views to go to a specific page.

    • @Andrew: i was about to ask the exact same thing…history is important.
      and on top of that i would love to see that working on an infinite scroll scenario as well….
      i was thinking about smt like: http://tumbledry.org/2011/05/12/screw_hashbangs_building

      …..although i can see the problem of loading a big portion of data (that would make us miss the all point of having a pagination) when a user types in something like http://www.example.com/1-666/ ….Anybody has yet found a solution for this?

      Trying to find a solution to this i was thinking about showing the user the requested “page/portion” (in my example would be page 666)
      but, at this point, the application would need to “activate” a “finite-scroll-towards-top” from 666 to 1 and still be able to let the infinite scroll working to move to page 667, 668, 669, etc.

      Just an idea…i haven’t really made any deep thoughts or tests this.

      @addy: What are your thoughts on this subject? Anyone?

      • The use case is that a request comes in from outside the app for a result set in the middle of an infinite load. Then scroll is used to go back up the page (or down the page).

        I would load the current results and the prior set and next set ( of 20? 1? depends on content type), then lazy load as usual but backwards.

        Treat it like a carousel. Always have the previous and next set loaded.

    • Hey Ben!

      Not at all :) I’m always more than happy to take on your feedback and if there are any improvements you think can be made to the requestPager in particular, I’d be eager to hear them. It’s great to know that for the clientPager portion, the codebase isn’t hugely different as I was concerned I might have been going a little off track!

      Cheers for the input as always!

  1. Pingback: Backbone.Paginator – New Pagination Components For Backbone.js « that dismal science

  2. I really enjoyed this article as well as the backbone plugin*. Pretty cool – but man Addy, I wish you’d do more videos (i tend to watch better than i read). You’re a great speaker, if you ever get a chance – i’d love to watch some presentations on backbone from you.

  3. Love to see your thoughts on reverse paging when in infinite scroll…ie dom cleanup, and how visible elements might differ from total collection size. I’m focused on this right now in order to manage dom size on mobile in an infinite scroll scenario,

    • I too am looking at this this problem and would love to get some more input from others. The dom cleanup is crucial in my embedded html client.

  4. I was doing pagination with creating new collection with the different url
    for example:

      Items = Backbone.Collection.extend({
       model: item,
       url: "api/items",
       initialize: function(page) {
         if(page) {  this.url = "api/items/page/"+page; }
       }
    

    but it’s get tricky when you need to change/delete item

  5. Pingback: 《JavaScript 每周导读》【第二期】 « NeverBest!我还能做的更好 – 007boy | im007boy

  6. Pingback: Backbone.Paginator not working with custom api that returns JSON | jasondenney.com

  7. Very good plugin. Waiting for example with history.
    A question: Is it enough stable to use it in a production application ?
    Best regards.
    Ramzi.

  8. Addy, Thanks a ton for this plugin.

    When using this plugin whats the best way to deal with thousands or more records in your database? It doesn’t appear the pagesInRange parameter works with requestPager.

Leave a Reply

Required fields are marked *.