Offline installation of npm packages
January 2, 2016
Occasionally, you may need to install npm packages while offline. This could be due to a flaky network connection, being on a flight or during a workshop. Getting this working has been a dream for a while.
They say you should follow your dreams. Unless it's been a while and they haven't followed you back, in which case unfollow them and move on. Luckily, there are a few options available to get npm working offline today.
First, a built-in (but ultimately incomplete) option. In theory, when you want to force installation from the
npm cache, you can use the
--cache-min flag with a high value. To install a package from the cache, run:
$ npm --cache-min 9999999 install <package-name>
npm --cache-min Infinity. Typing this out every time you need it is a little tedious, so alias it in your dotfiles. In my bash files, I've previously used:
npmo ="npm --cache-min 9999999 "
In an ideal world, the
npm client would just alias this to
--offline. So, why isn't this just a default in npm3?
It's the hero Node deserves, but not the one it needs right now
--cache-min hack has a number of pretty undesirable shortcomings.
When using it, npm still connects to the registry over the network if a package has dependencies that aren't already in the cache. It will also fail if you install a package that depends on a newer version of a package than the one installed in the past.
npm team have concerns about shipping a proper offline flag as-is as most users aren't going to understand the subtleties of how the npm cache works. This is understandable. From what I gather, they are open to eventually shipping full support for an offline experience but until then, let's look at some third party stop gap solutions.
A significantly more robust solution to
--cache-min is Nolan Lawson's offline-first local-npm package (which I love). It's a Node server that acts like a local npm mirror (without needing to do a complete replication of the whole npm registry).
npm installs are fetched from the registry and then modules and their deps get stored in a local PouchDB database. This caches them so subsequent
npm installs use the local cache rather than calling to the network.
local-npm also takes care of keeping modules updated when they change. It does this by listening for changes to the remote registry so you don't have to worry about staleness.
local-npm installed, run:
$ npm install -g local-npm
npm is built on top of CouchDB.
local-npm replicates the skimdb part of this database to a local PouchDB instance. Running:
will begin the replication process. There's no need to wait for the complete skimdb download (which can take a few hours). It will fall back to the network for any libraries that aren't yet replicated. As mentioned,
local-npm is offline-first. It has upfront replication for metadata and tarballs download the first time you install a specific version.
To complete setup, you'll want to go ahead and set
npm to point to the local server
local-npm is running:
$ npm set registry http://127.0.0.1:5080
Awesome. You should now be able to open up a new tab and just
npm install any dependencies you require using the caching proxy.
Here's what a successful offline
npm install should look like:
and for posterity, here's a tab where I'm running
local-npm and we can see it correctly proxying through requests for both the
To test everything works as expected:
- Check your network connection is up
- Run an
npm installto grab a module or collection of dependencies from a package.json file. For example,
- At this point,
local-npmwill have a cached version of these modules stored
- Clear the npm cache using
npm cache cleanor delete the modules from your
- Turn off your wifi/kill your network connection
- Try running that
npm installagain. Everything should install correctly through the
local-npmcache without hitting the network at all
A handy extra that comes with
local-npm's server setup is a simple in-browser UI for browsing local modules and searching for them. You can access this at http://localhost:5080/_browse.
Publishing packages from
local-npm doesn't require any change of registry configuration. Just ensure you're online and everything should work as expected.
As a reminder, in case you need to switch back to the main registry or your preferred mirror for any reason just use
npm set once again:
$ npm set registry https://registry.npmjs.org
local-npm is pretty great, it's not specifically the direction the npm team want to head in for their offline support. Regardless, it's still a pretty solid stop gap for today and I strongly recommend giving it a shot.
Since publishing this post, Nolan has also shared some speed tests comparing
local-npm to regular npm. In short, regular is faster the first time you
npm install a package, but after this
local-npm is consistently faster by an order of 2-3 times.
Using local-npm with other tools
local-npm with Yeoman and Yahoo's generator-fluxible, which only uses npm for package management. As you can see below, everything including dependency installation worked fine and I was able to preview the output from a cached install without issues:
Another popular lazy caching proxy for npm is npm_lazy. Similar to
local-npm, once setup any modules requested that aren't in the local cache are fetched from the npm registry lazily.
npm_lazy, the only things cached are package index metadata and package tarfiles. Pretty much all other endpoints get transparently proxied. This means an
npm install for cached packages will always work and (mostly the same as
local-npm), more exotic endpoints won't work if the main registry or mirrors are down - they'll act as their non-npm_lazy equivalents.
$ npm install -g npm_lazy
To start the server, run:
Similar to local-npm, the registry can be set to the local server running as follows:
$ npm config set registry http://localhost:8080/
npm_lazy has a relatively decent level of configuration, allowing customisation of the lifespan of the cache, maximum retries, HTTP timeouts and whether HTTPS requests are checked against Node's list of certificate authorities.
There have been numerous other packages written that aim to offer a caching proxy for npm (such as Sinopia), but I've tried to keep the list short to avoid choice paralysis.
In this post I've looked at offline package installation, but you may also be interested in offline package bundling. This is where you want to bundle a package and all of its dependencies into a single archive for sharing with a system that might be offline. This comes up a lot in workshops.
There's an open issue on npm to try improving the bundling story around npm pack and bundledDependencies. Until this comes to fruition, I recommend trying out either:
- Freight - can bundle both npm and bower dependencies into a compressed archive
- npmbox - similarly supports creating/installing dependencies from a single archive
- bundle-dependencies - deeply bundles all module dependencies for a package into a monolithic package you would later publish to npm