Speed up npm install with a local registry to cache packages

Remember the last time you ran the npm install command, and you had time to grab three coffees, learn Russian and read Dostoievski before you could do anything productive?

During the installation of a node app, the npm install step is doubtless the most time consuming one. Assuming your network is somehow slow it can be endless… It’s inefficient because this command will download packages that you might have already downloaded tens of times for this app or another.

The npm cache already saves approximately 30% of the installation time: when packages are in the npm cache, the initial metadata request sends the cached ETag for the package, and in the vast majority of cases, the registry will return a 304 and the package tarball won’t need to be downloaded again. However, the number of HTTP requests is still the same so there is some time that can still be saved.

Although I do recognize the social gain of taking a coffee break with your teammates, I will tell you how to avoid taking a coffee each time you run npm install thanks to Nexus!

Nexus

Nexus is a a repository manager developed by Sonatype. In other words it can simulate the npm registry on your host. It supports other repositories such as RPM, Maven, Docker Registry…

Install Nexus

The simplest way to use Nexus is to pull the official Docker image.

docker run -d --name nexus-data sonatype/nexus
docker run -d -p 8081:8081 --name nexus --volumes-from nexus-data sonatype/nexus

It can take some time (2-3 minutes) for the service to launch in a new container. You can tail the log to determine once Nexus is ready:

docker logs -f nexus

Otherwise following the official installation guide would take approximately 30 minutes.

Here are the main steps for ubuntu 15.10 (easily adaptable for other OS):

# install Java 8 JRE
sudo add-apt-repository ppa:webupd8team/java
sudo apt-get update
sudo apt-get install oracle-java8-installer

# Download Nexus archive
sudo mkdir -p /opt/sonatype
sudo wget http://download.sonatype.com/nexus/oss/nexus-installer-3.0.0-m6-unix-archive.tar.gz /opt/sonatype
sudo tar xvzf nexus-installer-3.0.0-m6-unix-archive.tar.gz --directory /opt/sonatype && rm nexus-installer-3.0.0-m6-unix-archive.tar.gz
sudo ln -s /opt/sonatype/nexus-3.0.0-b2015110601 /opt/sonatype/nexus

# Launch the Nexus server:
/opt/sonatype/nexus/bin/nexus run

Then you can access the web interface here: http://localhost:8081/

Set up the Nexus server

The next steps are based on Nexus guide for npm.

this video shows how to do the steps below!

1. Log in as ‘admin’ with the default password ‘admin123’ (you should change it as explained in the post install checklist)
2. Create a npmjs proxy:
   • Create a new repository: Parameters > Repositories > Create repository
   • Select “npm proxy” (the “npm hosted” allow to store private packages and is not in the scope of this article)
   • Fill the form (name: “npm-proxy”, remote storage: “https://registry.npmjs.org” check “Use the Nexus truststore”, select “default” for the blobstore) and click on “create repository”
3. Create a repository group
   • Then create a new repository “npm group”
   • I named it “npm” and fill the simple form including the repository you just created

Tell npm to hit the Nexus repository

In ~/.npmrc file, replace the line by:

# add the URL of the repository
registry = http://localhost:8081/content/groups/npm/

Now try to install a package. Then you’ll find it in the list of components in the nexus npm repository. You can disconnect from the Internet, remove your node modules and re-install. Voilà!

Unfortunately the official docker image does not provide the version 3 of Nexus so you cannot browse the npm packages.

Bonus: configure your local repository as a service

To avoid starting the nexus server each time you reboot, you can configure nexus as a service that will start during the boot phase.

If you’re using a docker image, you’ll just have to use the --restart=always option in your run commands.

Otherwise, the Nexus guide explain how to do it but it is not simple and the doc is not up to date (this is actually the reason why I started using Docker).

Alternatives

There are alternatives to Nexus to cache npm packages. Here are two interesting ones:

• npm-cache wraps npm and includes cache utilities but it breaks the standard way to install node apps. Who expects to run npm-cache install instead of npm install to install a node app?
• npm-proxy-cache is a simple node app doing the same job as Nexus but I don’t like the fact it’s not a formal registry (have you used it? Your feedback would be great).

Conclusion

Using Nexus saves approximately 30% of the installation time of packages already in the npm cache.

This is one of the many use cases of Nexus. You can easily set up this tool to store private packages and thus improve your installation process. This tool can be useful for companies with security concerns that do not want to access the Internet during the installation process.

So install Nexus but don’t forget to heavily test your app to run npm test and get another excuse to take a coffee break with your teammates!

[joinus]

[yikes-mailchimp form=“1”]