July 03, 2016

Docker for Gulp Build Tasks

I've found that I like running Gulp inside of a container to both run Node inside of something ephemeral and because Docker doesn't miss file changes (like Virtualbox or NFS file sharing does). See how I do that here!

I run my development environments inside of a Vagrant virtual machine. Most of my projects use a gulp pipeline to build static assets.

Generally, you can decide to run Gulp in one of two places:

  1. On the host machine: Install Node (via NVM or not) on your host machine (perhaps your Mac computer) and run gulp watch from the host machine.
  2. On the guest machine Install Node on the guest (the virtual machine) and run gulp watch from there.

Here are some pros and cons of that. Some of these cons are purely subjective (my personal preference). You can disagree with them, I won't mind.

NodeJS On the Host

The Pro: NodeJS on the host machine doesn't miss file changes. Since the files are actually on the host machine file system (not shared into the VM), inotify and friends are generally working as they should. This means that gulp watch doesn't bat an eye.

The Con: NodeJS is one of those tools I often fight with - either npm install in inexplicably failing, or NodeJS versions aren't matching up, or, even if everything should work, it just refuses to install correctly.

Therefore I HATE having it installed directly on my Mac. NVM doesn't really solves this for me. I instead prefer to install development related software on a virtual machine, as these can be destroyed and re-created very easily. NodeJS, especially, has a fast-moving community. It's always changing.

NodeJS on the Guest

The Pro: NodeJS on the guest let's me worry a LOT less about testing and installing things. VM's are servers that I can destroy and re-create easily. I don't mind experimenting with tools, or throwing multiple versions of binaries around inside of it willy-nilly.

(Personal preference, remember!)

The Con: The default file sharing that Virtualbox uses can easily miss file changes, making gulp watch frustrating to work with.

NFS make this situation better, but is not infallible. It should be noted that I've tried LOTS of various NFS configurations, including using cachefilesd).

Anything else we can try?

Agreed!

Docker

I've recently been getting reacquainted with Docker and it's new tools. I realized I could use it effectively here.

My goals in using Docker to run Gulp build steps were:

  1. Have NodeJS in something super ephemeral - something I wouldn't mind making mistakes on.
  2. Have gulp watch work more reliably.

The following setup does both of these things! Note that point 2 works because Docker's volume mounting is much better than either NFS or Virtualhost's default file share.

This is possible on my Mac because of the Docker app (in beta), which is solving the sticky bits of running Docker on Mac's kernel. It no longer runs in a Virtualbox VM thanks to magic.

Let's see how Docker helped me here:

Dockerfile

First, we'll create new Docker image that has NodeJS in it. I install Node 5, since that's what my project happened to expect. Node 6 is the latest and might be what you want instead.

FROM ubuntu:16.04

MAINTAINER Chris Fidao

WORKDIR /opt

ADD setup_5.x /tmp/setup_5.x
RUN bash /tmp/setup_5.x

RUN apt-get update
RUN apt-get install -y build-essential
RUN apt-get install -y nodejs
RUN /usr/bin/npm install -g gulp
RUN /usr/bin/npm install -g bower

VOLUME ["/opt"]
CMD ["gulp", "watch"]

All we do here is grab the latest Ubuntu LTS and install NodeJS. I then globally install Gulp and Bower. This is taken straight out of the Homestead install script.

Three things to note:

  1. I add a file called setup_5.x. This is directly from the Nodesource Node 5.x installer, which I downloaded and saved as a file first.
  2. The working directory is set to /opt, so any commands run will be relative to that directory
  3. Using CMD ["gulp", "watch"] means that if I don't specify any other command, this container will run gulp watch. I can, however, specify any other command I want. And I will! You'll see why.

Once this Dockerfile is saved, we can build a new image. I'll name (tag) it simply gulp:

$ cd /path/to/dir/with/Dockerfile

$ docker build -t gulp .
# ...it builds...

$ docker images
REPOSITORY    TAG       IMAGE ID        CREATED         SIZE
gulp          latest    59fe57f1d14a    17 hours ago    460.6 MB
ubuntu        16.04     2fa927b5cdd3    5 weeks ago     122 MB

Building node_modules

There's two steps to actually using this:

  1. Use the container to build the node_modules directory
  2. Use the container to run gulp watch

Let's first assume your project has not had npm install run yet. (I backed up and deleted my old node_modules directory).

# Get old node_modules dir out of the way
$ cd ~/Sites/some-project
$ cp node_modules ~/some-project-node_modules
$ rm -rf node_modules # everyone's familiar with this command

# Use our container to build a new node_modules dir
$ docker run --rm -v ~/Sites/some-project:/opt gulp npm install

Let's cover what's happening there:

  1. docker run - run a container
  2. --rm - Delete the container when it exits. We don't need this container to persist.
  3. -v ~/Sites/some-project:/opt - Share our host computer's some-project dir with the /opt dir, which we set as the working directory of the image (and will continue to be the working directory when we use this image to make a new container)
  4. gulp - specify the image from which the container should be created
  5. npm install - The default CMD set was gulp watch, but we're over-riding it in this case so we can have it install our npm dependencies first. This way, anything specific to the container's node version and architecture (Node 5.x, Ubuntu 16.04 x64) will be created correctly.

Running "Gulp Watch"

Once that's done, a new node_modules directory will exist! We're ready to run gulp watch now:

$ docker run --rm -v ~/Sites/some-project:/opt gulp

This is almost the exact same command, except we didn't give the container a command to run. Since we defined CMD ["gulp", "watch"] in the Dockerfile, the gulp watch command will be run by default.

Don't get the fact that I named the image gulp confused with the commands we're running in that container. The container first ran npm install, and our second container ran gulp watch.

I use --rm on both containers since the containers don't need to persist - the results of their actions are saved on our host computer thanks to the volume mounting.

When you're done with that container, you can open a new shell and run docker ps to get the docker ID or name, and then run docker stop <container-id-or-name> to stop it.

All Topics