Migrating from Travis CI to GitHub Actions: A first look

GitHub Actions, the new offering for CI/CD and other automated workflows, is still in beta just out of beta and looks promising.

In order to get myself acquainted with GitHub Actions, I used the nodenv test suite as a chance to try it out. So let’s see what it looks like to migrate a typical test suite from Travis CI to GitHub Actions.

Aside: The nodenv utility is written exclusively in bash shell script, but leverages npm for managing test dependencies, running tasks, versioning, and automating releases. Despite not being written in JavaScript itself, it still represents a typical Node.js library insofar as the CI process is essentially npm ci && npm test.

You can find out more about the core concepts for GitHub Actions, but I’ll summarize here:

• Actions are the building blocks of automation. GitHub provides some basic actions for things like setting up language runtimes and checking out a project’s source code from github, but there are a plethora of actions written by others, and you can write your own.
• Workflows are defined by a workflow file and define your project-specific automations. They are triggered in response to events (like a Push or Pull Request event).
• Workflows define one or more Jobs, which are comprised of a series of steps—each step being an Action or shell command.

With that out of the way, here is the .travis.yml we will be migrating from:

language: node_js
node_js: node
cache: npm

jobs:
  include:
    - stage: test # test with native ext
    - before_script: npm run clean # test without native ext

Creating the workflow

Per the documentation, we need to create a workflow file, so I create .github/workflows/ci.yml. The documentation for actions also includes some starter workflows for reference. We’ll use the Node.js workflow as a starting point. I begin by paring down the starter workflow to some minimal steps we can build upon.

name: Test
on: [push]

jobs:
  test:
    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v1

We could have our workflow triggered by push or pull request, but as I’m just experimenting, I’d like to be able to iterate on this workflow in a branch. To do that, I set the workflow to be triggered by push events so it should run on every push to our workflow-experiment branch. We could do some filtering on specific tags or branches, or even based on the pathnames of the files modified, but for now we’ll let it run on every push.

The checkout action (provided by GitHub) accepts some additional configuration—for instance, to check out a different branch or even a different project—but by default it will check out the sha that triggered the push, which is precisely what we want.

Success!

Aiming for minimal configuration

I notice the documentation states the name property is optional, and it will fall back to using the workflow filename if omitted. Since the name doesn’t really add any value for us, I’d prefer to omit it if we can.

Gotcha #1: If the name is omitted, the fallback is the full filename of the workflow file, including the path. I would have hoped that the filename fallback would have intelligently derived something like “ci”; but it appears they are being quite literal about using the workflow filename!

Since the workflow name doesn’t need to have any bearing on the filename, I’m curious what happens if two different workflows have the same name.

Gotcha #2: Duplicate-named workflows will each be listed regardless of the name collision. This isn’t really a gotcha, as I consider it the desired behavior. But since there isn’t any apparent way to distinguish them in the left-hand menu, it is certainly something to be aware of.

Since our test workflow only has a single job, it would be nice if we didn’t have to nest it under both the jobs mapping and its own test key – especially because this is essentially a thrice-redundant bit of information (the workflow filename, the name property, and the job key).

Gotcha #3: The nesting of a job under the jobs mapping cannot be removed; nor can jobs be a simple sequence of mappings.

Lastly, the on mapping accepts a sequence (array) of events which trigger the workflow. If the workflow is only triggered by a single event, say push, we can omit the array brackets.‍


  name: Test
- on: [push]
+ on: push

We now have what is likely the most minimal workflow configuration. Let’s continue porting from .travis.yml.

Building and running the test suite

The starter Node.js workflow demonstrates setting up the Node.js runtime using GitHub’s setup-node action. This is primarily useful for Node.js projects where one would want to run the test suite across multiple versions of Node.js; or to be explicit about which version of Node.js is required. However, because nodenv is actually bash shell, the tests are written using bats. Nodenv just uses npm as the task runner to invoke bats, so we don’t need to run on multiple versions of Node.js, nor do we care which version.

At this point I realize that these workflow environments already come with Node.js because most actions themselves are written in JavaScript. After some digging in the docs, it appears the current environment ships with Node.js 12, which is fine for this workflow. This means we can skip the setup-node action entirely and just leverage the out-of-the-box version of Node.js. Now we just need to install our dependencies and run the tests:‍

 name: Test
 on: push

 jobs:
   test:
     runs-on: ubuntu-latest

     steps:
     - uses: actions/checkout@v1
+    - run: npm ci
+    - run: npm test

Alright! We have a basic, functioning, single-OS CI workflow!

Let’s see what else we need to do to make our new workflow a complete replacement of the old Travis job.

Caching node_modules for a more efficient workflow

Our Travis setup is leveraging Travis CI’s dependency caching to speed up the npm-install process. GitHub provides the cache action for that.

However, after looking through the docs, it doesn’t appear to have any built-in knowledge about node dependencies. That means we are responsible for explicitly declaring how the cache key is computed and what files should be cached. That’s a bummer, because both of those bits of configuration would be conventional for any given language platform.

Gotcha #4: The first-party caching action requires explicit configuration—it does not infer any conventional, language-specific behavior.

We want to cache the node_modules directory, and the cache key is dependent on both the OS (due to dependencies that leverage native extensions) and package-lock.json.‍


 name: Test
 on: push

 jobs:
   test:
     runs-on: ubuntu-latest

     steps:
     - uses: actions/checkout@v1
+    - uses: actions/cache@v1
+      with:
+        path: node_modules
+        key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}
     - run: npm ci
     - run: npm test

The documentation for the cache action indicates that one of the action’s outputs is a boolean indicating if we have a cache hit or not. Technically, this would allow us to skip the install step entirely on a cache hit. However, if you take a peek at nodenv’s package.json, you’ll notice that we have an install hook script which compiles a native extension at install time. The extension needs to be compiled before the test suite is run, so we still need to run npm install regardless of the cache status. Therefore we can omit the cache-hit conditional logic and run npm ci unconditionally.

Matrix builds

So now we have our project checked out, dependencies installed, native extension compiled, and test suite run. The only remaining bit of configuration from Travis is that before_script.

This Travis config is rather obtuse, so to summarize: it runs two different jobs. One job runs npm test while the other job runs npm run clean && npm test. As such, the test suite is run once with the native extension (which is compiled automatically during npm ci by way of the prepare hook script) and a second time without the extension (which is removed by the clean script).

How do we replicate this in GitHub Actions?

The docs show how to configure a build matrix, to test across multiple operating systems, platforms, versions, or configurations at a time. For our workflow, we want to run the tests both with and without the native extension compiled. To do this, I’ve added two explicitly-named npm scripts:‍


  "build": "src/configure && make -C src",
  "clean": "rm -f libexec/*.dylib",
  "install": "npm run build",
  "test": "bats ${CI:+--tap} test",
+ "test:with_native_ext": "npm run build && npm test",
+ "test:without_native_ext": "npm run clean && npm test",

test:with_native_ext ensures the native extension is built prior to running the tests and test:without_native_ext ensures the native extension is removed prior to running the tests. Now we can add this native extension vector to our build matrix and run the corresponding npm script instead of npm test.‍


+ strategy:
+   matrix:
+     native_ext: [with_native_ext, without_native_ext]

  steps:
  - uses: actions/checkout@v1
  - uses: actions/cache@v1
    with:
      path: node_modules
      key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}
  - run: npm ci
- - run: npm test
+ - run: npm run test:${{ matrix.native_ext }}

Now we see the workflow lists two jobs instead of one. And to distinguish the jobs, GitHub includes the matrix “variable” in parenthesis as part of the job name.

While we’re using the build matrix to generate multiple jobs, let’s run the tests on macOS as well. It is a supported platform of nodenv, after all.‍‍

- runs-on: ubuntu-latest
+ runs-on: ${{ matrix.os }}

  strategy:
    matrix:
+     os: [ubuntu-latest, macOS-latest]
      native_ext: [with_native_ext, without_native_ext]

Gotcha #5: The version suffix on the OS identifier is required (i.e., ubuntu is not a valid OS; ubuntu-latest is). This is marginally disappointing because it means our job names will overflow the space allotted and will be harder to distinguish in the UI. As a workaround to this, you can change the order of the keys to the matrix mapping. It appears that the job name derives the parenthetical in the same order as the keys appear under matrix.

So now we have our test suite running automatically on every git push, on both macOS and Ubuntu, with and without the compiled native extension, and with cached node_modules for a faster build. As you can see, the workflow configuration isn’t quite as brief as the original Travis configuration due to GitHub Actions being less convention-driven.

However, it’s still rather straightforward and pleasant to work with; and perhaps more third-party actions will be developed that leverage convention more heavily. There is also a nice consistency by remaining on github.com when interacting with CI.

All in all, I already consider GitHub Actions a great replacement for trivial Travis CI setups, and I expect it to only get better as it leaves beta. In a later installment we’ll look into running workflows outside of a pull request by using Scheduled Events.