GitLab provides a continuous integration service, which is pretty nice for building, testing, and packaging your software and having all the UI integrated in GitLab. If you’re just using the free GitLab.com hosting, you get to utilise their Docker-based runner. (If your build process requires a non-Linux OS you’ll have to provide your own runner.)

Getting a basic build up and running is pretty simple. For example, here’s one job named test that only runs make check:

# .gitlab-ci.yml
test:
  script:
    - make check

If your test suite can measure code coverage, GitLab can also show it in the UI. At the moment this feature is rather rudimentary and requires you to go to the project settings and enter a regular expression to find the coverage amount in the build output.

The following is an example that works with coverage.py when you only have a single Python file. I haven’t tried it with multiple files; it may require a wrapper script that calculates the total coverage amount.

# .gitlab-ci.yml
test:
  image: python:3-alpine
  script:
    - pip install coverage
    - coverage run foo.py
    - coverage report -m
# Regex that matches the coverage amount:
# ^\S+\.py\s+\d+\s+\d+\s+(\d+\%)

A few lessons learnt from setting up test hooks for a small Python app:

• There is no way to test changes to your build script without pushing a commit. And then the build results will stay in your project page forever with no way to clean up the irrelevant ones. You can run builds locally with gitlab-runner, e.g. gitlab-runner exec shell test to run the test job on the local shell (replace shell with docker to use Docker). (Thanks Evan Felix for the info about gitlab-runner.)

• GitLab.com’s default Docker image (ruby:2.1 at the time of writing) is really fast to spin up, possibly because it’s cached. However, you should still explicitly name a Docker image in case the default changes.

• Installing packages is slower than downloading a Docker image. It’s not worth going out of your way to use the default image if you then have to call apt-get. See if your compiler has an official Docker image that has all the packages you need (please don’t run Docker images by random people). That said, we’re talking about differences of about ten seconds, so just choose the method that is most convenient.

• The ruby:2.1 image has Python 2 but not Python 3.

• The official Python repository on Docker Hub lists a number of Alpine-based images. As you would expect, these are smaller and slightly faster to download than the other (Debian-based) images.

• The coverage regex requires the percent sign to be escaped (\%).