Gitlab CI Docker Build

Easy building docker images with Gitlab CI

Posted by Kai Pazdzewicz on 12 December 2019

Intro

Gitlab.com is a free cloud or selfhosted git solution with a great CI. I use this CI to build my dockerfiles, to test my applications, etc. Pretty much what I want to execute after I wrote my code.

In this article we will build a simple docker image based on redis-server, lint this Dockerfile and build it in our ci. Also we are using the Gitlab Docker Registry to store our images.

This CI script will create docker images tagged after the commit SHA and also tag the master branch with latest. I use this commit SHA to deploy my applications to kubernetes.

Dockerfile

So we use the following Dockerfile to install redis-server on debian buster and build our own configuration file into it.

Dockerfile:

FROM debian:buster

EXPOSE 6379

RUN apt update -y && \
    apt install -y gnupg apt-transport-https curl ca-certificates software-properties-common lsb-release && \
    apt clean

RUN curl https://www.dotdeb.org/dotdeb.gpg | apt-key add - && \
    echo "deb http://ftp.hosteurope.de/mirror/packages.dotdeb.org/ stable all" > /etc/apt/sources.list.d/dotdeb.list && \
    apt-get update -y && \
    apt-get install -y redis-server

COPY redis.conf /etc/redis/redis.conf

CMD ["/usr/bin/redis-server", "/etc/redis/redis.conf"]

Redis Configuration

To configure our redis we use this example configuration.

redis.conf:

# REDIS CONFIGURATION
# Binde Redis auf 0.0.0.0:6379
bind 0.0.0.0
port 6379

# Standard Einstellungen, aka PID File, etc
pidfile "/var/run/redis/redis-server.pid"
timeout 0
daemonize no
supervised systemd
dir "/var/lib/redis"

# Anzahl maximaler Datenbanken
databases 16

# Logging für Incidents, etc
loglevel notice
logfile ""
slowlog-log-slower-than 10000
slowlog-max-len 128

# Erlaube keine DEBUG Kommandos
protected-mode yes

# LRU Cache Einstellungen
maxmemory 2gb
maxmemory-policy allkeys-lru
maxmemory-samples 5


# Performance Optimierung
tcp-backlog 65536
tcp-keepalive 60
list-max-ziplist-entries 512
list-max-ziplist-value 64
list-max-ziplist-size -2
zset-max-ziplist-entries 128
zset-max-ziplist-value 64
set-max-intset-entries 512
list-compress-depth 0

# Persistenz
# Append only file
appendonly yes
appendfilename "appendonly.aof"
appendfsync everysec
no-appendfsync-on-rewrite no
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
aof-rewrite-incremental-fsync yes
aof-load-truncated yes
# RDB file für regelmäßige Snapshots
save 900 1
save 300 10
save 60 10000
stop-writes-on-bgsave-error yes
rdbcompression yes
rdbchecksum yes
dbfilename "dump.rdb"

# Replikation
slave-serve-stale-data yes
slave-read-only yes
replica-serve-stale-data yes
replica-read-only yes
repl-diskless-sync no
repl-diskless-sync-delay 5
repl-disable-tcp-nodelay no
replica-priority 100
client-output-buffer-limit normal 0 0 0
client-output-buffer-limit replica 256mb 64mb 60
client-output-buffer-limit pubsub 32mb 8mb 60

# Redis LUA Funktionen
lua-time-limit 5000

# MISC Einstellungen
latency-monitor-threshold 0
notify-keyspace-events ""
hll-sparse-max-bytes 3000
activerehashing yes
hz 10
dynamic-hz yes

# DYNAMIC REDIS CONFIGURATION

Gitlab-CI

So this part is the main part of this document.

We want to build three stages, where the last stage is only run on the master branch. This will allow testing & building on other branches without touching the latest tag, what you then use in your compose files.

The first stage is linting with hadolint. Pretty simple but will ensure that our Dockerfile is best practice.

The second stage is building the dockerfile, tagging the image with the current commit hash and pushing it into the registry.

The third stage is run only on the master branch and will rebuild the image with a cache from the second stage. This will speed up the build process. And then tag the immage with the latest tag and push it to the registry.

.gitlab-ci.yml:

variables:
  DOCKER_HOST: tcp://docker:2375
  DOCKER_DRIVER: overlay2
  CONTAINER_IMAGE: registry.gitlab.com/<gitlab-username>/$CI_PROJECT_NAME

image: docker:latest

stages:
  - lint
  - build
  - build_latest

####
# Lint Dockerfile
####

redis_lint:
  stage: lint
  image: hadolint/hadolint:latest-debian
  script:
    - hadolint Dockerfile
  allow_failure: true

####
# Build Dockerfile
####

redis_build_and_push:
  stage: build
  image: docker:latest
  services:
    - docker:dind
  script:
    - docker login -u gitlab-ci-token -p ${CI_BUILD_TOKEN} registry.gitlab.com
    - docker build -t ${CONTAINER_IMAGE}:${CI_COMMIT_SHA} ./
    - docker push ${CONTAINER_IMAGE}:${CI_COMMIT_SHA}

####
# retag to LATEST (only if master)
####

redis_build_and_push_latest:
  stage: build_latest
  services:
    - docker:dind
  script:
    - docker login -u gitlab-ci-token -p ${CI_BUILD_TOKEN} registry.gitlab.com
    - docker pull ${CONTAINER_IMAGE}:${CI_COMMIT_SHA}
    - docker build --cache-from ${CONTAINER_IMAGE}:${CI_COMMIT_SHA} -t ${CONTAINER_IMAGE}:latest ./
    - docker push ${CONTAINER_IMAGE}:latest
  only:
    - master

Now we just need to commit our changes to the repository and push it to Gitlab.com.