Deploy images to distinct environments

Problem

You have a project running live on the Google Cloud Platform, but only one environment. It´s a simple stack organized with docker-compose.

Suddenly you need to create a new environment to separate Live and development or even Live, QA and development.

Solution

Requirements

1. You build your Docker images with GCP Cloudbuild (or manually).
2. The docker images are in GCP Container Registry Repository.
3. GitHub holds your source code repositories and is connected to GCP.
4. The project has a frontend and a backend repository.
5. Nginx serves as a simple application gateway.
6. You have one main repository per environment (e.g. staging, pre-production, master)

Implementation

Concepts

Tag Images with a tag

To be able to use this Solution, a Git commit needs to be related to a Docker image.
To achieve this we use the internal variables from cloud build to use the Git commit hash as the Docker Tag on the Image that was created.

Run specific Images per environment

By not specifying the tag or specifying the latest tag, there will be Images shared between environments. This is not what you want.

Therefore on each deployment, you need to specify the Tag of the Docker image that was created.

Templated Docker-Compose

Docker always uses the latest image if no Tag is specified or the image with the Tag you request. This means a config is absolute.

Docker doesn´t really allow variables in the docker-compose file. To still use variables e.g. for the Tag, you need to add a bit of scripting and console magic.

Then you can have a config with variables in your GitHub repository, but on each deployment, the variables are replaced with real values thanks to the console magic.

Automate deployment

You don´t want information silos/dependencies on a specific person just for deployments. You and your colleagues won´t like it and the person who you depend on also won´t like it.

For that goal, GCP Cloudbuild is used and a simple way to do a deployment is provided.

Files

docker-compose.template.yml

This is the upper template file. It is separated from the bottom to not have to use more complex Logic.

Look at https://github.com/thestrugglingdeveloper/templateddockercompose/blob/master/docker-compose.template.yml.

Do the following replacements:

1. Replace asia.gcr.io with the domain for your zone
2. Replace ${projectId} with the name of your project
3. Replace ${gcp-container-registry-name} with the name of the repository use used in GCP Container Registry (we assume you have the same name plus suffix -frontend/-backend)

$BACKENDTAG and $FRONTENDTAG will be replaced by our script rewrite.sh.

docker-compose.lower_template.yml

This is the lower part of the template file. In this setup, it has the network definition and the Nginx configuration.

Do the following replacements:

${path_to_compiled_code} with the path to the compiled code (often the dist folder).

rewrite.sh

This script uses the previous version, updates front- or backend version, and writes a new docker-compose.yml.

Line 36 does the actual replacing magic with the help of the eval tool.

It expects a TAG env var for the new tag of the image and the only parameter frontend or backend .

So a call looks like TAG=a1b2c3 bash rewrite.sh frontend .

You won´t use the script directly, it will be called in the deployment.

deploy.sh

https://github.com/thestrugglingdeveloper/templateddockercompose/blob/master/deploy.sh

This script expects the same call as the rewrite.sh. The difference is that the previous script just updates the docker-compose.yml.

Deploy.sh handles:

1. pulling the new image
2. stopping the previous stack with the old images
3. starting the stack with the new image
4. deleting the old image

cloudbuild.yml

Last but not least the cloudbuild.yml.

Do the following replacements:

1. ${gcp-container-registry-name} with the name you use on Container registry
2. ${user} with the user who has the code in his path.
3. ${vm} with the VM name of the project you want to deploy to
4. ${zone} with the zone where the VM is located
5. $path with the path the folder where the files are located on the server.
6. In the line 26 replace only backend if it is a different image. E.g. the frontend.

Do not (!) change $SHORT_SHA . This is an internal variable from GCP Cloudbuild. It is required for:

1. Tag Images with a tag
2. Run specific Images per environment

nginx/config.conf

This file configures how Nginx works as a Gateway. It sets various timeouts to one day, instead of 1 minutes.

It does not configure SSL (via CloudFlare or other).

Do these replacements:

1. ${server_name} with the DNS domain name you want Nginx to server
2. ${backend_path} with the path that the backend expects

Explanation

Configure Cloudbuild to react as you need it and use these files as per your requirements.

For example: If you have multiple microservices, extend the 2 scripts and the docker-compose.template.yml.

Let me know your questions.

Best,

Frank