Docker-compose

This page describes how to setup a self-managed instance of R2Deovps using Docker-compose.

💻 Requirements

The system is requiring a Linux server. It runs in 🐳 Docker containers using a docker-compose configuration. Specifications:

• OS: Ubuntu or Debian
• Hardware
  • CPU x86_64/amd64 with at least 2 cores
  • 4 GB RAM
  • 30 GB of storage for R2Devops
• Network
  • Public IP
  • Ingress access on TCP ports 80, 443 and 22
  • Egress access on all ports
• Installed software
  • Git
  • Docker
  • Note: both compose plugin (docker compose) and docker-compose CLI are working. The first one is installed by default with Docker

🛠️ Installation

📥 Setup your environment

1. Set secret variable provided by R2Devops in your shell

   export REGISTRY_TOKEN="REPLACE_ME"
   

2. Clone the repository on your server

   git clone https://github.com/r2devops/self-managed.git r2devops
   cd r2devops
   

3. Create your configuration files

   cp .env.example .env
   cp .docker/r2devops/config.json.example .docker/r2devops/config.json
   

4. Add your license key (provided by R2Devops): edit the .env file by updating value of PROJECTS_LIMIT_KEY variable

   License key

   If you do not have a license-key, you can let the variable PROJECTS_LIMIT_KEY empty (value: ""). Your R2Devops instance will be limited to 10 projects.

   .env

   PROJECTS_LIMIT_KEY="<license-key>"
   

📄 Domain name

1. Edit the .env file by updating value of DOMAIN_NAME, CERTIFICATE_EMAIL and JOBS_GITLAB_URL variables

   .env

   DOMAIN_NAME="r2devops.<domain_name>"
   CERTIFICATE_EMAIL="<your_email>"
   JOBS_GITLAB_URL="<url_of_your_gitlab_instance>"
   

   Example with domain name mydomain.com

   DOMAIN_NAME="r2devops.mydomain.com"
   CERTIFICATE_EMAIL="tech@mydomain.com"
   JOBS_GITLAB_URL="gitlab.mydomain.com"
   

2. Edit the .docker/r2devops/config.json file by updating apiUrl, apiUrlIdentities and gitLabApiUrl parameters

   {
       "appTitle": "R2Devops",
       "apiUrl": "https://api.r2devops.<domain_name>",
       "apiUrlIdentities": "https://api.r2devops.<domain_name>/kratos/public",
       "gitLabApiUrl": "https://<gitlab_intance_domain>",
       "selfHosted": true,
       "docUrl": "https://docs.r2devops.io"
   }
   

   Example with domain name mydomain.com

   "apiUrl": "https://api.r2devops.mydomain.com",
   "apiUrlIdentities": "https://api.r2devops.mydomain.com/kratos/public",
   "gitLabApiUrl": "https://gitlab.mydomain.com",
   

3. Create DNS records

   The application needs two DNS records to work properly:

   1. Frontend
      • Name: r2devops.<domain_name>
      • Type: A
      • Content: <your-server-public-ip>
   2. Backend
      • Name: api.r2devops.<domain_name>
      • Type: A
      • Content: <your-server-public-ip>

Domains & Certificates

For example, if you have the domain name mydomain.com:

• The frontend URL will be https://r2devops.mydomain.com
• The backend URL will be https://api.r2devops.mydomain.com

All certificates will be auto-generated using Let's encrypt at the application launch

🦊 GitLab OIDC

R2Devops uses GitLab as an OIDC (OpenID Connect) provider to authenticate users. Let's see how to connect it to your GitLab instance.

Create an application

Choose a group on your GitLab instance to create an application. It can be any group. Open the chosen group in GitLab interface and navigate through Settings > Applications:

Then, create an application with the following information :

Warning

You need to replace <API_URL> below with the API URL consistently with what you have configured as backend DNS record and what you have in your .env file

• Name: R2Devops self-managed
• Redirect URI : https://<API_URL>/kratos/public/self-service/methods/oidc/callback/gitlab
• Confidential: true (let the box checked)
• Scopes: openid, email

Click on Save Application and you should see the following screen:

Update the configuration

In .env file, in the OIDC section:

1. Copy/paste the Application ID and the Secret from the application you just created
2. Update the GITLAB_INSTANCE_DOMAIN value with the domain of your GitLab self-managed instance

.env

# OIDC
GITLAB_OIDC='
[
  {
    "id": "gitlab",
    "provider": "gitlab",
    "issuer_url": "<GITLAB_INSTANCE_DOMAIN>",
    "client_id": "<APPLICATION_ID>",
    "client_secret": "<APPLICATION_SECRET>",
    "mapper_url": "file:///etc/config/kratos/oidc.gitlab.jsonnet",
    "scope": [
      "openid",
      "email"
    ]
  }
]
'

Example with domain name mydomain.com

.env

# OIDC
GITLAB_OIDC='
[
  {
    "id": "gitlab",
    "provider": "gitlab",
    "issuer_url": "https://gitlab.mydomain.com",
    "client_id": "d6678248ebba5f5a6a0c5730dbbf67e8c0943e134f45151af73f96a53c47e6d2",
    "client_secret": "89e1bd86c0d65f2f5a7898af9d02f5efce85088f0e835cc0d26c6a8cf119ae6a",
    "mapper_url": "file:///etc/config/kratos/oidc.gitlab.jsonnet",
    "scope": [
      "openid",
      "email"
    ]
  }
]
'

🔐 Generate secrets

Generate random secrets for all components:

sed -i "s/REPLACE_ME_BY_JOBS_DB_PASSWORD/$(openssl rand -hex 16)/g" .env
sed -i "s/REPLACE_ME_BY_JOBS_REDIS_PASSWORD/$(openssl rand -hex 16)/g" .env
sed -i "s/REPLACE_ME_BY_S3_SECRET_KEY/$(openssl rand -hex 16)/g" .env
sed -i "s/REPLACE_ME_BY_KRATOS_DB_PASSWORD/$(openssl rand -hex 16)/g" .env
sed -i "s/REPLACE_ME_BY_SECRET_COOKIE/$(openssl rand -hex 16)/g" .env
sed -i "s/REPLACE_ME_BY_SECRET_CIPHER/$(openssl rand -hex 16)/g" .env
sed -i "s/REPLACE_ME_BY_SECRET_DEFAULT/$(openssl rand -hex 16)/g" .env

🐳 Docker login

Login to R2Devops registry

echo $REGISTRY_TOKEN | docker login --username r2devops --password-stdin https://registry.gitlab.com/v2/r2devops

🚀 Launch the application

Congratulations

You have successfully installed R2Devops on your server 🎉

Now you can launch the application and ensure everything works as expected.

Run the following command to start the system

docker compose up -d

Reconfigure

If you need to reconfigure some files and relaunch the application, after your updates you can simply run the command again to do so.

docker compose up -d

What's next

Now that you have finished this tutorial, here are some simple tasks you should give a try :

• 📈 Learn how to use the platform by reading the documentation
• 📕 Import your first job, here is the tutorial

Not the same behavior

Did you encounter a problem during the installation process ? See the troubleshooting section.

🔄 Backup and restore

Data required to fully backup and restore a R2Devops system are the following:

• Configuration file: .env
• Databases:
  • PostgreSQL database of Jobs service
  • PostgreSQL database of Kratos service
• Files data:
  • Files stored in the Minio service
  • File storing data about certificate for Traefik service

All these data can be easily backup and restored using 2 scripts from the installation git repository:

• backup.sh
• restore.sh

💽 Backup

To backup the system, go to your installation git repository and run the following command:

./backup.sh

The script will create a backups directory and create a backup archive inside it prefixed with the date (backup_r2-$DATE)

Regular backup

You can use a cron job to perform regular backups. Here is a cron job that launch a backup every day at 2am:

0 2 * * * /r2devops/backup.sh

It can be added to your crontab with the command crontab -e. Check more information about cron jobs here.

🛳️ Restore

To restore a backup from scratch on a new system, follow this process:

1. Be sure that your new system is compliant with requirements
2. Copy the backup file on your new server
3. Setup your environment

   export REGISTRY_TOKEN="REPLACE_ME"
   

4. Clone the installation repository

   git clone https://gitlab.com/r2devops/self-managed.git r2devops
   cd r2devops/docker-compose
   

5. If the IP address of your server changed from your previous installation, update your DNS records. See section 2 of domain configuration
6. Login to R2Devops registry

   echo $REGISTRY_TOKEN | docker login --username r2devops --password-stdin https://registry.gitlab.com/v2/r2devops
   

7. Launch the restore script

   ./restore.sh <path_to_your_backup_file>
   

Any errors during the restore process ?

Did you encounter a problem during the restore process ? See the troubleshooting section.