Skip to main content

Podman

info

This method is supported by R2DevOps community.

This page describes how to setup a self-managed instance of R2Devops using podman.

💻 Requirements​

The system requires a Linux server. It runs in 🕸 podman containers using a yaml configuration. Specifications:

  • OS: Ubuntu or Debian

  • Hardware

    • CPU x86_64/amd64 with at least 2 cores
    • 4 GB RAM
    • 30 GB of storage for R2Devops

  • Network

    • Users must be able to reach the R2Devops server on TCP ports 80 and 443
    • The R2Devops server must be able to access internet
    • The R2Devops server must be able to communicate with GitLab instance
    • The installation process required to have a write access to the DNS Zone to setup R2Devops domain
    • If the server is not reachable from internet or if you want to use your own certificate for HTTPS, you need to be able to generate certificate during the installation process for R2Devops domain

  • Installed software

    • Git
    • podman- Docker hub registry must be resolved by podman in file /etc/containers/registries.conf
/etc/containers/registries.conf
unqualified-search-registries = ["docker.io"]

🛠️ Installation​

📥 Setup your environment​

  1. Clone the repository on your server 
    git clone https://github.com/r2devops/self-managed.git r2devops
    cd r2devops
  2. Create your configuration files 
    cp .env.example .env
    cp .docker/r2devops/config.json.example .docker/r2devops/config.json

📋 Organization​

In your .env file:

  1. Add your organization

    • If you use a SaaS version of GitLab (like gitlab.com): add the path of your organization top-level group in ORGANIZATION variable
    .env
    ORGANIZATION="<top-level-group-name>"
    • Else, let the ORGANIZATION variable empty
    .env
    ORGANIZATION=""

  2. (Optional) Add your license

    License key

    If you do not have a license key, you can let the variable LICENSE empty (value: ""). Your R2Devops instance will be limited to 5 projects.

    Add your license key (provided by R2Devops): edit the .env file by updating value of LICENSE variable:

    .env
    LICENSE="<license-key>"

📄 Domain name​

  1. Edit the .env file by updating value of DOMAIN_NAME, CERTIFICATE_EMAIL and JOBS_GITLAB_URL variables

    .env
    DOMAIN_NAME="r2devops.<domain_name>"
    CERTIFICATE_EMAIL="<your_email>"
    JOBS_GITLAB_URL="https://<url_of_your_gitlab_instance>"
    Example with domain name 'mydomain.com'
    DOMAIN_NAME="r2devops.mydomain.com"
    CERTIFICATE_EMAIL="tech@mydomain.com"
    JOBS_GITLAB_URL="https://gitlab.mydomain.com"

  2. Edit the .docker/r2devops/config.json file by updating apiUrl, apiUrlIdentities and gitLabApiUrl parameters

    warning

    Set allowExternalQueries to false if you want to prevent R2Devops from initiating queries to sources other than backend and GitLab.

    {
        "appTitle": "R2Devops",
        "apiUrl": "https://r2devops.<domain_name>/api",
        "gitLabApiUrl": "https://<gitlab_intance_domain>",
        "selfHosted": true,
        "docUrl": "https://docs.r2devops.io",
        "debug": false,
        "allowExternalQueries": true
    }
    Example with domain name 'mydomain.com'
    "apiUrl": "https://r2devops.mydomain.com/api",
    "gitLabApiUrl": "https://gitlab.mydomain.com",

  3. Create DNS record

    • Name: r2devops.<domain_name>
    • Type: A
    • Content: <your-server-public-ip>
Certificate

A certificate will be auto-generated using Let's encrypt at the application launch

🦊 GitLab OIDC​

R2Devops uses GitLab as an OAuth2 provider to authenticate users. Let's see how to connect it to your GitLab instance.

Create an application​

Choose a group on your GitLab instance to create an application. It can be any group. Open the chosen group in GitLab interface and navigate through Settings > Applications:

Profile_Menu

Then, create an application with the following information :

  • Name: R2Devops self-managed
  • Redirect URI : https://r2devops.<domain_name>/api/auth/gitlab/callback
  • Confidential: true (let the box checked)
  • Scopes: api

Click on Save Application and you should see the following screen:

Application

Update the configuration​

In .env file:

  1. Copy/paste the Application ID and the Secret from the application you just created
    .env
    GITLAB_OAUTH2_CLIENT_ID="<application-id>"
    GITLAB_OAUTH2_CLIENT_SECRET="<application-secret>"

🔐 Generate secrets​

Generate random secrets for all components:

sed -i "s/REPLACE_ME_BY_SECRET_KEY/$(openssl rand -hex 32)/g" .env
sed -i "s/REPLACE_ME_BY_JOBS_DB_PASSWORD/$(openssl rand -hex 16)/g" .env
sed -i "s/REPLACE_ME_BY_JOBS_REDIS_PASSWORD/$(openssl rand -hex 16)/g" .env
sed -i "s/REPLACE_ME_BY_S3_SECRET_KEY/$(openssl rand -hex 16)/g" .env

📋 (Optional) Add your custom CA​

If your GitLab instance is using a TLS certificate signed with your own Certificate authority (CA):

TEST-AND-EDIT

  • Add the CA certificate file in podman ?

📄 Prepare podman for launch​

Generate podman network:

podman network create intranet

Generate podman socket

systemctl --user start podman.socket
systemctl --user enable podman.socket

If you encounter this error Failed to connect to bus: No medium found use these commands with your user as sudoer:

sudo loginctl enable-linger <your_local_user>
sudo systemctl --user -M <your_local_user>@ start podman.socket
sudo systemctl --user -M <your_local_user>@ enable podman.socket

Generate podman config files:

set -a; source .env; set +a
export uid=$(id -u)
envsubst < podman.yml.example > podman.yml
envsubst < configmap.yml.example > configmap.yml

Allow port 80 and above in system for local user:

  1. Add this line to /etc/sysctl.conf file as sudo user or root: 
    net.ipv4.ip_unprivileged_port_start=80
  2. Restart sysctl 
    sudo systemctl restart systemd-sysctl

🚀 Launch the application​

Congratulations

You have successfully installed R2Devops on your server 🎉

Now you can launch the application and ensure everything works as expected.

Run the following command to start the system:

podman play kube podman.yml --configmap configmap.yml --network intranet
Reconfigure

If you need to reconfigure some files and relaunch the application, after your updates you can simply run the command again to do so.

podman play kube podman.yml --replace --configmap configmap.yml --network intranet
Not the same behavior

Did you encounter a problem during the installation process ? See the troubleshooting section.

⏫ Update​

Follow these steps to update your self-managed instance to a new version:

  1. Navigate to the location of your self-managed git repository
  2. Update it 
    git pull
  3. Open the .env.example file and copy the values of FRONTEND_IMAGE_TAG and BACKEND_IMAGE_TAG variables
  4. Edit the .env file by updating values of FRONTEND_IMAGE_TAG and BACKEND_IMAGE_TAG variables with the values previously copied
    .env
    FRONTEND_IMAGE_TAG="<new frontend version>"
    BACKEND_IMAGE_TAG="<new backend version>"
  5. Restart your containers 
    set -a; source .env; set +a
    export uid=$(id -u)
    envsubst < podman.local.yml.example > podman.yml
    envsubst < configmap.local.yml.example > configmap.yml
  6. You have successfully updated R2Devops on your server 🎉

🔄 Backup and restore​

Data required to fully backup and restore a R2Devops system are the following:

  • Configuration file: .env
  • Databases:
    • PostgreSQL database of Jobs service
  • Files data:
    • Files stored in the Minio service
    • File storing data about certificate for Traefik service

All these data can be easily backup and restored using 2 scripts from the installation git repository:

  • backup_podman.sh
  • restore_podman.sh

💽 Backup​

To backup the system, go to your installation git repository and run the following commands:

cp scripts/*_podman.sh .
./backup_podman.sh <postgres image version>

The script will create a backups directory and create a backup archive inside it prefixed with the date (backup_r2-$DATE)

Regular backup

You can use a cron job to perform regular backups. Here is a cron job that launch a backup every day at 2am:

0 2 * * * /r2devops/backup_podman.sh <postgres image version>

It can be added to your crontab with the command crontab -e. Check more information about cron jobs here.

🛳️ Restore​

To restore a backup from scratch on a new system, follow this process:

  1. Be sure that your new system is compliant with requirements
  2. Copy the backup file on your new server
  3. Clone the installation repository 
    git clone https://gitlab.com/r2devops/self-managed.git r2devops
    cd r2devops
    cp scripts/*_podman.sh .
  4. If the IP address of your server changed from your previous installation, update your DNS records. See section 2 of domain configuration
  5. Launch the restore script 
    ./restore_podman.sh <postgres image version> <path_to_your_backup_file>
Any errors during the restore process ?

Did you encounter a problem during the restore process ? See the troubleshooting section.