Skip to content

Kubernetes

This page describes how to run a self-managed instance of R2Devops on Kubernetes.

💻 Requirements

  • A Kubernetes cluster with:
    • One ingress controller(ex: Nginx or Traefik)
    • A certificate manager with a ACME provider: cert-manager
    • Only if you run external services (PostgreSQL, Redis, MinIO) in Kubernetes: the ability to provision persistent volumes in your cluster
  • Your local environment with CLI to interact with Kubernetes API:
  • Write access to the DNS zone of the domain to use with R2Devops
  • A user account on the GitLab instance

🛠️ Installation

The Helm chart used in this documentation allows installing all these services embedded in the chart as dependencies or to use external S3, PostgreSQL and/or Redis. Both alternatives are detailed below.

📥 Initialize your cluster

  1. Choose a namespace name and set it in a variable
    export R2DEVOPS_NS="r2devops"
    
  2. Create the namespace
    kubectl create ns $R2DEVOPS_NS
    
  3. Add R2Devops repo
    helm repo add r2devops https://charts.r2devops.io/
    

📄 Domain name

Info

You need a domain to run R2Devops. For example, if you have the domain name mydomain.com:

  • R2Devops URL will be https://r2devops.mydomain.com
  1. Create DNS record

    • Name: r2devops.<domain_name>
    • Type: A
    • Content: <your-cluster-public-ip>

🦊 GitLab OIDC

R2Devops uses GitLab as an OAuth2 provider to authenticate users. Let's see how to connect it to your GitLab instance.

  1. Choose a group on your GitLab instance to create an application. It can be any group. Open the chosen group in GitLab interface and navigate through Settings > Applications:

    Profile_Menu

  2. Then, create an application with the following information

    • Name: R2Devops self-managed
    • Redirect URI : https://r2devops.<domain_name>/api/auth/gitlab/callback
    • Confidential: true (let the box checked)
    • Scopes: api
  3. Click on Save Application and you should see the following screen:

    Application

  4. Store Application ID and Secret somewhere safe, we will need to use them in next step

⚙️ Configure values

This section describes how to configure your custom values file. The default values.yaml is available here. Two examples are available at the end of this documentation:

  1. Using only services embedded in the chart
  2. Using only external services

Note

For the following sections, we assume that your custom value file will be located in your current directory and be named custom_values.yaml

🔐 Secrets

This section is optional. You need to follow this section only if you want to store secrets values as kubernetes secrets instead of writing them in your custom value file.

  1. R2Devops secret

    Replace all occurrences of REDACTED by your R2Devops secrets encoded in base64 and create following secret:

    • license-key: your R2Devops license
    • secret-key: 256 bit secret key used to encrypt sensitive data (openssl rand -hex 32)
    • gitlab-oauth2-client-id: Application ID of the GitLab application
    • gitlab-oauth2-client-secret: Secret of the GitLab application
    apiVersion: v1
    kind: Secret
    metadata:
      name: r2devops-secret
      namespace: r2devops
    type: Opaque
    data:
      license-key: REDACTED
      secret-key: REDACTED
      gitlab-oauth2-client-id: REDACTED
      gitlab-oauth2-client-secret: REDACTED
    
  2. MinIO secret

    Replace both REDACTED values by your S3 storage key and secret encoded in base64. If you want to use minIO embedded in this chart, choose the values.

    apiVersion: v1
    kind: Secret
    metadata:
      name: minio-secret
      namespace: r2devops
    type: Opaque
    data:
      root-user: REDACTED
      root-password: REDACTED
    

  3. PostgreSQL secret

    Replace REDACTED by your postgres password encoded in base64. If you want to use postgres embedded in this chart, choose the value.

    apiVersion: v1
    kind: Secret
    metadata:
      name: postgresql-secret
      namespace: r2devops
    type: Opaque
    data:
      password: REDACTED
    

  4. Redis secret

    Replace REDACTED by your redis password encoded in base64. If you want to use redis embedded in this chart, choose the value.

    apiVersion: v1
    kind: Secret
    metadata:
      name: redis-secret
      namespace: r2devops
    type: Opaque
    data:
      password: REDACTED
    

🤖 R2Devops

Add R2Devops related configuration in your new values file custom_values.yaml:

  1. Add license key (provided by R2Devops) and R2Devops domain

    License key

    If you do not have a license key, you can let the variable LICENSE empty (value: ""). Your R2Devops instance will be limited to 5 projects.

    front:
      host: "r2devops.mydomain.com"
    
    jobs:
      host: "r2devops.mydomain.com"
    
      # Not using secret for license (comment if you use secret)
      extraEnv:
        - name: LICENSE
        value: "<license-key>"
        - name: SECRET_KEY
        value: "<secret-key>"
        - name: GITLAB_OAUTH2_CLIENT_ID
        value: "<gitlab-oauth2-client-id>"
        - name: GITLAB_OAUTH2_CLIENT_SECRET
        value: "<gitlab-oauth2-client-secret>"
    
      # Using existing secret for license (uncomment if you use secret)
      #extraEnv:
      #  - name: LICENSE
      #    valueFrom:
      #      secretKeyRef:
      #        name: "r2devops-secret"
      #        key: "license-key"
      #  - name: SECRET_KEY
      #    valueFrom:
      #      secretKeyRef:
      #        name: "r2devops-secret"
      #        key: "secret-key"
      #  - name: GITLAB_OAUTH2_CLIENT_ID
      #    valueFrom:
      #      secretKeyRef:
      #        name: "r2devops-secret"
      #        key: "gitlab-oauth2-client-id"
      #  - name: GITLAB_OAUTH2_CLIENT_SECRET
      #    valueFrom:
      #      secretKeyRef:
      #        name: "r2devops-secret"
      #        key: "gitlab-oauth2-client-secret"
    
    worker:
      replicaCount: 10 # Default is 5. Increase it depending of your needs
    
  2. Add your GitLab instance domain and organization

    organization

    If you use a SaaS version of GitLab (like gitlab.com): add the name of your organization top-level group in organization. Else, let it empty

    gitlab:
      domain: "https://gitlab.mydomain.com"
      organization: ""
    
  3. Add your Ingress configuration

    ingress:
      enabled: true
      className: ""   # Add class name for your ingress controller
      annotations: {} # Add annotation required by your ingress controller or certificate manager
    

📙 MinIO

You can choose between 2 options: use minIO embedded in this chart or use any external S3 service.

Add the following configuration (1 OR 2) in your custom_values.yaml file:

  1. Use minIO embedded in this chart

    minio:
      dependency:
        enabled: true
    
      # Not using secret for auth (comment if you use secret)
      auth:
        rootPassword: REPLACE_ME_BY_NEW_MINIO_PASSWORD
    
      # Using existing secret for auth (uncomment if you use secret)
      #auth:
      #  existingSecret: "minio-secret"
    
  2. Use an external S3 service (bucket must be created)

    minio:
      dependency:
        enabled: false
      custom:
        host: REPLACE_ME_BY_S3_ENDPOINT
        bucketName: REPLACE_ME_BY_S3_BUCKET_NAME
        region: REPLACE_ME_BY_BUCKET_REGION
        disableSSL: "false"
        forcePathStyle: "false"
    
      # Not using secret for auth (comment if you use secret)
      auth:
        rootUser: REPLACE_ME_BY_S3_ACCESS_KEY
        rootPassword: REPLACE_ME_BY_S3_SECRET_KEY
    
      # Using existing secret for auth (uncomment if you use secret)
      #auth:
      #  existingSecret: "minio-secret"
    

📘 PostgreSQL

You can choose between 2 options: use postgresql embedded in this chart or use an external postgresql.

Add following configuration (1 OR 2) in your custom_values.yaml file:

  1. Use postgresql embedded in this chart

    postgresql:
      dependency:
        enabled: true
      global:
        postgresql:
    
          # Not using secret for auth (comment if you use secret)
          auth:
            password: REPLACE_ME_BY_NEW_POSTGRESQL_PASSWORD
            postgresPassword: REPLACE_ME_BY_NEW_POSTGRESQL_PASSWORD
    
          # Using existing secret for auth (uncomment if you use secret)
          #auth:
          #  username: r2devops
          #  existingSecret: "postgresql-secret"
          #  secretKeys:
          #    adminPasswordKey: "password"
          #    userPasswordKey: "password"
    
  2. Use an external postgresql (database must be created)

    postgresql:
      dependency:
        enabled: false
      custom:
        host: REPLACE_ME_BY_POSTGRES_HOST
        dbName: REPLACE_ME_BY_POSTGRES_DB_NAME
        sslmode: "require"
        port: 5432
      global:
        postgresql:
    
          # Not using secret for auth (comment if you use secret)
          auth:
            username: REPLACE_ME_BY_POSTGRES_USERNAME
            postgresPassword: REPLACE_ME_BY_POSTGRES_PASSWORD
    
          # Using existing secret for auth password (uncomment if you use secret)
          #auth:
          #  username: r2devops
          #  existingSecret: "postgresql-secret"
          #  secretKeys:
          #    adminPasswordKey: "password"
          #    userPasswordKey: "password"
    

📕 Redis

You can choose between 2 options: use redis embedded in this chart or use an external redis.

Add following configuration (1 OR 2) in your custom_values.yaml file:

  1. Use redis embedded in this chart

    redis:
      dependency:
        enabled: true
    
      # Not using secret for auth (comment if you use secret)
      auth:
        password: REPLACE_ME_BY_NEW_REDIS_PASSWORD
    
      # Using existing secret for auth (uncomment if you use secret)
      #auth:
      #  existingSecret: "redis-secret"
      #  existingSecretPasswordKey: "password"
    
  2. Use an external redis (database must be created)

    redis:
      dependency:
        enabled: false
      custom:
        port: 6379
        host: REPLACE_ME_BY_REDIS_HOST
        user: REPLACE_ME_BY_REDIS_USENAME
        cert: |
          REPLACE_ME_BY_REDIS_TLS_CERTIFICATE
    
      # Not using secret for auth (comment if you use secret)
      auth:
        password: REPLACE_ME_BY_REDIS_PASSWORD
    
      # Using existing secret for auth (uncomment if you use secret)
      #auth:
      #  existingSecret: "redis-secret"
      #  existingSecretPasswordKey: "password"
    

🚀 Install the chart

Run the following command:

helm upgrade -n $R2DEVOPS_NS --install r2devops r2devops/r2devops -f custom_values.yaml

Congratulations

You have successfully installed R2Devops on your Kubernetes cluster 🎉

What's next

Now that you have finished this tutorial, here are some simple tasks you should give a try :

  • 📈 Learn how to use the platform by reading the documentation
  • 📕 Import your first job, here is the tutorial

Not the same behavior

Did you encounter a problem during the installation process ? See the troubleshooting section.

📚 Configuration example

Info

The following examples run in a Kubernetes cluster using nginx as ingressController, cert-manager and a clusterIssuer named letsencrypt-production"

📦 No external services

This is an example of custom values.yaml file using all services from chart dependencies.

Example (to expand)
front:
  host: "r2devops.mydomain.com"

jobs:
  host: "r2devops.mydomain.com"
  extraEnv:
    - name: LICENSE
    value: "REDACTED"
    - name: SECRET_KEY
    value: "REDACTED"
    - name: GITLAB_OAUTH2_CLIENT_ID
    value: "REDACTED"
    - name: GITLAB_OAUTH2_CLIENT_SECRET
    value: "REDACTED"

gitlab:
  domain: "https://gitlab.mydomain.com"

worker:
  replicaCount: 15

ingress:
  enabled: true
  className: "nginx"
  annotations:
    cert-manager.io/cluster-issuer: "letsencrypt-production"

# ref. https://github.com/minio/minio/blob/master/helm/minio/values.yaml
minio:
  dependency:
    enabled: true
  auth:
    rootPassword: REDACTED
  image:
    debug: true

# ref. https://github.com/bitnami/charts/blob/main/bitnami/postgresql/values.yaml
postgresql:
  dependency:
    enabled: true
  global:
    postgresql:
      auth:
        password: REDACTED
        postgresPassword: REDACTED

# ref. https://github.com/bitnami/charts/blob/main/bitnami/redis/values.yaml
redis:
  dependency:
    enabled: true
  auth:
    password: REDACTED

📥 External services

This is an example of custom values.yaml file using external services for MinIO, PostgreSQL and Redis.

Example (to expand)
front:
  host: "r2devops.mydomain.com"

jobs:
  host: "r2devops.mydomain.com"
  extraEnv:
    - name: LICENSE
    value: "REDACTED"
    - name: SECRET_KEY
    value: "REDACTED"
    - name: GITLAB_OAUTH2_CLIENT_ID
    value: "REDACTED"
    - name: GITLAB_OAUTH2_CLIENT_SECRET
    value: "REDACTED"

gitlab:
  domain: "https://gitlab.mydomain.com"

worker:
  replicaCount: 20

ingress:
  enabled: true
  className: "nginx"
  annotations:
    cert-manager.io/cluster-issuer: "letsencrypt-production"

# ref. https://github.com/minio/minio/blob/master/helm/minio/values.yaml
minio:
  dependency:
    enabled: false
  auth:
    rootUser: REDACTED
    rootPassword: REDACTED
  custom:
    host: "https://s3.us-east-1.amazonaws.com"
    bucketName: "r2devops-files"
    region: "us-east-1"
    disableSSL: "false"
    forcePathStyle: "false"

# ref. https://github.com/bitnami/charts/blob/main/bitnami/postgresql/values.yaml
postgresql:
  dependency:
    enabled: false
  global:
    postgresql:
      auth:
        username: REDACTED
        postgresPassword: REDACTED
  custom:
    host: "database-1.REDACTED.us-east-1.rds.amazonaws.com"
    port: 5432
    dbName: "r2devops"
    sslmode: "require"

# ref. https://github.com/bitnami/charts/blob/main/bitnami/redis/values.yaml
redis:
  dependency:
    enabled: false
  auth:
    password: REDACTED
  custom:
    port: 6379
    host: "REDACTED"
    user: "REDACTED"
    cert: |
      -----BEGIN CERTIFICATE-----
      REDACTED
      -----END CERTIFICATE-----