Contribute to R2Devops official CI/CD Catalog¶

This page describes how to create or update a template in the R2Devops/hub repository. It's the guide!

Contributing workflow¶

Follow the 3 simple steps below to contribute efficiently in the hub. You'll see, it will all go smoothly! 👇

🍴 Step 1: Fork!¶

The first step is to create your own copy of r2devops/hub repository, to be able to work on it before merging your update in the real project.

1. Go on the fork page creation: r2devops/hub.
2. Select the group in which you want to create the fork.

💻 Step 2: Work in your fork¶

1. If you want to create a new template:

   1. Create a new branch for your template and work on it 🎋
   2. Make sure that you have NPM installed, more about that here
   3. Install the package Cookicutter by executing this command:

      npm install -g cookiecutter
      

   4. In the hub folder, run this command and input the necessary information:

      npx cookiecutter
      

   Info

   Alternatively, you can do the last steps manually by creating a new directory dedicated to your template in templates/ folder if you want to add a new template. You can use the template as a starting point. If you want to modify an existing template, you don't have to create a new directory.

2. Be sure to respect the rules we describe in this guide.

3. Do not update the CI/CD configuration file .gitlab-ci.yml.

🧪 Step 3: Test your template¶

Before merging it into the hub, you should test it deeply to be sure it works as expected.

1. Find a suitable project for your template You can pick one of yours or use a GitLab template project. To simulate the GitLab template that will run on your project, there are several way to do it :

   1. With docker You can mount the repository folder inside the container:

      # Example if your template will use node:16.3-buster as Docker image
      docker run -v /path/to/your/repo:/mnt --entrypoint "/bin/sh" -it node:16.3-buster
      

   2. With GitLab Runner If you have GitLab Runner installed, you can test your template on your project. Copy the content of the template.yml file into your .gitlab-ci.yml and include the template's stage. Then test it via:

      gitlab-runner exec docker <template_name>
      

   3. By remoting your template Move your template configuration in your project root directory and include it locally in your .gitlab-ci.yml:

      include:
          - local: 'my-work-in-progress-template.yml'
      

      OR You can remote your template from your fork of the hub. On gitlab.com, find your template.yml in your repository and click on it. Then open the raw content by clicking on Open raw button and copy the url. Now go back on your project and remote it in your .gitlab-ci.yml:

      include:
        #change it by the url of your template
        - remote : 'https://gitlab.com/<project>/-/raw/<hash>/templates/<template_name>/<template_name>.yml'
      

      Add, commit and push your modifications and check the result of the pipeline on Git Lab, under CI/CD > Pipelines. After the merge request approval, you can remote it directly from the hub, check the quick use section.
   4. Test the behaviour of your template 🎲
   5. If you're creating a template from one of the hub's issue, there should be at least one scenario. Test each scenario separately and ensure the pipeline succeed.
   6. If you're creating a template on your own, think of each case, your template should be the most generic as possible.

🚀 Step 4: Merge request¶

Thanks a lot for your contribution 😀🎉 !

Now, we will take a look at your work and merge it if everything is ok. 👀 Meanwhile, you can join our Discord community to tell us more about your fresh new contribution. We love talking with our contributors and users!

Guidelines (Required)¶

.
└── templates
    └── <template_name>
        ├── <template_name>.yml        # template definition
        ├── CHANGELOG.md          # template changelog
        └── README.md             # template documentation

Each templates must be compliant with these following rules:

• Be compliant with the template structure
• Use the image option. The goal is to provide plug and play templates working in any environments thanks to containers.
• Use fixed tag for docker image and any external tool used inside the template. It shouldn't be latest or any tag/version that will be overwritten.
• Use only resource with license compatible with the template license, and permitting anyone to use it.
• Pass our Continuous Integration pipeline, which includes security check templates (the pipeline will be run automatically inside your fork 🎢).

Best practices (Optional)¶

🤖 Template definition¶

🧮 Variables¶

In order to be customizable, your template should use variables. This section allows to define environment variables, usable by the template script.

Example of relevant situation to use variable:

• File name.
• Path to files or folders.
• Any options/parameters used by the template.
• Enable or disable template features.
• Version of tools retrieved during the template.

Here is an example for newman template 👇

template content

newman:
    image: node:15.4.0
    stage: tests
    variables:
        NEWMAN_COLLECTION: "postman_collection.json"
        NEWMAN_GLOBALS_FILE: ""
        NEWMAN_ADDITIONAL_OPTIONS: ""
        NEWMAN_JUNIT_REPORT: "newman-report.xml"
        NEWMAN_FAIL_ON_ERROR: "true"
        NEWMAN_ITERATIONS_NUMBER: "2"
    script:
        - npm install -g newman newman-reporter-junitfull
        - if [[ ! -z ${NEWMAN_GLOBALS_FILE} ]]; then
        -   NEWMAN_ADDITIONAL_OPTIONS+=" -g ${NEWMAN_GLOBALS_FILE}"
        - fi
        - if [[ ! ${NEWMAN_FAIL_ON_ERROR} == "true" ]]; then
        -   NEWMAN_ADDITIONAL_OPTIONS+=" --suppress-exit-code"
        - fi
        - newman run ${NEWMAN_COLLECTION} -r cli,junitfull
          --reporter-junitfull-export ${NEWMAN_JUNIT_REPORT}
          -n ${NEWMAN_ITERATIONS_NUMBER} ${NEWMAN_ADDITIONAL_OPTIONS}
    artifacts:
        [...]

Variables

Name	Description	Default
NEWMAN_COLLECTION	Name of the Postman collection	postman_collection.json
NEWMAN_GLOBALS_FILE	Name of the Postman globals file for variables
NEWMAN_ADDITIONAL_OPTIONS	Other options you may want to use with Newman
NEWMAN_FAIL_ON_ERROR	Fail template on a request/test error	true
NEWMAN_ITERATIONS_NUMBER	Number of Newman iterations to run (see documentation)	2

🐳 Docker image¶

The better place to find a docker image is the docker hub. You can start your research there with the following steps:

General guidelines to choose the image

• The first thing to check is if the image is official (OFFICIAL IMAGE badge on docker hub): this is the perfect image for your use case .
• If it is not, the following points should be considered to choose an image:
  • The image must be versioned and not only with latest tag. If image isn't versioned: it's not usable for your template.
  • It should be actively maintained, with frequent updates, and should contain recent versions.
  • The image should be small, containing only required tools.
  • The image should be efficient to run the template.
  • A large usage of the image can be a good indicator, but be aware, it doesn't guarantee the quality neither the security of the image.

📦 Artifacts¶

The vast majority of templates produce a result. This result can be of different kinds:

• Test report.
• Build result.

In both case, the result should be exposed by the template using the artifact option. It permits passing an artifact to another template of the pipeline and expose results to users.

An artifact can be configured at different level of integration in Gitlab interface:

1. Best integration: Gitlab artifacts:reports

   This is a way to integrate a report result in an user-friendly way in Gitlab's interface. We encourage all template contributors to adapt their template output to a format compatible with a Gitlab report.

   Example of artifacts:reports:junit report

   template trivy_image that uses its output as junit report in artifacts:reports:junit section:

   trivy_image:
     [...]
     artifacts:
       reports:
         junit: "$TRIVY_OUTPUT"
   

2. Quick integration with artifacts:expose_as

   This is a way to quickly integrate any format of report in Gitlab Merge Request interface. Technically, you don't have to shape your report output in a specific format, but we recommend to use HTML format. In this way, the report is one-click readable from any Merge Request.

   Example of artifacts:expose_as report

   template nmap uses artifacts:expose_as to expose its HTML report:

   nmap:
     [...]
     artifacts:
       expose_as: "nmap-report"
       paths:
         - "${HTML_OUTPUT}"
       when: always
   

3. Simple artifact without integration

   Example of artifacts

   template that specify an artifact:

   template_name:
     [...]
     artifacts:
       paths:
         - "output"
       when: always
   

🔩 Keep your template generic¶

The templates of the hub should remain as generic as possible. In order to ensure it:

• Try to avoid using rules options, that is strongly linked to the context of the user and should be set by the user. Also, some features requiring specific workflows, as Gitlab Merge Trains, are more easily implemented by users if you don't specify rules in your template.
• Try to avoid using before_script and after_script to let users the possibility to redefine these options while exploiting to the maximum your template.

📖 Template documentation¶

As described in R2Devops/hub template structure, the documentation of a template is written inside its README.md file.

We recommend including at least the following sections in your documentation:

• Objective: describe the goal of your template.
• How to use it: a list of steps to quickly use your template.
• template details: describes details of the template (name, docker image, stage, etc).
• Variables: table listing variables used by the template (name, description, default value, mandatory if needed).
• Artifacts: describes artifact(s) defined by the template.

🚄 Compliance with another template¶

Several templates of the hub can be used together without any configuration. This is currently implemented for all templates producing an HTML output and the template pages which deploys the HTML on a webserver.

So, if your template produce a static website output, ensure to store the result of the build in ${CI_PROJECT_DIR}/website_build, and to configure this path as artifact. You can see an example in mkdocs template.