Contribute to R2Devops official CI/CD Catalog¶
This page describes how to create or update a template in the R2Devops/hub repository. It's the guide!
Info
In order to contribute efficiently, we recommend you to know the following topics:
Tip
If you are hesitant about which template to create/update, you can get inspired by our pre-made template issues on GitLab under the label Easy first step
.
Contributing workflow¶
Follow the 3 simple steps below to contribute efficiently in the hub. You'll see, it will all go smoothly! ๐
๐ด Step 1: Fork!¶
The first step is to create your own copy of r2devops/hub
repository, to be able to work on it before merging your update in the real project.
- Go on the fork page creation:
r2devops/hub
. - Select the group in which you want to create the fork.
๐ป Step 2: Work in your fork¶
Do not alter .gitlab-ci.yml
To leverage the R2Devops validity and security checks on your template, do not update the CI/CD configuration file in your fork (.gitlab-ci.yml
file).
If you alter it, we will not be able to merge your template in r2devops/hub
repository (yes, what a shame, after all your hard work...). ๐
-
If you want to create a new template:
- Create a new branch for your template and work on it ๐
- Make sure that you have NPM installed, more about that here
- Install the package
Cookicutter
by executing this command:npm install -g cookiecutter
- In the
hub
folder, run this command and input the necessary information:npx cookiecutter
Info
Alternatively, you can do the last steps manually by creating a new directory dedicated to your template in
templates/
folder if you want to add a new template. You can use the template as a starting point. If you want to modify an existing template, you don't have to create a new directory. -
Be sure to respect the rules we describe in this guide.
- Do not update the CI/CD configuration file
.gitlab-ci.yml
.
๐งช Step 3: Test your template¶
Before merging it into the hub, you should test it deeply to be sure it works as expected.
-
Find a suitable project for your template You can pick one of yours or use a GitLab template project. To simulate the GitLab template that will run on your project, there are several way to do it :
- With docker You can mount the repository folder inside the container:
# Example if your template will use node:16.3-buster as Docker image docker run -v /path/to/your/repo:/mnt --entrypoint "/bin/sh" -it node:16.3-buster
- With GitLab Runner If you have GitLab Runner installed, you can test your template on your project. Copy the content of the template.yml file into your .
gitlab-ci.yml
and include the template's stage. Then test it via:gitlab-runner exec docker <template_name>
- By remoting your template Move your template configuration in your project root directory and include it locally in your
.gitlab-ci.yml
:OR You can remote your template from your fork of the hub. On gitlab.com, find yourinclude: - local: 'my-work-in-progress-template.yml'
template.yml
in your repository and click on it. Then open the raw content by clicking onOpen raw
button and copy the url. Now go back on your project and remote it in your.gitlab-ci.yml
:Add, commit and push your modifications and check the result of the pipeline on Git Lab, under CI/CD > Pipelines. After the merge request approval, you can remote it directly from the hub, check the quick use section.include: #change it by the url of your template - remote : 'https://gitlab.com/<project>/-/raw/<hash>/templates/<template_name>/<template_name>.yml'
- Test the behaviour of your template ๐ฒ
- If you're creating a template from one of the hub's issue, there should be at least one scenario. Test each scenario separately and ensure the pipeline succeed.
- If you're creating a template on your own, think of each case, your template should be the most generic as possible.
- With docker You can mount the repository folder inside the container:
๐ You can customize it with variables
๐ Step 4: Merge request¶
1. Ensure that the last pipeline in your fork passed before going further (check it in CI/CD > Pipelines
)
2. Create a new merge request in your fork (Merge Requests > New merge request
) ๐
- Select branches
- As
Source branch
, select the branch in which you have worked in your fork (usuallylatest
). We encourage you to create a new branch for each template! - As
Target branch
, select latest inr2devops/hub
project. - Click on
Compare branches and continue
.
- As
- In
Title
: add short description of your contribution. - In
Description
:- Do not remove the default content, this is the Definition of Done (DoD)!
- Add a description of your contribution with all information permitting us to understand what you have done and why. If your contribution is related to an existing issue, add a reference:
## Contribution Addition of a new template permitting to build golang binaries. Issue related: r2devops/hub#945 ## Definition of Done [...]
- Add a link to your template running and working in a publicly accessible Gitlab project.
- Write a
comment
:- Ping a member of the team for asking a review.
- Share the link of the generated documentation. It could be found by browsing artifacts of the
mkdocs
templates. The path is usually :/website_build/templates/<stage>/<template_name>/index.html
. - For each scenarios written in the issue description : Test each case with corresponding variables fulfil and ensure the pipeline succeed. Share the link in the comment and specify the scenario associated.
- If you want to allow commits from hub maintainers inside your fork branch, check the box
Contribution
(this isn't available for protected branches likelatest
).
3. In the newly created MR, ensure to fulfill all the steps of the template's Definition of Done, and tick the related boxes
Thanks a lot for your contribution ๐๐ !
Now, we will take a look at your work and merge it if everything is ok. ๐ Meanwhile, you can join our Discord community to tell us more about your fresh new contribution. We love talking with our contributors and users!
Guidelines (Required)¶
Structure of official template repo
Our templates are stored in the R2Devops hub repository inside the templates
folder, and follow this standardized structure:
.
โโโ templates
โโโ <template_name>
โโโ <template_name>.yml # template definition
โโโ CHANGELOG.md # template changelog
โโโ README.md # template documentation
A template of template is available in the R2Devops hub repository.
Warning
Following these guidelines is required to contribute to R2Devops/hub repository.
Each templates must be compliant with these following rules:
- Be compliant with the template structure
- Use the
image
option. The goal is to provide plug and play templates working in any environments thanks to containers. - Use fixed tag for docker image and any external tool used inside the template. It shouldn't be
latest
or any tag/version that will be overwritten. - Use only resource with license compatible with the template license, and permitting anyone to use it.
- Pass our Continuous Integration pipeline, which includes security check templates (the pipeline will be run automatically inside your fork ๐ข).
Best practices (Optional)¶
Info
Following these best practices is recommended to contribute to R2Devops/hub repository.
๐ค Template definition¶
๐งฎ Variables¶
In order to be customizable, your template should use variables
. This section allows to define environment variables, usable by the template script.
Tip
Set default values for your variables, to reflect the most common use-case. With this, your template will remain plug-and-play while being customizable!
Example of relevant situation to use variable:
- File name.
- Path to files or folders.
- Any options/parameters used by the template.
- Enable or disable template features.
- Version of tools retrieved during the template.
Here is an example for newman
template ๐
newman:
image: node:15.4.0
stage: tests
variables:
NEWMAN_COLLECTION: "postman_collection.json"
NEWMAN_GLOBALS_FILE: ""
NEWMAN_ADDITIONAL_OPTIONS: ""
NEWMAN_JUNIT_REPORT: "newman-report.xml"
NEWMAN_FAIL_ON_ERROR: "true"
NEWMAN_ITERATIONS_NUMBER: "2"
script:
- npm install -g newman newman-reporter-junitfull
- if [[ ! -z ${NEWMAN_GLOBALS_FILE} ]]; then
- NEWMAN_ADDITIONAL_OPTIONS+=" -g ${NEWMAN_GLOBALS_FILE}"
- fi
- if [[ ! ${NEWMAN_FAIL_ON_ERROR} == "true" ]]; then
- NEWMAN_ADDITIONAL_OPTIONS+=" --suppress-exit-code"
- fi
- newman run ${NEWMAN_COLLECTION} -r cli,junitfull
--reporter-junitfull-export ${NEWMAN_JUNIT_REPORT}
-n ${NEWMAN_ITERATIONS_NUMBER} ${NEWMAN_ADDITIONAL_OPTIONS}
artifacts:
[...]
Name | Description | Default |
---|---|---|
NEWMAN_COLLECTION | Name of the Postman collection | postman_collection.json |
NEWMAN_GLOBALS_FILE | Name of the Postman globals file for variables |
|
NEWMAN_ADDITIONAL_OPTIONS | Other options you may want to use with Newman |
|
NEWMAN_FAIL_ON_ERROR | Fail template on a request/test error | true |
NEWMAN_ITERATIONS_NUMBER | Number of Newman iterations to run (see documentation) | 2 |
๐ณ Docker image¶
Info
As described in our guidelines, all templates are run inside a container instance, so they must specify the Docker image to use. Depending on your template, it can be tricky to find the perfect image.
The better place to find a docker image is the docker hub. You can start your research there with the following steps:
1. Search for an image prepared with the tool you want to run
- This is the best situation: a ready-to-use docker image that doesn't require any additional installation.
- Example for
mkdocs
template:squidfunk/mkdocs-material
. - If you find it, check it with the general guidelines below.
2. If there isn't any image prepared for the tool you want to run, search for more general images
- This case will require to install needed tool as first step of your template. If the installation is long or heavy, you should considers to build your own image with the tool already installed.
- The vast majority of operating system and languages provides official images, choose the more appropriate for your template. Some examples:
alpine
,debian
,ubuntu
,python
,node
.
3. If you decide to build your own image: the image must be stored in a publicly reachable registry, like Docker hub or GitLab registry
General guidelines to choose the image
- The first thing to check is if the image is official (
OFFICIAL IMAGE
badge on docker hub): this is the perfect image for your use case.
- If it is not, the following points should be considered to choose an image:
- The image must be versioned and not only with
latest
tag. If image isn't versioned: it's not usable for your template. - It should be actively maintained, with frequent updates, and should contain recent versions.
- The image should be small, containing only required tools.
- The image should be efficient to run the template.
- A large usage of the image can be a good indicator, but be aware, it doesn't guarantee the quality neither the security of the image.
- The image must be versioned and not only with
๐ฆ Artifacts¶
The vast majority of templates produce a result. This result can be of different kinds:
- Test report.
- Build result.
In both case, the result should be exposed by the template using the artifact
option. It permits passing an artifact to another template of the pipeline and expose results to users.
Info
- In the case of a test report, you need to use
when: always
option underartifacts
if you want to expose result even if template fails. - You can combine both a build result and a test report by using both
artifacts:paths
andartifacts:reports
.
An artifact can be configured at different level of integration in Gitlab interface:
-
Best integration: Gitlab
artifacts:reports
This is a way to integrate a report result in an user-friendly way in Gitlab's interface. We encourage all template contributors to adapt their template output to a format compatible with a Gitlab report.
Example of
artifacts:reports:junit
reporttemplate
trivy_image
that uses its output asjunit
report inartifacts:reports:junit
section:trivy_image: [...] artifacts: reports: junit: "$TRIVY_OUTPUT"
-
Quick integration with
artifacts:expose_as
This is a way to quickly integrate any format of report in Gitlab Merge Request interface. Technically, you don't have to shape your report output in a specific format, but we recommend to use
HTML
format. In this way, the report is one-click readable from any Merge Request.Example of
artifacts:expose_as
reporttemplate
nmap
usesartifacts:expose_as
to expose itsHTML
report:nmap: [...] artifacts: expose_as: "nmap-report" paths: - "${HTML_OUTPUT}" when: always
-
Simple artifact without integration
Example of
artifacts
template that specify an
artifact
:template_name: [...] artifacts: paths: - "output" when: always
๐ฉ Keep your template generic¶
The templates of the hub should remain as generic as possible. In order to ensure it:
- Try to avoid using
rules
options, that is strongly linked to the context of the user and should be set by the user. Also, some features requiring specific workflows, as Gitlab Merge Trains, are more easily implemented by users if you don't specifyrules
in your template. - Try to avoid using
before_script
andafter_script
to let users the possibility to redefine these options while exploiting to the maximum your template.
Info
The templates of the hub can be dynamically customized by users.
๐ Template documentation¶
As described in R2Devops/hub template structure, the documentation of a template is written inside its README.md
file.
Tip
Don't hesitate to copy the documentation from another template as starting point. For example, the raw content of openapi README.md
!
We recommend including at least the following sections in your documentation:
- Objective: describe the goal of your template.
- How to use it: a list of steps to quickly use your template.
- template details: describes details of the template (name, docker image, stage, etc).
- Variables: table listing variables used by the template (name, description, default value, mandatory if needed).
- Artifacts: describes artifact(s) defined by the template.
๐ Compliance with another template¶
Several templates of the hub can be used together without any configuration. This is currently implemented for all templates producing an HTML
output and the template pages
which deploys the HTML
on a webserver.
Info
This feature is empowered by the artifacts
option: templates producing a static website output give it to the pages
template through an artifact stored in a standard path: ${CI_PROJECT_DIR}/website_build
.
So, if your template produce a static website output, ensure to store the result of the build in ${CI_PROJECT_DIR}/website_build
, and to configure this path as artifact. You can see an example in mkdocs
template.
Congratulation, you did it!
You went through all our guideline. ๐ฅณ
If never something feel unclear or you're having a doubt, join us on Discord to ask us anything! We'll be more than happy to help.