Skip to content

gcix - Write your GitLab CI pipelines in X languages

Apache License 2.0

GitLab tag (self-managed) GitLab pipeline status GitLab last commit GitLab contributors

PyPI - Downloads npm - Downloads Docker Pulls

Matrix

The complete documentation is found at https://docs.gcix.dev

The GitLab CI X Library (gcix) is a library to create dynamic pipelines for GitLab CI.

With the gcix you can write your GitLab CI pipelines in multiple languages.

Supported languages

  • Typescript/JavaScript (native)
  • Python

A simple pipeline could look like...

...this in Typescript

1
2
3
4
5
6
7
import { Pipeline, Job } from "gcix"

const pipeline = new Pipeline()
const job = new Job({stage: "build", scripts: ["docker build ."]})

pipeline.addChildren({jobsOrJobCollections: [job]})
pipeline.writeYaml()

...this in Python

1
2
3
4
5
6
7
from gcip import Pipeline, Job

pipeline = Pipeline()
job      = Job(stage="build", scripts=["docker build ."])

pipeline.add_children(jobs_or_job_collections=[job])
pipeline.write_yaml()

Intro

The gcix is a rewrite of the gcip, the reason behind the rewrite is, that I wanted to learn Typescript, and give developers the choice to use the language they are familiar with. This is because I chose Typescript and a tool called jsii made by AWS. jsii translates the different language API's to typescript. The user which uses Python is using Pythons syntax, behind the scenes they are getting translated to Typescript.

Examples

For a more complex and real world example, just check out our projects .gitlab-ci.ts. The .gitlab-ci.ts is the written example of a working gcix in Typescript. The Typescript code is getting rendered and stored as an artifact in the pipeline see generated yaml file.

The gcix is using the GitLab feature of dynamic child pipelines. First the .gitlab-ci.ts generates the common GitLab CI yaml file which is then started as child pipeline. To get the generated pipeline it is necessary to invoke the .gitlab-ci.ts with ts-node. To get an idea of a dynamic child pipelines setup, you can have a look into .gitlab-ci.yml.

Creating your pipelines in any supported programming language code allows you all the features of that language, like:

  • re-use code (DRY - Don't repeat yourself)
  • use variables, control flow (if-then-else, loops, ...), complex data structures, input/output, error handling, ...
  • programming paradigms like object-oriented or functional programming
  • use 3rd party libraries in your pipelines
  • test driven development of pipelines with Jest(Typescript), pytest(Python)...
  • package management and distribution of your pipeline code
  • ... anything you can imagine to do with Python code

Documentation

Please read the User Documentation to get a quick introduction into most features of the gcix.

You can consult the API Reference to get an overview of all classes and methods and a deeper view into their parameters.

For more information on how to create your own library based on gcix, please read the Developer Documentation to learn about good practices and conventions.

Why pipelines as code?

There has been a really good discussion on Hacker News about configuration vs code. Comments stating that when struggling around with the limitations of static configuration we start writing pipelines as code, until we hassle around with complex code and start re-writing complex pipelines in static configuration and so forth.

It is absolutely right that pipelines as code are not new and have lots of drawbacks. Chances are good that you have the one guru in you company loving this project and is writing weird pipelines that nobody else understands. Also comments state that pipelines are those basic things that build and test your code and shouldn't be in code too, because then you might have a pipeline that build and test your pipeline code.

All those statements are quite true. However all those statements could be applied to static configuration. We started this project because of GitLab CI yaml files growing over thousands of lines. We tried to implement some logic with clunky rule sets. We tried to re-use code with yaml templates. We tried to write predefined pipelines by using includes. We started to write bash scripts that do bulk loads of work within a job. All in all those felt like a bad workaround, while having in mind how much cleaner this might be, writing our pipelines in code. That is why we started this project and since using it with great success.

However it is absolutely important to understand the gcix as a supplement and not as a substitution to GitLab CI yaml files. As long you are fine with having a couple of jobs you could easily describe in static configuration, just do it. If you feel that you can't manage the complexity of you static yaml configuration and started to build lots of helper scripts, you could consider writing your pipelines in code. This considerations should include, if the pipeline code you write is definitely an advantage over the static scripts you had before - and not just for you but also for your colleagues and the company you are writing the code for.

The gcix should be a choice - not a standard.

Thanks

First of all, I have to thank Thomas Steinbach he created the GitLab CI Python Library (gcip) and started a great journey with GitLab CI pipelines as code. Another thank you goes to the DB Systel GmbH.

Author

gcix was created by Daniel von Eßen in 2023.

License

The content of this repository is licensed under the Apache 2.0 license.

Copyright Daniel von Eßen