GitLab CI/CD Variables: A Comprehensive Guide

By /

GitLab CI/CD Variables: A Comprehensive Guide

GitLab CI/CD pipelines are powerful tools for automating your software development lifecycle. A key component of that power comes from the flexibility offered by CI/CD variables. These variables allow you to customize your pipelines, making them reusable, secure, and adaptable to different environments and situations. This guide provides a comprehensive overview of GitLab CI/CD variables, covering their types, scopes, precedence, security considerations, and practical usage examples.

1. What are CI/CD Variables?

CI/CD variables are dynamic values that can be used within your .gitlab-ci.yml file (your pipeline configuration) and in your scripts that are executed during pipeline stages. Think of them as environment variables for your CI/CD process. They allow you to:

  • Avoid hardcoding values: Instead of embedding sensitive information (like API keys) or environment-specific details directly in your code or .gitlab-ci.yml, you use variables as placeholders.
  • Parameterize your pipelines: Make your pipelines reusable across different projects or environments by changing the values of variables without modifying the pipeline definition itself.
  • Control pipeline behavior: Use variables to conditionally execute jobs or stages, or to modify the behavior of your scripts based on external factors.
  • Secure sensitive information: GitLab provides mechanisms to securely store and mask sensitive variables, preventing them from being exposed in logs or the repository.

2. Types of CI/CD Variables

GitLab offers two primary types of variables based on their data type:

  • Variable (Simple): These are the most common type, representing simple key-value pairs where the value is a string. They are suitable for storing most configuration data.

    yaml
    variables:
    DATABASE_URL: "postgres://user:password@host:port/database"
    DEPLOYMENT_ENVIRONMENT: "staging"

  • File: These variables store the content of a file as their value. This is crucial for handling sensitive information like certificates, private keys, or configuration files that shouldn’t be directly embedded in your .gitlab-ci.yml or as a simple variable. When a file variable is used, GitLab creates a temporary file on the runner containing the variable’s value and sets the variable to the path of that temporary file.

    Example (using the GitLab UI – see “Where to Define Variables” below): You would select “File” as the Type, provide a Key (e.g., MY_SECRET_KEY), and paste the contents of your secret key file into the Value field.

    In your script, you’d reference it like this:

    “`bash

    Accessing the file variable in a script

    openssl rsa -in $MY_SECRET_KEY -pubout # $MY_SECRET_KEY will contain the path to the temporary file
    “`

3. Where to Define Variables (and Their Scope)

The location where you define a variable determines its scope – which parts of your pipeline can access it. GitLab provides several levels of scope:

  • Project-Level Variables (Settings > CI/CD > Variables): These variables are available to all pipelines within a specific project. They are ideal for project-specific configuration settings that apply to all branches and environments. This is the most common and recommended place to define most variables. You define them through the GitLab UI, making them easy to manage.

  • Group-Level Variables (Group Settings > CI/CD > Variables): These variables are available to all projects within a GitLab group (and its subgroups). They are useful for sharing common configurations across multiple projects, such as API keys for shared services or organization-wide settings. This is also defined via the GitLab UI.

  • Instance-Level Variables (Admin Area > Settings > CI/CD > Variables): These variables are available to all projects on the GitLab instance. They are typically used for global configurations, such as the address of a shared Docker registry or default settings that apply to all users. Use with extreme caution, as changing these affects every project on the instance. Defined via the GitLab UI in the Admin Area.

  • Job-Level Variables (within .gitlab-ci.yml): These variables are defined within a specific job in your .gitlab-ci.yml file and are only available to that particular job. They are useful for overriding global or project-level variables or for defining job-specific settings.

    yaml
    deploy:
    stage: deploy
    variables:
    DEPLOYMENT_ENVIRONMENT: "production" # Overrides any project-level setting
    script:
    - echo "Deploying to $DEPLOYMENT_ENVIRONMENT"

  • Pipeline-Level Variables (Triggering Pipelines): You can define variables when manually triggering a pipeline, either through the GitLab UI (“Run pipeline” button) or via the API. These variables override any existing variables with the same name for that specific pipeline run. This is excellent for testing different configurations without permanently changing your project settings.

  • Predefined Variables: GitLab provides a set of predefined variables that contain information about the pipeline, the project, the runner, and the environment. These variables are automatically available in every pipeline and job. Examples include:

    • CI_COMMIT_SHA: The commit SHA being built.
    • CI_JOB_ID: The unique ID of the current job.
    • CI_PIPELINE_ID: The unique ID of the current pipeline.
    • CI_PROJECT_DIR: The full path where the project is cloned on the runner.
    • CI_COMMIT_BRANCH: The branch name (if applicable).
    • CI_COMMIT_TAG: The tag name (if applicable).
    • GITLAB_USER_LOGIN: The username of the user who triggered the pipeline.
    • And many more…

  • Runtime Variables: Created during the runtime of a job using shell commands. These variables are only available for subsequent commands within the same job.

    yaml
    my_job:
    script:
    - export MY_RUNTIME_VAR="some_value"
    - echo "The runtime variable is: $MY_RUNTIME_VAR"
    # MY_RUNTIME_VAR is not available in other jobs

4. Variable Precedence (Which Variable Wins?)

When multiple variables with the same name are defined at different scopes, GitLab uses a specific order of precedence to determine which value is used. The order, from highest to lowest precedence, is:

  1. Trigger variables/API-defined variables: Variables set when manually triggering a pipeline or via the API.
  2. Scheduled pipeline variables: Variables set for scheduled pipelines.
  3. Job-level variables (defined in .gitlab-ci.yml): Variables defined within the specific job.
  4. Project-level variables: Variables defined in the project settings.
  5. Group-level variables: Variables defined in the group settings.
  6. Instance-level variables: Variables defined in the instance settings.
  7. Predefined variables: Variables provided by GitLab.
  8. Runtime Variables Have the lowest precedence, and are only valid in the job they were defined in.

In essence, variables defined closer to the execution context (e.g., job-level) override those defined further away (e.g., project-level).

5. Securing Variables

GitLab provides features to protect sensitive variables:

  • Masked Variables: When you mark a variable as “Masked” in the GitLab UI, its value will be replaced with [MASKED] in the job logs. This prevents accidental exposure of sensitive information like passwords or API keys. This is crucial for security. Masking only applies to the logs; the variable’s value is still available to your scripts. The value must be plain text and meet certain requirements (e.g., minimum length, specific characters allowed) to be successfully masked. Refer to GitLab’s documentation for the exact masking rules.

  • Protected Variables: You can mark a variable as “Protected,” which means it will only be available to pipelines running on protected branches or protected tags. This is useful for restricting access to sensitive credentials needed for deployments to production environments, for example.

  • File Variables: as mentioned before, using the File variable type is the most secure way to handle sensitive files.

  • Variable expansion: You can prevent unintended variable expansion by using the $$ syntax. This will pass the literal string $VARIABLE_NAME to the script, instead of expanding the value of VARIABLE_NAME. This is useful when you need to pass a variable name to a tool that handles its own variable expansion.
    yaml
    script:
    - command_that_uses_its_own_variables --variable=$$MY_VARIABLE

6. Practical Examples

Here are some practical examples illustrating the use of CI/CD variables:

  • Deploying to Different Environments:

    “`yaml
    variables:
    DEPLOYMENT_ENVIRONMENT: “staging” # Default environment

    deploy_staging:
    stage: deploy
    script:
    – echo “Deploying to $DEPLOYMENT_ENVIRONMENT”
    # … deployment commands for staging …
    only:
    – staging

    deploy_production:
    stage: deploy
    variables:
    DEPLOYMENT_ENVIRONMENT: “production” # Override for production
    script:
    – echo “Deploying to $DEPLOYMENT_ENVIRONMENT”
    # … deployment commands for production …
    only:
    – main
    “`

  • Using API Keys:

    In GitLab Project Settings > CI/CD > Variables:

    • Key: MY_API_KEY
    • Value: (Your actual API key)
    • Masked: (Checked)
    • Protected: (Checked if appropriate)

    In .gitlab-ci.yml:

    yaml
    my_job:
    script:
    - curl -H "Authorization: Bearer $MY_API_KEY" https://api.example.com/data

  • Using File Variables (e.g., a private key):
    In GitLab Project Settings > CI/CD > Variables:

    • Key: SSH_PRIVATE_KEY
    • Type: File
    • Value: (Paste the contents of your private key file)
    • Masked: (Checked)
    • Protected: (Checked if appropriate)

    In .gitlab-ci.yml:

    yaml
    deploy:
    script:
    - ssh -i $SSH_PRIVATE_KEY user@server "deploy_command"

  • Using Predefined Variables:

    yaml
    build:
    script:
    - echo "Building commit $CI_COMMIT_SHA on branch $CI_COMMIT_BRANCH"

  • Passing Variables Between Jobs (using artifacts):

    While variables defined in one job are not directly accessible in another, you can use artifacts to pass data.
    “`yaml
    job1:
    script:
    – echo “VALUE=$(date +%s)” > output.txt
    artifacts:
    paths:
    – output.txt

    job2:
    needs: [job1]
    script:
    – export MY_VALUE=$(cat output.txt)
    – echo “The value from job1 is: $MY_VALUE”
    “`

7. Best Practices

  • Use Project-Level Variables: For most configurations, define variables at the project level through the GitLab UI.
  • Mask Sensitive Variables: Always mask variables that contain sensitive information.
  • Use Protected Variables: Protect variables used for deployments to sensitive environments.
  • Avoid Hardcoding: Never hardcode sensitive information or environment-specific details in your .gitlab-ci.yml file or scripts.
  • Document Your Variables: Clearly document the purpose and expected values of your variables, either in comments within your .gitlab-ci.yml file or in a separate document.
  • Use Descriptive Variable Names: Use meaningful names that clearly indicate the purpose of the variable.
  • Consider Group-Level Variables: For configurations shared across multiple projects, use group-level variables.
  • Be Mindful of Precedence: Carefully consider variable precedence when defining variables at different scopes.
  • Use file type for secrets: Secrets that come in file format, such as certificates, should be stored as File variables.
  • Review and Audit: Regularly review your CI/CD variables to ensure they are still relevant, secure, and up-to-date.

By mastering GitLab CI/CD variables, you can create flexible, secure, and maintainable pipelines that streamline your software development workflow. This comprehensive guide provides the foundation you need to effectively use variables in your projects. Remember to consult the official GitLab documentation for the most up-to-date information and detailed specifications.

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top