Pipeline Builds in a DevOps Environment

Pipeline Builds in a DevOps Environment

Tuesday, October 10, 2017

The Jenkins Pipeline, or simply “Pipeline”, is a real game changer for Jenkins users and DevOps teams. Your development cycle that starts with writing code and ends with running your code in a production environment can be scripted with this handy plugin. This way the full process is automated and no manual interventions are needed, which drastically reduces the chance of user errors in the release process.

Pipelines are not only useful to automate your release cycle, but also make your developers responsible for bringing their code to production. Instead of only one or a few persons that are bothered with the release process, the whole team is aware of the development cycle and responsible for delivering working code.

DevOps environments result in shorter development and release cycles, higher productivity and better code quality. The cross-functional teams benefit from the improved communication, knowledge sharing, learning, innovation and experimentation.

Which steps are needed to create a release?

A good starting point to define a pipeline is to think which different steps are needed to bring your code to productions.

In your projects the following steps are defined:

  • Set-up / Configure a build environment.
  • Checkout your code.

  • Build your code. In all our projects the build process is independent of the environment on which it will be installed, so make sure you don’t use any environment specific settings.

  • Perform quality controls. This step consists out of two main tasks: running tests and perform code quality checks.

  • Deploy your code on a Continuous Integration environment.

  • Run functional tests. Since we now have a CI environment at our disposal, we can run some Selenium or other kinds tests.

  • Deploy the code on the test environment.

  • Deploy the code on the user acceptance environment. 

  • Deploy the code on the production environment.

Deployments on test, acceptance and production environments shouldn’t automatically be triggered by anyone at any time. Before these deployments are performed, someone should approve the deployment. Next to that, also not all code should be able to make it to the acceptance or production environment. Only code that is ready for a release should make it till these steps.

This way our Jenkins Pipeline looks like this:

Develop your pipeline as code

Your project should have store a “Jenkinsfile” in your version control system (eg. Git). That Jenkinsfile contains the pipeline and is tested like the rest of your code. Treating your pipeline as code enforces good discipline and also opens up a new world of features and capabilities like multi-branch, pull request detection, …

At the top of the Jenkinsfile we define one parameter: DEPLOYS. If the pipeline build is automatically triggered by code changes in your SCM (every 10 minutes a check is performed), you don’t wan this code to be deployed on your test, acceptance or production environment. The build is only triggered to see if the new code breaks your release cycle. In case you manually trigger the build, the parameter can be set by the user.

The pipeline also needs to be aware of the build environment. We need to tell it what tools we want to have at our disposal, together with other environment variables. In our example we want to use Maven 3 in combination with Java 8.

pipeline {
  agent any
  parameters {
    booleanParam(name: 'DEPLOYS', defaultValue: false, description: 'Use this build for deployment.')
 }
  triggers {
    pollSCM('H/10 * * * * ')
  }
  environment {
    JAVA_HOME="${tool 'Java8'}"
    PATH="${env.JAVA_HOME}/bin:${env.PATH}"
  }
  tools {
    maven "Maven3"
    jdk "Java8"
  }
  stages {
  }
}

Script your stages

Our build and release process consists out of multiple stages. Each stage has a unique name and steps that must be performed. A step can be for example:

  • Checkout your code
  • Run a shell command
  • Send a Hipchat message
  • Wait for user input
  • ...

The prepare stage checks out our SCM code

stage('Prepare') {
      steps {
        checkout scm
      }
    }

, while the build stage performs a maven build.

  stage('Build') {
      steps {
        sh "mvn -T 4 -P theme clean install"
      }
    }

These stages are followed by ‘Quality Control’, ‘Deploy to CI’ and ‘Functional Tests’ stages. All these steps are executed if the build was triggered by a code change or manually. This way the steps are useful for both continuous integration as release builds. If a step fails, the build process fails and the developer is informed about the failure.

In the end, if everything went well and you manually triggered the build, you want to deploy your code on the test environment. In this case the user was asked if he wanted to use the build for a deployment. If he said yes, the steps of the approval stage are executed and the user is asked to approve the deployment to the test environment. He has one hour to react on the question: if he hasn’t approved the deployment in time, the build process is ended.

  stage('Approve deployment on TST') {
      when {
        expression { return params.DEPLOYS }
      }
      steps {
        timeout(time: 1, unit: 'HOURS') {
          script {
            env.DEPLOY_TST = input message: 'Approve deployment', parameters: [
              [$class: 'BooleanParameterDefinition', defaultValue: false, description: '', name: 'Approve deployment on TST']
            ]
          }
        }
      }
    }

When the deployment was approved, the build steps can be executed.

stage('Deploy on TST') {
      when {
        environment name: 'DEPLOY_TST', value: "true"
      }
      steps {
        …
      }
    }

Only in case of a successful deployment on the test environment a deployment on the acceptance environment can be done. The same is the case for production: only a successful deployment on acceptance can lead to a deployment on production.

stage('Approve deployment on UAT') {
      when {
        environment name: 'DEPLOY_TST', value: "true"
      }
      steps {
        timeout(time: 7, unit: 'DAYS') {
          script {
           env.DEPLOY_UAT = input message: 'Approve deployment', parameters: [
              [$class: 'BooleanParameterDefinition', defaultValue: false, description: '', name: 'Approve deployment on UAT']
            ]
          }
        }
      }
    }

Multi-pipeline project when using GitFlow

Of course your project contains not only a master version but a lot of branches can exist. Therefore a Multi-pipeline project is always recommended. Such a project scans your SMC for all branches and can trigger a pipeline build on each branch. This way you’re always sure that a branch doesn’t break your master code.

Each change on a branch triggers a continuous integration build for that branch. But you don’t want every branch to be deployable on all environments. If a user manually triggers a feature branch, this incomplete feature can accidentally be deployed on your UAT, or even worse, your production environment.

Using GitFlow or another type of branching model can come the the rescue in this case. Only release or master branches should offer you the possibility to deploy your code on acceptance, and only the master branch should be deployable on production. So the 'Approve deployment on PRD’ stage should look something like:

    stage('Approve deployment on PRD') {
      when {
        branch 'master'
        environment name: 'DEPLOY_TST', value: "true"
        environment name: 'DEPLOY_UAT', value: "true"
      }
      steps {
        timeout(time: 14, unit: 'DAYS') {
          script {
           env.DEPLOY_PRD = input message: 'Approve deployment', parameters: [
              [$class: 'BooleanParameterDefinition', defaultValue: false, description: '', name: 'Approve deployment on PRD']
            ]
          }
        }
      }
    }