OpenShift Commons

Support for OpenShift Commons calls & demos

Openshift Pipelines

OpenShift Pipelines is a cloud-native, continuous integration and delivery (CI/CD) solution for building pipelines using [Tekton](https://tekton.dev). Tekton is a flexible, Kubernetes-native, open-source CI/CD framework that enables automating deployments across multiple platforms (Kubernetes, serverless, VMs, etc) by abstracting away the underlying details.

OpenShift Pipelines features:

  • Standard CI/CD pipeline definition based on Tekton
  • Build images with Kubernetes tools such as S2I, Buildah, Buildpacks, Kaniko, etc
  • Deploy applications to multiple platforms such as Kubernetes, serverless and VMs
  • Easy to extend and integrate with existing tools
  • Scale pipelines on-demand
  • Portable across any Kubernetes platform
  • Designed for microservices and decentralized teams
  • Integrated with the OpenShift Developer Console

Prerequisite

You need an OpenShift 4 cluster in order to complete this tutorial. If you don’t have an existing cluster, go to http://try.openshift.com and register for free in order to get an OpenShift 4 cluster up and running on AWS within minutes.

You will also use the Tekton CLI (tkn) through out this tutorial. Download the Tekton CLI and copy it to a location on your PATH.

Concepts

Tekton defines a number of Kubernetes custom resources as building blocks in order to standardize pipeline concepts and provide a terminology that is consistent across CI/CD solutions. These custom resources (CR) are an extension of Kubernetes that let users create and interact with these objects using kubectl and other Kubernetes tools.

The custom resources needed to define a pipeline are:

  • Task: a reusable, loosely coupled number of steps that perform a specific task (e.g., building a container image)
  • Pipeline: the definition of the pipeline and the ~Task~s that it should perform
  • PipelineResource: inputs (e.g., git repository) and outputs (e.g., image registry) to and out of a pipeline or task
  • TaskRun: the result of running an instance of task
  • PipelineRun: the result of running an instance of pipeline, which includes a number of ~TaskRun~s

Deploy sample application

Create the project

We are creating a project, openshift-comons. Building container images using build tools such as S2I, Buildah, Kaniko, etc require privileged access to the cluster. OpenShift default security settings do not allow privileged containers unless specifically configured. Create a service account for running pipelines and enable it to run privileged pods for building images. 

oc new-project pipeline-demo
oc create serviceaccount pipeline
oc adm policy add-scc-to-user privileged -z pipeline
oc adm policy add-role-to-user edit -z pipeline

The application

We will use the Spring PetClinic sample application during this tutorial, which is a simple Spring Boot application. 

---
apiVersion: image.openshift.io/v1
kind: ImageStream
metadata:
  labels:
    app: spring-petclinic
  name: spring-petclinic
---
apiVersion: apps.openshift.io/v1
kind: DeploymentConfig
metadata:
  labels:
    app: spring-petclinic
  name: spring-petclinic
spec:
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    app: spring-petclinic
    deploymentconfig: spring-petclinic
  strategy:
    activeDeadlineSeconds: 21600
    resources: {}
    rollingParams:
      intervalSeconds: 1
      maxSurge: 25%
      maxUnavailable: 25%
      timeoutSeconds: 600
      updatePeriodSeconds: 1
    type: Rolling
  template:
    metadata:
      labels:
        app: spring-petclinic
        deploymentconfig: spring-petclinic
    spec:
      containers:
      - image: spring-petclinic:latest
        imagePullPolicy: Always
        livenessProbe:
          failureThreshold: 3
          httpGet:
            path: /
            port: 8080
            scheme: HTTP
          initialDelaySeconds: 45
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 1
        name: spring-petclinic
        ports:
        - containerPort: 8080
          protocol: TCP
        - containerPort: 8443
          protocol: TCP
        - containerPort: 8778
          protocol: TCP
        readinessProbe:
          failureThreshold: 3
          httpGet:
            path: /
            port: 8080
            scheme: HTTP
          initialDelaySeconds: 45
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 5
        resources: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      terminationGracePeriodSeconds: 30
  test: false
  triggers:
  - imageChangeParams:
      containerNames:
      - spring-petclinic
      from:
        kind: ImageStreamTag
        name: spring-petclinic:latest
        namespace: pipeline-demo
    type: ImageChange
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: spring-petclinic
  name: spring-petclinic
spec:
  ports:
  - name: 8080-tcp
    port: 8080
    protocol: TCP
    targetPort: 8080
  - name: 8443-tcp
    port: 8443
    protocol: TCP
    targetPort: 8443
  - name: 8778-tcp
    port: 8778
    protocol: TCP
    targetPort: 8778
  selector:
    app: spring-petclinic
    deploymentconfig: spring-petclinic
  sessionAffinity: None
  type: ClusterIP
---
apiVersion: route.openshift.io/v1
kind: Route
metadata:
  labels:
    app: spring-petclinic
  name: spring-petclinic
spec:
  port:
    targetPort: 8080-tcp
  to:
    kind: Service
    name: spring-petclinic
    weight: 100

The tasks

  • openshift-client from the upstream catalog (see here) 

    apiVersion: tekton.dev/v1alpha1
kind: Task
metadata:
  name: openshift-client
spec:
  inputs:
    params:
      - name: ARGS
        description: The OpenShift CLI arguments to run
        default: help
  steps:
    - name: oc
      image: quay.io/openshift-pipeline/openshift-cli:0.5.0
      command: ["/usr/local/bin/oc"]
      args:
        - "${inputs.params.ARGS}"

  • s2i-java from our downstream catalog (see here) 

    apiVersion: tekton.dev/v1alpha1
kind: Task
metadata:
  name: s2i-java-8
spec:
  inputs:
    resources:
      - name: source
        type: git
    params:
      - name: PATH_CONTEXT
        description: The location of the path to run s2i from.
        default: .
      - name: TLSVERIFY
        description: Verify the TLS on the registry endpoint (for push/pull to a non-TLS registry)
        default: "true"
  outputs:
    resources:
      - name: image
        type: image
  steps:
    - name: generate
      image: quay.io/openshift-pipeline/s2i
      workingdir: /workspace/source
      command: ['s2i', 'build', '${inputs.params.PATH_CONTEXT}', 'registry.access.redhat.com/redhat-openjdk-18/openjdk18-openshift', '--image-scripts-url', 'image:///usr/local/s2i', '--as-dockerfile', '/gen-source/Dockerfile.gen']
      volumeMounts:
        - name: gen-source
          mountPath: /gen-source
    - name: build
      image: quay.io/buildah/stable
      workingdir: /gen-source
      command: ['buildah', 'bud', '--tls-verify=${inputs.params.TLSVERIFY}', '--layers', '-f', '/gen-source/Dockerfile.gen', '-t', '${outputs.resources.image.url}', '.']
      volumeMounts:
        - name: varlibcontainers
          mountPath: /var/lib/containers
        - name: gen-source
          mountPath: /gen-source
      securityContext:
        privileged: true
    - name: push
      image: quay.io/buildah/stable
      command: ['buildah', 'push', '--tls-verify=${inputs.params.TLSVERIFY}', '${outputs.resources.image.url}', 'docker://${outputs.resources.image.url}']
      volumeMounts:
        - name: varlibcontainers
          mountPath: /var/lib/containers
      securityContext:
        privileged: true
  volumes:
    - name: varlibcontainers
      emptyDir: {}
    - name: gen-source
      emptyDir: {}

The pipeline

A pipeline defines a number of tasks that should be executed and how they interact with each other via their inputs and outputs.

Here is the YAML file that represents the above pipeline: 

apiVersion: tekton.dev/v1alpha1
kind: Pipeline
metadata:
  name: petclinic-deploy-pipeline
spec:
  resources:
  - name: app-git
    type: git
  - name: app-image
    type: image
  tasks:
  - name: build
    taskRef:
      name: s2i-java-8
    params:
      - name: TLSVERIFY
        value: "false"
    resources:
      inputs:
      - name: source
        resource: app-git
      outputs:
      - name: image
        resource: app-image
  - name: deploy
    taskRef:
      name: openshift-client
    runAfter:
      - build
    params:
    - name: ARGS
      value: "rollout latest spring-petclinic"

This pipeline performs the following:

  1. Clones the source code of the application from a Git repository (app-git resource)
  2. Builds the container image using the s2i-java-8 task that generates a Dockerfile for the application and uses Buildah to build the image
  3. The application image is pushed to an image registry (app-image resource)
  4. The new application image is deployed on OpenShift using the openshift-cli

You might have noticed that there are no references to the PetClinic Git repository and its image in the registry. That’s because ~Pipeline~s in Tekton are designed to be generic and re-usable across environments and stages through the application’s lifecycle. ~Pipeline~s abstract away the specifics of the Git source repository and image to be produced as ~resource~s. When triggering a pipeline, you can provide different Git repositories and image registries to be used during pipeline execution. Be patient! You will do that in a little bit in the next section.

The resources

Now that the pipeline is created, you can trigger it to execute the tasks specified in the pipeline. Triggering pipelines is an area that is under development and in the next release it will be possible to be done via the OpenShift web console and Tekton CLI. In this tutorial, you will trigger the pipeline through creating the Kubernetes objects (the hard way!) in order to learn the mechanics of triggering.

First, you should create a number of ~PipelineResource~s that contain the specifics of the Git repository and image registry to be used in the pipeline during execution. Expectedly, these are also reusable across multiple pipelines. 

---
apiVersion: tekton.dev/v1alpha1
kind: PipelineResource
metadata:
  name: petclinic-image
spec:
  type: image
  params:
  - name: url
    value: image-registry.openshift-image-registry.svc:5000/pipeline-demo/spring-petclinic
---
apiVersion: tekton.dev/v1alpha1
kind: PipelineResource
metadata:
  name: petclinic-git
spec:
  type: git
  params:
  - name: url
    value: https://github.com/spring-projects/spring-petclinic

Run the pipeline

A PipelineRun is how you can start a pipeline and tie it to the Git and image resources that should be used for this specific invocation. You can start the pipeline using the CLI: 

tkn pipeline start petclinic-deploy-pipeline \
          -r app-git=petclinic-git \
          -r app-image=petclinic-image \
          -s pipeline

tkn features

tkn pipeline logs -f
tkn pipeline start petclinic-deploy-pipeline --last

tkn resource list
tkn task list
tkn taskrun list
tkn pipeline list
tkn pipelinerun list

# same with logs, descibe, …

Additionnals Tasks and Pipeline

Tasks

apiVersion: tekton.dev/v1alpha1
kind: Task
metadata:
  name: golangci-lint
spec:
  inputs:
    params:
    - name: package
      description: base package (and its children) under validation
    - name: flags
      description: flags to use for the test command
      default: --verbose
    - name: version
      default: golangci-lint version to use
      default: "v1.17.1"
    - name: GOOS
      description: "running operating system target"
      default: linux
    - name: GOARCH
      description: "running architecture target"
      default: amd64
    - name: GO111MODULE
      description: "value of module support"
      default: auto
    resources:
    - name: source
      type: git
      targetPath: src/${inputs.params.package}
  steps:
  - name: lint
    image: golangci/golangci-lint:${inputs.params.version}
    workingdir: /workspace/src/${inputs.params.package}
    command:
    - /bin/bash
    args:
    - -c
    - "golangci-lint run ${inputs.params.flags}"
    env:
    - name: GOPATH
      value: /workspace
    - name: GOOS
      value: "${inputs.params.GOOS}"
    - name: GOARCH
      value: "${inputs.params.GOARCH}"
    - name: GO111MODULE
      value: "${inputs.params.GO111MODULE}"
---
apiVersion: tekton.dev/v1alpha1
kind: Task
metadata:
  name: golang-test
spec:
  inputs:
    params:
    - name: package
      description: package (and its children) under test
    - name: packages
      description: "packages to test (default: ./...)"
      default: "./..."
    - name: version
      description: golang version to use for tests
      default: "1.12"
    - name: flags
      description: flags to use for the test command
      default: -race -cover -v
    - name: GOOS
      description: "running program's operating system target"
      default: linux
    - name: GOARCH
      description: "running program's architecture target"
      default: amd64
    - name: GO111MODULE
      description: "value of module support"
      default: auto
    resources:
    - name: source
      type: git
      targetPath: src/${inputs.params.package}
  steps:
  - name: unit-test
    image: golang:${inputs.params.version}
    workingdir: /workspace/src/${inputs.params.package}
    command:
    - /bin/bash
    args:
    - -c
    - "go test ${inputs.params.flags} ${inputs.params.packages}"
    env:
    - name: GOPATH
      value: /workspace
    - name: GOOS
      value: "${inputs.params.GOOS}"
    - name: GOARCH
      value: "${inputs.params.GOARCH}"
    - name: GO111MODULE
      value: "${inputs.params.GO111MODULE}"
---
apiVersion: tekton.dev/v1alpha1
kind: Task
metadata:
  name: golang-build
spec:
  inputs:
    params:
    - name: package
      description: base package to build in
    - name: packages
      description: "packages to build (default: ./cmd/...)"
      default: "./cmd/..."
    - name: version
      description: golang version to use for builds
      default: "1.12"
    - name: flags
      description: flags to use for the test command
      default: -v
    - name: GOOS
      description: "running program's operating system target"
      default: linux
    - name: GOARCH
      description: "running program's architecture target"
      default: amd64
    - name: GO111MODULE
      description: "value of module support"
      default: auto
    resources:
    - name: source
      type: git
      targetPath: src/${inputs.params.package}
  steps:
  - name: build
    image: golang:${inputs.params.version}
    workingdir: /workspace/src/${inputs.params.package}
    command:
    - /bin/bash
    args:
    - -c
    - "go build ${inputs.params.flags} ${inputs.params.packages}"
    env:
    - name: GOPATH
      value: /workspace
    - name: GOOS
      value: "${inputs.params.GOOS}"
    - name: GOARCH
      value: "${inputs.params.GOARCH}"
    - name: GO111MODULE
      value: "${inputs.params.GO111MODULE}"

Pipelines

---
apiVersion: tekton.dev/v1alpha1
kind: Pipeline
metadata:
  name: cli-build-pipeline
spec:
  params:
  - name: package
    description: package to release
    default: github.com/tektoncd/cli
  resources:
  - name: source-repo
    type: git
  tasks:
    - name: lint
      taskRef:
        name: golangci-lint
      params:
        - name: package
          value: ${params.package}
        - name: flags
          value: -v
      resources:
        inputs:
          - name: source
            resource: source-repo
    - name: unit-tests
      runAfter: [lint]
      taskRef:
        name: golang-test
      params:
        - name: package
          value: ${params.package}
      resources:
        inputs:
          - name: source
            resource: source-repo
    - name: build
      runAfter: [lint]
      taskRef:
        name: golang-build
      params:
        - name: package
          value: ${params.package}
      resources:
        inputs:
          - name: source
            resource: source-repo

Resources

apiVersion: tekton.dev/v1alpha1
kind: PipelineResource
metadata:
  name: tektoncd-cli-git
spec:
  type: git
  params:
  - name: revision
    value: master
  - name: url
    value: https://github.com/tektoncd/cli