OpenShift Commons
Support for OpenShift Commons calls & demos

Table of Contents

1 Openshift Pipelines

OpenShift Pipelines is a cloud-native, continuous integration and delivery (CI/CD) solution for building pipelines using [Tekton](https://tekton.dev). Tekton is a flexible, Kubernetes-native, open-source CI/CD framework that enables automating deployments across multiple platforms (Kubernetes, serverless, VMs, etc) by abstracting away the underlying details.

OpenShift Pipelines features:

  • Standard CI/CD pipeline definition based on Tekton
  • Build images with Kubernetes tools such as S2I, Buildah, Buildpacks, Kaniko, etc
  • Deploy applications to multiple platforms such as Kubernetes, serverless and VMs
  • Easy to extend and integrate with existing tools
  • Scale pipelines on-demand
  • Portable across any Kubernetes platform
  • Designed for microservices and decentralized teams
  • Integrated with the OpenShift Developer Console

1.1 Prerequisite

You need an OpenShift 4 cluster in order to complete this tutorial. If you don’t have an existing cluster, go to http://try.openshift.com and register for free in order to get an OpenShift 4 cluster up and running on AWS within minutes.

You will also use the Tekton CLI (tkn) through out this tutorial. Download the Tekton CLI and copy it to a location on your PATH.

1.2 Concepts

Tekton defines a number of Kubernetes custom resources as building blocks in order to standardize pipeline concepts and provide a terminology that is consistent across CI/CD solutions. These custom resources (CR) are an extension of Kubernetes that let users create and interact with these objects using kubectl and other Kubernetes tools.

The custom resources needed to define a pipeline are:

  • Task: a reusable, loosely coupled number of steps that perform a specific task (e.g., building a container image)
  • Pipeline: the definition of the pipeline and the ~Task~s that it should perform
  • PipelineResource: inputs (e.g., git repository) and outputs (e.g., image registry) to and out of a pipeline or task
  • TaskRun: the result of running an instance of task
  • PipelineRun: the result of running an instance of pipeline, which includes a number of ~TaskRun~s

1.3 Deploy sample application

1.3.1 Create the project

We are creating a project, openshift-comons. Building container images using build tools such as S2I, Buildah, Kaniko, etc require privileged access to the cluster. OpenShift default security settings do not allow privileged containers unless specifically configured. Create a service account for running pipelines and enable it to run privileged pods for building images.

oc new-project pipeline-demo
oc create serviceaccount pipeline
oc adm policy add-scc-to-user privileged -z pipeline
oc adm policy add-role-to-user edit -z pipeline

1.3.2 The application

We will use the Spring PetClinic sample application during this tutorial, which is a simple Spring Boot application.

---
apiVersion: image.openshift.io/v1
kind: ImageStream
metadata:
  labels:
    app: spring-petclinic
  name: spring-petclinic
---
apiVersion: apps.openshift.io/v1
kind: DeploymentConfig
metadata:
  labels:
    app: spring-petclinic
  name: spring-petclinic
spec:
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    app: spring-petclinic
    deploymentconfig: spring-petclinic
  strategy:
    activeDeadlineSeconds: 21600
    resources: {}
    rollingParams:
      intervalSeconds: 1
      maxSurge: 25%
      maxUnavailable: 25%
      timeoutSeconds: 600
      updatePeriodSeconds: 1
    type: Rolling
  template:
    metadata:
      labels:
        app: spring-petclinic
        deploymentconfig: spring-petclinic
    spec:
      containers:
      - image: spring-petclinic:latest
        imagePullPolicy: Always
        livenessProbe:
          failureThreshold: 3
          httpGet:
            path: /
            port: 8080
            scheme: HTTP
          initialDelaySeconds: 45
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 1
        name: spring-petclinic
        ports:
        - containerPort: 8080
          protocol: TCP
        - containerPort: 8443
          protocol: TCP
        - containerPort: 8778
          protocol: TCP
        readinessProbe:
          failureThreshold: 3
          httpGet:
            path: /
            port: 8080
            scheme: HTTP
          initialDelaySeconds: 45
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 5
        resources: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      terminationGracePeriodSeconds: 30
  test: false
  triggers:
  - imageChangeParams:
      containerNames:
      - spring-petclinic
      from:
        kind: ImageStreamTag
        name: spring-petclinic:latest
        namespace: pipeline-demo
    type: ImageChange
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: spring-petclinic
  name: spring-petclinic
spec:
  ports:
  - name: 8080-tcp
    port: 8080
    protocol: TCP
    targetPort: 8080
  - name: 8443-tcp
    port: 8443
    protocol: TCP
    targetPort: 8443
  - name: 8778-tcp
    port: 8778
    protocol: TCP
    targetPort: 8778
  selector:
    app: spring-petclinic
    deploymentconfig: spring-petclinic
  sessionAffinity: None
  type: ClusterIP
---
apiVersion: route.openshift.io/v1
kind: Route
metadata:
  labels:
    app: spring-petclinic
  name: spring-petclinic
spec:
  port:
    targetPort: 8080-tcp
  to:
    kind: Service
    name: spring-petclinic
    weight: 100

1.3.3 The tasks

  • openshift-client from the upstream catalog (see here)

    apiVersion: tekton.dev/v1alpha1
    kind: Task
    metadata:
      name: openshift-client
    spec:
      inputs:
        params:
          - name: ARGS
            description: The OpenShift CLI arguments to run
            default: help
      steps:
        - name: oc
          image: quay.io/openshift-pipeline/openshift-cli:0.5.0
          command: ["/usr/local/bin/oc"]
          args:
            - "${inputs.params.ARGS}"
    
  • s2i-java from our downstream catalog (see here)

    apiVersion: tekton.dev/v1alpha1
    kind: Task
    metadata:
      name: s2i-java-8
    spec:
      inputs:
        resources:
          - name: source
            type: git
        params:
          - name: PATH_CONTEXT
            description: The location of the path to run s2i from.
            default: .
          - name: TLSVERIFY
            description: Verify the TLS on the registry endpoint (for push/pull to a non-TLS registry)
            default: "true"
      outputs:
        resources:
          - name: image
            type: image
      steps:
        - name: generate
          image: quay.io/openshift-pipeline/s2i
          workingdir: /workspace/source
          command: ['s2i', 'build', '${inputs.params.PATH_CONTEXT}', 'registry.access.redhat.com/redhat-openjdk-18/openjdk18-openshift', '--image-scripts-url', 'image:///usr/local/s2i', '--as-dockerfile', '/gen-source/Dockerfile.gen']
          volumeMounts:
            - name: gen-source
              mountPath: /gen-source
        - name: build
          image: quay.io/buildah/stable
          workingdir: /gen-source
          command: ['buildah', 'bud', '--tls-verify=${inputs.params.TLSVERIFY}', '--layers', '-f', '/gen-source/Dockerfile.gen', '-t', '${outputs.resources.image.url}', '.']
          volumeMounts:
            - name: varlibcontainers
              mountPath: /var/lib/containers
            - name: gen-source
              mountPath: /gen-source
          securityContext:
            privileged: true
        - name: push
          image: quay.io/buildah/stable
          command: ['buildah', 'push', '--tls-verify=${inputs.params.TLSVERIFY}', '${outputs.resources.image.url}', 'docker://${outputs.resources.image.url}']
          volumeMounts:
            - name: varlibcontainers
              mountPath: /var/lib/containers
          securityContext:
            privileged: true
      volumes:
        - name: varlibcontainers
          emptyDir: {}
        - name: gen-source
          emptyDir: {}
    

1.3.4 The pipeline

A pipeline defines a number of tasks that should be executed and how they interact with each other via their inputs and outputs.

Here is the YAML file that represents the above pipeline:

apiVersion: tekton.dev/v1alpha1
kind: Pipeline
metadata:
  name: petclinic-deploy-pipeline
spec:
  resources:
  - name: app-git
    type: git
  - name: app-image
    type: image
  tasks:
  - name: build
    taskRef:
      name: s2i-java-8
    params:
      - name: TLSVERIFY
        value: "false"
    resources:
      inputs:
      - name: source
        resource: app-git
      outputs:
      - name: image
        resource: app-image
  - name: deploy
    taskRef:
      name: openshift-client
    runAfter:
      - build
    params:
    - name: ARGS
      value: "rollout latest spring-petclinic"

This pipeline performs the following:

  1. Clones the source code of the application from a Git repository (app-git resource)
  2. Builds the container image using the s2i-java-8 task that generates a Dockerfile for the application and uses Buildah to build the image
  3. The application image is pushed to an image registry (app-image resource)
  4. The new application image is deployed on OpenShift using the openshift-cli

You might have noticed that there are no references to the PetClinic Git repository and its image in the registry. That’s because ~Pipeline~s in Tekton are designed to be generic and re-usable across environments and stages through the application’s lifecycle. ~Pipeline~s abstract away the specifics of the Git source repository and image to be produced as ~resource~s. When triggering a pipeline, you can provide different Git repositories and image registries to be used during pipeline execution. Be patient! You will do that in a little bit in the next section.

1.3.5 The resources

Now that the pipeline is created, you can trigger it to execute the tasks specified in the pipeline. Triggering pipelines is an area that is under development and in the next release it will be possible to be done via the OpenShift web console and Tekton CLI. In this tutorial, you will trigger the pipeline through creating the Kubernetes objects (the hard way!) in order to learn the mechanics of triggering.

First, you should create a number of ~PipelineResource~s that contain the specifics of the Git repository and image registry to be used in the pipeline during execution. Expectedly, these are also reusable across multiple pipelines.

---
apiVersion: tekton.dev/v1alpha1
kind: PipelineResource
metadata:
  name: petclinic-image
spec:
  type: image
  params:
  - name: url
    value: image-registry.openshift-image-registry.svc:5000/pipeline-demo/spring-petclinic
---
apiVersion: tekton.dev/v1alpha1
kind: PipelineResource
metadata:
  name: petclinic-git
spec:
  type: git
  params:
  - name: url
    value: https://github.com/spring-projects/spring-petclinic

1.3.6 Run the pipeline

A PipelineRun is how you can start a pipeline and tie it to the Git and image resources that should be used for this specific invocation. You can start the pipeline using the CLI:

tkn pipeline start petclinic-deploy-pipeline \
          -r app-git=petclinic-git \
          -r app-image=petclinic-image \
          -s pipeline

1.3.7 tkn features

tkn pipeline logs -f
tkn pipeline start petclinic-deploy-pipeline --last

tkn resource list
tkn task list
tkn taskrun list
tkn pipeline list
tkn pipelinerun list

# same with logs, descibe, …

1.4 Additionnals Tasks and Pipeline

1.4.1 Tasks

apiVersion: tekton.dev/v1alpha1
kind: Task
metadata:
  name: golangci-lint
spec:
  inputs:
    params:
    - name: package
      description: base package (and its children) under validation
    - name: flags
      description: flags to use for the test command
      default: --verbose
    - name: version
      default: golangci-lint version to use
      default: "v1.17.1"
    - name: GOOS
      description: "running operating system target"
      default: linux
    - name: GOARCH
      description: "running architecture target"
      default: amd64
    - name: GO111MODULE
      description: "value of module support"
      default: auto
    resources:
    - name: source
      type: git
      targetPath: src/${inputs.params.package}
  steps:
  - name: lint
    image: golangci/golangci-lint:${inputs.params.version}
    workingdir: /workspace/src/${inputs.params.package}
    command:
    - /bin/bash
    args:
    - -c
    - "golangci-lint run ${inputs.params.flags}"
    env:
    - name: GOPATH
      value: /workspace
    - name: GOOS
      value: "${inputs.params.GOOS}"
    - name: GOARCH
      value: "${inputs.params.GOARCH}"
    - name: GO111MODULE
      value: "${inputs.params.GO111MODULE}"
---
apiVersion: tekton.dev/v1alpha1
kind: Task
metadata:
  name: golang-test
spec:
  inputs:
    params:
    - name: package
      description: package (and its children) under test
    - name: packages
      description: "packages to test (default: ./...)"
      default: "./..."
    - name: version
      description: golang version to use for tests
      default: "1.12"
    - name: flags
      description: flags to use for the test command
      default: -race -cover -v
    - name: GOOS
      description: "running program's operating system target"
      default: linux
    - name: GOARCH
      description: "running program's architecture target"
      default: amd64
    - name: GO111MODULE
      description: "value of module support"
      default: auto
    resources:
    - name: source
      type: git
      targetPath: src/${inputs.params.package}
  steps:
  - name: unit-test
    image: golang:${inputs.params.version}
    workingdir: /workspace/src/${inputs.params.package}
    command:
    - /bin/bash
    args:
    - -c
    - "go test ${inputs.params.flags} ${inputs.params.packages}"
    env:
    - name: GOPATH
      value: /workspace
    - name: GOOS
      value: "${inputs.params.GOOS}"
    - name: GOARCH
      value: "${inputs.params.GOARCH}"
    - name: GO111MODULE
      value: "${inputs.params.GO111MODULE}"
---
apiVersion: tekton.dev/v1alpha1
kind: Task
metadata:
  name: golang-build
spec:
  inputs:
    params:
    - name: package
      description: base package to build in
    - name: packages
      description: "packages to build (default: ./cmd/...)"
      default: "./cmd/..."
    - name: version
      description: golang version to use for builds
      default: "1.12"
    - name: flags
      description: flags to use for the test command
      default: -v
    - name: GOOS
      description: "running program's operating system target"
      default: linux
    - name: GOARCH
      description: "running program's architecture target"
      default: amd64
    - name: GO111MODULE
      description: "value of module support"
      default: auto
    resources:
    - name: source
      type: git
      targetPath: src/${inputs.params.package}
  steps:
  - name: build
    image: golang:${inputs.params.version}
    workingdir: /workspace/src/${inputs.params.package}
    command:
    - /bin/bash
    args:
    - -c
    - "go build ${inputs.params.flags} ${inputs.params.packages}"
    env:
    - name: GOPATH
      value: /workspace
    - name: GOOS
      value: "${inputs.params.GOOS}"
    - name: GOARCH
      value: "${inputs.params.GOARCH}"
    - name: GO111MODULE
      value: "${inputs.params.GO111MODULE}"

1.4.2 Pipelines

---
apiVersion: tekton.dev/v1alpha1
kind: Pipeline
metadata:
  name: cli-build-pipeline
spec:
  params:
  - name: package
    description: package to release
    default: github.com/tektoncd/cli
  resources:
  - name: source-repo
    type: git
  tasks:
    - name: lint
      taskRef:
        name: golangci-lint
      params:
        - name: package
          value: ${params.package}
        - name: flags
          value: -v
      resources:
        inputs:
          - name: source
            resource: source-repo
    - name: unit-tests
      runAfter: [lint]
      taskRef:
        name: golang-test
      params:
        - name: package
          value: ${params.package}
      resources:
        inputs:
          - name: source
            resource: source-repo
    - name: build
      runAfter: [lint]
      taskRef:
        name: golang-build
      params:
        - name: package
          value: ${params.package}
      resources:
        inputs:
          - name: source
            resource: source-repo

1.4.3 Resources

apiVersion: tekton.dev/v1alpha1
kind: PipelineResource
metadata:
  name: tektoncd-cli-git
spec:
  type: git
  params:
  - name: revision
    value: master
  - name: url
    value: https://github.com/tektoncd/cli

Emacs 24.3.50.3 (Org mode 8.0.3)

Vincent Demeester. Last Updated 2019-09-27 Fri 10:27 (exported 2019-10-08 Tue 16:46).