| | |
| | | # An Enslaved Hope |
| | | |
| | | > In this exercise we'll break free from the chains of point'n'click Jenkins by introducing pipeline as code in the form of `Jenkinsfile`. Following this we will introduce some new Jenkins slaves that will be used in later labs. |
| | | > In this exercise we'll break free from the chains of point'n'click Jenkins by introducing pipeline as code in the form of `Jenkinsfile`. Following this we will introduce some new Jenkins slaves that will be used in later exercises. |
| | | |
| | | ![jenkins-fail-meme](../images/exercise4/jenkins-fail-meme.jpeg) |
| | | |
| | | There are a number of ways pipeline as code can be achieved in Jenkins. |
| | | * The Job DSL Plugin - this is a slightly older but very functional DSL mechanism to create reusable pipelines. Create a `groovy` file to run Jenkins Domain Specific Language to create jobs, functions and other items. In Jenkins; you then can execute this file which will build all of the config.xml files needed for each Job. |
| | |
| | | As a learner you will be able to |
| | | - Use a Jenkinsfile to create a declarative pipeline to build, bake and deploy the Todolist App |
| | | - Identify the differences between scripted, declarative and DSL pipelines |
| | | - Create Jenkins slave nodes for use in builds in future labs |
| | | - Create Jenkins slave nodes for use in builds in future exercises |
| | | |
| | | ## Tools and Frameworks |
| | | > Name of tool - short description and link to docs or website |
| | |
| | | _____ |
| | | |
| | | ## 10,000 Ft View |
| | | > The goal of this exercise is to move to using the Jenkinsfile in the todolist-api and todolist-fe projects. Additionally we will create new slaves for use in the next lab |
| | | > The goal of this exercise is to move to using the Jenkinsfile in the todolist-api and todolist-fe projects. Additionally we will create new slaves for use in the next exercise. |
| | | |
| | | 2. On Jenkins; create a multibranch pipeline project to scan the GitLab endpoint for each app. Use the Jenkinsfile provided to run the stages. Replace the `<YOUR_NAME>` with appropriate variable. |
| | | |
| | |
| | | > This is a fairly structured guide with references to exact filenames and sections of text to be added. |
| | | |
| | | ### Part 1 - The Jenkinsfile |
| | | > _In this exercise we'll replace the Pipeline we created in Lab 2 with a Jenkinsfile approach_ |
| | | > _In this exercise we'll replace the Pipeline we created in Exercise 2 with a Jenkinsfile approach_ |
| | | |
| | | 2. On your terminal navigate to your `todolist-api` project and checkout the pipeline feature branch that's been already created for you. |
| | | ```bash |
| | | $ git checkout feature/jenkinsfile |
| | | git checkout feature/jenkinsfile |
| | | ``` |
| | | |
| | | 2. Open up your `todolist-api` application in your favourite editor and move to the `Jenkinsfile` in the root of the project. The highlevel structure of the file is shown collapsed below. |
| | |
| | | * `post {}` hook is used to specify the post-build-actions. Jenkins declarative provides very useful callbacks for `success`, `failure` and `always` which are useful for controlling the job flow |
| | | * `when {}` is used for flow control. It can be used at stage level and be used to stop pipeline entering that stage. eg when branch is master; deploy to `test` environment. |
| | | |
| | | 2. The Jenkinsfile is mostly complete to do all the testing etc that was done in previous labs. Some minor changes will be needed to orchestrate namespaces. Find and replace all instances of `<YOUR_NAME>` in the Jenkinsfile. Update the `<GIT_USERNAME>` to the one you login to the cluster with; this variable is used in the namespace of your git projects when checking out code etc. Ensure the `GITLAB_DOMAIN` matches your git host. |
| | | 2. The Jenkinsfile is mostly complete to do all the testing etc that was done in previous exercises. Some minor changes will be needed to orchestrate namespaces. Find and replace all instances of `<YOUR_NAME>` in the Jenkinsfile. Update the `<GIT_USERNAME>` to the one you login to the cluster with; this variable is used in the namespace of your git projects when checking out code etc. Ensure the `GITLAB_DOMAIN` matches your git host. |
| | | ```groovy |
| | | environment { |
| | | // GLobal Vars |
| | |
| | | |
| | | GIT_SSL_NO_VERIFY = true |
| | | GIT_CREDENTIALS = credentials('jenkins-git-creds') |
| | | GITLAB_DOMAIN = "gitlab.apps.lader.rht-labs.com" |
| | | GITLAB_DOMAIN = "gitlab.apps.<SOME_DOMAIN>.com" |
| | | GITLAB_PROJECT = "<GIT_USERNAME>" |
| | | } |
| | | ``` |
| | | |
| | | 2. With these changes in place, push your changes to the `feature/jenkinsfile` branch. |
| | | ```bash |
| | | $ git add Jenkinsfile |
| | | $ git commit -m "ADD - namespace and git repo to pipeline" |
| | | $ git push |
| | | git add Jenkinsfile |
| | | ``` |
| | | ```bash |
| | | git commit -m "ADD - namespace and git repo to pipeline" |
| | | ``` |
| | | ```bash |
| | | git push |
| | | ``` |
| | | |
| | | 2. When the changes have been successfully pushed; Open Jenkins. |
| | |
| | | 2. On the job's configure page; set the Branch Sources to `git` |
| | | ![multibranch-select-git](../images/exercise4/multibranch-select-git.png) |
| | | |
| | | 2. Fill in the Git settings with your `todolist-api` GitLab url and set the credentials as you've done before. `https://gitlab.apps.<SOMEDOMAIN>.com/<YOUR_NAME>/todolist-api.git` |
| | | 2. Fill in the Git settings with your `todolist-api` GitLab url and set the credentials as you've done before. `https://gitlab.apps.lader.rht-labs.com/<YOUR_NAME>/todolist-api.git` |
| | | ![multibranch-git](../images/exercise4/multibranch-git.png) |
| | | |
| | | 2. Set the `Scan Multibranch Pipeline Triggers` to be periodic and the interval to 1 minute. This will poll the gitlab instance for new branches or change sets to build. |
| | |
| | | |
| | | 2. The pipeline file is setup to only run `bake` & `deploy` stages when on `master` or `develop` branch. This is to provide us with very fast feedback for team members working on feature or bug fix branches. Each time someone commits or creates a new branch a basic build with testing occurs to give very rapid feedback to the team. Let's now update our `master` and `develop` branches to include the Jenkinsfile and delete the feature branch. |
| | | ```bash |
| | | $ git checkout develop |
| | | $ git merge feature/jenkinsfile |
| | | git checkout develop |
| | | ``` |
| | | ```bash |
| | | git merge feature/jenkinsfile |
| | | # you may get merge conflicts at this point |
| | | $ git add . |
| | | $ git commit -m "Jenkinsfile updates" |
| | | $ git checkout master |
| | | $ git merge develop |
| | | $ git push -u origin --all |
| | | ``` |
| | | ```bash |
| | | git add . |
| | | ``` |
| | | ```bash |
| | | git commit -m "Jenkinsfile updates" |
| | | ``` |
| | | ```bash |
| | | git checkout master |
| | | ``` |
| | | ```bash |
| | | git merge develop |
| | | ``` |
| | | ```bash |
| | | git push -u origin --all |
| | | ``` |
| | | ```bash |
| | | # this is to delete the branch from the remote |
| | | $ git push origin :feature/jenkinsfile |
| | | git push origin :feature/jenkinsfile |
| | | ``` |
| | | |
| | | 2. Back on Jenkins we should see our `todolist-api` pipelines have changed with the `develop` and `master` now appearing. The feature was deleted so this job should have gone away. |
| | |
| | | |
| | | 2. We can move on to the `todolist-fe` job. The process is the same as before, checkout the feature branch |
| | | ```bash |
| | | $ cd todolist-fe |
| | | $ git checkout feature/jenkinsfile |
| | | cd todolist-fe |
| | | ``` |
| | | ```bash |
| | | git checkout feature/jenkinsfile |
| | | ``` |
| | | |
| | | 2. Open up your `todolist-fe` application in your favourite editor and move to the `Jenkinsfile` in the root of the project. Update all `<YOUR_NAME>` and `<GIT_USERNAME>` as you did before, including in the prepare environment steps. Check the `GITLAB_DOMAIN` is set too. |
| | |
| | | |
| | | 2. Commit your changes to your feature branch as you did previously. |
| | | ```bash |
| | | $ git add Jenkinsfile |
| | | $ git commit -m "ADD - namespace and git repo to pipeline" |
| | | $ git push |
| | | git add Jenkinsfile |
| | | ``` |
| | | ```bash |
| | | git commit -m "ADD - namespace and git repo to pipeline" |
| | | ``` |
| | | ```bash |
| | | git push |
| | | ``` |
| | | |
| | | 2. This time update your master and develop branches before creating config in Jenkins |
| | | ``` |
| | | $ git checkout develop |
| | | $ git merge feature/jenkinsfile |
| | | git checkout develop |
| | | ``` |
| | | ```bash |
| | | git merge feature/jenkinsfile |
| | | # you may get merge conflicts at this point |
| | | $ git add . |
| | | $ git commit -m "Jenkinsfile updates" |
| | | $ git checkout master |
| | | $ git merge develop |
| | | ``` |
| | | ```bash |
| | | git add . |
| | | ``` |
| | | ```bash |
| | | git commit -m "Jenkinsfile updates" |
| | | ``` |
| | | ```bash |
| | | git checkout master |
| | | ``` |
| | | ```bash |
| | | git merge develop |
| | | ``` |
| | | ```bash |
| | | # this is to delete the branch from the remote |
| | | $ git push origin :feature/jenkinsfile |
| | | $ git push -u origin --all |
| | | git push origin :feature/jenkinsfile |
| | | ``` |
| | | ```bash |
| | | git push -u origin --all |
| | | ``` |
| | | |
| | | 2. On Jenkins; create a new `Multibranch Pipeline` job called `todolist-fe`. |
| | |
| | | |
| | | 2. Open the `params/ocp-pipeline` file and update `PIPELINE_SOURCE_REPOSITORY_URL` with the git url of your project (Don't forget to add the `.git` at the end). For example: |
| | | ``` |
| | | PIPELINE_SOURCE_REPOSITORY_URL=https://gitlab.apps.lader.rht-labs.com/<GIT_USERNAME>/todolist-fe.git |
| | | PIPELINE_SOURCE_REPOSITORY_URL=https://gitlab.apps.<SOME_DOMAIN>.com/<GIT_USERNAME>/todolist-fe.git |
| | | PIPELINE_SOURCE_REPOSITORY_REF=develop |
| | | NAME=todolist-fe |
| | | ``` |
| | |
| | | |
| | | 2. Use the OpenShift Applier to create the cluster content |
| | | ```bash |
| | | $ cd .openshift-applier |
| | | $ ansible-playbook apply.yml -i inventory/ \ |
| | | cd .openshift-applier |
| | | ``` |
| | | ```bash |
| | | ansible-playbook apply.yml -i inventory/ \ |
| | | -e "filter_tags=pipeline" |
| | | ``` |
| | | |
| | | 2. With these changes in place, commit your changes to GitLab |
| | | ```bash |
| | | $ git add . |
| | | $ git commit -m "ADD - ocp pipeline in git repo" |
| | | $ git push |
| | | git add . |
| | | ``` |
| | | ```bash |
| | | git commit -m "ADD - ocp pipeline in git repo" |
| | | ``` |
| | | ```bash |
| | | git push |
| | | ``` |
| | | |
| | | 2. Login to your OpenShift Cluster and go to the `<YOUR_NAME>-ci-cd` namespace. On the side menu; hit Builds > Pipeline to see your newly created pipeline running in OCP Land. |
| | |
| | | |
| | | 2. Update the `todolist-api/.openshift-applier/params/ocp-pipeline` |
| | | ``` |
| | | PIPELINE_SOURCE_REPOSITORY_URL=https://gitlab.apps.lader.rht-labs.com/<GIT_USERNAME>/todolist-api.git |
| | | PIPELINE_SOURCE_REPOSITORY_URL=https://gitlab.apps.<SOME_DOMAIN>.com/<GIT_USERNAME>/todolist-api.git |
| | | PIPELINE_SOURCE_REPOSITORY_REF=develop |
| | | NAME=todolist-api |
| | | ``` |
| | | |
| | | 2. Use the OpenShift Applier to create the cluster content |
| | | ```bash |
| | | $ cd todolist-api/.openshift-applier |
| | | $ ansible-playbook apply.yml -i inventory/ \ |
| | | cd todolist-api/.openshift-applier |
| | | ``` |
| | | ```bash |
| | | ansible-playbook apply.yml -i inventory/ \ |
| | | -e "filter_tags=pipeline" |
| | | ``` |
| | | |
| | |
| | | |
| | | 2. Commit your changes to GitLab |
| | | ```bash |
| | | $ git add . |
| | | $ git commit -m "ADD - ocp pipeline in git repo" |
| | | $ git push |
| | | git add . |
| | | ``` |
| | | ```bash |
| | | git commit -m "ADD - ocp pipeline in git repo" |
| | | ``` |
| | | ```bash |
| | | git push |
| | | ``` |
| | | |
| | | ### Part 3 - Security Scanning Slaves |
| | |
| | | #### 3a - OWASP ZAP |
| | | > _OWASP ZAP (Zed Attack Proxy) is a free open source security tool used for finding security vulnerabilities in web applications._ |
| | | |
| | | 3. On your terminal; move to the `enablement-ci-cd` repo. We already have the `templates/jenkins-slave-generic-template.yml` template we're going to re-use from the previous lab so all we need is to check out the params file |
| | | 3. On your terminal; move to the `enablement-ci-cd` repo. We need to checkout a template for OpenShift to build our Jenkins Slave images and some parameters for the `zap` slave. |
| | | ```bash |
| | | $ git checkout exercise4/zap-and-arachni params/jenkins-slave-zap |
| | | git checkout exercise4/zap-and-arachni params/jenkins-slave-zap templates/jenkins-slave-generic-template.yml |
| | | ``` |
| | | |
| | | 3. This should have created the following files which we will fill out. We will use a `ZAP` image hosted on the `rht-labs/ci-cd` repo so there will be no `Dockerfile` needed as we did with the `jenkins-slave-npm` in exercise 2: |
| | | 3. This should have created the following files which we will fill out. We will use a `ZAP` image hosted on the `rht-labs/ci-cd` repo so there will be no `Dockerfile` needed: |
| | | - `params/jenkins-slave-zap` |
| | | |
| | | 3. Create an object in `inventory/host_vars/ci-cd-tooling.yml` called `jenkins-slave-zap` and add the following content: |
| | |
| | | tags: |
| | | - zap |
| | | ``` |
| | | ![zap-object](../images/exercise4/zap-project.png) |
| | | ![zap-object](../images/exercise4/zap-object.png) |
| | | |
| | | 3. Run the ansible playbook filtering with tag `zap` so only the zap build pods are run. |
| | | ```bash |
| | | $ ansible-playbook apply.yml -e target=tools \ |
| | | ansible-playbook apply.yml -e target=tools \ |
| | | -i inventory/ \ |
| | | -e "filter_tags=zap" |
| | | ``` |
| | | |
| | | 3. Head to (https://console.somedomain.com/console/project/<YOUR_NAME>-ci-cd/browse/builds) on Openshift and you should see `jenkins-slave-zap`. |
| | | 3. Head to https://console.lader.rht-labs.com on Openshift and move to your ci-cd project > builds. You should see `jenkins-slave-zap` has been built. |
| | | ![zap-build](../images/exercise4/zap-build.png) |
| | | |
| | | #### 3b - Arachni Scan |
| | |
| | | |
| | | 3. On your terminal; checkout the params and Docker file. The Dockerfile for the `Arachni` scanner is included here and we will point the build to it. |
| | | ```bash |
| | | $ git checkout exercise4/zap-and-arachni params/jenkins-slave-arachni docker/jenkins-slave-arachni |
| | | git checkout exercise4/zap-and-arachni params/jenkins-slave-arachni docker/jenkins-slave-arachni |
| | | ``` |
| | | |
| | | 3. Create an object in `inventory/host_vars/ci-cd-tooling.yml` called `jenkins-slave-arachni` with the following content: |
| | |
| | | |
| | | 3. With these changes in place, push your changes to the `master` branch. |
| | | ```bash |
| | | $ git add . |
| | | $ git commit -m "ADD - Arachni scanning image" |
| | | $ git push |
| | | git add . |
| | | ``` |
| | | ```bash |
| | | git commit -m "ADD - Arachni scanning image" |
| | | ``` |
| | | ```bash |
| | | git push |
| | | ``` |
| | | |
| | | 3. Run the ansible playbook filtering with tag `arachni` so only the arachni build pods are run. |
| | | ```bash |
| | | $ ansible-playbook apply.yml -e target=tools \ |
| | | ansible-playbook apply.yml -e target=tools \ |
| | | -i inventory/ \ |
| | | -e "filter_tags=arachni" |
| | | ``` |
| | | |
| | | 3. Head to (https://console.somedomain.com/console/project/<YOUR_NAME>-ci-cd/browse/builds) on Openshift and you should see `jenkins-slave-arachni`. |
| | | ![todolist-fe-multi](../images/exercise4/builds-zap-arachni.png) |
| | | 3. Head to https://console.lader.rht-labs.com on Openshift and move to your ci-cd project > builds. You should see `jenkins-slave-arachni`. |
| | | ![builds-zap-arachni](../images/exercise4/builds-zap-arachni.png) |
| | | |
| | | _____ |
| | | |