| commit | author | age | ||
| 74d1f1 | 1 | # The Manual Menace |
| 0f4d08 | 2 | |
| 74d1f1 | 3 | > In this lab learners will use Ansible to drive automated provisioning of Projects, Access Control, Git, Jenkins and Nexus |
| D | 4 | |
| 5 | _____ | |
| 0f4d08 | 6 | |
| D | 7 | ## Learning Outcomes |
| 8 | As a learner you will be able to | |
| 74d1f1 | 9 | |
| D | 10 | 1. Run the OpenShift Applier to automate creating cluster content |
| 11 | 1. Create and admin project namespaces in OpenShift | |
| 12 | 1. Deploy commonly used applications to support the development process | |
| 0f4d08 | 13 | |
| D | 14 | ## Tools and Frameworks |
| 15 | ||
| 74d1f1 | 16 | * [GitLab](https://about.gitlab.com/) - Community driven Git server now with integrated DevOps Toolchain. |
| D | 17 | * [Nexus](https://www.sonatype.com/nexus-repository-sonatype) - Repository manager for storing lots of application types. Can also host `npm` and `Docker` registries. |
| 18 | * [Jenkins](https://jenkins.io/) - OpenSource Build automation server. Highly customisable with plugins. | |
| 19 | * [Ansible](https://www.ansible.com/) - IT Automation tool used to provision and manage state of cloud and physical infrastructure. | |
| 20 | * [OpenShift Applier](https://github.com/redhat-cop/openshift-applier) - Community driven Git server | |
| 0f4d08 | 21 | |
| 74d1f1 | 22 | ## Big Picture |
| D | 23 | This exercise begins with an empty Cluster |
| 24 | > TODO - add big picture here... | |
| 25 | ||
| 26 | _____ | |
| 0f4d08 | 27 | |
| D | 28 | ## 10,000 Ft View |
| 74d1f1 | 29 | > This lab is aimed at the creation of the tooling that will be used to support the rest of the Exercises. The highlevel goal is to create a collection of project namespaces and populate them with Git, Jenkins & Nexus. |
| D | 30 | |
| 31 | If you're feeling confident and don't want to follow the step-by-step guide these highlevel instructions should provide a challenge for you: | |
| 32 | ||
| 33 | 2. Clone the repo `git@github.com:rht-labs/enablement-ci-cd.git` which contains the scaffold of the project. | |
| 92099a | 34 | |
| 74d1f1 | 35 | 2. Create `<your-name>-ci-cd`, `<your-name>-dev` and `<your-name>-test` project namespaces using the inventory and run them with the OpenShift Applier to populate the cluster |
| 92099a | 36 | |
| D | 37 | 2. Use the templates provided to create build and deployment configs in `<your-name>-ci-cd` for. Templates are on a branch called `exercise1/git-nexus` && `exercise1/jenkins`: |
| 74d1f1 | 38 | * Nexus |
| D | 39 | * GitLab |
| 92099a | 40 | * Jenkins (using an s2i to pre-configure jenkins) |
| D | 41 | |
| 74d1f1 | 42 | 2. Commit your `enablement-ci-cd` repository to the GitLab Instance you've created |
| 92099a | 43 | |
| D | 44 | 2. Burn it all down and re-apply your inventory proving config-as-code works. |
| 0f4d08 | 45 | |
| D | 46 | ## Step by Step Instructions |
| bd7806 | 47 | > This is a structured guide with references to exact filenames and explanations. |
| 0f4d08 | 48 | |
| bd7806 | 49 | ### Part 1 - Create OpenShift Projects |
| D | 50 | 3. Clone the scaffold project to your local machine and open it in your favourite editor. |
| 51 | ```bash | |
| 52 | git clone git@github.com:rht-labs/enablement-ci-cd.git | |
| 0f4d08 | 53 | ``` |
| D | 54 | |
| bd7806 | 55 | 3. The project is laid out as follows |
| D | 56 | ``` |
| 57 | . | |
| 58 | ├── README.md | |
| 59 | ├── docker | |
| 60 | │ └── jenkins-slave-node | |
| 61 | ├── inventory | |
| 62 | │ ├── group_vars | |
| 63 | │ │ └── all.yml | |
| 64 | │ └── hosts | |
| 65 | ├── jenkins-s2i | |
| 66 | │ ├── configuration | |
| 67 | ├── params | |
| 68 | │ └── project-requests-ci-cd | |
| 69 | ├── requirements.yml | |
| 70 | └── templates | |
| 71 | └── project-requests.yml | |
| 72 | ``` | |
| 73 | * `docker` folder contains our jenkins-slave images that will be used by the builds. | |
| 74 | * `jenkins-s2i` contains the configuration and plugins we want to bring jenkins to life with | |
| 75 | * `params` houses the variables we will load the templates with | |
| 76 | * `templates` is a collection of OpenShift templates | |
| 77 | * `inventory/group_vars/all.yml` is the collection of objects we want to insert into the cluster. | |
| 78 | * `requirements.yml` is a manifest which contains the ansible modules needed to run the playbook | |
| 79 | Open the `inventory/group_vars/all.yml` file; you should see a some variables setup to create the `ci-cd` namespace. This calls the `templates/project-requests.yml` template with the `params/project-requests-ci-cd` parameters. We will add some additional content here but first let's explore the parameters and the template | |
| 80 | ||
| 92099a | 81 | 3. Open the `params/project-requests-ci-cd` and replace the `<YOUR_NAME or initials>` with your name to create the correstponding projects in the cluster. |
| 1c9328 | 82 |  |
| bd7806 | 83 | |
| 92099a | 84 | 3. Create another two params files for `params/project-requests-dev` & `params/project-requests-test` and add the `NAMESPACE=<YOUR_NAME>-dev` && `NAMESPACE=<YOUR_NAME>-test` and update their Display names. |
| bd7806 | 85 | |
| D | 86 | 3. In the `inventory/group_vars/all.yml` file; add the new inventory items for the projects you want to create (dev & test) by adding another object to the content array. You can copy and paste them from the `ci-cd` example and update them accordingly eg |
| 87 | ```yaml | |
| 1c9328 | 88 | - name: <YOUR_NAME>-dev |
| D | 89 | template: "{{ inventory_dir }}/../templates/project-requests.yml" |
| 90 | template_action: create | |
| 91 | params: "{{ inventory_dir }}/../params/project-requests-dev" | |
| 92 | tags: | |
| 93 | - projects | |
| 94 | - name: <YOUR_NAME>-test | |
| 95 | template: "{{ inventory_dir }}/../templates/project-requests.yml" | |
| 96 | template_action: create | |
| 97 | params: "{{ inventory_dir }}/../params/project-requests-test" | |
| 98 | tags: | |
| 99 | - projects | |
| bd7806 | 100 | ``` |
| 1c9328 | 101 |  |
| bd7806 | 102 | |
| D | 103 | 3. With the configuration in place; install the OpenShift Applier dependency |
| 104 | ```bash | |
| 105 | $ ansible-galaxy install -r requirements.yml --roles-path=roles | |
| 106 | ``` | |
| 107 | ||
| 108 | 3. Apply the inventory by logging into OpenShift and then running | |
| 109 | ```bash | |
| 110 | $ oc login -p <password> -u <user> <cluster_url> | |
| 92099a | 111 | $ ansible-playbook roles/openshift-applier/playbooks/openshift-cluster-seed.yml -i inventory/ |
| bd7806 | 112 | ``` |
| D | 113 | |
| 1c9328 | 114 | 3. Once successful you should see an output similar to this  |
| bd7806 | 115 | |
| D | 116 | ### Part 2 - Nexus and GitLab |
| da55a5 | 117 | > _Now that we have our Projects setup; we can start to populate them with Apps to be used in our dev lifecycle_ |
| bd7806 | 118 | |
| 92099a | 119 | 4. In the `enablement-ci-cd` repo, checkout the templates for Nexus by running |
| da55a5 | 120 | ```bash |
| 92099a | 121 | $ git checkout exercise1/git-nexus templates/nexus.yml |
| D | 122 | ``` |
| 123 | The tempate contains all the things needed to setup a persistent nexus server, exposing a service and route while also creating the persistent volume needed. Have a read through the template; at the bottom you'll see a collection of parameters we will pass to the template. | |
| 124 | ||
| 125 | 4. Add some parameters for running the template by creating a new file in the `params` directory. | |
| 126 | ```bash | |
| 127 | $ touch params/nexus | |
| da55a5 | 128 | ``` |
| D | 129 | |
| 92099a | 130 | 4. The essential params to inclue in this file are: `params` directory. |
| D | 131 | ```bash |
| 132 | VOLUME_CAPACITY=5Gi | |
| 133 | MEMORY_LIMIT=2Gi | |
| 134 | ``` | |
| 135 | ||
| 1c9328 | 136 | 4. Create a new object in the inventory variables called `ci-cd-deployments` and populate it's `content` is as follows (swapping `<YOUR_NAME>-ci-cd` for the namespace you created earlier) |
| 92099a | 137 | |
| D | 138 | ```yaml |
| 1c9328 | 139 | - object: ci-cd-deployments |
| D | 140 | content: |
| 141 | - name: "nexus" | |
| 142 | namespace: "<YOUR_NAME>-ci-cd" | |
| 143 | template: "{{ inventory_dir }}/../templates/nexus.yml" | |
| 144 | params: "{{ inventory_dir }}/../params/nexus" | |
| 145 | tags: | |
| 146 | - nexus | |
| 92099a | 147 | ``` |
| 1c9328 | 148 |  |
| 92099a | 149 | |
| D | 150 | 4. Run the OpenShift applier, specifying the tag `nexus` to speed up it's execution. |
| 151 | ```bash | |
| 152 | $ ansible-playbook roles/openshift-applier/playbooks/openshift-cluster-seed.yml \ | |
| 153 | -i inventory/ \ | |
| 1c9328 | 154 | -e "filter_tags=nexus" |
| 92099a | 155 | ``` |
| D | 156 | |
| 1c9328 | 157 | 4. Once successful; login to the cluster and navigate to the `<YOUR_NAME>-ci-cd`. You should see Nexus up and running. You can login with default credentials (admin / admin123)  |
| 92099a | 158 | |
| D | 159 | 4. Now lets do the same thing for GitLab to get it up and running. Checkout the template provided by running |
| 160 | ```bash | |
| 161 | $ git checkout exercise1/gitlab-nexus templates/gitlab.yml | |
| 162 | ``` | |
| 163 | Explore the template; it contains the PVC, buildConfig and services. The DeploymentConfig is made up of these apps | |
| 164 | - Redis (3.2.3) | |
| 165 | - PostgreSQL (9.4) | |
| 166 | - GitLab CE (v10.2.3) | |
| 167 | ||
| 168 | 4. Add a new params file in the `params` folder called `gitlab` | |
| 169 | ```bash | |
| 170 | $ touch params/gitlab | |
| 171 | ``` | |
| 172 | ||
| 173 | 4. Open the `params/gitlab` file and add the following params | |
| 174 | ``` | |
| 175 | LDAP_BIND_DN=uid=<BIND_USER>,ou=People,dc=<YOUR_DOMAIN>,dc=com | |
| 176 | LDAP_USER_FILTER=(memberof=CN=YourGroup,OU=Users,DC=<YOUR_DOMAIN>,DC=com) | |
| 177 | LDAP_PASSWORD=<BIND_USER_PASSWORD> | |
| 178 | LDAP_HOST=<LDAP_HOST> | |
| 179 | LDAP_BASE=ou=People,dc=<YOUR_DOMAIN>,dc=com | |
| 180 | LDAP_LABEL="<LDAP_DESCRIPTION>" | |
| 181 | GITLAB_ROOT_PASSWORD=<GITLAB_ROOT_USER_PASSWORD> | |
| 182 | GITLAB_DATA_VOL_SIZE=2Gi | |
| 183 | POSTGRESQL_VOL_SIZE=1Gi | |
| 184 | APPLICATION_HOSTNAME=<GITLAB_URL> | |
| 4feb7f | 185 | NAMESPACE=<YOUR_NAME>-ci-cd |
| 92099a | 186 | ``` |
| D | 187 | where the following need to be replaced by actual values: |
| 188 | * `<BIND_USER>` is the user used to query the LDAP | |
| 189 | * `<BIND_USER_PASSWORD>` is the password used when querying the LDAP | |
| 190 | * `<YOUR_DOMAIN>` is the domain the LDAP is hosted on | |
| 191 | * `<LDAP_HOST>` is fqdn of the LDAP server | |
| 192 | * `<LDAP_DESCRIPTION>` is the description to be used on the sign-in header for GitLab eg "Name LDAP Login" | |
| 193 | * `<GITLAB_ROOT_USER_PASSWORD>` is the root user for GOD access on the GitLab instance eg password123 | |
| 194 | * `<GITLAB_URL>` is the endpoint for gitlab. It will take the form `gitlab-<YOUR_NAME>-ci-cd.apps.<ENV_ID>.<YOUR_DOMAIN>.com` | |
| 195 | ||
| 196 | 4. Create another object in the inventory `all_vars.yml` file to run the build & deploy of this template. Add the following and update the `namespace:` accordingly | |
| 197 | ```yaml | |
| 1c9328 | 198 | - name: "gitlab" |
| D | 199 | namespace: "<YOUR_NAME>-ci-cd" |
| 200 | template: "{{ inventory_dir }}/../templates/gitlab.yml" | |
| 201 | params: "{{ inventory_dir }}/../params/gitlab" | |
| 202 | tags: | |
| 203 | - gitlab | |
| 92099a | 204 | ``` |
| D | 205 | |
| 206 | 4. Run the OpenShift applier, specifying the tag `gitlab` to speed up it's execution. | |
| 207 | ```bash | |
| 208 | $ ansible-playbook roles/openshift-applier/playbooks/openshift-cluster-seed.yml \ | |
| 209 | -i inventory/ \ | |
| 1c9328 | 210 | -e "filter_tags=gitlab" |
| 92099a | 211 | ``` |
| D | 212 | |
| 1c9328 | 213 | 4. Once successful; login to the cluster and navigate to the `<YOUR_NAME>-ci-cd`. You should see GitLab up and running.  |
| 4feb7f | 214 | |
| D | 215 | 4. Navigate to gitlab. You can login with using your cluster credentials using the LDAP tab displaying your `<LDAP_DESCRIPTION>` from previous steps |
| 216 | ||
| 1c9328 | 217 |  |
| 4feb7f | 218 | |
| 1c9328 | 219 | 4. Once logged in create a new project called `enablement-ci-cd` and mark it as public. Once created; copy out the `git remote add origin ...` instructions for use on the next step. |
| D | 220 |  |
| 221 | <p class="tip"> | |
| 222 | Note - we would not normally make the project public here on residency but are for simplicity of the exercise | |
| 223 | </p> | |
| 224 | ||
| 225 | 4. Commit your local project to this new origin by first removing the existing origin (github) where the the project was cloned from. Remember to substitute `<YOUR_NEW_GIT_PROJECT>` accordingly | |
| 226 | ```bash | |
| 227 | $ git remote remove origin | |
| 228 | $ git remote add origin <YOUR_NEW_GIT_PROJECT> | |
| 229 | $ git add . | |
| 230 | $ git commit -m "Adding git and nexus config" | |
| 231 | $ git push -u origin --all | |
| 232 | ``` | |
| bd7806 | 233 | |
| D | 234 | ### Part 3 - Jenkins & s2i |
| 1c9328 | 235 | 5. Add the Jenkins Build & Deployment configs to the `enablement-ci-cd` repo by merging the contents `exercise1/jenkins` in |
| D | 236 | ```bash |
| 237 | $ git checkout exercise1/jenkins templates/jenkins.yml | |
| 238 | ``` | |
| bd7806 | 239 | |
| 1c9328 | 240 | 5. Add the Jenkins Build & Deployment configs to the `enablement-ci-cd` repo by merging the contents `exercise1/jenkins` in |
| D | 241 | ```bash |
| 242 | $ git checkout exercise1/jenkins templates/jenkins.yml | |
| 243 | ``` | |
| 244 | The Jenkins template is essentially the standard persistent jenkins one with OpenShift. | |
| 245 | ||
| 246 | 5. As before; create a new set of params by creating a `params/jenkins` file and adding some overrides to the template and updating the `NAMESPACE` value. | |
| 247 | ```bash | |
| 248 | MEMORY_LIMIT=8Gi | |
| 249 | VOLUME_CAPACITY=5Gi | |
| 250 | JVM_ARCH=x86_64 | |
| 251 | NAMESPACE=<YOUR_NAME>-ci-cd | |
| 252 | JENKINS_OPTS=--sessionTimeout=720 | |
| 253 | ``` | |
| 254 | 5. Add a `jenkins` variable to the ansible inventory underneath the git and nexus ones. Remember to replace `<YOUR_NAME>` with the appropriate value. | |
| 255 | ```yaml | |
| 256 | - name: "jenkins" | |
| 257 | namespace: "<YOUR_NAME>-ci-cd" | |
| 258 | template: "{{ inventory_dir }}/../templates/jenkins.yml" | |
| 259 | params: "{{ inventory_dir }}/../params/jenkins" | |
| 260 | tags: | |
| 261 | - jenkins | |
| 262 | ``` | |
| 263 | This configuration if applied now; it will create the deployment configuration needed for Jenkins but the `${NAMESPACE}:${JENKINS_IMAGE_STREAM_TAG}` in the template won't exist yet. | |
| 264 | ||
| 265 | 5. To create this image we will take the supported OpenShift Jenkins Image and bake into it some extra configuration using an [s2i](https://github.com/openshift/source-to-image) builder image. More information on Jenkins s2i is found on the [openshift/jenkins](https://github.com/openshift/jenkins#installing-using-s2i-build) github page. To create an s2i configuration for jenkins, check out the pre-canned configuration source in the `enablement-ci-cd` repo | |
| 266 | ```bash | |
| 267 | $ git checkout exercise1/jenkins-s2i jenkins-s2i | |
| 268 | ``` | |
| 269 | The structure of the jenkins s2i config is | |
| 270 | ``` | |
| 271 | jenkins-s2i | |
| 272 | ├── README.md | |
| 273 | ├── configuration | |
| 274 | │ ├── build-failure-analyzer.xml | |
| 275 | │ ├── init.groovy | |
| 276 | │ ├── jenkins.plugins.slack.SlackNotifier.xml | |
| 277 | │ └── jobs | |
| 278 | │ └── seed-multibranch-job | |
| 279 | │ └── config.xml | |
| 280 | └── plugins.txt | |
| 281 | ``` | |
| 282 | * `plugins.txt` is a list of `pluginId:version` for Jenkins to pre-install when starting | |
| 283 | * `./configuration` contains content that is placed in `${JENKINS_HOME}`. A `config.xml` could be placed in here to control the bulk of Jenkins configuration. | |
| 284 | * `./configuration/jobs/*` contains job names and xml config that jenkins loads when starting. The seed job in there we will return to in later lessons. | |
| 285 | * `build-failure-analyzer.xml` is config for the plugin to read the logs and look for key items based on a Regex | |
| 286 | * `init.groovy` contains a collection of settings jenkins configures itself with when launching | |
| 287 | ||
| 288 | 5. Let's add a plugin for Jenkins to be started with, [green-balls](https://plugins.jenkins.io/greenballs). This simply changes the default `SUCCESS` status of Jenkins from Blue to Green. Append the `plugins.txt` file with | |
| 289 | ```txt | |
| 290 | greenballs:1.15 | |
| 291 | ``` | |
| 292 |  | |
| 293 | Why does Jenkins have Blue Balls? More can be found [on reddit](https://www.reddit.com/r/programming/comments/4lu6q8/why_does_jenkins_have_blue_balls/) or the [jenkins blog](https://jenkins.io/blog/2012/03/13/why-does-jenkins-have-blue-balls/) | |
| 294 | ||
| 295 | 5. Before building and deploying Jenkins; add git credentials to the s2i by either adding them to the `template/jenkins.yml` and `params/jenkins` or for simplicity just replace the `<USERNAME>` && `<PASSWORD>` with your ldap credentials. | |
| 296 | <p class="tip"> | |
| 297 | Note in a residency we would not use your GitCredentials for pushing and pulling from Git, A service user would be created here. | |
| 298 | </p> | |
| 299 | ```groovy | |
| 300 | gitUsername = System.getenv("GIT_USERNAME") ?: "<USERNAME>" | |
| 301 | gitPassword = System.getenv("GIT_PASSWORD") ?: "<PASSWORD>" | |
| 302 | ``` | |
| 303 | ||
| 304 | 5. Add a new params file in the `params` folder called `jenkins-s2i` | |
| 305 | ```bash | |
| 306 | $ touch params/jenkins-s2i | |
| 307 | ``` | |
| 308 | ||
| 309 | 5. Open the `params/jenkins-s2i` file and add the following content; replacing variables as appropriate | |
| 310 | ``` | |
| 311 | SOURCE_REPOSITORY_URL=<YOUR_ENABLEMENT_REPO> | |
| 312 | NAME=jenkins | |
| 313 | SOURCE_REPOSITORY_CONTEXT_DIR=jenkins-s2i | |
| 314 | IMAGE_STREAM_NAMESPACE=<YOUR_NAME>-ci-cd | |
| 315 | ``` | |
| 316 | where | |
| 317 | * `<YOUR_ENABLEMENT_REPO>` is the full path clone path of the repo where this project is stored (including the https && .git) | |
| 318 | * `<YOUR_NAME>` is the prefix for your `-ci-cd` project. | |
| 319 | * Explore some of the other parameters in `templates/jenkins-s2i.yml` | |
| 320 | ||
| 321 | 5. Create a new object `ci-cd-builds` in the ansible `all.yml` to drive the s2i build configuration. | |
| 322 | ```yaml | |
| 323 | - object: ci-cd-builds | |
| 324 | content: | |
| 325 | - name: "jenkins-s2i" | |
| 326 | namespace: "<YOUR_NAME>-ci-cd" | |
| 327 | template: "{{ inventory_dir }}/../templates/jenkins-s2i.yml" | |
| 328 | params: "{{ inventory_dir }}/../params/jenkins-s2i" | |
| 329 | tags: | |
| 330 | - jenkins | |
| 331 | ``` | |
| 332 | ||
| 333 | 5. Commit your code to your GitLab instance | |
| 334 | ```bash | |
| 335 | $ git add . | |
| 336 | $ git commit -m "Adding Jenkins and Jenkins s2i" | |
| 337 | $ git push | |
| 338 | ``` | |
| 339 | ||
| 340 | 5. When your code is commited; run the OpenShift Applier to add the config to the cluster | |
| 341 | ```bash | |
| 342 | $ ansible-playbook roles/openshift-applier/playbooks/openshift-cluster-seed.yml \ | |
| 343 | -i inventory/ \ | |
| 344 | -e "filter_tags=jenkins" | |
| 345 | ``` | |
| 346 | ||
| 347 | ### Part 4 - Live, Die, Repeat | |
| bd7806 | 348 | 6. Commit your code to the new repo in GitLab |
| D | 349 | |
| 1c9328 | 350 | 6. Burn your OCP content to the ground |
| 92099a | 351 | |
| D | 352 | 6. Re-apply the inventory! |
| 0f4d08 | 353 | |
| 74d1f1 | 354 | _____ |
| D | 355 | |
| 0f4d08 | 356 | ## Extension Tasks |
| D | 357 | > Ideas for go-getters. Advanced topic for doers to get on with if they finish early. These will usually not have a solution and are provided for additional scope. |
| 358 | ||
| 92099a | 359 | - Add more secure access for Nexus (ie not admin / admin123) using the automation to drive secret creation |
| 1c9328 | 360 | - Add a SonarQube persistent deployment to the `ci-cd-deployments` section. |
| D | 361 | - Add `jenkins.plugins.slack.SlackNotifier.xml` to `jenkins-s2i/configuration` to include URL of Slack for team build notifications and rebuild Jenkins S2I |
| 0f4d08 | 362 | |
| 74d1f1 | 363 | _____ |
| D | 364 | |
| 0f4d08 | 365 | ## Additional Reading |
| D | 366 | > List of links or other reading that might be of use / reference for the exercise |
