RedHatTraining/rht-labs-docs.git

Tim Beattie

2018-04-28 b5d705d1d88f2313dd34f991d47d72a7bd674539

commit \| author \| age
2b8436	1	# Attack of the Pipelines
c951f7	2
b5d705	3	> In this exercise we will explore the sample TODO List application and create a pipeline in Jenkins to build and deploy our code.
2b8436	4
D	5	![jenkins-time](../images/exercise2/jenkins-time.jpg)
	6
14cd2d	7	## Exercise Intro
D	8	This lesson is focused on creating a pipeline for our application. What is a pipeline? A pipeline is a series of steps or stages that takes our code from source to a deployed application. There can be many stages to a pipeline but a simple flow is to run a `build > bake > deploy`. Usually the first stage is trigger by something like a git commit. There could be many steps in each of these stages; such as compiling code, running tests and linting. All of these are done to try and drive up code quality and give more assurance that what is deployed is behaving as expected. In the exercise we will create Jenkins pipeline by configuring it through the UI, this will create an un-gated pathway to production
	9
	10	First we will explore the sample application and get it running locally. The sample app is a `todolist` app - the `Hello World` app of the modern day.
	11
	12	#### Why create pipelines
	13	* Assurance - drive up code quality and remove the need for dedicated deployment / release management teams
	14	* Freedom - allow developers to take ownership of how and when code gets built and shipped
3f6da0	15	* Reliability - pipelines are a bit boring; they execute the same way each and every time they're run!
14cd2d	16	* A pathway to production:
D	17	- Puts the product in the hands of the customer quicker
	18	- Enables seamless and repeatable deploys
	19	- More prod like infrastructure increases assurance
	20	- “We have already done it” behavior de-risks go live
	21
43f2f2	22	_____
c951f7	23
D	24	## Learning Outcomes
2b8436	25	As a learner by the end of this lesson you will be able to
D	26
	27	- Build and run the full stack of the TODO List application locally
	28	- Create an un-gated pipeline using the Jenkins UI for the backend and frontend
	29	- Add branching to the pipeline to target specific namespace
c951f7	30
D	31	## Tools and Frameworks
cebd30	32	> The following tools are used throughout this exercise. Familiarity with them is not required but knowing what they are may help. You will not need to install Vue or Mongodb they are taken care of by our `todolist` app.
c951f7	33
D	34	1. [Jenkins](https://jenkins.io/) - OpenSource build automation server; highly customisable through plugins
2b8436	35	1. [NodeJS](https://nodejs.org/en/) - Node.js® is a JavaScript runtime built on Chrome's V8 JavaScript engine. Node.js uses an event-driven, non-blocking I/O model that makes it lightweight and efficient. Node.js' package ecosystem, npm, is the largest ecosystem of open source libraries in the world.
D	36	1. [MongoDB](https://www.mongodb.com/what-is-mongodb) - MongoDB stores data in flexible, JSON-like documents, meaning fields can vary from document to document and data structure can be changed over time
	37	1. [VueJS](https://vuejs.org/) - Vue (pronounced /vjuː/, like view) is a progressive framework for building user interfaces. It is designed from the ground up to be incrementally adoptable, and can easily scale between a library and a framework depending on different use cases. It consists of an approachable core library that focuses on the view layer only, and an ecosystem of supporting libraries that helps you tackle complexity in large Single-Page Applications.
c951f7	38
43f2f2	39	## Big Picture
2b8436	40	> From the previous exercise; we created some supporting tooling needed by our app/
43f2f2	41
D	42	_____
c951f7	43
D	44	## 10,000 Ft View
2b8436	45	> _This lab requires users to take the sample TODO app and create a build pipeline in Jenkins by clicking your way to success ending up with an app deployed to each of the namespaces created previously_
43f2f2	46
2b8436	47	2. Import the projects into your gitlab instance. See README of each for build instructions
43f2f2	48
2b8436	49	2. Deploy a `MongoDB` using the provided template to all project namespace.
D	50
	51	2. Create 2 pipline with three stages (`build`, `bake`, `deploy`) in jenkins for `develop` & `master` branches on the `todolist-fe` such that:
e919d0	52	* a `Build` job is responsible for compiling and packaging our code:
2b8436	53	1. Checkout from source code (`develop` for `<yourname>-dev` & `master` for `<yourname>-test`)
D	54	2. Install node dependencies and run a build / package
e919d0	55	3. Send the package to Nexus
D	56	4. Archive the workspace to persist the workspace in case of failure
	57	4. Tag the GitLab repository with the `${JOB_NAME}.${BUILD_NUMBER}` from Jenkins. This is our `${BUILD_TAG}` which will be used on downstream jobs.
	58	5. Trigger the `bake` job with the `${BUILD_TAG}` param
	59	* a `Bake` job should take the package and put it in a Linux Container
2b8436	60	1. Take an input of the previous jobs `${BUILD_TAG}` ie `${JOB_NAME}.${BUILD_NUMBER}`.
D	61	2. Checkout the binary from Nexus and unzip it's contents
e919d0	62	3. Run an oc start-build of the App's BuildConfig and tag it's imagestream with the provided `${BUILD_TAG}`
2b8436	63	4. Trigger a deploy job using the parameter `${BUILD_TAG}`
e919d0	64	* a `deploy` job should roll out the changes by updating the image tag in the DC:
2b8436	65	1. Take an input of the `${BUILD_TAG}`
D	66	2. Patch / set the DeploymentConfig to the image's `${BUILD_TAG}`
	67	3. Rollout the changes
	68	4. Verify the deployment
	69
	70	2. Repeat the above setup for the backend `todolist-fe`. TIP - use the copy config to speed things up!
	71
	72	2. Verify that both apps and the DB are talking to one another as expected.
c951f7	73
D	74	## Step by Step Instructions
7383de	75	> This is a fairly structured guide with references to exact filenames and sections of text to be added.
c951f7	76
f016b7	77	### Part 1 - Explore the Todo List App
7383de	78	> _In this part of the exercise we will explore the sample application, become familiar with it locally before building and deploying in OCP Land_
43f2f2	79
d4f1fa	80	#### 1a Todolist-fe
4c8010	81
7383de	82	2. Git clone the `todolist-fe` project to somewhere sensible and checkout the `develop` branch.
D	83	```bash
4acca2	84	$ git clone https://github.com/rht-labs/todolist-fe.git
91cc51	85	$ cd todolist-fe
7383de	86	$ git checkout develop
4c8010	87	```
A	88	Followed by;
	89	```
	90	$ for branch in `git branch -a \| grep remotes \| grep -v HEAD \| grep -v master`; do
	91	git branch --track ${branch#remotes/origin/} $branch
	92	done
c951f7	93	```
7383de	94
10416f	95	2. Open up Gitlab and login. Create a new project (internal) in GitLab called `todolist-fe` to host your clone of the project and copy it's remote address. ![new-gitlab-proj](../images/exercise2/new-gitlab-proj.png)
7383de	96
D	97	2. In your local clone of the `todolist-fe`, remove the origin and add the GitLab origin by replacing `<YOUR_GIT_LAB_PROJECT>`. Push your app to GitLab
	98	```bash
10416f	99	$ git remote set-url origin <YOUR_GIT_LAB_PROJECT>
A	100	# verify the origin has been updated
	101	$ git remote -v
7383de	102	$ git push -u origin --all
c951f7	103	```
7383de	104
D	105	2. To get the app running locally; first check you've got node and npm installed
	106	```bash
	107	$ node -v
	108	$ npm -v
	109	```
	110	<p class="tip" >
	111	NOTE - If you are missing these dependencies; install them with ease using the [Node Version Manager](https://github.com/creationix/nvm)
	112	</p>
	113	![node-version](../images/exercise2/node-version.png)
	114
	115	2. The `todolist-fe` has a package.json at the root of the project, this defines some configuration for the app including it's dependencies, dev dependencies, scripts and other configuration. Install the apps dependencies
	116	```bash
	117	$ npm install
	118	```
	119
abb0a5	120	2. The `todolist-fe` has some scripts defined in the package.json at the root of the project. A snippet of the npm scripts are shown below. To run any of these scripts run `npm run <SCRIPT_NAME>`. Let's start by serving our application
7383de	121	![npm-scripts](../images/exercise2/npm-scripts.png)
D	122	```bash
	123	npm run serve
	124	```
	125
10416f	126	2. This will take sometime to execute; but once done it should open the browser for you displaying the homepage of the `todolist-fe` app.
7383de	127	![todo-list-app](../images/exercise2/todo-list-app.png)
10416f	128	* Click 'Todo' at the top of the home page to get to the above page.
abb0a5	129	* The server hosting it live reloads; so if you make changes to your code base the app will live update
7383de	130	* The Data you see in the screen is dummy / stubbed data. This is served up when there is no backend connection found
D	131
abb0a5	132	2. The app is a todolist manager built in Vue.js. Play around with the App. You will notice when you add todos they appear and clear as expected. If you refresh the page you'll lose all additions. This is because there is no persistence layer. We will add one in the next part.
7383de	133
D	134	3. The structure of the `todolist-fe` is as follows.
	135	```bash
	136	todolist-fe
	137	├── jest.config.js
	138	├── jsconfig.json
	139	├── nightwatch.config.js
	140	├── node_modules
	141	├── package.json
	142	├── public
	143	│ ├── favicon.ico
	144	│ ├── img
	145	│ ├── index.html
	146	│ └── manifest.json
	147	├── src
	148	│ ├── App.vue
	149	│ ├── assets
	150	│ ├── components
10416f	151	│ │ └── *
7383de	152	│ ├── config
D	153	│ ├── main.js
	154	│ ├── registerServiceWorker.js
	155	│ ├── router.js
	156	│ ├── scss
	157	│ ├── services
	158	│ ├── store
10416f	159	│ │ └── *
7383de	160	│ └── views
10416f	161	│ └── *
7383de	162	├── tests
D	163	│ ├── e2e
	164	│ └── unit
	165	└── vue.config.js
	166	```
	167	where the following are the important things:
	168	* `./src` is the main collection of files needed by the app. The entrypoint is the `main.js` which is used to load the root `App.vue` file.
	169	* `./node_modules` is where the dependencies are stored
b5d705	170	* `./test` contains our end-to-end tests and unit tests. More covered on these in later exercises.
10416f	171	* `./src/components` contains small, lightweight reusable components for our app. For example, the `NewTodo` component which encapsulates the styling, logic and data for adding a new todo to our list
7383de	172	* `./src/store` is the `vuex` files for managing application state and backend connectivity
D	173	* `./src/views` is the view containers; which are responsible for loading components and managing their interactions.
	174	* the `./src/router.js` controls routing logic. In our case the app only has one real endpoint.
	175	* `./src/scss` contains custom SCSS used in the application.
	176	* `./*.js` is mostly config files for running and managing the app and the tests
c951f7	177
b815cc	178	2. To prepare Nexus to host the binaries created by the frontend and backend builds we need to run a prepare-nexus script. Before we do this we need to export some variables change `<YOUR_NAME>` and `somedomain` accordingly.
4c8010	179	```bash
A	180	export NEXUS_SERVICE_HOST=nexus-<YOUR_NAME>-ci-cd.apps.somedomain.com
	181	export NEXUS_SERVICE_PORT=80
	182	npm run prepare-nexus
	183	```
	184	<p class="tip">
	185	NOTE - This step in a residency would be automated by a more complex nexus deployment in the ci-cd project
	186	</p>
	187
d4f1fa	188	#### 1b Todolist-api
456daa	189
d2e708	190	2. Now let's move on to the `todolist-api` and wire them together. As with the `todolist-fe` we need to clone the repo and add it to our GitLab in the cluster.
D	191	```bash
4acca2	192	$ git clone https://github.com/rht-labs/todolist-api.git
abb0a5	193	$ cd todolist-api
d2e708	194	$ git checkout develop
4c8010	195	```
A	196	Followed by;
	197	```
	198	$ for branch in `git branch -a \| grep remotes \| grep -v HEAD \| grep -v master`; do
	199	git branch --track ${branch#remotes/origin/} $branch
	200	done
d2e708	201	```
D	202
abb0a5	203	2. On GitLab; create a new project (internal) called `todolist-api` to host your clone of the project and copy it's remote address as you did for the previous repositories.
d2e708	204
D	205	2. In your local clone of the `todolist-api`, remove the origin and add the GitLab origin by replacing `<YOUR_GIT_LAB_PROJECT>`. Push your app to GitLab
	206	```bash
10416f	207	$ git remote set-url origin <YOUR_GIT_LAB_PROJECT>
d2e708	208	$ git push -u origin --all
D	209	```
	210
	211	2. Once pushed; explore the application. It is a NodeJS application with the Express.js framework and MongoDB for persistent storage. Same as before, the `package.json` defines most of the configuration etc. Install the dependencies
	212	```bash
10416f	213	# npm i === npm install
d2e708	214	$ npm i
D	215	```
	216
	217	2. While the dependencies are being installed; explore the project structure.
	218	```bash
	219	todolist-api
	220	├── Dockerfile
	221	├── Gruntfile.js
	222	├── README.md
	223	├── node_modules
	224	├── package-lock.json
	225	├── package.json
	226	├── server
	227	│ ├── api
	228	│ │ └── todo
	229	│ ├── app.js
	230	│ ├── components
	231	│ │ └── errors
	232	│ ├── config
	233	│ │ ├── environment
	234	│ │ ├── express.js
	235	│ │ ├── local.env.sample.js
	236	│ │ └── seed.js
	237	│ ├── mocks
	238	│ │ ├── mock-routes-config.json
	239	│ │ ├── mock-routes.js
	240	│ │ └── mock-routes.spec.js
	241	│ ├── routes.js
	242	│ └── views
	243	│ └── 404.html
	244	└── tasks
	245	└── perf-test.js
	246	```
	247	where the following are the important things:
	248	* `./server` is the main collection of files needed by the app. The entrypoint is the `app.js`
	249	* `./node_modules` is where the dependencies are stored
	250	* `./server/api` is where the api's controller, data model & unit test are stored.
	251	* `./server/mocks` is a mock server used for when there is no DB access
10416f	252	* `./server/config` stores our Express JS config, header information and other middleware.
A	253	* `./server/config/environment` stores enviromnent specific config; such as connectivity to backend services like MongoDB.
b5d705	254	* `./tasks` is a collection of additional `Grunt` tasks which will be used in later exercises
10416f	255	* `Grunt` is a taskrunner for use with Node.JS projects
d2e708	256	* `package.json` contains the dependency list and a lot of very helpful scripts for managing the app lifecycle
D	257
df8dea	258	2. A snippet of the npm scripts are shown below. There are application start scripts, build and test items which will be used in the build. The ones for MongoDB are just provided for convenience and require Docker installed to execute.
d2e708	259	```json
D	260	"scripts": {
	261	"start": "node server/app.js",
	262	"dev": "./node_modules/.bin/grunt serve",
	263	"jshint": "./node_modules/.bin/grunt jshint",
	264	"clean": "rm -rf reports package-contents*",
	265	"package": "zip -r package-contents.zip package-contents",
	266	"test": "node_modules/.bin/nyc node_modules/.bin/mocha server/*/.spec.js --exit",
	267	"mongo" : "docker run -i -d --name mongo-local -p 27017:27017 mongo",
	268	"mongo:drop" : "npm run mongo:stop && docker rm mongo-local",
	269	"mongo:stop" : "docker stop mongo-local",
	270	"mongo:start" : "docker start mongo-local"
	271	},
456daa	272	```
A	273
abb0a5	274	2. To run the application; start a new instance of the MongoDB by running the following. This will pull a mongodb image from Dockerhub and then start it for our API to connect to.
d2e708	275	```bash
D	276	$ npm run mongo
	277	```
	278	<p class="tip">
60ff08	279	NOTE - `npm run mongo:drop` is used to completely remove the running container. `npm run mongo:stop` & `npm run mongo:start` will preserve data in the container
d2e708	280	</p>
D	281
	282	2. Fire up the `todolist-api` by running.
	283	```bash
	284	$ npm run start
	285	```
	286	![node-app-started](../images/exercise2/node-app-started.png)
	287
10416f	288	2. Check things are up and running by testing the API with a `curl`. The API should return some seeded data (stored in `server/config/seed.js`)
d2e708	289	```bash
D	290	$ curl localhost:9000/api/todos
	291	```
	292	```json
	293	[{
	294	"_id": "5ac8ff1fdfafb02138698948",
	295	"title": "Learn some stuff about MongoDB",
	296	"completed": false,
	297	"__v": 0
	298	},
	299	{
	300	"_id": "5ac8ff1fdfafb02138698949",
	301	"title": "Play with NodeJS",
	302	"completed": true,
	303	"__v": 0
	304	}]
	305	```
	306
10416f	307	2. Now let's check out `todolist-fe` app by reloading the browser. We should now see our dummy front end data is replaced by the backend seed data. Adding new todos will add them in the backend, these will persist when the page is refreshed.
d2e708	308	![fullstack-app](../images/exercise2/fullstack-app.png)
D	309
	310
9fc88c	311	### Part 2 - Create a NodeJS Build slave
399baa	312	> _In this exercise; we will create a build configuration to generate a slave for Jenkins to use in it's builds_
9fc88c	313
399baa	314	3. In order for Jenkins to be able to run `npm` builds and installs as we have done locally, we must configure a `jenkins-build-slave` for Jenkins to use. This slave will be dynamically provisioned when we run a build. It needs to have NodeJS and npm installed in it. In your `enablement-cd-cd` repository, checkout the template and configuration. This will bring in the template, the params & the `Dockerfile`.
9fc88c	315	```bash
da614f	316	$ git checkout exercise2/jenkins-slave docker/ templates/ params/jenkins-slave-npm
9fc88c	317	```
D	318
8eeb96	319	3. Open the `params/jenkins-slave-npm` file and update `<GIT_URL>` accordingly. This set of parameters will clone from the enablement repo and run a docker build of the Dockerfile stored in `docker/jenkins-slave-npm`.
9fc88c	320	```bash
8eeb96	321	SOURCE_REPOSITORY_URL=<GIT_URL>
9fc88c	322	SOURCE_CONTEXT_DIR=docker/jenkins-slave-npm
b815cc	323	NAME=jenkins-slave-npm
9fc88c	324	```
D	325
06fc7b	326	3. Create an item in the `inventory/host_vars/ci-cd-tooling.yml` under the `ci-cd-builds` object to run the template with.
9fc88c	327	```yaml
b815cc	328	- name: "jenkins-slave-npm"
06fc7b	329	namespace: "{{ ci_cd_namespace }}"
D	330	template: "{{ playbook_dir }}/templates/jenkins-slave-generic-template.yml"
	331	params: "{{ playbook_dir }}/params/jenkins-slave-npm"
9fc88c	332	tags:
D	333	- jenkins-slave
	334	```
	335	![jenkins-slave-ansible](../images/exercise2/jenkins-slave-ansible.png)
	336
da614f	337	3. Commit your changes to the `enablement-ci-cd` repository!
D	338	```bash
	339	$ git add .
	340	$ git commit -m "ADD npm slave node for Jenkins"
	341	$ git push
	342	```
	343
9fc88c	344	3. Run the OpenShift Applier to trigger a build of this jenkins slave image.
D	345	```bash
06fc7b	346	$ ansible-playbook apply.yml -e target=tools \
9fc88c	347	-i inventory/ \
D	348	-e "filter_tags=jenkins-slave"
	349	```
	350
	351	3. Verify the build executed successfully by logging into the cluster and checking the `builds` tab of the `<YOUR_NAME>-ci-cd` project.
	352	![jenkins-slave-npm-build](../images/exercise2/jenkins-slave-npm-build.png)
	353
b5d705	354	3. You should now be able to apply the label `jenkins-slave-npm` to a build job to run a build on this newly created slave as we will see in the rest of this exercise
9fc88c	355	<p class="tip">
D	356	NOTE - Jenkins may need to be restarted for the configuration to appear. To do this; navigate to your jenkins instance and add `/restart` to the url.
	357	</p>
	358
	359	### Part 3 - Add configs to cluster
f016b7	360	> _In this exercise; we will use the OpenShift Applier to drive the creation of cluster content required by the app such as MongoDB and the Apps Build / Deploy Config_
9fc88c	361
3dce53	362	4. On your terminal navigate to the root of the `todolist-fe` application. The app contains a hidden folder called `.openshift-applier`. Move into this `.openshift-applier` directory and you should see a familiar looking directory structure for an ansible playbook.
cdcafd	363	```
D	364	.openshift-applier
	365	├── README.md
	366	├── apply.yml
	367	├── inventory
	368	│ ├── group_vars
	369	│ │ └── all.yml
	370	│ └── hosts
	371	├── params
	372	│ ├── build
	373	│ ├── dev
68d81a	374	│ ├── ocp-pipeline
cdcafd	375	│ └── test
D	376	├── requirements.yml
	377	└── templates
68d81a	378	├── ocp-pipeline.yml
D	379	├── todolist-fe-build.yml
	380	└── todolist-fe-deploy.yml
cdcafd	381	```
D	382	where the following
bc2216	383	* the `apply.yml` file is the entrypoint.
cdcafd	384	* the `inventory` contains the objects to populate the cluster with.
D	385	* the `params` contains the variables we'll apply to the `templates`
f016b7	386	* the `templates` required by the app. These include the Build, Deploy configs as well as the services, health checks, and other app definitions.
cdcafd	387
bc2216	388	4. There are a few updates to these manifests we need to make before applying the cluster content. In the `apply.yml` update the namespace `<YOUR_NAME>` variables accordingly.
cdcafd	389	```yaml
D	390	ci_cd_namespace: donal-ci-cd
	391	dev_namespace: donal-dev
	392	test_namespace: donal-test
	393	```
	394
bc2216	395	4. In the `params` folder update the `dev` and `test` files with the correct `<YOUR_NAME>` as you've done above. Example for the `dev` file:
cdcafd	396	```bash
D	397	PIPELINES_NAMESPACE=donal-ci-cd
c58300	398	NAME=todolist-fe
cdcafd	399	DEPLOYER_USER=jenkins
D	400	APP_TAG=latest
	401	NAMESPACE=donal-dev
	402	```
	403
f016b7	404	4. With those changes in place we can now run the playbook. First install the `openshift-applier` dependency and then run the playbook (from the `.openshift-applier` directory). This will populate the cluster with all the config needed for the front end app.
cdcafd	405	```bash
D	406	$ ansible-galaxy install -r requirements.yml --roles-path=roles
	407	$ ansible-playbook apply.yml -i inventory/
	408	```
071905	409	![ansible-success](../images/exercise2/ansible-success.png)
cdcafd	410
f016b7	411	4. Once successful, `commit` and `push` your changes to gitlab.
68d81a	412	```bash
D	413	$ git add .
	414	$ git commit -m "UPDATE - change namespace vars to donal"
	415	$ git push
	416	```
f016b7	417
e90e9c	418	4. Back on your terminal navigate to the root of the `todolist-api` application. Open the `.openshift-applier` directory. The same layout as seen in `todolist-fe` should be visible with one noticeable difference; the api requires `MongoDB` to connect to at runtime.
c58300	419
D	420	4. In the `apply.yml` update the namespace `<YOUR_NAME>` variables accordingly. For example:
	421	```yaml
	422	ci_cd_namespace: donal-ci-cd
	423	dev_namespace: donal-dev
	424	test_namespace: donal-test
	425	```
	426
	427	4. In the `params` folder update the `dev` and `test` files with the correct `<YOUR_NAME>` as you've done above. Example for the `dev` file:
	428	```bash
	429	PIPELINES_NAMESPACE=donal-ci-cd
	430	NAME=todolist-api
	431	DEPLOYER_USER=jenkins
	432	APP_TAG=latest
	433	NAMESPACE=donal-dev
	434	```
	435
bfd54c	436	4. Finally; run the Openshift Applier and install its dependencies to run the content into the cluster
c58300	437	```bash
D	438	$ ansible-galaxy install -r requirements.yml --roles-path=roles
	439	$ ansible-playbook apply.yml -i inventory/
	440	```
9fc88c	441
68d81a	442	4. Once successful, `commit` and `push` your changes to gitlab.
D	443	```bash
	444	$ git add .
	445	$ git commit -m "UPDATE - change namespace vars to donal"
	446	$ git push
	447	```
e90e9c	448
68d81a	449	4. Validate the build and deploy configs have been created in Openshift by checking `<YOUR_NAME> CI-CD builds` for the `BuildConfigs`
D	450	![ocp-app-bc](../images/exercise2/ocp-app-bc.png)
	451
	452	4. Check `<YOUR_NAME>-dev` to see the deployment configs are in place
6c8424	453	![ocp-app-dc](../images/exercise2/ocp-app-dc.png)
e90e9c	454
9fc88c	455	### Part 4 - Build > Bake > Deploy
0e648a	456	> _In this exercise; we take what we have working locally and get it working in OpenShift_
9fc88c	457
41217d	458	This exercise will involve creating three stages (or items) in our pipeline, each of these is detailed below at a very high level. Move on to the next step to begin implementation.
D	459	* a build job is responsible for compiling and packaging our code:
	460	1. Checkout from source code (`develop` for `<yourname>-dev` & `master` for `<yourname>-test`)
	461	2. Install node dependencies and run a build / package
	462	3. Send the package to Nexus
	463	4. Archive the workspace to persist the workspace in case of failure
	464	4. Tag the GitLab repository with the `${JOB_NAME}.${BUILD_NUMBER}` from Jenkins. This is our `${BUILD_TAG}` which will be used on downstream jobs.
	465	5. Trigger the `bake` job with the `${BUILD_TAG}` param
	466	* a bake job should take the package and put it in a Linux Container
	467	1. Take an input of the previous jobs `${BUILD_TAG}` ie `${JOB_NAME}.${BUILD_NUMBER}`.
	468	2. Checkout the binary from Nexus and unzip it's contents
	469	3. Run an oc start-build of the App's BuildConfig and tag it's imagestream with the provided `${BUILD_TAG}`
	470	4. Trigger a deploy job using the parameter `${BUILD_TAG}`
	471	* a deploy job should roll out the changes by updating the image tag in the DC:
	472	1. Take an input of the `${BUILD_TAG}`
	473	2. Patch / set the DeploymentConfig to the image's `${BUILD_TAG}`
	474	3. Rollout the changes
	475	4. Verify the deployment
76d54e	476	* We will now go through these steps in detail.
9fc88c	477
d4f1fa	478	#### 4a - Build
9fc88c	479
c58300	480	5. With the BuildConfig and DeployConfig in place for both our apps (`-fe` & `-api`) from previous steps; Log into Jenkins and create a `New Item`. This is just jenkins speak for a new job configuration. ![new-item](../images/exercise2/new-item.png)
9fc88c	481
579436	482	5. Name this job `dev-todolist-fe-build` and select `Freestyle Job`. All our jobs will take the form of `<ENV>-<APP_NAME>-<JOB_PURPOSE>`. ![freestyle-job](../images/exercise2/freestyle-job.png)
9fc88c	483
b815cc	484	5. The page that loads is the Job Configuration page and it can be returned to at anytime from Jenkins. Let's start configuring our job. To conserve space; we will make sure Jenkins only keeps the last builds artifacts. Tick the `Discard old builds` checkbox, then `Advanced` and set `Max # of builds to keep with artifacts` to 1 as indicated below
D	485	![keep-artifacts](../images/exercise2/keep-artifacts.png)
43f2f2	486
579436	487	5. Our NodeJS build needs to be run on the `jenkins-slave-npm` we created earlier. Specify this in the box labelled `Restrict where this project can be run` ![label-jenkins-slave](../images/exercise2/label-jenkins-slave.png)
c951f7	488
1b285d	489	5. On the Source Code Management tab, select the Git radio button, specify the endpoint for our GitLab `todolist-fe` Project and specify your credentials from the dropdown box. Set the Branch Specifier to `develop`. ![git-scm](../images/exercise2/git-scm.png)
c951f7	490
579436	491	5. Scroll down to the Build Environment tab and select the `Color ANSI Console Output` checkbox ![ansi](../images/exercise2/ansi.png)
e919d0	492
76d54e	493	5. Move on to the Build section and select `Add build step`. From the dropdown select `Execute Shell`. On the box that appears; insert the following, to build package and deploy our app to Nexus:
e919d0	494	```bash
D	495	set -o xtrace
4acca2	496	npm install
D	497	npm run build:ci:dev
	498	npm run package
	499	npm run publish
e919d0	500	```
D	501	![build-step](../images/exercise2/build-step.png)
	502
579436	503	5. Scroll to the final section; the Post-build Actions. Add a new post-build action from the dropdown called `Archive the artifacts` and specify `**` in the box. This will zip the entire workspace and copy it back to Jenkins for inspection if needed. ![archive-artifacts](../images/exercise2/archive-artifacts.png)
e919d0	504
579436	505	5. On the Post-build Actions; Add another post-build action from the dropdown called `Git Publisher`. This is useful for tying the git check-in to the feature in your tracking tool to the built product.
e919d0	506	* Tick the box `Push Only If Build Succeeds`
D	507	* Add the Tag to push of
	508	```bash
	509	${JOB_NAME}.${BUILD_NUMBER}
	510	```
	511	* Specify the commit message to be
	512	```bash
	513	Automated commit by jenkins from ${JOB_NAME}.${BUILD_NUMBER}
	514	```
1b285d	515
A	516	* Check `Create New Tag` and set `Target remote name` to `origin`
e919d0	517	![git-publisher](../images/exercise2/git-publisher.png)
D	518
1b285d	519	5. Finally; add the trigger for the next job in the pipeline. This is to trigger the bake job with the current build tag. Add another post-build action from the dropdown called `Trigger parameterized build on other projects`.
e919d0	520	* Set the project to build to be `dev-todolist-fe-bake`
D	521	* Set the condition to be `Stable or unstable but not failed`.
	522	* Click Add Parameters dropdown and select Predefined parameters.
	523	* In the box, insert our BUILD_TAG as follows
	524	```bash
	525	BUILD_TAG=${JOB_NAME}.${BUILD_NUMBER}
	526	```
	527	![param-trigger](../images/exercise2/param-trigger.png)
	528	<p class="tip">
3b5f91	529	NOTE - Jenkins might say "No such project ‘dev-todolist-fe-bake’. Did you mean ...." at this point. Don't worry; it's because we have not created the next job yet.
e919d0	530	</p>
D	531
579436	532	5. Hit `save` which will take you to the job overview page - and that's it; our build phase is complete!
e919d0	533
d4f1fa	534	#### 4b - Bake
41217d	535
579436	536	5. Next we will setup our bake phase; which is a little simpler. Go to Jenkins home and create another Freestyle Job (as before) called `dev-todolist-fe-bake`.
e919d0	537
1b285d	538	5. This job will take in the BUILD_TAG from the previous one so check the `This project is parameterized` box on the General tab.
3b5f91	539	* Add string parameter type
1b285d	540	* set the Name to `BUILD_TAG`. This will be available to the job as an Enviroment Variable.
3b5f91	541	* You can set `dev-todolist-fe-build.` as the default value for ease when triggering manually.
D	542	* The description is not required but a handy one for reference would be `${JOB_NAME}.${BUILD_NUMBER} of previous build eg dev-todolist-fe-build.1232`
2ff842	543	<p class="tip">
A	544	NOTE - Don't forget to include the `.` after `dev-todolist-fe-build` in the Default Value box.
	545	</p>
	546
3b5f91	547	![param-trigger-bake](../images/exercise2/param-trigger-bake.png)
c951f7	548
579436	549	5. This time set the `Restrict where this project can be run` label to `master`.
3b5f91	550	<p class="tip">
2ff842	551	NOTE - `Master` is the default node that jobs run on. We don't want jenkins to execute the bake on any other nodes if the `master` is busy so it is always safer to specify it here.
3b5f91	552	</p>
D	553
579436	554	5. There is no Git or SCM needed for this job so move down to the Build Environment and tick `Delete workspace before build starts`
3b5f91	555
579436	556	5. Scroll down to the Build Environment tab and select the `Color ANSI Console Output` checkbox ![delete-ansi](../images/exercise2/delete-ansi.png)
3b5f91	557
579436	558	5. Move on to the Build section and select `Add build step`. From the dropdown select `Execute Shell`. On the box the appears; insert the following, to pull the package from Nexus. We patch the BuildConfig with the Jenkins Tag to get traceablility from feature to source code to built item. Finally; the oc start-build command is run:
1b285d	559	Remember to replace `<YOUR_NAME>` accordingly.
3b5f91	560	```bash
D	561	#!/bin/bash
d00794	562	curl -v -f http://admin:admin123@${NEXUS_SERVICE_HOST}:${NEXUS_SERVICE_PORT}/repository/zip/com/redhat/todolist/${BUILD_TAG}/package-contents.zip -o package-contents.zip
3b5f91	563	unzip package-contents.zip
cebd30	564	oc project <YOUR_NAME>-ci-cd
3b5f91	565	NAME=todolist-fe
9adaf7	566	oc patch bc ${NAME} -p "{\"spec\":{\"output\":{\"to\":{\"kind\":\"ImageStreamTag\",\"name\":\"${NAME}:${BUILD_TAG}\"}}}}"
3b5f91	567	oc start-build ${NAME} --from-dir=package-contents/ --follow
D	568	```
	569	![bake-step](../images/exercise2/bake-step.png)
	570
579436	571	5. Finally; add the trigger for the next job in the pipeline. Add a post-build action from the dropdown called `Trigger parameterized build on other projects`.
3b5f91	572	* Set the project to build to be `dev-todolist-fe-deploy`
D	573	* Set the condition to be `Stable`.
3dce53	574	* Click Add Parameters dropdown and select `Current build parameters`. This will pass the `${BUILD_TAG}` to the downstream job which we will create next.
3b5f91	575	![downstream-trigger-deploy](../images/exercise2/downstream-trigger-deploy.png)
D	576
579436	577	5. Hit save! That's our bake phase done! Finally; on to our deploy
3b5f91	578
d4f1fa	579	#### 4c - Deploy
41217d	580
1b285d	581	5. Next we will setup our deploy phase. This job is very similar in setup to the bake phase so this time go to Jenkins home and create `dev-todolist-fe-deploy` Job but scroll to the bottom and Copy from `dev-todolist-fe-bake`.
579436	582	![copy-from](../images/exercise2/copy-from.png)
D	583
1b285d	584	5. The only two differences between these jobs is the Build Step and there are no Post Build Actions. First to the Build tab and add the following to the shell box. The process for running the deploy is to tag the image created previously for use in the `ci-cd` namespace for use in the dev project. Then update the DeploymentConfig to use the Jenkins Tag which kicked the process off. Once successful; the changes are rolled out. Remember to change `<YOUR_NAME>` accordingly.
579436	585	```bash
D	586	#!/bin/bash
	587	set -o xtrace
	588	# VARS
1b285d	589	PIPELINES_NAMESPACE=<YOUR_NAME>-ci-cd
A	590	NAMESPACE=<YOUR_NAME>-dev
579436	591	NAME=todolist-fe
D	592	oc project ${NAMESPACE}
	593	oc tag ${PIPELINES_NAMESPACE}/${NAME}:${BUILD_TAG} ${NAMESPACE}/${NAME}:${BUILD_TAG}
	594	oc set env dc ${NAME} NODE_ENV=dev
	595	oc set image dc/${NAME} ${NAME}=docker-registry.default.svc:5000/${NAMESPACE}/${NAME}:${BUILD_TAG}
	596	oc rollout latest dc/${NAME}
	597	```
	598	![deploy-step](../images/exercise2/deploy-step.png)
	599
3dce53	600	5. When a deployment has completed; OpenShift can verify it's success. Add another step by clicking the `Add build Step` on the Build tab then `Verify OpenShift Deployment` including the following:
579436	601	* Set the Project to your `<YOUR_NAME>-dev`
D	602	* Set the DeploymentConfig to your app's name `todolist-fe`
	603	* Set the replica count to `1`
	604	![verify-deployment](../images/exercise2/verify-deployment.png)
	605
	606	5. Finally; delete the Post Build Action to trigger another job (by hitting the red X). Save the configuration. We're almost ready to run the pipeline!
	607
d4f1fa	608	#### 4d - Pipeline
41217d	609
3dce53	610	5. With our Jenkins setup in place; now move to our `todolist-fe` app's source code. We have to add our configuration to the frontend to tell it where the API layer will be hosted. Open the source in your favourite editor and navigate to `src/config/dev.js`.
D	611
	612	5. Update `<YOUR_NAME>` accordingly with the route where the Todo List API will live when it is deployed, update the `somedomain` to the clusters domain. The correct full URL can also be found on the OpenShift Console; if you copy it from there remember to append `/api/todos` to the URL. For example:
41217d	613	![fe-dev-config](../images/exercise2/fe-dev-config.png)
D	614
3dce53	615	5. Repeat this for `src/config/test.js` file. If you copy the URL from the previous step; change `dev` to `test`. Once done; commit your chanages and push them to GitLab
41217d	616	```bash
D	617	$ git add .
	618	$ git commit -m "ADD config for api"
	619	$ git push
	620	```
	621
cebd30	622	5. Back on Jenkins; We can tie all the jobs in the pipeline together into a nice single view using the Build Pipeline view. Back on the Jenkins home screen Click the + beside the all tab on the top.
D	623	![add-view](../images/exercise2/add-view.png)
	624
	625	5. On the view that loads; Give the new view a sensible name like `dev-todolist-fe-pipeline` and select Build Pipeline
	626	![new-pipeline](../images/exercise2/new-pipeline.png)
579436	627
D	628	5. Set the Pipeline Flow's Inital Job to `dev-todolist-fe-build` and save.
	629	![pipeline-flow](../images/exercise2/pipeline-flow.png)
	630
cebd30	631	5. You should now see the pipeline view. Run the pipeline by hitting run (you can move onto the next part while it is running as it may take some time).
af284d	632	![dev-pipeline-view](../images/exercise2/dev-pipeline-view.jpeg)
579436	633
D	634	### Part 5 - Backend Pipeline
1b285d	635	> In this exercise we will use the Jobs created for the `todolist-fe` as a template to create a pipeline for the `todolist-api` app by copying the config.
579436	636
1b285d	637	6. On Jenkins home; create a new job for our backend build called `dev-todolist-api-build`. Use the `Copy from` section to copy all the configuration from the `dev-todolist-fe-build`.
d00794	638	![copy-fe-build](../images/exercise2/copy-fe-build.png)
579436	639
d00794	640	6. When this has loaded; find and replace both occurrences `-fe` with `-api` within the Job's configuration. Places to make sure you check are:
D	641	* The GitLab project URL
	642	* Projects to build on the Post Build Action
579436	643
1b285d	644	6. On the Build tab; remove the `:dev` from the `npm run build:ci:dev` so the line reads.
A	645	The rest of the instructions can be left as they are.
d00794	646	```bash
4acca2	647	npm run build:ci
d00794	648	```
cebd30	649	![api-build-step](../images/exercise2/api-build-step.png)
579436	650
d00794	651	6. Save the configuration for `dev-todolist-api-build`
579436	652
1b285d	653	6. On Jenkins home; create a new job for our backend bake called `dev-todolist-api-bake`. Use the Copy from section to copy all the configuration from the `dev-todolist-fe-bake` as you've just done.
d00794	654
D	655	6. When this has loaded; find and replace the occurrences `-fe` with `-api` within the Job's configuration. Places to make sure you check are:
	656	* The BUILD_TAG default value and description
	657	* NAME in the execute shell step
	658	* Projects to build on the Post Build Action
	659
	660	6. Save the configuration for `dev-todolist-api-build`
	661
	662	6. On Jenkins home; create a new job for our backend build called `dev-todolist-api-deploy`. Use the Copy from section to copy all the configuration from the `dev-todolist-fe-deploy` as you've just done.
	663
	664	6. When this has loaded; find and replace the occurrences `-fe` with `-api` within the Job's configuration. Places to make sure you check are:
	665	* The BUILD_TAG default value and description
	666	* NAME in the execute shell step
	667	* The name of the DeploymentConfig to validate in the Verify OpenShift Deployment
	668
b815cc	669	6. Save the configuration for `dev-todolist-api-deploy` and that's it for wiring together our `todolist-api` pipeline.
d00794	670
cebd30	671	6. Run the `dev-todolist-api-build` to trigger the backend pipeline. While this is building, check our front end app and see if it has deployed successfully.
D	672
3dce53	673	6. To check the deployment in OpenShift; open the console and go to your `dev` namespace. You should see the deployment was successful; hit the URL to open the app (the screenshot below has both apps deployed).
cebd30	674	![ocp-deployment](../images/exercise2/ocp-deployment.png)
D	675
	676	6. If it has been a success we should see our dummyData. This is because there is no backend deployed.
d00794	677	![no-backend-app](../images/exercise2/no-backend-app.png)
D	678
456daa	679	6. When `dev-todolist-api-build` has completed we should see the sample data has changed on refresh.
d00794	680	![with-backend-app](../images/exercise2/with-backend-app.png)
c951f7	681
106095	682	### Part 6 - GitLab Webhooks
b00f5a	683	> _In this exercise we will link GitLab to Jenkins so that new build jobs are triggered on each push to the `develop` branch._
11b338	684
b00f5a	685	<p class="tip" >
D	686	NOTE - This section is optional! Git webhooks are useful but not needed for course completion.
	687	</p>
	688
b815cc	689	7. In order to allow GitLab trigger Jenkins (because of the OpenShift Auth Plugin), we need to allow the `Anonymous` user trigger builds. Head to your Jenkins Dashboard and click on `Manage Jenkins` on the left hand side. Then scroll down and click `Configure Global Security`. Alternatively, type in `https://jenkins-<YOUR_NAME>-ci-cd.apps.some.domain.com/configureSecurity/` . You should see a screen like so:
106095	690	![jenkins-global-security](../images/exercise2/jenkins-global-security.png)
A	691
	692	7. Scroll down to the `Authorization` section and allow `Anonymous` to create jobs. Do this by navigating through the matrix of checkboxes and check `Build` and `Cancel` under the Job heading. Leave all other user behaviour as is. Anonymous is the user that GitLab will act as so this allows the WebHook to trigger builds. (The screenshot has been cropped to bring Job further to the left.) Hit `Save` or `Apply`.
	693	![jenkins-anon-permissions](../images/exercise2/jenkins-anon-permissions.png)
	694
7c720c	695	7. Go to your `dev-todolist-fe-build` and head to the `configure` section (`https://jenkins-<YOUR_NAME>-ci-cd.apps.some.domain.com/job/dev-todolist-fe-build/configure`). Scroll down to the `Build Triggers` section and check the `Build when a change is pushed to GitLab` box. Leave all the other settings as they are but copy the `GitLab webhook URL`. `https://jenkins-<YOUR_NAME>-ci-cd.apps.some.domain.com/project/dev-todolist-fe-build`. Remember to Save and Apply this change.
0c725b	696	![jenkins-build-triggers-gitlab](../images/exercise2/jenkins-build-triggers-gitlab.png)
11b338	697
b311e1	698	7. Switch over to GitLab and select your `todolist-fe` repository. On the left hand task bar hover over the settings cog and select `integrations`. (`https://gitlab-<YOUR_NAME>-ci-cd.apps.some.domain.com/<YOUR_NAME>/todolist-fe/settings/integrations`)
0c725b	699	![gitlab-integrations](../images/exercise2/gitlab-integrations.png)
11b338	700
0c725b	701	7. Paste the `GitLab webhook URL` that we copied earlier into the `URL` field. Check Push events as the trigger, and make sure you `uncheck` the `SSL verification` checkbox. Click Add webhook at the bottom.
A	702	![gitlab-integrations-details](../images/exercise2/gitlab-integrations-details.png)
11b338	703
0c725b	704	7. Before we move on let's test the webhook. Select the Test drop down and click `Push events`. This will trigger the test and return a status code at the top of the page. If all goes well it should be a cool blue 200.
A	705	![gitlab-integrations-details](../images/exercise2/gitlab-webhook-test.png)
11b338	706
0c725b	707	7. We can now test this properly by heading into the `todolist-fe` repository through <YOUR_FAVOURITE_EDITOR>. Make a small change to your code, then commit and push it, ensuring you're on the develop branch. Then head over to Jenkins and wait until the `dev-todolist-fe-build` job has been triggered.
11b338	708
7c720c	709	7. All that's left to do is to repeat the same steps for `todolist-api` (Starting from step 3):
0c725b	710	Create Build Trigger:
b311e1	711	`https://jenkins-<YOUR_NAME>-ci-cd.apps.some.domain.com/job/dev-todolist-api-build/configure`
0c725b	712	Create GitLab Integration:
b311e1	713	`https://gitlab-<YOUR_NAME>-ci-cd.apps.some.domain.com/donal/todolist-api/settings/integrations`
0c725b	714	Check your build status and you should see something like this. With `Started by Gitlab push by <YOUR_NAME>`:
A	715	![jenkins-gitlab-webhook-success](../images/exercise2/jenkins-gitlab-webhook-success.png)
11b338	716
0c725b	717	7. We now have a working GitLab webhook so any time we push code it will automatically build! Next up we'll show you how to add tests to your pipeline.
b00f5a	718
43f2f2	719	_____
D	720
c951f7	721	## Extension Tasks
cf415b	722	> _Ideas for go-getters. Advanced topic for doers to get on with if they finish early. These will usually not have a solution available and are provided for additional scope._
c951f7	723
d00794	724	- Pipeline Tasks
456daa	725	* Add pipeline for `master` branch for each project. Use `test-` instead of `dev-` across all config and names in the pipeline
A	726	* Do the `.openshift-applier` steps as part of the pipeline for greater end to end automation.
e919d0	727	- Promote build
456daa	728	* Create a _promote-to-uat_ phase after the `master` branch deploy
cf415b	729	* Create a `uat` env using the OpenShift Applier as seen before
e919d0	730	* Tag and promote the image without rebuilding after the `test-**-deploy`
c58300	731	- MongoDB tasks
e919d0	732	* Add MongoDB Stateful set for the UAT environment (or test)
D	733	* Inject MongoDB config into the NodeJS app using config map & secrets.
cf415b	734	* Improve the security of the DB by making the user /passwords randomly generated
e919d0	735	- Setup Nexus as an `npm` mirror registry and use it in the builds to speed up the build time
c951f7	736
D	737	## Additional Reading
43f2f2	738	> List of links or other reading that might be of use / reference for the exercise
7383de	739
43f2f2	740	## Slide links
7c832b	741
01c4da	742	- [Intro](https://docs.google.com/presentation/d/1t1CONuy-_IRPZYmU010Qgk2rshiDJTennvLyQR8GllE)
RH	743	- [Wrap-up](https://docs.google.com/presentation/d/1kZ8SV6iJnrKk_AqPpyPuNZifv7VzItHOB9HYdOnNJjI)
5d4563	744	- [All Material](https://drive.google.com/drive/folders/1lf66ks2tT0eQ4A9RSU48u0ZhvBXzoHWJ)