RedHatTraining/rht-labs-docs.git

donal

2018-05-09 58480f18403a3e74f9afa99d15c8a89ffaeeec2a

commit \| author \| age
3772d9	1	# Attack of the Pipelines
0f4d08	2
867471	3	> In this exercise we will explore the sample TODO List application and create a pipeline in Jenkins to build and deploy our code.
3772d9	4
D	5	![jenkins-time](../images/exercise2/jenkins-time.jpg)
	6
78b569	7	## Exercise Intro
D	8	This lesson is focused on creating a pipeline for our application. What is a pipeline? A pipeline is a series of steps or stages that takes our code from source to a deployed application. There can be many stages to a pipeline but a simple flow is to run a `build > bake > deploy`. Usually the first stage is trigger by something like a git commit. There could be many steps in each of these stages; such as compiling code, running tests and linting. All of these are done to try and drive up code quality and give more assurance that what is deployed is behaving as expected. In the exercise we will create Jenkins pipeline by configuring it through the UI, this will create an un-gated pathway to production
	9
	10	First we will explore the sample application and get it running locally. The sample app is a `todolist` app - the `Hello World` app of the modern day.
	11
	12	#### Why create pipelines
	13	* Assurance - drive up code quality and remove the need for dedicated deployment / release management teams
	14	* Freedom - allow developers to take ownership of how and when code gets built and shipped
2059d3	15	* Reliability - pipelines are a bit boring; they execute the same way each and every time they're run!
78b569	16	* A pathway to production:
D	17	- Puts the product in the hands of the customer quicker
	18	- Enables seamless and repeatable deploys
	19	- More prod like infrastructure increases assurance
	20	- “We have already done it” behavior de-risks go live
	21
5a16fd	22	_____
0f4d08	23
D	24	## Learning Outcomes
3772d9	25	As a learner by the end of this lesson you will be able to
D	26
	27	- Build and run the full stack of the TODO List application locally
	28	- Create an un-gated pipeline using the Jenkins UI for the backend and frontend
	29	- Add branching to the pipeline to target specific namespace
0f4d08	30
D	31	## Tools and Frameworks
8a47e6	32	> The following tools are used throughout this exercise. Familiarity with them is not required but knowing what they are may help. You will not need to install Vue or Mongodb they are taken care of by our `todolist` app.
0f4d08	33
D	34	1. [Jenkins](https://jenkins.io/) - OpenSource build automation server; highly customisable through plugins
3772d9	35	1. [NodeJS](https://nodejs.org/en/) - Node.js® is a JavaScript runtime built on Chrome's V8 JavaScript engine. Node.js uses an event-driven, non-blocking I/O model that makes it lightweight and efficient. Node.js' package ecosystem, npm, is the largest ecosystem of open source libraries in the world.
D	36	1. [MongoDB](https://www.mongodb.com/what-is-mongodb) - MongoDB stores data in flexible, JSON-like documents, meaning fields can vary from document to document and data structure can be changed over time
	37	1. [VueJS](https://vuejs.org/) - Vue (pronounced /vjuː/, like view) is a progressive framework for building user interfaces. It is designed from the ground up to be incrementally adoptable, and can easily scale between a library and a framework depending on different use cases. It consists of an approachable core library that focuses on the view layer only, and an ecosystem of supporting libraries that helps you tackle complexity in large Single-Page Applications.
0f4d08	38
5a16fd	39	## Big Picture
3772d9	40	> From the previous exercise; we created some supporting tooling needed by our app/
5a16fd	41
D	42	_____
0f4d08	43
D	44	## 10,000 Ft View
3772d9	45	> _This lab requires users to take the sample TODO app and create a build pipeline in Jenkins by clicking your way to success ending up with an app deployed to each of the namespaces created previously_
5a16fd	46
3772d9	47	2. Import the projects into your gitlab instance. See README of each for build instructions
5a16fd	48
3772d9	49	2. Deploy a `MongoDB` using the provided template to all project namespace.
D	50
	51	2. Create 2 pipline with three stages (`build`, `bake`, `deploy`) in jenkins for `develop` & `master` branches on the `todolist-fe` such that:
c3a934	52	* a `Build` job is responsible for compiling and packaging our code:
3772d9	53	1. Checkout from source code (`develop` for `<yourname>-dev` & `master` for `<yourname>-test`)
D	54	2. Install node dependencies and run a build / package
c3a934	55	3. Send the package to Nexus
D	56	4. Archive the workspace to persist the workspace in case of failure
	57	4. Tag the GitLab repository with the `${JOB_NAME}.${BUILD_NUMBER}` from Jenkins. This is our `${BUILD_TAG}` which will be used on downstream jobs.
	58	5. Trigger the `bake` job with the `${BUILD_TAG}` param
	59	* a `Bake` job should take the package and put it in a Linux Container
3772d9	60	1. Take an input of the previous jobs `${BUILD_TAG}` ie `${JOB_NAME}.${BUILD_NUMBER}`.
D	61	2. Checkout the binary from Nexus and unzip it's contents
c3a934	62	3. Run an oc start-build of the App's BuildConfig and tag it's imagestream with the provided `${BUILD_TAG}`
3772d9	63	4. Trigger a deploy job using the parameter `${BUILD_TAG}`
c3a934	64	* a `deploy` job should roll out the changes by updating the image tag in the DC:
3772d9	65	1. Take an input of the `${BUILD_TAG}`
D	66	2. Patch / set the DeploymentConfig to the image's `${BUILD_TAG}`
	67	3. Rollout the changes
	68	4. Verify the deployment
	69
	70	2. Repeat the above setup for the backend `todolist-fe`. TIP - use the copy config to speed things up!
	71
	72	2. Verify that both apps and the DB are talking to one another as expected.
0f4d08	73
D	74	## Step by Step Instructions
f2b1b4	75	> This is a fairly structured guide with references to exact filenames and sections of text to be added.
0f4d08	76
14268c	77	### Part 1 - Explore the Todo List App
f2b1b4	78	> _In this part of the exercise we will explore the sample application, become familiar with it locally before building and deploying in OCP Land_
5a16fd	79
752f2a	80	#### 1a Todolist-fe
256ca9	81
f2b1b4	82	2. Git clone the `todolist-fe` project to somewhere sensible and checkout the `develop` branch.
D	83	```bash
58480f	84	cd ~/innovation-labs
D	85	```
	86	```bash
ad8436	87	git clone https://github.com/rht-labs/todolist-fe.git
58480f	88	```
D	89	```bash
ad8436	90	cd todolist-fe
256ca9	91	```
ad8436	92	```bash
D	93	./git-pull-all.sh
58480f	94	```
D	95	```bash
	96	git checkout develop
0f4d08	97	```
f2b1b4	98
486224	99	2. Open up Gitlab and login. Create a new project (internal) in GitLab called `todolist-fe` to host your clone of the project and copy it's remote address. ![new-gitlab-proj](../images/exercise2/new-gitlab-proj.png)
f2b1b4	100
D	101	2. In your local clone of the `todolist-fe`, remove the origin and add the GitLab origin by replacing `<YOUR_GIT_LAB_PROJECT>`. Push your app to GitLab
	102	```bash
ad8436	103	git remote set-url origin <YOUR_GIT_LAB_PROJECT>
486224	104	# verify the origin has been updated
ad8436	105	git remote -v
D	106	git push -u origin --all
0f4d08	107	```
f2b1b4	108
D	109	2. To get the app running locally; first check you've got node and npm installed
	110	```bash
ad8436	111	node -v
D	112	npm -v
f2b1b4	113	```
D	114	<p class="tip" >
	115	NOTE - If you are missing these dependencies; install them with ease using the [Node Version Manager](https://github.com/creationix/nvm)
	116	</p>
	117	![node-version](../images/exercise2/node-version.png)
	118
	119	2. The `todolist-fe` has a package.json at the root of the project, this defines some configuration for the app including it's dependencies, dev dependencies, scripts and other configuration. Install the apps dependencies
	120	```bash
ad8436	121	npm install
f2b1b4	122	```
D	123
514d1d	124	2. The `todolist-fe` has some scripts defined in the package.json at the root of the project. A snippet of the npm scripts are shown below. To run any of these scripts run `npm run <SCRIPT_NAME>`. Let's start by serving our application
f2b1b4	125	![npm-scripts](../images/exercise2/npm-scripts.png)
D	126	```bash
	127	npm run serve
	128	```
	129
486224	130	2. This will take sometime to execute; but once done it should open the browser for you displaying the homepage of the `todolist-fe` app.
f2b1b4	131	![todo-list-app](../images/exercise2/todo-list-app.png)
486224	132	* Click 'Todo' at the top of the home page to get to the above page.
514d1d	133	* The server hosting it live reloads; so if you make changes to your code base the app will live update
f2b1b4	134	* The Data you see in the screen is dummy / stubbed data. This is served up when there is no backend connection found
D	135
514d1d	136	2. The app is a todolist manager built in Vue.js. Play around with the App. You will notice when you add todos they appear and clear as expected. If you refresh the page you'll lose all additions. This is because there is no persistence layer. We will add one in the next part.
f2b1b4	137
D	138	3. The structure of the `todolist-fe` is as follows.
	139	```bash
	140	todolist-fe
	141	├── jest.config.js
	142	├── jsconfig.json
	143	├── nightwatch.config.js
	144	├── node_modules
	145	├── package.json
	146	├── public
	147	│ ├── favicon.ico
	148	│ ├── img
	149	│ ├── index.html
	150	│ └── manifest.json
	151	├── src
	152	│ ├── App.vue
	153	│ ├── assets
	154	│ ├── components
486224	155	│ │ └── *
f2b1b4	156	│ ├── config
D	157	│ ├── main.js
	158	│ ├── registerServiceWorker.js
	159	│ ├── router.js
	160	│ ├── scss
	161	│ ├── services
	162	│ ├── store
486224	163	│ │ └── *
f2b1b4	164	│ └── views
486224	165	│ └── *
f2b1b4	166	├── tests
D	167	│ ├── e2e
	168	│ └── unit
	169	└── vue.config.js
	170	```
	171	where the following are the important things:
	172	* `./src` is the main collection of files needed by the app. The entrypoint is the `main.js` which is used to load the root `App.vue` file.
	173	* `./node_modules` is where the dependencies are stored
867471	174	* `./test` contains our end-to-end tests and unit tests. More covered on these in later exercises.
486224	175	* `./src/components` contains small, lightweight reusable components for our app. For example, the `NewTodo` component which encapsulates the styling, logic and data for adding a new todo to our list
f2b1b4	176	* `./src/store` is the `vuex` files for managing application state and backend connectivity
D	177	* `./src/views` is the view containers; which are responsible for loading components and managing their interactions.
	178	* the `./src/router.js` controls routing logic. In our case the app only has one real endpoint.
	179	* `./src/scss` contains custom SCSS used in the application.
	180	* `./*.js` is mostly config files for running and managing the app and the tests
0f4d08	181
1937a3	182	2. To prepare Nexus to host the binaries created by the frontend and backend builds we need to run a prepare-nexus script. Before we do this we need to export some variables change `<YOUR_NAME>` and `somedomain` accordingly.
256ca9	183	```bash
12ca4a	184	export NEXUS_SERVICE_HOST=$(oc get route nexus --template='{{.spec.host}}' -n <YOUR_NAME>-ci-cd)
256ca9	185	export NEXUS_SERVICE_PORT=80
A	186	npm run prepare-nexus
	187	```
	188	<p class="tip">
	189	NOTE - This step in a residency would be automated by a more complex nexus deployment in the ci-cd project
	190	</p>
	191
752f2a	192	#### 1b Todolist-api
f9e311	193
3691a1	194	2. Now let's move on to the `todolist-api` and wire them together. As with the `todolist-fe` we need to clone the repo and add it to our GitLab in the cluster.
D	195	```bash
58480f	196	cd ~/innovation-labs
D	197	```
	198	```bash
ad8436	199	git clone https://github.com/rht-labs/todolist-api.git
58480f	200	```
D	201	```bash
ad8436	202	cd todolist-api
256ca9	203	```
ad8436	204	```bash
D	205	./git-pull-all.sh
58480f	206	```
D	207	```bash
	208	git checkout develop
3691a1	209	```
D	210
514d1d	211	2. On GitLab; create a new project (internal) called `todolist-api` to host your clone of the project and copy it's remote address as you did for the previous repositories.
3691a1	212
D	213	2. In your local clone of the `todolist-api`, remove the origin and add the GitLab origin by replacing `<YOUR_GIT_LAB_PROJECT>`. Push your app to GitLab
	214	```bash
ad8436	215	git remote set-url origin <YOUR_GIT_LAB_PROJECT>
D	216	```
	217	```bash
	218	git push -u origin --all
3691a1	219	```
D	220
	221	2. Once pushed; explore the application. It is a NodeJS application with the Express.js framework and MongoDB for persistent storage. Same as before, the `package.json` defines most of the configuration etc. Install the dependencies
	222	```bash
ad8436	223	npm i
3691a1	224	```
D	225
	226	2. While the dependencies are being installed; explore the project structure.
	227	```bash
	228	todolist-api
	229	├── Dockerfile
	230	├── Gruntfile.js
	231	├── README.md
	232	├── node_modules
	233	├── package-lock.json
	234	├── package.json
	235	├── server
	236	│ ├── api
	237	│ │ └── todo
	238	│ ├── app.js
	239	│ ├── components
	240	│ │ └── errors
	241	│ ├── config
	242	│ │ ├── environment
	243	│ │ ├── express.js
	244	│ │ ├── local.env.sample.js
	245	│ │ └── seed.js
	246	│ ├── mocks
	247	│ │ ├── mock-routes-config.json
	248	│ │ ├── mock-routes.js
	249	│ │ └── mock-routes.spec.js
	250	│ ├── routes.js
	251	│ └── views
	252	│ └── 404.html
	253	└── tasks
	254	└── perf-test.js
	255	```
	256	where the following are the important things:
	257	* `./server` is the main collection of files needed by the app. The entrypoint is the `app.js`
	258	* `./node_modules` is where the dependencies are stored
	259	* `./server/api` is where the api's controller, data model & unit test are stored.
	260	* `./server/mocks` is a mock server used for when there is no DB access
486224	261	* `./server/config` stores our Express JS config, header information and other middleware.
A	262	* `./server/config/environment` stores enviromnent specific config; such as connectivity to backend services like MongoDB.
867471	263	* `./tasks` is a collection of additional `Grunt` tasks which will be used in later exercises
486224	264	* `Grunt` is a taskrunner for use with Node.JS projects
3691a1	265	* `package.json` contains the dependency list and a lot of very helpful scripts for managing the app lifecycle
D	266
1d9ebc	267	2. A snippet of the npm scripts are shown below. There are application start scripts, build and test items which will be used in the build. The ones for MongoDB are just provided for convenience and require Docker installed to execute.
3691a1	268	```json
D	269	"scripts": {
	270	"start": "node server/app.js",
	271	"dev": "./node_modules/.bin/grunt serve",
	272	"jshint": "./node_modules/.bin/grunt jshint",
	273	"clean": "rm -rf reports package-contents*",
	274	"package": "zip -r package-contents.zip package-contents",
	275	"test": "node_modules/.bin/nyc node_modules/.bin/mocha server/*/.spec.js --exit",
	276	"mongo" : "docker run -i -d --name mongo-local -p 27017:27017 mongo",
	277	"mongo:drop" : "npm run mongo:stop && docker rm mongo-local",
	278	"mongo:stop" : "docker stop mongo-local",
	279	"mongo:start" : "docker start mongo-local"
	280	},
f9e311	281	```
A	282
514d1d	283	2. To run the application; start a new instance of the MongoDB by running the following. This will pull a mongodb image from Dockerhub and then start it for our API to connect to.
3691a1	284	```bash
ad8436	285	npm run mongo
3691a1	286	```
D	287	<p class="tip">
e8cd20	288	NOTE - `npm run mongo:drop` is used to completely remove the running container. `npm run mongo:stop` & `npm run mongo:start` will preserve data in the container
3691a1	289	</p>
D	290
	291	2. Fire up the `todolist-api` by running.
	292	```bash
ad8436	293	npm run start
3691a1	294	```
D	295	![node-app-started](../images/exercise2/node-app-started.png)
	296
486224	297	2. Check things are up and running by testing the API with a `curl`. The API should return some seeded data (stored in `server/config/seed.js`)
3691a1	298	```bash
ad8436	299	curl localhost:9000/api/todos
3691a1	300	```
D	301	```json
	302	[{
	303	"_id": "5ac8ff1fdfafb02138698948",
	304	"title": "Learn some stuff about MongoDB",
	305	"completed": false,
	306	"__v": 0
	307	},
	308	{
	309	"_id": "5ac8ff1fdfafb02138698949",
	310	"title": "Play with NodeJS",
	311	"completed": true,
	312	"__v": 0
	313	}]
	314	```
	315
486224	316	2. Now let's check out `todolist-fe` app by reloading the browser. We should now see our dummy front end data is replaced by the backend seed data. Adding new todos will add them in the backend, these will persist when the page is refreshed.
3691a1	317	![fullstack-app](../images/exercise2/fullstack-app.png)
D	318
	319
273e4c	320	### Part 2 - Create a NodeJS Build slave
374f63	321	> _In this exercise; we will create a build configuration to generate a slave for Jenkins to use in it's builds_
273e4c	322
7dca92	323	3. In order for Jenkins to be able to run `npm` builds and installs as we have done locally, we must configure a `jenkins-build-slave` for Jenkins to use. This slave will be dynamically provisioned when we run a build. It needs to have NodeJS and npm installed in it. In your `enablement-ci-cd` repository, checkout the template and configuration. This will bring in the template, the params & the `Dockerfile`.
273e4c	324	```bash
ad8436	325	git checkout exercise2/jenkins-slave docker/ templates/ params/jenkins-slave-npm
273e4c	326	```
D	327
d82125	328	3. Open the `params/jenkins-slave-npm` file and update `<GIT_URL>` accordingly. This set of parameters will clone from the enablement repo and run a docker build of the Dockerfile stored in `docker/jenkins-slave-npm`.
273e4c	329	```bash
d82125	330	SOURCE_REPOSITORY_URL=<GIT_URL>
273e4c	331	SOURCE_CONTEXT_DIR=docker/jenkins-slave-npm
1937a3	332	NAME=jenkins-slave-npm
273e4c	333	```
D	334
b92135	335	3. Create an item in the `inventory/host_vars/ci-cd-tooling.yml` under the `ci-cd-builds` object to run the template with.
273e4c	336	```yaml
1937a3	337	- name: "jenkins-slave-npm"
b92135	338	namespace: "{{ ci_cd_namespace }}"
D	339	template: "{{ playbook_dir }}/templates/jenkins-slave-generic-template.yml"
	340	params: "{{ playbook_dir }}/params/jenkins-slave-npm"
273e4c	341	tags:
D	342	- jenkins-slave
	343	```
	344	![jenkins-slave-ansible](../images/exercise2/jenkins-slave-ansible.png)
	345
e23af1	346	3. Commit your changes to the `enablement-ci-cd` repository!
D	347	```bash
ad8436	348	git add .
D	349	```
	350	```bash
	351	git commit -m "ADD npm slave node for Jenkins"
	352	```
	353	```bash
	354	git push
e23af1	355	```
D	356
273e4c	357	3. Run the OpenShift Applier to trigger a build of this jenkins slave image.
D	358	```bash
ad8436	359	ansible-playbook apply.yml -e target=tools \
273e4c	360	-i inventory/ \
D	361	-e "filter_tags=jenkins-slave"
	362	```
	363
	364	3. Verify the build executed successfully by logging into the cluster and checking the `builds` tab of the `<YOUR_NAME>-ci-cd` project.
	365	![jenkins-slave-npm-build](../images/exercise2/jenkins-slave-npm-build.png)
	366
867471	367	3. You should now be able to apply the label `jenkins-slave-npm` to a build job to run a build on this newly created slave as we will see in the rest of this exercise
273e4c	368	<p class="tip">
D	369	NOTE - Jenkins may need to be restarted for the configuration to appear. To do this; navigate to your jenkins instance and add `/restart` to the url.
	370	</p>
	371
	372	### Part 3 - Add configs to cluster
14268c	373	> _In this exercise; we will use the OpenShift Applier to drive the creation of cluster content required by the app such as MongoDB and the Apps Build / Deploy Config_
273e4c	374
ac41ef	375	4. On your terminal navigate to the root of the `todolist-fe` application. The app contains a hidden folder called `.openshift-applier`. Move into this `.openshift-applier` directory and you should see a familiar looking directory structure for an ansible playbook.
d98f40	376	```
D	377	.openshift-applier
	378	├── README.md
	379	├── apply.yml
	380	├── inventory
	381	│ ├── group_vars
	382	│ │ └── all.yml
	383	│ └── hosts
	384	├── params
	385	│ ├── build
	386	│ ├── dev
3f9666	387	│ ├── ocp-pipeline
d98f40	388	│ └── test
D	389	├── requirements.yml
	390	└── templates
3f9666	391	├── ocp-pipeline.yml
D	392	├── todolist-fe-build.yml
	393	└── todolist-fe-deploy.yml
d98f40	394	```
2310e7	395	with the following
d1565e	396	* the `apply.yml` file is the entrypoint.
d98f40	397	* the `inventory` contains the objects to populate the cluster with.
D	398	* the `params` contains the variables we'll apply to the `templates`
14268c	399	* the `templates` required by the app. These include the Build, Deploy configs as well as the services, health checks, and other app definitions.
d98f40	400
d1565e	401	4. There are a few updates to these manifests we need to make before applying the cluster content. In the `apply.yml` update the namespace `<YOUR_NAME>` variables accordingly.
d98f40	402	```yaml
D	403	ci_cd_namespace: donal-ci-cd
	404	dev_namespace: donal-dev
	405	test_namespace: donal-test
	406	```
	407
d1565e	408	4. In the `params` folder update the `dev` and `test` files with the correct `<YOUR_NAME>` as you've done above. Example for the `dev` file:
d98f40	409	```bash
D	410	PIPELINES_NAMESPACE=donal-ci-cd
261f19	411	NAME=todolist-fe
d98f40	412	DEPLOYER_USER=jenkins
D	413	APP_TAG=latest
	414	NAMESPACE=donal-dev
	415	```
	416
14268c	417	4. With those changes in place we can now run the playbook. First install the `openshift-applier` dependency and then run the playbook (from the `.openshift-applier` directory). This will populate the cluster with all the config needed for the front end app.
d98f40	418	```bash
ad8436	419	ansible-galaxy install -r requirements.yml --roles-path=roles
D	420	```
	421	```bash
	422	ansible-playbook apply.yml -i inventory/
d98f40	423	```
69a4e4	424	![ansible-success](../images/exercise2/ansible-success.png)
d98f40	425
14268c	426	4. Once successful, `commit` and `push` your changes to gitlab.
3f9666	427	```bash
ad8436	428	git add .
D	429	```
	430	```bash
	431	git commit -m "UPDATE - change namespace vars to donal"
	432	```
	433	```bash
	434	git push
3f9666	435	```
14268c	436
176e08	437	4. Back on your terminal navigate to the root of the `todolist-api` application. Open the `.openshift-applier` directory. The same layout as seen in `todolist-fe` should be visible with one noticeable difference; the api requires `MongoDB` to connect to at runtime.
261f19	438
D	439	4. In the `apply.yml` update the namespace `<YOUR_NAME>` variables accordingly. For example:
	440	```yaml
	441	ci_cd_namespace: donal-ci-cd
	442	dev_namespace: donal-dev
	443	test_namespace: donal-test
	444	```
	445
	446	4. In the `params` folder update the `dev` and `test` files with the correct `<YOUR_NAME>` as you've done above. Example for the `dev` file:
	447	```bash
	448	PIPELINES_NAMESPACE=donal-ci-cd
	449	NAME=todolist-api
	450	DEPLOYER_USER=jenkins
	451	APP_TAG=latest
	452	NAMESPACE=donal-dev
	453	```
	454
d362dc	455	4. Finally; run the Openshift Applier and install its dependencies to run the content into the cluster
261f19	456	```bash
ad8436	457	ansible-galaxy install -r requirements.yml --roles-path=roles
D	458	```
	459	```bash
	460	ansible-playbook apply.yml -i inventory/
261f19	461	```
273e4c	462
3f9666	463	4. Once successful, `commit` and `push` your changes to gitlab.
D	464	```bash
ad8436	465	git add .
D	466	```
	467	```bash
	468	git commit -m "UPDATE - change namespace vars to donal"
	469	```
	470	```bash
	471	git push
3f9666	472	```
176e08	473
3f9666	474	4. Validate the build and deploy configs have been created in Openshift by checking `<YOUR_NAME> CI-CD builds` for the `BuildConfigs`
D	475	![ocp-app-bc](../images/exercise2/ocp-app-bc.png)
	476
	477	4. Check `<YOUR_NAME>-dev` to see the deployment configs are in place
ee4a9b	478	![ocp-app-dc](../images/exercise2/ocp-app-dc.png)
176e08	479
273e4c	480	### Part 4 - Build > Bake > Deploy
b15f06	481	> _In this exercise; we take what we have working locally and get it working in OpenShift_
273e4c	482
40ab32	483	This exercise will involve creating three stages (or items) in our pipeline, each of these is detailed below at a very high level. Move on to the next step to begin implementation.
D	484	* a build job is responsible for compiling and packaging our code:
	485	1. Checkout from source code (`develop` for `<yourname>-dev` & `master` for `<yourname>-test`)
	486	2. Install node dependencies and run a build / package
	487	3. Send the package to Nexus
	488	4. Archive the workspace to persist the workspace in case of failure
	489	4. Tag the GitLab repository with the `${JOB_NAME}.${BUILD_NUMBER}` from Jenkins. This is our `${BUILD_TAG}` which will be used on downstream jobs.
	490	5. Trigger the `bake` job with the `${BUILD_TAG}` param
	491	* a bake job should take the package and put it in a Linux Container
	492	1. Take an input of the previous jobs `${BUILD_TAG}` ie `${JOB_NAME}.${BUILD_NUMBER}`.
	493	2. Checkout the binary from Nexus and unzip it's contents
	494	3. Run an oc start-build of the App's BuildConfig and tag it's imagestream with the provided `${BUILD_TAG}`
	495	4. Trigger a deploy job using the parameter `${BUILD_TAG}`
	496	* a deploy job should roll out the changes by updating the image tag in the DC:
	497	1. Take an input of the `${BUILD_TAG}`
	498	2. Patch / set the DeploymentConfig to the image's `${BUILD_TAG}`
	499	3. Rollout the changes
	500	4. Verify the deployment
19df4d	501	* We will now go through these steps in detail.
273e4c	502
752f2a	503	#### 4a - Build
273e4c	504
261f19	505	5. With the BuildConfig and DeployConfig in place for both our apps (`-fe` & `-api`) from previous steps; Log into Jenkins and create a `New Item`. This is just jenkins speak for a new job configuration. ![new-item](../images/exercise2/new-item.png)
273e4c	506
dd060d	507	5. Name this job `dev-todolist-fe-build` and select `Freestyle Job`. All our jobs will take the form of `<ENV>-<APP_NAME>-<JOB_PURPOSE>`. ![freestyle-job](../images/exercise2/freestyle-job.png)
273e4c	508
1937a3	509	5. The page that loads is the Job Configuration page and it can be returned to at anytime from Jenkins. Let's start configuring our job. To conserve space; we will make sure Jenkins only keeps the last builds artifacts. Tick the `Discard old builds` checkbox, then `Advanced` and set `Max # of builds to keep with artifacts` to 1 as indicated below
D	510	![keep-artifacts](../images/exercise2/keep-artifacts.png)
5a16fd	511
dd060d	512	5. Our NodeJS build needs to be run on the `jenkins-slave-npm` we created earlier. Specify this in the box labelled `Restrict where this project can be run` ![label-jenkins-slave](../images/exercise2/label-jenkins-slave.png)
0f4d08	513
19f0bd	514	5. On the Source Code Management tab, select the Git radio button, specify the endpoint for our GitLab `todolist-fe` Project and specify your credentials from the dropdown box. Set the Branch Specifier to `develop`. ![git-scm](../images/exercise2/git-scm.png)
0f4d08	515
dd060d	516	5. Scroll down to the Build Environment tab and select the `Color ANSI Console Output` checkbox ![ansi](../images/exercise2/ansi.png)
c3a934	517
19df4d	518	5. Move on to the Build section and select `Add build step`. From the dropdown select `Execute Shell`. On the box that appears; insert the following, to build package and deploy our app to Nexus:
c3a934	519	```bash
D	520	set -o xtrace
7a3088	521	npm install
D	522	npm run build:ci:dev
	523	npm run package
	524	npm run publish
c3a934	525	```
D	526	![build-step](../images/exercise2/build-step.png)
	527
dd060d	528	5. Scroll to the final section; the Post-build Actions. Add a new post-build action from the dropdown called `Archive the artifacts` and specify `**` in the box. This will zip the entire workspace and copy it back to Jenkins for inspection if needed. ![archive-artifacts](../images/exercise2/archive-artifacts.png)
c3a934	529
dd060d	530	5. On the Post-build Actions; Add another post-build action from the dropdown called `Git Publisher`. This is useful for tying the git check-in to the feature in your tracking tool to the built product.
c3a934	531	* Tick the box `Push Only If Build Succeeds`
D	532	* Add the Tag to push of
	533	```bash
	534	${JOB_NAME}.${BUILD_NUMBER}
	535	```
	536	* Specify the commit message to be
	537	```bash
	538	Automated commit by jenkins from ${JOB_NAME}.${BUILD_NUMBER}
	539	```
19f0bd	540
A	541	* Check `Create New Tag` and set `Target remote name` to `origin`
c3a934	542	![git-publisher](../images/exercise2/git-publisher.png)
D	543
19f0bd	544	5. Finally; add the trigger for the next job in the pipeline. This is to trigger the bake job with the current build tag. Add another post-build action from the dropdown called `Trigger parameterized build on other projects`.
c3a934	545	* Set the project to build to be `dev-todolist-fe-bake`
D	546	* Set the condition to be `Stable or unstable but not failed`.
	547	* Click Add Parameters dropdown and select Predefined parameters.
	548	* In the box, insert our BUILD_TAG as follows
	549	```bash
	550	BUILD_TAG=${JOB_NAME}.${BUILD_NUMBER}
	551	```
	552	![param-trigger](../images/exercise2/param-trigger.png)
	553	<p class="tip">
eaf747	554	NOTE - Jenkins might say "No such project ‘dev-todolist-fe-bake’. Did you mean ...." at this point. Don't worry; it's because we have not created the next job yet.
c3a934	555	</p>
D	556
dd060d	557	5. Hit `save` which will take you to the job overview page - and that's it; our build phase is complete!
c3a934	558
752f2a	559	#### 4b - Bake
40ab32	560
dd060d	561	5. Next we will setup our bake phase; which is a little simpler. Go to Jenkins home and create another Freestyle Job (as before) called `dev-todolist-fe-bake`.
c3a934	562
19f0bd	563	5. This job will take in the BUILD_TAG from the previous one so check the `This project is parameterized` box on the General tab.
eaf747	564	* Add string parameter type
19f0bd	565	* set the Name to `BUILD_TAG`. This will be available to the job as an Enviroment Variable.
eaf747	566	* You can set `dev-todolist-fe-build.` as the default value for ease when triggering manually.
D	567	* The description is not required but a handy one for reference would be `${JOB_NAME}.${BUILD_NUMBER} of previous build eg dev-todolist-fe-build.1232`
363974	568	<p class="tip">
A	569	NOTE - Don't forget to include the `.` after `dev-todolist-fe-build` in the Default Value box.
	570	</p>
	571
eaf747	572	![param-trigger-bake](../images/exercise2/param-trigger-bake.png)
0f4d08	573
dd060d	574	5. This time set the `Restrict where this project can be run` label to `master`.
eaf747	575	<p class="tip">
363974	576	NOTE - `Master` is the default node that jobs run on. We don't want jenkins to execute the bake on any other nodes if the `master` is busy so it is always safer to specify it here.
eaf747	577	</p>
D	578
dd060d	579	5. There is no Git or SCM needed for this job so move down to the Build Environment and tick `Delete workspace before build starts`
eaf747	580
dd060d	581	5. Scroll down to the Build Environment tab and select the `Color ANSI Console Output` checkbox ![delete-ansi](../images/exercise2/delete-ansi.png)
eaf747	582
dd060d	583	5. Move on to the Build section and select `Add build step`. From the dropdown select `Execute Shell`. On the box the appears; insert the following, to pull the package from Nexus. We patch the BuildConfig with the Jenkins Tag to get traceablility from feature to source code to built item. Finally; the oc start-build command is run:
19f0bd	584	Remember to replace `<YOUR_NAME>` accordingly.
eaf747	585	```bash
D	586	#!/bin/bash
436657	587	curl -v -f http://admin:admin123@${NEXUS_SERVICE_HOST}:${NEXUS_SERVICE_PORT}/repository/zip/com/redhat/todolist/${BUILD_TAG}/package-contents.zip -o package-contents.zip
eaf747	588	unzip package-contents.zip
8a47e6	589	oc project <YOUR_NAME>-ci-cd
eaf747	590	NAME=todolist-fe
62d4b9	591	oc patch bc ${NAME} -p "{\"spec\":{\"output\":{\"to\":{\"kind\":\"ImageStreamTag\",\"name\":\"${NAME}:${BUILD_TAG}\"}}}}"
eaf747	592	oc start-build ${NAME} --from-dir=package-contents/ --follow
D	593	```
	594	![bake-step](../images/exercise2/bake-step.png)
	595
dd060d	596	5. Finally; add the trigger for the next job in the pipeline. Add a post-build action from the dropdown called `Trigger parameterized build on other projects`.
eaf747	597	* Set the project to build to be `dev-todolist-fe-deploy`
D	598	* Set the condition to be `Stable`.
ac41ef	599	* Click Add Parameters dropdown and select `Current build parameters`. This will pass the `${BUILD_TAG}` to the downstream job which we will create next.
eaf747	600	![downstream-trigger-deploy](../images/exercise2/downstream-trigger-deploy.png)
D	601
dd060d	602	5. Hit save! That's our bake phase done! Finally; on to our deploy
eaf747	603
752f2a	604	#### 4c - Deploy
40ab32	605
19f0bd	606	5. Next we will setup our deploy phase. This job is very similar in setup to the bake phase so this time go to Jenkins home and create `dev-todolist-fe-deploy` Job but scroll to the bottom and Copy from `dev-todolist-fe-bake`.
dd060d	607	![copy-from](../images/exercise2/copy-from.png)
D	608
19f0bd	609	5. The only two differences between these jobs is the Build Step and there are no Post Build Actions. First to the Build tab and add the following to the shell box. The process for running the deploy is to tag the image created previously for use in the `ci-cd` namespace for use in the dev project. Then update the DeploymentConfig to use the Jenkins Tag which kicked the process off. Once successful; the changes are rolled out. Remember to change `<YOUR_NAME>` accordingly.
dd060d	610	```bash
D	611	#!/bin/bash
	612	set -o xtrace
	613	# VARS
19f0bd	614	PIPELINES_NAMESPACE=<YOUR_NAME>-ci-cd
A	615	NAMESPACE=<YOUR_NAME>-dev
dd060d	616	NAME=todolist-fe
D	617	oc project ${NAMESPACE}
	618	oc tag ${PIPELINES_NAMESPACE}/${NAME}:${BUILD_TAG} ${NAMESPACE}/${NAME}:${BUILD_TAG}
	619	oc set env dc ${NAME} NODE_ENV=dev
	620	oc set image dc/${NAME} ${NAME}=docker-registry.default.svc:5000/${NAMESPACE}/${NAME}:${BUILD_TAG}
	621	oc rollout latest dc/${NAME}
	622	```
	623	![deploy-step](../images/exercise2/deploy-step.png)
	624
ac41ef	625	5. When a deployment has completed; OpenShift can verify it's success. Add another step by clicking the `Add build Step` on the Build tab then `Verify OpenShift Deployment` including the following:
dd060d	626	* Set the Project to your `<YOUR_NAME>-dev`
D	627	* Set the DeploymentConfig to your app's name `todolist-fe`
	628	* Set the replica count to `1`
	629	![verify-deployment](../images/exercise2/verify-deployment.png)
	630
	631	5. Finally; delete the Post Build Action to trigger another job (by hitting the red X). Save the configuration. We're almost ready to run the pipeline!
	632
752f2a	633	#### 4d - Pipeline
40ab32	634
ac41ef	635	5. With our Jenkins setup in place; now move to our `todolist-fe` app's source code. We have to add our configuration to the frontend to tell it where the API layer will be hosted. Open the source in your favourite editor and navigate to `src/config/dev.js`.
D	636
	637	5. Update `<YOUR_NAME>` accordingly with the route where the Todo List API will live when it is deployed, update the `somedomain` to the clusters domain. The correct full URL can also be found on the OpenShift Console; if you copy it from there remember to append `/api/todos` to the URL. For example:
40ab32	638	![fe-dev-config](../images/exercise2/fe-dev-config.png)
D	639
ac41ef	640	5. Repeat this for `src/config/test.js` file. If you copy the URL from the previous step; change `dev` to `test`. Once done; commit your chanages and push them to GitLab
40ab32	641	```bash
ad8436	642	git add .
D	643	```
	644	```bash
	645	git commit -m "ADD config for api"
	646	```
	647	```bash
	648	git push
40ab32	649	```
D	650
8a47e6	651	5. Back on Jenkins; We can tie all the jobs in the pipeline together into a nice single view using the Build Pipeline view. Back on the Jenkins home screen Click the + beside the all tab on the top.
D	652	![add-view](../images/exercise2/add-view.png)
	653
	654	5. On the view that loads; Give the new view a sensible name like `dev-todolist-fe-pipeline` and select Build Pipeline
	655	![new-pipeline](../images/exercise2/new-pipeline.png)
dd060d	656
D	657	5. Set the Pipeline Flow's Inital Job to `dev-todolist-fe-build` and save.
	658	![pipeline-flow](../images/exercise2/pipeline-flow.png)
	659
8a47e6	660	5. You should now see the pipeline view. Run the pipeline by hitting run (you can move onto the next part while it is running as it may take some time).
204a7b	661	![dev-pipeline-view](../images/exercise2/dev-pipeline-view.jpeg)
dd060d	662
D	663	### Part 5 - Backend Pipeline
19f0bd	664	> In this exercise we will use the Jobs created for the `todolist-fe` as a template to create a pipeline for the `todolist-api` app by copying the config.
dd060d	665
19f0bd	666	6. On Jenkins home; create a new job for our backend build called `dev-todolist-api-build`. Use the `Copy from` section to copy all the configuration from the `dev-todolist-fe-build`.
436657	667	![copy-fe-build](../images/exercise2/copy-fe-build.png)
dd060d	668
436657	669	6. When this has loaded; find and replace both occurrences `-fe` with `-api` within the Job's configuration. Places to make sure you check are:
D	670	* The GitLab project URL
	671	* Projects to build on the Post Build Action
dd060d	672
19f0bd	673	6. On the Build tab; remove the `:dev` from the `npm run build:ci:dev` so the line reads.
A	674	The rest of the instructions can be left as they are.
436657	675	```bash
7a3088	676	npm run build:ci
436657	677	```
8a47e6	678	![api-build-step](../images/exercise2/api-build-step.png)
dd060d	679
436657	680	6. Save the configuration for `dev-todolist-api-build`
dd060d	681
19f0bd	682	6. On Jenkins home; create a new job for our backend bake called `dev-todolist-api-bake`. Use the Copy from section to copy all the configuration from the `dev-todolist-fe-bake` as you've just done.
436657	683
D	684	6. When this has loaded; find and replace the occurrences `-fe` with `-api` within the Job's configuration. Places to make sure you check are:
	685	* The BUILD_TAG default value and description
	686	* NAME in the execute shell step
	687	* Projects to build on the Post Build Action
	688
	689	6. Save the configuration for `dev-todolist-api-build`
	690
	691	6. On Jenkins home; create a new job for our backend build called `dev-todolist-api-deploy`. Use the Copy from section to copy all the configuration from the `dev-todolist-fe-deploy` as you've just done.
	692
	693	6. When this has loaded; find and replace the occurrences `-fe` with `-api` within the Job's configuration. Places to make sure you check are:
	694	* The BUILD_TAG default value and description
	695	* NAME in the execute shell step
	696	* The name of the DeploymentConfig to validate in the Verify OpenShift Deployment
	697
1937a3	698	6. Save the configuration for `dev-todolist-api-deploy` and that's it for wiring together our `todolist-api` pipeline.
436657	699
8a47e6	700	6. Run the `dev-todolist-api-build` to trigger the backend pipeline. While this is building, check our front end app and see if it has deployed successfully.
D	701
ac41ef	702	6. To check the deployment in OpenShift; open the console and go to your `dev` namespace. You should see the deployment was successful; hit the URL to open the app (the screenshot below has both apps deployed).
8a47e6	703	![ocp-deployment](../images/exercise2/ocp-deployment.png)
D	704
	705	6. If it has been a success we should see our dummyData. This is because there is no backend deployed.
436657	706	![no-backend-app](../images/exercise2/no-backend-app.png)
D	707
f9e311	708	6. When `dev-todolist-api-build` has completed we should see the sample data has changed on refresh.
436657	709	![with-backend-app](../images/exercise2/with-backend-app.png)
0f4d08	710
f50a94	711	### Part 6 - GitLab Webhooks
b770c6	712	> _In this exercise we will link GitLab to Jenkins so that new build jobs are triggered on each push to the `develop` branch._
13e8fd	713
b770c6	714	<p class="tip" >
570732	715	NOTE - This section is optional! Git webhooks are useful but not needed for Enablement completion.
b770c6	716	</p>
D	717
1937a3	718	7. In order to allow GitLab trigger Jenkins (because of the OpenShift Auth Plugin), we need to allow the `Anonymous` user trigger builds. Head to your Jenkins Dashboard and click on `Manage Jenkins` on the left hand side. Then scroll down and click `Configure Global Security`. Alternatively, type in `https://jenkins-<YOUR_NAME>-ci-cd.apps.some.domain.com/configureSecurity/` . You should see a screen like so:
f50a94	719	![jenkins-global-security](../images/exercise2/jenkins-global-security.png)
A	720
	721	7. Scroll down to the `Authorization` section and allow `Anonymous` to create jobs. Do this by navigating through the matrix of checkboxes and check `Build` and `Cancel` under the Job heading. Leave all other user behaviour as is. Anonymous is the user that GitLab will act as so this allows the WebHook to trigger builds. (The screenshot has been cropped to bring Job further to the left.) Hit `Save` or `Apply`.
	722	![jenkins-anon-permissions](../images/exercise2/jenkins-anon-permissions.png)
	723
30f5d9	724	7. Go to your `dev-todolist-fe-build` and head to the `configure` section (`https://jenkins-<YOUR_NAME>-ci-cd.apps.some.domain.com/job/dev-todolist-fe-build/configure`). Scroll down to the `Build Triggers` section and check the `Build when a change is pushed to GitLab` box. Leave all the other settings as they are but copy the `GitLab webhook URL`. `https://jenkins-<YOUR_NAME>-ci-cd.apps.some.domain.com/project/dev-todolist-fe-build`. Remember to Save and Apply this change.
c50c7b	725	![jenkins-build-triggers-gitlab](../images/exercise2/jenkins-build-triggers-gitlab.png)
13e8fd	726
db60b7	727	7. Switch over to GitLab and select your `todolist-fe` repository. On the left hand task bar hover over the settings cog and select `integrations`. (`https://gitlab-<YOUR_NAME>-ci-cd.apps.some.domain.com/<YOUR_NAME>/todolist-fe/settings/integrations`)
c50c7b	728	![gitlab-integrations](../images/exercise2/gitlab-integrations.png)
13e8fd	729
c50c7b	730	7. Paste the `GitLab webhook URL` that we copied earlier into the `URL` field. Check Push events as the trigger, and make sure you `uncheck` the `SSL verification` checkbox. Click Add webhook at the bottom.
A	731	![gitlab-integrations-details](../images/exercise2/gitlab-integrations-details.png)
13e8fd	732
c50c7b	733	7. Before we move on let's test the webhook. Select the Test drop down and click `Push events`. This will trigger the test and return a status code at the top of the page. If all goes well it should be a cool blue 200.
A	734	![gitlab-integrations-details](../images/exercise2/gitlab-webhook-test.png)
13e8fd	735
c50c7b	736	7. We can now test this properly by heading into the `todolist-fe` repository through <YOUR_FAVOURITE_EDITOR>. Make a small change to your code, then commit and push it, ensuring you're on the develop branch. Then head over to Jenkins and wait until the `dev-todolist-fe-build` job has been triggered.
13e8fd	737
30f5d9	738	7. All that's left to do is to repeat the same steps for `todolist-api` (Starting from step 3):
c50c7b	739	Create Build Trigger:
db60b7	740	`https://jenkins-<YOUR_NAME>-ci-cd.apps.some.domain.com/job/dev-todolist-api-build/configure`
c50c7b	741	Create GitLab Integration:
db60b7	742	`https://gitlab-<YOUR_NAME>-ci-cd.apps.some.domain.com/donal/todolist-api/settings/integrations`
c50c7b	743	Check your build status and you should see something like this. With `Started by Gitlab push by <YOUR_NAME>`:
A	744	![jenkins-gitlab-webhook-success](../images/exercise2/jenkins-gitlab-webhook-success.png)
13e8fd	745
c50c7b	746	7. We now have a working GitLab webhook so any time we push code it will automatically build! Next up we'll show you how to add tests to your pipeline.
b770c6	747
5a16fd	748	_____
D	749
0f4d08	750	## Extension Tasks
a4f42c	751	> _Ideas for go-getters. Advanced topic for doers to get on with if they finish early. These will usually not have a solution available and are provided for additional scope._
0f4d08	752
436657	753	- Pipeline Tasks
f9e311	754	* Add pipeline for `master` branch for each project. Use `test-` instead of `dev-` across all config and names in the pipeline
A	755	* Do the `.openshift-applier` steps as part of the pipeline for greater end to end automation.
c3a934	756	- Promote build
f9e311	757	* Create a _promote-to-uat_ phase after the `master` branch deploy
a4f42c	758	* Create a `uat` env using the OpenShift Applier as seen before
c3a934	759	* Tag and promote the image without rebuilding after the `test-**-deploy`
261f19	760	- MongoDB tasks
c3a934	761	* Add MongoDB Stateful set for the UAT environment (or test)
D	762	* Inject MongoDB config into the NodeJS app using config map & secrets.
a4f42c	763	* Improve the security of the DB by making the user /passwords randomly generated
c3a934	764	- Setup Nexus as an `npm` mirror registry and use it in the builds to speed up the build time
0f4d08	765
D	766	## Additional Reading
5a16fd	767	> List of links or other reading that might be of use / reference for the exercise
f2b1b4	768
5a16fd	769	## Slide links
9eed0b	770
4f0295	771	- [Intro](https://docs.google.com/presentation/d/1t1CONuy-_IRPZYmU010Qgk2rshiDJTennvLyQR8GllE)
RH	772	- [Wrap-up](https://docs.google.com/presentation/d/1kZ8SV6iJnrKk_AqPpyPuNZifv7VzItHOB9HYdOnNJjI)
3207b9	773	- [All Material](https://drive.google.com/drive/folders/1lf66ks2tT0eQ4A9RSU48u0ZhvBXzoHWJ)