| commit | author | age | ||
| 3772d9 | 1 | # Attack of the Pipelines |
| 0f4d08 | 2 | |
| 3772d9 | 3 | > In this lab we will explore the sample TODO List application and create a pipeline in Jenkins to build and deploy our code. |
| D | 4 | |
| 5 |  | |
| 6 | ||
| 5a16fd | 7 | _____ |
| 0f4d08 | 8 | |
| D | 9 | ## Learning Outcomes |
| 3772d9 | 10 | As a learner by the end of this lesson you will be able to |
| D | 11 | |
| 12 | - Build and run the full stack of the TODO List application locally | |
| 13 | - Create an un-gated pipeline using the Jenkins UI for the backend and frontend | |
| 14 | - Add branching to the pipeline to target specific namespace | |
| 0f4d08 | 15 | |
| D | 16 | ## Tools and Frameworks |
| 3772d9 | 17 | > The following tools are used throughout this exercise. Familiarity with them is not required but knowing what they are may help! |
| 0f4d08 | 18 | |
| D | 19 | 1. [Jenkins](https://jenkins.io/) - OpenSource build automation server; highly customisable through plugins |
| 3772d9 | 20 | 1. [NodeJS](https://nodejs.org/en/) - Node.js® is a JavaScript runtime built on Chrome's V8 JavaScript engine. Node.js uses an event-driven, non-blocking I/O model that makes it lightweight and efficient. Node.js' package ecosystem, npm, is the largest ecosystem of open source libraries in the world. |
| D | 21 | 1. [MongoDB](https://www.mongodb.com/what-is-mongodb) - MongoDB stores data in flexible, JSON-like documents, meaning fields can vary from document to document and data structure can be changed over time |
| 22 | 1. [VueJS](https://vuejs.org/) - Vue (pronounced /vjuː/, like view) is a progressive framework for building user interfaces. It is designed from the ground up to be incrementally adoptable, and can easily scale between a library and a framework depending on different use cases. It consists of an approachable core library that focuses on the view layer only, and an ecosystem of supporting libraries that helps you tackle complexity in large Single-Page Applications. | |
| 0f4d08 | 23 | |
| 5a16fd | 24 | ## Big Picture |
| 3772d9 | 25 | > From the previous exercise; we created some supporting tooling needed by our app/ |
| 5a16fd | 26 | |
| D | 27 | _____ |
| 0f4d08 | 28 | |
| D | 29 | ## 10,000 Ft View |
| 3772d9 | 30 | > _This lab requires users to take the sample TODO app and create a build pipeline in Jenkins by clicking your way to success ending up with an app deployed to each of the namespaces created previously_ |
| 5a16fd | 31 | |
| 3772d9 | 32 | 2. Import the projects into your gitlab instance. See README of each for build instructions |
| 5a16fd | 33 | |
| 3772d9 | 34 | 2. Deploy a `MongoDB` using the provided template to all project namespace. |
| D | 35 | |
| 36 | 2. Create 2 pipline with three stages (`build`, `bake`, `deploy`) in jenkins for `develop` & `master` branches on the `todolist-fe` such that: | |
| 37 | * `Build` job should: | |
| 38 | 1. Checkout from source code (`develop` for `<yourname>-dev` & `master` for `<yourname>-test`) | |
| 39 | 2. Install node dependencies and run a build / package | |
| 40 | 3. Send the package to Nexus (as a zip) | |
| 41 | 4. Tag the git repository with the `${JOB_NAME}.${BUILD_NUMBER}` from Jenkins | |
| 42 | 5. Trigger the `bake` job | |
| 43 | * `Bake` job should: | |
| 44 | 1. Take an input of the previous jobs `${BUILD_TAG}` ie `${JOB_NAME}.${BUILD_NUMBER}`. | |
| 45 | 2. Checkout the binary from Nexus and unzip it's contents | |
| 46 | 3. Run and OC build of the App and tag it's imagestream with the provided `${BUILD_TAG}` | |
| 47 | 4. Trigger a deploy job using the parameter `${BUILD_TAG}` | |
| 48 | * `deploy` job should: | |
| 49 | 1. Take an input of the `${BUILD_TAG}` | |
| 50 | 2. Patch / set the DeploymentConfig to the image's `${BUILD_TAG}` | |
| 51 | 3. Rollout the changes | |
| 52 | 4. Verify the deployment | |
| 53 | ||
| 54 | 2. Repeat the above setup for the backend `todolist-fe`. TIP - use the copy config to speed things up! | |
| 55 | ||
| 56 | 2. Verify that both apps and the DB are talking to one another as expected. | |
| 0f4d08 | 57 | |
| D | 58 | ## Step by Step Instructions |
| f2b1b4 | 59 | > This is a fairly structured guide with references to exact filenames and sections of text to be added. |
| 0f4d08 | 60 | |
| f2b1b4 | 61 | ### Part 1 - Explore the Sample App |
| D | 62 | > _In this part of the exercise we will explore the sample application, become familiar with it locally before building and deploying in OCP Land_ |
| 5a16fd | 63 | |
| f2b1b4 | 64 | 2. Git clone the `todolist-fe` project to somewhere sensible and checkout the `develop` branch. |
| D | 65 | ```bash |
| 66 | $ git clone https://github.com/springdo/todolist-fe.git | |
| 67 | $ git checkout develop | |
| 0f4d08 | 68 | ``` |
| f2b1b4 | 69 | |
| D | 70 | 2. Create a new project (internal) in GitLab called `todolist-fe` to host your clone of the project and copy it's remote address.  |
| 71 | ||
| 72 | 2. In your local clone of the `todolist-fe`, remove the origin and add the GitLab origin by replacing `<YOUR_GIT_LAB_PROJECT>`. Push your app to GitLab | |
| 73 | ```bash | |
| 74 | $ git remote remove origin | |
| 75 | $ git remote add origin <YOUR_GIT_LAB_PROJECT> | |
| 76 | $ git push -u origin --all | |
| 0f4d08 | 77 | ``` |
| f2b1b4 | 78 | |
| D | 79 | 2. To get the app running locally; first check you've got node and npm installed |
| 80 | ```bash | |
| 81 | $ node -v | |
| 82 | $ npm -v | |
| 83 | ``` | |
| 84 | <p class="tip" > | |
| 85 | NOTE - If you are missing these dependencies; install them with ease using the [Node Version Manager](https://github.com/creationix/nvm) | |
| 86 | </p> | |
| 87 |  | |
| 88 | ||
| 89 | 2. The `todolist-fe` has a package.json at the root of the project, this defines some configuration for the app including it's dependencies, dev dependencies, scripts and other configuration. Install the apps dependencies | |
| 90 | ```bash | |
| 91 | $ npm install | |
| 92 | ``` | |
| 93 | ||
| 94 | 2. The `todolist-fe` has some scripts defined in the package.json at the root of the project. To Run any of these scripts run `npm run <SCRIPT_NAME>`. Let's start by serving our application | |
| 95 |  | |
| 96 | ```bash | |
| 97 | npm run serve | |
| 98 | ``` | |
| 99 | ||
| 100 | 2. This will take sometime to execute; but once done it should open the browser for you displaying the `todolist-fe` app. | |
| 101 |  | |
| 102 | * The server hosting it live reloads; so as you make changes to your code; the app will live update | |
| 103 | * The Data you see in the screen is dummy / stubbed data. This is served up when there is no backend connection found | |
| 104 | ||
| 3691a1 | 105 | 2. The app is a todolist manager built in Vue.js. Play around with the App. You will notice when you add todos they appear and clear as expected. If you refresh the page you'll loose all additions. This is because there is persistence |
| f2b1b4 | 106 | |
| D | 107 | 3. The structure of the `todolist-fe` is as follows. |
| 108 | ```bash | |
| 109 | todolist-fe | |
| 110 | ├── jest.config.js | |
| 111 | ├── jsconfig.json | |
| 112 | ├── nightwatch.config.js | |
| 113 | ├── node_modules | |
| 114 | ├── package.json | |
| 115 | ├── public | |
| 116 | │ ├── favicon.ico | |
| 117 | │ ├── img | |
| 118 | │ ├── index.html | |
| 119 | │ └── manifest.json | |
| 120 | ├── src | |
| 121 | │ ├── App.vue | |
| 122 | │ ├── assets | |
| 123 | │ ├── components | |
| 124 | │ ├── config | |
| 125 | │ ├── main.js | |
| 126 | │ ├── registerServiceWorker.js | |
| 127 | │ ├── router.js | |
| 128 | │ ├── scss | |
| 129 | │ ├── services | |
| 130 | │ ├── store | |
| 131 | │ └── views | |
| 132 | ├── tests | |
| 133 | │ ├── e2e | |
| 134 | │ └── unit | |
| 135 | └── vue.config.js | |
| 136 | ``` | |
| 137 | where the following are the important things: | |
| 138 | * `./src` is the main collection of files needed by the app. The entrypoint is the `main.js` which is used to load the root `App.vue` file. | |
| 139 | * `./node_modules` is where the dependencies are stored | |
| 140 | * `./test` contains our end-to-end tests and unit tests. More covered on these in later labs. | |
| 141 | * `./src/components` contains small, lightweight reusable components for our app. These include the `NewTodo` component which encapsulates the styling, logic and data for adding a new todo to our list | |
| 142 | * `./src/store` is the `vuex` files for managing application state and backend connectivity | |
| 143 | * `./src/views` is the view containers; which are responsible for loading components and managing their interactions. | |
| 144 | * the `./src/router.js` controls routing logic. In our case the app only has one real endpoint. | |
| 145 | * `./src/scss` contains custom SCSS used in the application. | |
| 146 | * `./*.js` is mostly config files for running and managing the app and the tests | |
| 0f4d08 | 147 | |
| 3691a1 | 148 | 2. Now let's move on to the `todolist-api` and wire them together. As with the `todolist-fe` we need to clone the repo and add it to our GitLab in the cluster. |
| D | 149 | ```bash |
| 150 | $ git clone https://github.com/springdo/todolist-api.git | |
| 151 | $ git checkout develop | |
| 152 | ``` | |
| 153 | ||
| 154 | 2. Create a new project (internal) in GitLab called `todolist-api` to host your clone of the project and copy it's remote address. | |
| 155 | ||
| 156 | 2. In your local clone of the `todolist-api`, remove the origin and add the GitLab origin by replacing `<YOUR_GIT_LAB_PROJECT>`. Push your app to GitLab | |
| 157 | ```bash | |
| 158 | $ git remote remove origin | |
| 159 | $ git remote add origin <YOUR_GIT_LAB_PROJECT> | |
| 160 | $ git push -u origin --all | |
| 161 | ``` | |
| 162 | ||
| 163 | 2. Once pushed; explore the application. It is a NodeJS application with the Express.js framework and MongoDB for persistent storage. Same as before, the `package.json` defines most of the configuration etc. Install the dependencies | |
| 164 | ```bash | |
| 165 | $ npm i | |
| 166 | ``` | |
| 167 | ||
| 168 | 2. While the dependencies are being installed; explore the project structure. | |
| 169 | ```bash | |
| 170 | todolist-api | |
| 171 | ├── Dockerfile | |
| 172 | ├── Gruntfile.js | |
| 173 | ├── README.md | |
| 174 | ├── node_modules | |
| 175 | ├── package-lock.json | |
| 176 | ├── package.json | |
| 177 | ├── server | |
| 178 | │ ├── api | |
| 179 | │ │ └── todo | |
| 180 | │ ├── app.js | |
| 181 | │ ├── components | |
| 182 | │ │ └── errors | |
| 183 | │ ├── config | |
| 184 | │ │ ├── environment | |
| 185 | │ │ ├── express.js | |
| 186 | │ │ ├── local.env.sample.js | |
| 187 | │ │ └── seed.js | |
| 188 | │ ├── mocks | |
| 189 | │ │ ├── mock-routes-config.json | |
| 190 | │ │ ├── mock-routes.js | |
| 191 | │ │ └── mock-routes.spec.js | |
| 192 | │ ├── routes.js | |
| 193 | │ └── views | |
| 194 | │ └── 404.html | |
| 195 | └── tasks | |
| 196 | └── perf-test.js | |
| 197 | ``` | |
| 198 | where the following are the important things: | |
| 199 | * `./server` is the main collection of files needed by the app. The entrypoint is the `app.js` | |
| 200 | * `./node_modules` is where the dependencies are stored | |
| 201 | * `./server/api` is where the api's controller, data model & unit test are stored. | |
| 202 | * `./server/mocks` is a mock server used for when there is no DB access | |
| 203 | * `./server/config` stores our Express JS config, header information and other middlewear. | |
| 204 | * `./server/config/environment` stores enviromnent specific config; such as connectivity to backend services like the MongoDB. | |
| 205 | * `./tasks` is a collection of additional `Grunt` tasks which will be used in later labs | |
| 206 | * `package.json` contains the dependency list and a lot of very helpful scripts for managing the app lifecycle | |
| 207 | ||
| 208 | 2. The npm scripts are shown below. There are application start scripts, build and test items which will be used in the build. The ones for MongoDB are just provided for convenience and require Docker installed to execute. | |
| 209 | ```json | |
| 210 | "scripts": { | |
| 211 | "start": "node server/app.js", | |
| 212 | "dev": "./node_modules/.bin/grunt serve", | |
| 213 | "jshint": "./node_modules/.bin/grunt jshint", | |
| 214 | "jshint:ci": "./node_modules/.bin/grunt jshint:ci_server", | |
| 215 | "clean": "rm -rf reports package-contents*", | |
| 216 | "build": "mkdir -p package-contents && cp -vr server Dockerfile package.json package-contents", | |
| 217 | "package": "zip -r package-contents.zip package-contents", | |
| 218 | "test": "node_modules/.bin/nyc node_modules/.bin/mocha server/**/*.spec.js --exit", | |
| 219 | "test:ci": "export MOCHA_FILE='reports/server/mocha/test-results.xml' && export NODE_ENV=ci && node_modules/.bin/nyc node_modules/.bin/mocha server/**/*.spec.js -R mocha-junit-reporter --exit", | |
| 220 | "mongo" : "docker run -i -d --name mongo-local -p 27017:27017 mongo", | |
| 221 | "mongo:drop" : "npm run mongo:stop && docker rm mongo-local", | |
| 222 | "mongo:stop" : "docker stop mongo-local", | |
| 223 | "mongo:start" : "docker start mongo-local" | |
| 224 | }, | |
| 225 | ``` | |
| 226 | ||
| 227 | 2. To run the application; start a new instance of the MongoDB by running. | |
| 228 | ```bash | |
| 229 | $ npm run mongo | |
| 230 | ``` | |
| 231 | <p class="tip"> | |
| 232 | `npm run mongo:drop` is used to completely remove the running container. `npm run mongo:stop` & `npm run mongo:start` will preserve data in the container | |
| 233 | </p> | |
| 234 | ||
| 235 | 2. Fire up the `todolist-api` by running. | |
| 236 | ```bash | |
| 237 | $ npm run start | |
| 238 | ``` | |
| 239 |  | |
| 240 | ||
| 241 | 2. Check things are responding correctly by running and checking the response. It contains some seeded data (stored in `server/config/seed.js`) | |
| 242 | ```bash | |
| 243 | $ curl localhost:9000/api/todos | |
| 244 | ``` | |
| 245 | ```json | |
| 246 | [{ | |
| 247 | "_id": "5ac8ff1fdfafb02138698948", | |
| 248 | "title": "Learn some stuff about MongoDB", | |
| 249 | "completed": false, | |
| 250 | "__v": 0 | |
| 251 | }, | |
| 252 | { | |
| 253 | "_id": "5ac8ff1fdfafb02138698949", | |
| 254 | "title": "Play with NodeJS", | |
| 255 | "completed": true, | |
| 256 | "__v": 0 | |
| 257 | }] | |
| 258 | ``` | |
| 259 | ||
| 260 | 2. Now let's check out `todolist-fe` app by reloading the browser. We should now see our dummy front end data is replaced by the backend seed data | |
| 261 |  | |
| 262 | ||
| 263 | ||
| b15f06 | 264 | ### Part 2 - Build > Bake > Deploy |
| D | 265 | > _In this exercise; we take what we have working locally and get it working in OpenShift_ |
| 5a16fd | 266 | |
| 0f4d08 | 267 | 3. Do other things |
| D | 268 | |
| 5a16fd | 269 | _____ |
| D | 270 | |
| 0f4d08 | 271 | ## Extension Tasks |
| 5a16fd | 272 | > _Ideas for go-getters. Advanced topic for doers to get on with if they finish early. These will usually not have a solution and are provided for additional scope._ |
| 0f4d08 | 273 | |
| f2b1b4 | 274 | - Add a GitHub Webhook to trigger your build on each commit |
| 3772d9 | 275 | - Create a _promote-to-uat_ phase after the <master> branch deploy |
| D | 276 | - Add MongoDB Stateful set for the UAT environment (or test) |
| 0f4d08 | 277 | - Do some other stuff |
| D | 278 | |
| 279 | ## Additional Reading | |
| 5a16fd | 280 | > List of links or other reading that might be of use / reference for the exercise |
| D | 281 | |
| f2b1b4 | 282 | - What's in a package.json? |
| D | 283 | |
| 5a16fd | 284 | ## Slide links |
| D | 285 | > link back to the deck for the supporting material |
