| | |
|---|
| | | |
|---|
| | | There are many tools out there for supporting these testing types but often they are left to the end of a delivery. Many traditional projects will leave Performance testing or security sign off to a few weeks before Go Live. This raises the question of what do we do if things do not pass these tests? Do we hold off the Go Live or accept the risk? In most cases we can learn earlier if things will be show stoppers and more importantly we can automate them. |
|---|
| | | |
|---|
| | | For example; imagine a developer called `Sam` has checked in some poor performing function into an API that spikes its response time. Let's call this `Sam Code`. This `Sam Code` may not be caught by our Unit Tests but could have very big impact on application usability. Before building code on top this and it becoming more of an issue at the end of a project; we could code and capture metrics around API response time and track these over time and hopefully spot regressions earlier. |
|---|
| | | For example; imagine a developer called `Sam` has checked in some poorly performing function into an API that spikes its response time. Let's call this `Sam Code`. This `Sam Code` may not be caught by our Unit Tests but could have very big impact on application usability. Before building code on top this and it becoming more of an issue at the end of a project; we could code and capture metrics around API response time and track these over time and hopefully spot regressions earlier. |
|---|
| | | |
|---|
| | | Another one of the age old questions is "How do we know we're testing enough?". Well the simple answer is you can never do enough testing! But how do we know we are testing the right things? Code coverage metrics can be run on our application while running the tests. They can identify what files are being tested and a line by line of how many times a test executes against a block of code. Reporting these metrics in our pipeline gives a greater handle on the quality of our testing. |
|---|
| | | |
|---|
| | |
|---|
| | | 1. [stryker](http://stryker-mutator.io/) - Mutation testing! What is it? Bugs, or mutants, are automatically inserted into your production code. Your tests are run for each mutant. If your tests fail then the mutant is killed. If your tests passed, the mutant survived. The higher the percentage of mutants killed, the more effective your tests are. It's really that simple. |
|---|
| | | |
|---|
| | | ## Big Picture |
|---|
| | | This exercise begins cluster containing blah blah |
|---|
| | | > From the previous exercise; we introduced pipeline-as-code and new Jenkins Slave nodes. This exercise focuses on extending the pipeline with Non-functional testin and some automated security testing. |
|---|
| | | |
|---|
| | |  |
|---|
| | | |
|---|
| | | _____ |
|---|
| | | |
|---|
| | |
|---|
| | | |
|---|
| | | 2. Open the `todolist-fe` application's `Jenkinsfile` in your favourite editor. The file is stored in the root of the project. |
|---|
| | | |
|---|
| | | 2. The file is layed out with a collection of stages that correspond to each part of our build as seen below. We will create a new stage to execute in parallel. |
|---|
| | | 2. The file is laid out with a collection of stages that correspond to each part of our build as seen below. We will create a new stage to execute in parallel. |
|---|
| | |  |
|---|
| | | |
|---|
| | | 2. Create a new Parallel Stage called `security scanning` underneath the `stage("e2e test") { }` section as shown below. The contents of the `e2e test` have been removed for simplicity. |
|---|
| | |
|---|
| | | } |
|---|
| | | ``` |
|---|
| | | |
|---|
| | | 2. Finally add the reporting for Jenkins in `post` hook of our Declarative Pipeline. This is to report the findings of the scan in Jenkins as a HTML report. |
|---|
| | | 2. Finally add the reporting for Jenkins in `post` hook of our Declarative Pipeline. This is to report the findings of the scan in Jenkins as an HTML report. |
|---|
| | | ```groovy |
|---|
| | | stage('OWASP Scan') { |
|---|
| | | agent { |
|---|
| | |
|---|
| | | |
|---|
| | | 4. Click `add build step` and select `execute shell` and add the following to it, replacing `<YOUR_NAME>` as expected. We will just test the `create` and `show` API for the moment. We are grabbing the response code of the perf-test to keep Jenkins running both shells steps and then exiting with whichever fails: |
|---|
| | | ```bash |
|---|
| | | export E2E_TEST_ROUTE=todolist-api-<YOUR_NAME>-dev.apps.lader.rht-labs.com |
|---|
| | | export E2E_TEST_ROUTE=todolist-api-<YOUR_NAME>-dev.<APPS_URL> |
|---|
| | | npm install |
|---|
| | | set +e |
|---|
| | | npm run perf-test:create |
|---|
