Running tests and creating code coverage reports for React / NodeJS project continuously with Jenkins Pipelines, Jest & Cobertura or Jest-JUnit and push to Artifactory
The foundation for a healthy project is a high standard of automated tests. When your code is well tested, with tests that cover actual scenarios and user behaviors, modifying the code is not as risky task as it was otherwise. Of course it doesn’t mean your project is bulletproof, but the potential damage to your users decreases.
When you’re working in a team, running the automated tests continuously has great benefits (“OMG don’t update your code from the source control, you won’t be able to build the project until a fix is checked-in!” for example). Nonetheless, continuously keeping track of the code coverage your tests provide also has benefits (developers keep checking-in code that future modifications in it would be risky, for example).
This guide will walk you through continuously running tests and creating code coverage reports in Jenkins.
It is written with React in mind (+TypeScript), but it can work with any Jest-supported JavaScript framework (one can even use Istanbul directly without Jest).
It does assume that you previously setup your project to run tests (e.g. following React + Jest + Enzyme + TypeScript guide).
I originally wanted to have the code coverage report parsed by JaCoCo so the Jenkins job list can show the total number of lines covered. I did not find a way to do this, and this is why this guide is using Cobertura. As an alternative, if this number is not important to you and you don’t want to install the Cobertura Jenkins Plugin, you can use the Jest-JUnit alternative, also described in this guide.
Project structure and tools in use
- ReactJS, but can be any JavaScript framework
- Jest (tested with v22) — testing solution for React made by Facebook that has IstanbulJS built-in, to produce a code coverage report
- Webpack — bundler, but can be any bundler
- NodeJS (tested with v10.4.1)
- NPM (tested with v6.1.0, notice some older versions such as 3–5 have some quirks with running Jest tests sometimes)
- Jenkins (with Pipelines) — continuous integration tool, running processes based on code (Jenkinsfile)
- Artifactory + Artifactory Jenkins Plugin — artifact management
Reporters
We have three ways to produce a code coverage report:
- Cobertura Jenkins Plugin — a utility for code coverage that will collect the report from Jest
- Jest-JUnit — Exports the Jest code coverage report to XML formatted as a Java JUnit report
- Plain HTML — exports the code coverage report as an IstanbulJS-formatted HTML
I’ll exemplify the use of all, separately.
Option 1: Code coverage with Cobertura
Cobertura Jenkins Plugin has a nice feature, that it can show the total percentage of code-lines covered in Jenkins views. “Bottom lines” are always more convenient for glimpses, but this is also comfortable when you have a physical monitor displaying Jenkins’ lines coverage to the developers and/or the managers.
Step 1: Setup report creation
Edit your package.json and add the following lines:
{
"name": "continuous-test-code-coverage-guide",
"scripts": {
"start": "webpack",
"test": "jest --coverage --coverageDirectory=output/coverage/jest"
},
...
"jest": {
"coverageReporters": [
"text",
"cobertura"
]
...
}
}Now, running npm run test for this package.json will produce two code coverage reports in output/coverage/jest — one is in text format, and the other in “cobertura” format.
We use the text format because it prints a nice report to the console, which is comfortable to developers running the test script on their dev-machine.
We use the “cobertura” format so the result can be parsed by the Jenkins Cobertura Plugin.
Step 2: Create a Jenkins pipeline file
Create a Jenkinsfile file, and put the following content (with the relevant modifications):
#!/bin/groovy
pipeline {
tools {
nodejs 'default-nodejs'
}
stages {
stage('Startup') {
steps {
script {
sh 'npm install'
}
}
}
stage('Test') {
steps {
script {
sh 'npm run test'
}
}
post {
always {
step([$class: 'CoberturaPublisher', coberturaReportFile: 'output/coverage/jest/cobertura-coverage.xml'])
}
}
}
stage('Build') {
steps {
script {
sh 'npm start'
sh 'npm pack'
}
}
}
stage('Deploy') {
when {
expression {
currentBuild.result == null || currentBuild.result == 'SUCCESS'
}
}
steps {
script {
def server = Artifactory.server 'My_Artifactory'
uploadArtifact(server)
}
}
}
}
}def uploadArtifact(server) {
def uploadSpec = """{
"files": [
{
"pattern": "continuous-test-code-coverage-guide*.tgz",
"target": "npm-stable/"
}
]
}"""
server.upload(uploadSpec)
def buildInfo = Artifactory.newBuildInfo()
server.upload spec: uploadSpec, buildInfo: buildInfo
server.publishBuildInfo buildInfo
}
The line calling CoberturaPublisher will publish the report that npm run test created (see previous step).
npm start created the dist/ folder with the bundled code (see previous step), and npm pack created a tgz file that wraps it.
uploadArtifact will upload the tgz file (on successful builds) to your JFrog Artifactory. I recommend using the Artifactory Plugin to setup the Artifactory parameters in Jenkins > Manage Jenkins > Configure System> Artifactory, but nonetheless, the Artifactory help guide for setting up the server’s parameters within the Jenkinfile is simple enough.
Step 3: Create the Jenkins Pipeline job
Go to Jenkins, and create a new job. Choose a Pipeline job. Choose to run the pipeline from the Jenkinsfile we created in step 2.
If you need options here, try installing some Jenkins plugins (e.g. BlueOcean).
The end results for running this job:
- Use Jenkins > Edit View > Columns > Add column > Cobertura coverage, to view the total numbers in a glimps on your Jenkins jobs list.
- Inside this pipeline job we created, you should have a “Test Result Trend” column in the right-hand side, with data in it.
- Inside this pipeline job we created, you should have a “Coverage Report” and “Test Result Analyzer” options on the left-hand menu, with data in them.
- Same as 2+3, specific for each build in this pipeline job we created.
- Inside each build you should have an Artifactory Build Info option on the left-hand menu.
Notice that after Option 2 part of the guide, there’s a section for tips.
Option 2: Code coverage with Jest-JUnit
The Jest-JUnit integration with Jenkins lacks some functionality, but it doesn’t require installing the Cobertura Jenkins Plugin (for example if its license is not approved by your company).
Setting up code coverage report creation
Install Jest-JUnit as a devDependency of your project:npm install — dev jest-junit
Then, edit your package.json and add the following lines:
{
"name": "continuous-test-code-coverage-guide",
"scripts": {
"start": "webpack",
"test": "jest --coverage --coverageDirectory=output/coverage/jest"
},
...
"jest": {
"coverageReporters": [
"text"
],
"reporters": [
"default",
"jest-junit"
],
...
},
"jest-junit": {
"output": "output/coverage/junit/junit.xml",
"usePathForSuiteName": "true"
}
}Now, running npm run test for this package.json will produce a code coverage report in output/coverage/junit/junit.xml (thanks to reporters: jest-junit) and on the console (thanks to coverageReportes: text + reporters: default).
Step 2: Create a Jenkins pipeline file
Create a Jenkinsfile file, and put the following content (with the relevant modifications):
#!/bin/groovy
pipeline {
tools {
nodejs 'default-nodejs'
}
stages {
stage('Startup') {
steps {
script {
sh 'npm install'
}
}
}
stage('Test') {
steps {
script {
sh 'npm run test'
}
}
post {
always {
junit 'output/coverage/junit/junit.xml'
}
}
}
stage('Build') {
steps {
script {
sh 'npm start'
sh 'npm pack'
}
}
}
stage('Deploy') {
when {
expression {
currentBuild.result == null || currentBuild.result == 'SUCCESS'
}
}
steps {
script {
def server = Artifactory.server 'My_Artifactory'
uploadArtifact(server)
}
}
}
}
}def uploadArtifact(server) {
def uploadSpec = """{
"files": [
{
"pattern": "continuous-test-code-coverage-guide*.tgz",
"target": "npm-stable/"
}
]
}"""
server.upload(uploadSpec) def buildInfo = Artifactory.newBuildInfo()
server.upload spec: uploadSpec, buildInfo: buildInfo
server.publishBuildInfo buildInfo
}
The line calling junit will publish the report that npm run test created (see previous step).
npm start created the dist/ folder with the bundled code (see previous step), and npm pack created a tgz file that wraps it.
uploadArtifact will upload the tgz file to your JFrog Artifactory. I recommend using the Artifactory Plugin to setup the Artifactory parameters in Jenkins > Manage Jenkins > Configure System> Artifactory, but nonetheless, the Artifactory help guide for setting up the server’s parameters within the Jenkinfile is simple enough.
Step 3: Create the Jenkins Pipeline job
Go to Jenkins, and create a new job. Choose a Pipeline job. Choose to run the pipeline from the Jenkinsfile we created in step 2.
If you need options here, try installing some Jenkins plugins (e.g. BlueOcean).
The end results for running this job:
- Inside the pipeline job we created, there a “Test Result” with data on the left-hand menu.
- Inside each build you should have an Artifactory Build Info option on the left-hand menu.
Option 3: Plain HTML
If you don’t want to install any additional plugin anywhere, you can simply post the Jest (Istanbul) HTML report into Jenkins. By this you are relying on the Istanbul report format, which might be comfortable to most (but is different than Java reports, in case your team is used to them).
Setting up code coverage report creation
Edit your package.json and add the following lines:
{
"name": "continuous-test-code-coverage-guide",
"scripts": {
"start": "webpack",
"test": "jest --coverage --coverageDirectory=output/coverage/jest"
},
...
"jest": {
"coverageReporters": [
"text",
"html"
]
...
}
}Now, running npm run test for this package.json will produce a code coverage report in output/coverage/jest/index.html (thanks to coverageReporters: html) and on the console (thanks to coverageReportes: text).
Hint: you can change “html” to “lcov” as well, if you like its format better. It is also an HTML report.
Step 2: Create a Jenkins pipeline file
Create a Jenkinsfile file, and put the following content (with the relevant modifications):
#!/bin/groovy
pipeline {
tools {
nodejs 'default-nodejs'
}
stages {
stage('Startup') {
steps {
script {
sh 'npm install'
}
}
}
stage('Test') {
steps {
script {
sh 'npm run test'
}
}
post {
always {
publishHTML target: [
allowMissing : false,
alwaysLinkToLastBuild: false,
keepAll : true,
reportDir : 'output/coverage/jest',
reportFiles : 'index.html',
reportName : 'Test Report'
]
}
}
}
stage('Build') {
steps {
script {
sh 'npm start'
sh 'npm pack'
}
}
}
stage('Deploy') {
when {
expression {
currentBuild.result == null || currentBuild.result == 'SUCCESS'
}
}
steps {
script {
def server = Artifactory.server 'My_Artifactory'
uploadArtifact(server)
}
}
}
}
}def uploadArtifact(server) {
def uploadSpec = """{
"files": [
{
"pattern": "continuous-test-code-coverage-guide*.tgz",
"target": "npm-stable/"
}
]
}"""
server.upload(uploadSpec)def buildInfo = Artifactory.newBuildInfo()
server.upload spec: uploadSpec, buildInfo: buildInfo
server.publishBuildInfo buildInfo
}
The line calling publishHTML will publish the report that npm run test created (see previous step).
Hint: if you used lcov in the previous step, here replace the value of reportDir with output/coverage/jest/lcov-report.
npm start created the dist/ folder with the bundled code (see previous step), and npm pack created a tgz file that wraps it.
uploadArtifact will upload the tgz file to your JFrog Artifactory. I recommend using the Artifactory Plugin to setup the Artifactory parameters in Jenkins > Manage Jenkins > Configure System> Artifactory, but nonetheless, the Artifactory help guide for setting up the server’s parameters within the Jenkinfile is simple enough.
Step 3: Create the Jenkins Pipeline job
Go to Jenkins, and create a new job. Choose a Pipeline job. Choose to run the pipeline from the Jenkinsfile we created in step 2.
If you need options here, try installing some Jenkins plugins (e.g. BlueOcean).
The end results for running this job:
- Inside the pipeline job we created, there a “Test Result” with data on the left-hand menu.
- Inside each build you should have an Artifactory Build Info option on the left-hand menu.
Tips
Fail low coverage results
You can set Jest to fail for low coverage results (for example, if developers kept checking-in code without testing it).
Read the documentation for this.
