jenkinsrobotframeworkjacocofunctional-testing

Code coverage report for testcases written in robot framework for Java application

I have a scenario where I am trying to get the coverage report of the functional testcases we have written using robot framework. Application backend is written in java and UI code is written Vue.js Application is deployed in K8s cluster running as a container. And application ULS is exposed using Ingress controller. Robot test cases we trigger using jenkins from another server. I want to get the code coverage of the functional testcases we run. Any suggestions on what tool we can use and how to get it would be really appreciated.

FYI - I am already using JaCoCo for unit test case coverage.

Thanks, Utkarsh

I followed below URL. Jacoco: Find code coverage for external tests I added this to my build.gradle

jacocoTestReport { 
reports { 
xml.enabled false 
csv.enabled false 
html.destination file("$buildDir/reports/coverage") 
}

I updated my docker file as below, before I build the docker image for my app java -jar myApp.jar -javaagent:/some/path/jacocoagent.jar

And then I started the execution of functional testcase from Jenkins server.

After that I am not able to understand how to generate the coverage report. May be I need to execute below command inside pod to get the coverage report. But I am not sure if it's a good idea to pack gradle binaries in the container as it will increase the size of docker image. How do I automate this process from jenkins to show coverage report on job page. ./gradlew jacocoTestReport

Any suggestion will be helpful.

Solution

  • I am able to get the coverage for my regression test suite. I did this manually just to proof the concept first. My application is running as a pod in Kubernetes cluster and application URL (https://example.com) is exposed via ingress controller. My testcases are triggered from a different server using Jenkins and we use robot framework. When the test case are triggered it access URL example.com and start executing the test cases.

    Steps I followed:

    1. Added jacocoagent.jar and jacococli.jar in the pod or container.

    2. Set below JVM parameter while deploying the application on K8s through YML file. You can set this via dockerfile as well. javaagent:/tmp/jacocoagent.jar=output=tcpserver,includes=com/x/y/z/*,address='asterisk symbol',port=6300,append=false

    3. I triggered my regression suite.

    4. Went inside the pod and dump the exec file using jacococli.jar

      java -jar jacococli.jar dump --address localhost --port 6300 --destfile jacoco-server.exec

    5. Copied above exec file 'jacoco-server.exec' to my local.

    6. Converted the exec file to a html report.

      java -jar jacococli.jar report jacoco-server.exec --classfiles classes/ --html html**

    NOTE: Class file path is must here.

    This is what I did to prove the concept. To automate this I will be using output mode as 'file' instead of 'tcpserver' and will set a 'destpath' where the report will be created. With mode as 'file' jacoco.exec file will written to the destpath on VM/pod termination. I still have to test output mode as file option with K8s.