Configuring Coveralls.io Reports for Maven Builds with GitHub Actions
Coveralls.io is a very popular tool to capture and track code coverage of open source projects. Recently, I wanted to give it a try and decided to configure the generation of a report for one of my Java projects built via Maven.
While there is out of the box support for any code coverage tool that generates LCOV data via the Coveralls GitHub Action, I wanted to publish my report out of the code coverage metrics generated by JaCoCo, which uses its own format. I am used to JaCoCo and I did not want to change code coverage tool which was already integrated into my Maven builds.
Coveralls already support JaCoCo and we can integrated its services via the associated plugin —coveralls-maven-plugin — that works out of the box with the JaCoCo plugin. The setup is pretty simple, the only information required to the plugin is the coveralls repository token, which can be found in the detail page of the repository information in the coveralls.io website. As this information is sensitive, it is not a good idea to store this detail in the pom.xml file or any other file in the GitHub repository as it will provide anyone that has read access to the repository with the ability to push code coverage data to coveralls.io on your behalf.
The problem was then how to supply the repository token to plugin without any leaving any trace of it neither in the build logs or in the repository file system. It turned out that with the configuration of a secret and some minimal bash scripting this was pretty easy to do.
Step 1 — Configuring JaCoCo for Maven Builds
The first step entails providing code coverage information to configuring a plugin that generates such data. I have chosen to use JaCoCo, but Coveralls support also Cobertura, and Saga. The process is rather identical for all the three plugins as described here.
Below is the configuration that I have used for my project using JaCoCo:
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.8.6</version>
<executions>
<execution>
<id>prepare-agent</id>
<goals>
<goal>prepare-agent</goal>
</goals>
</execution>
</executions>
</plugin>The following command allows verifying that the plugin works as expected:
mvn test jacoco:reportAn important thing to note here is the version of the plugin. I am running Java 11 and if we use any 0.7.xx version we will be having the following fatal error:
FATAL ERROR in native method: processing of -javaagent failed, processJavaStart failed
....
Caused by: java.lang.RuntimeException: Class java/util/UUID could not be instrumented.This is due to the fact that the JaCoCo java agent is unable to instrument some of the classes in the bytecode, due to the fact that is not the first Java agent being run during the Maven build. This issue is better described here. Version 0.8.6 solves this problem.
Step 2 — Configuring Coveralls for Maven Builds
To configure Coveralls report generation and push in the build process we can add the Coveralls Maven Plugin (I am using version 4.3.0) that only requires the specification of the repository token that is being analysed for code coverage. All the other settings are worked out automatically, including the location of the JaCoCo code coverage report files.
<plugin>
<groupId>org.eluder.coveralls</groupId>
<artifactId>coveralls-maven-plugin</artifactId>
<version>4.3.0</version>
<configuration>
<repoToken>yourcoverallsprojectrepositorytoken</repoToken>
</configuration>
</plugin>As shown in the listing the plugin requires specifying the repository token. For now we just defer this settings and try to run the plugin to see whether there is any other problem besides pushing the report. The following command allows validating that the plugin works as expected:
mvn test jacoco:report coveralls:reportOne of the possible problems that may arise in this case is the following error:
Execution default-cli of goal org.eluder.coveralls:coveralls-maven-plugin:4.3.0:report failed: A required class was missing while executing org.eluder.coveralls:coveralls-maven-plugin:4.3.0:report: javax/xml/bind/DatatypeConverterThis error identifies a missing dependency for the plugin, which can be easily fixed by adding the artifact java.xml.bind:jaxb-api as a dependency for the plugin as shown below:
<plugin>
<groupId>org.eluder.coveralls</groupId>
<artifactId>coveralls-maven-plugin</artifactId>
<version>4.3.0</version>
<configuration>
<repoToken>yourcoverallsprojectrepositorytoken</repoToken>
</configuration>
<dependencies>
<dependency>
<groupId>javax.xml.bind</groupId>
<artifactId>jaxb-api</artifactId>
<version>2.3.1</version>
</dependency>
</dependencies>
</plugin>This should resolve the problem and leave the execution only with the HTTP 422 error due to the plugin not being able to match the value of the supplied token (i.e. yourcoverallsprojectrepositorytoken) to any of the tokens in your coveralls profile.
Step3 — Setting Up Coveralls Secret in GitHub
In order to enable access to the repository token from the build environment we need to expose a secret containing the value of the repository token mapping the repository we are analysing. The value of the token can be retrieved by repository page in your coveralls profile (i.e. https://coveralls.io/github/<owner-name>/<repository-name>).
Once we have copied the token, we can go to the Settings page of the repository in GitHub, access the Secrets section, and then click “New repository secret” as shown in the figure.
In line with the guidelines for naming secrets I have named it COVERALL_REPO_SECRET and pasted the content previously copied from the Coveralls.io repository page.
We can choose to configure it either as Environment Secret or Repository Secret. The former requires the existence of an environment first where to expose the secrete, the latter is global to the repository and accessible to any collaborator to the repository.
Step4 — Adding Actions to Push Coverage Report to Coveralls.io
What is left is now is tie all together and pass the information exposed in the COVERALL_REPO_SECRET secret to the Coveralls Maven Plugin before its execution.
We can easily use this with some simple bash scripting that performs the following steps:
- before running the plugin, injects the value of the secret into the configuration of the plugin
- after running the plugin, restores the configuration of the plugin to its original setup, containing yourcoverallsprojectrepositorytoken.
Assuming that we already have a workflow that builds Java projects with Maven we highlight here the additional steps required to configure the integration with coveralls.io.
name: Build and Test Pipeline
on: [push]
jobs:
Main:
runs-on: ubuntu-latest
steps:
- name: 🧺 Checkout Repository Code
uses: actions/checkout@v2
- name: 🛠 Set up JDK 11
uses: actions/setup-java@v2
with:
java-version: '11'
distribution: 'adopt'
cache: maven
- name: 📦 Build with Maven
run: mvn --batch-mode --update-snapshots test jacoco:report
- name: 🛠 Coveralls Coverage Report Setup
run: sed -i.bak 's/yourcoverallsprojectrepositorytoken/${{ secrets.COVERALL_REPO_TOKEN }}/g' pom.xml - name: 🚀 Coveralls Coverage Report Submission
run: mvn coveralls:report
- name: 🧹 Cleanup Repository Token
if: ${{ always() }}
run: if [ -f "pom.xml.bak" ]; then mv pom.xml.bak pom.xml; fi
The listing above shows the inclusion of three additional steps to the standard set of actions to execute a Maven build:
- Coveralls Coverage Report Setup: uses sed to backup the current content of the pom.xml and replace the placeholder for the repository token with the value of the secret.
- Coveralls Coverage Report Submission: invokes
mvn coveralls:report. This command assumes that a previous code coverage report has been generated and pushes the corresponding report to the Coveralls.io API. - Cleanup Repository Token: checks whether there is backed up pom.xml and copies back the file to its original state to prevent from leaving any trace of the token in the file system. It is worth noting the conditional expression
if: ${{ always() }}that ensures that the step is always run regardless of the success of the previous action. This is quite important as the submission of the report entails a network communication with coveralls.io which may fail.
That’s it! …nothing else is required, except for ensuring that we generate the code coverage report prior to executing the plugin. This is done in the step that precedes the setup of the plugin, which executes the Maven command: mvn test jacoco:report .
** Update ***
It turns out that the Coveralls Maven Plugin supports the specification of the repository token via the system property repoToken. In a Maven Project system properties can be either statically defined in the <properties></properties> section of the pom.xml or dynamically via the command argument --define (-D) . By utilising the second option, we can then:
- remove the placeholder associated to
<repoToken></repoToken>in the configuration section of the plugin; - include the
--define repoToken=${{ secrets.COVERALL_REPO_TOKEN }}switch to the command that submits the report to Coveralls.io; and - remove the backup of the
pom.xmlfile and the bash scripts that replaces the token.
This solution relies upon the same protection mechanism that is used by the previous approach by which secrets references are obscured in the console log of GitHub Actions.
The modified files to support this approach include the pom.xml where the definition of the plugin result as follows:
<plugin>
<groupId>org.eluder.coveralls</groupId>
<artifactId>coveralls-maven-plugin</artifactId
<version>4.3.0</version>
<dependencies>
<dependency>
<group>javax.xml.bind</group>
<artifact>jaxb-api</artifact>
<version>2.3.1</version>
</dependency>
</dependencies>
</plugin>The other change introduced is in the GitHub Action file, where all the scripting is removing in favour of the command line specification:
name: Build and Test Pipeline
on: [push]
jobs:
Main:
runs-on: ubuntu-latest
steps:
- name: 🧺 Checkout Repository Code
uses: actions/checkout@v2
- name: 🛠 Set up JDK 11
uses: actions/setup-java@v2
with:
java-version: '11'
distribution: 'adopt'
cache: maven
- name: 📦 Build with Maven
run: mvn --batch-mode --update-snapshots test jacoco:report
- name: 🚀 Coveralls Coverage Report Submission
run: mvn coveralls:report --define repoToken=${{ secrets.COVERALL_REPO_TOKEN }}This solution is much cleaner and shorter. Thank You Phred for pointing this out!
That’s All Folks!
While there isn’t yet support out of the box for Maven builds via Github Actions, the steps required to configure the build environment and the plugin are rather simple, but there are a few things to consider.
By adding the repository token as a secret in the build environment we make that information accessible to anyone that can modify GitHub actions to the repository. This may or may not be appropriate in all scenarios, but this applies to every piece of information that is exposed as a secret. Having the proper security in place in terms of access control is an appropriate measure to take not only in this case but always.
The process described here, works also for Saga or Cobertura since the problem being solved is how to feed the plugin with the repository token and not how configure the plugin with other tools, which are supported rather out of the box.
If you want to see a working example of this have a look at the GitHub Actions and the POM file in this repository: https://github.com/hyp0th3rmi4/hacker-rank-coding-practice/.
NOTE: The master branch contains the corrected version which feeds the Coveralls Maven Plugin via command line through the definition of a system property.
