The meta-shift plugin for Jenkins

About

A plugin for Jenkins to capture and visualize various software quality metrics for Yocto based projects using the reports generated by the meta-shift layer.

The list of software quality metric:

• Unit test
• Statement coverage
• Branch coverage
• Mutation test

Note (0.6.0): The static-analysis, cache, code-size, and tested-recipes metrics were removed because the current meta-shift layer no longer produces their reports. This plugin now targets the current meta-shift (scarthgap) layer. Build results and job threshold settings from older versions need to be re-run / re-entered.

Note (0.7.0): The per-metric detail pages were merged into the build and recipe pages (the distribution now renders inside each metric card), and the annotated source page was replaced by a per-file annotation summary (uncovered line ranges, partially covered branches, and mutation details). Source files are no longer stored with the build. Reports from builds published by older versions need to be re-run to be viewable.

Note (0.8.0): The annotated source page is back: the current meta-shift layer writes a metadata.json (the BitBake S variable) into each task report directory, so the plugin now stores the reported source files with the build at publish time (capped at 1MB per file) and paints coverage, branch and mutation annotations directly on the source. The per-metric file tables of the recipe page were merged into a single per-file table, unit tests link to their source lines, the build overview highlights the worst failing recipes, and the CSV/Excel export buttons were removed. Builds published by older versions fall back to the annotation summary on the file page.

You may find more information about the meta-shift layer on https://github.com/shift-left-test/meta-shift

Getting started

Configuration

The meta-shift plugin is configured at the job level by adding a meta-shift report post build action. The configuration parameters include:

• Report Root Path: Specify the path to meta-shift report files in the Ant glob syntax, such as **/build/test-reports/*.xml.
• Unstable: If this option is checked, then the plugin marks the build status as unstable if the value of the metric does not satisfy the threshold.
• Threshold: The threshold represent the minimum value which the metric is required to be qualified

You may also configure the parameters on the Jenkins system configuration panel, which are globally available in the whole pipeline by default.

Archiving the raw report files

The plugin stores only the parsed data with the build; the HTML report pages generated by the meta-shift layer are not archived. To keep them, use the built-in archiveArtifacts step:

archiveArtifacts artifacts: 'build/test-reports/**', allowEmptyArchive: true

To browse them from a sidebar link, use the HTML Publisher plugin instead:

publishHTML(target: [
  reportDir: 'build/test-reports',
  reportFiles: '*/coverage/index.html, */test/index.html, */checktest/index.html',
  reportName: 'meta-shift raw reports',
  keepAll: true,
  includes: '**/*.html',
])

Note: Jenkins serves build-generated HTML with a restrictive default Content-Security-Policy, so the reports render unstyled and the mutation report appears blank. Configure a Resource Root URL to view them as intended.

Development

Requirements

• Maven (3 or above)
• Java development kit (9 or above)

How to build the plugin

$ mvn package

How to test

$ mvn verify

How to run locally (invokes a local Jenkins instance with the plugin installed)

$ mvn hpi:run -Dhost=0.0.0.0 -Djetty.port=12345

License

This project source code is available under MIT license. See LICENSE.