Evaluator Background Information

Evaluator Background Information#

The evaluator checks the quality gate status fetched by the fetcher and returns RED if any of the gate checks doesn’t have an OK status.

For example, the result of evaluating the fetched quality gates status below will be RED, because new_coverage and bugs metrics don’t have OK status.

{
  "projectStatus": {
    "status": "ERROR",
    "conditions": [
      {
        "status": "OK",
        "metricKey": "new_reliability_rating",
        "comparator": "GT",
        "periodIndex": 1,
        "errorThreshold": "1",
        "actualValue": "1"
      },
      {
        "status": "OK",
        "metricKey": "new_security_rating",
        "comparator": "GT",
        "periodIndex": 1,
        "errorThreshold": "1",
        "actualValue": "1"
      },
      {
        "status": "ERROR",
        "metricKey": "new_coverage",
        "comparator": "LT",
        "periodIndex": 1,
        "errorThreshold": "80",
        "actualValue": "0.0"
      },
      {
        "status": "ERROR",
        "metricKey": "bugs",
        "comparator": "GT",
        "errorThreshold": "50",
        "actualValue": "153"
      }
    ],
    "periods": [
      {
        "index": 1,
        "mode": "PREVIOUS_VERSION",
        "date": "2022-04-14T06:29:06+0000"
      }
    ],
    "ignoredConditions": false,
    "period": { "mode": "PREVIOUS_VERSION", "date": "2022-04-14T06:29:06+0000" }
  }
}

Environment variables#

SONARQUBE_ONLY_FAILED_METRICS#

If set to true, only failing metrics will be listed in the run report. Default value: false

Example config#

Below is an example configuration file that runs SonarQube evaluator. The autopilot is configured in lines: 7-17. Required environment variables are read from provided run environment variables or secrets. Then the autopilot is used by the check 1.1 in line 32 which is part of requirement 5.1.

 1metadata:
 2  version: v1
 3header:
 4  name: MACMA
 5  version: 1.16.0
 6autopilots:
 7  sonarqube-autopilot:
 8    run: |
 9      sonarqube-fetcher
10      sonarqube-evaluator
11    env:
12      SONARQUBE_ENDPOINT: ${{ vars.SONARQUBE_ENDPOINT }}
13      SONARQUBE_HOSTNAME: ${{ vars.SONARQUBE_HOSTNAME }}
14      SONARQUBE_PORT: ${{ vars.SONARQUBE_PORT }}
15      SONARQUBE_PROJECT_KEY: ${{ vars.SONARQUBE_PROJECT_KEY }}
16      SONARQUBE_PROJECT_TOKEN: ${{ secrets.SONARQUBE_PROJECT_TOKEN }}
17      SONARQUBE_PROTOCOL: https
18finalize:
19  run: |
20    html-finalizer
21chapters:
22  "1":
23    title: Project management
24    requirements:
25      "5.1":
26        title: The test strategy was defined/adhered to.
27        text: The test strategy was defined/adhered to.
28        checks:
29          "1.1":
30            title: Sonarqube QG Report
31            automation:
32              autopilot: sonarqube-autopilot