Fetcher Background Information

Fetcher Background Information#

A fetcher that fetches git resources data from a git server and stores it in the evidence path. The evidence path is set during the execution of a run and read as an environment variable by any evaluator used to evaluate the data fetched by git fetcher.

At the moment, the fetcher supports fetching pull-requests from GitHub and Bitbucket servers. Additionally, fetching branch and tag meta data from Bitbucket servers is also supported.

Note

You can also use the GitHub CLI to fetch data from GitHub servers. For more details, see Getting Started with GitHub CLI.

Prerequisites#

The user id that is set for the fetcher in GIT_FETCHER_USERNAME or the username whose API token is used in GIT_FETCHER_API_TOKEN must have access to the git project in order for the fetcher to work.

Environment variables#

GIT_FETCHER_SERVER_TYPE#

Type of the git server. Supported types are: [github, bitbucket]

GIT_FETCHER_SERVER_API_URL#

Git server api url.

Examples:

GIT_FETCHER_SERVER_AUTH_METHOD#

Authentication method to use.

Supported methods:

  • basic: username & password

  • token: bearer token

GIT_FETCHER_API_TOKEN#

Bearer token if token authentication method is defined.

GIT_FETCHER_USERNAME#

Username if basic authentication method is defined.

GIT_FETCHER_PASSWORD#

Password if basic authentication method is defined.

GIT_FETCHER_OUTPUT_FILE_PATH#

Filename to which the fetched data will be stored in the evidence path. If not specified, default value is git-fetcher-data.json.

GIT_FETCHER_CONFIG_FILE_PATH#

The path to the fetcher’s config file. If not specified, default value is git-fetcher-config.yml. More details about the config file can be found right below.

The fetcher’s config file#

The definition of which resources to query from the git server is given by a yaml file. The location of this file is then referenced by the GIT_FETCHER_CONFIG_FILE_PATH environment variable.

This config file should have the following structure:

org: github-org    # Organization name
repo: github-repo  # Repo name
resource: pull-requests  # Resource to fetch. One of: pull-requests, branches, tags (branches and tags are only supported for Bitbucket)
labels: # Optional. Labels of the resources to fetch. Only works with GitHub
  - release
filter:
  # date/hash/tag filters can't be combined with one another.
  # However, it is possible to combine exactly one of these filter types with other, non-mutually-exclusive
  # filters like 'state' filter.
  state: ALL # Optional. One of: DECLINED, MERGED, OPEN, ALL. Only works with Bitbucket.
  startDate: 01-06-2020 # Optional. Format dd-MM-yyyy. Only works with Bitbucket.
  endDate: 31-12-2022 # Optional. Format dd-MM-yyyy. Works only if startDate was provided. Only works with Bitbucket.
  startHash: c1b611a # Optional. Not working together with date filter. Only works with Bitbucket.
  endHash: d916ec2a # Optional. Works only if startHash was provided. Only works with Bitbucket.
  startTag: release/0.1.0 # Optional. Only works with Bitbucket.
  endTag: release/0.2.0 # Optional. Only works if startTag was provided. Only works with Bitbucket.

Example config#

Below is an example configuration file that runs git fetcher and json evaluator. The autopilot (git fetcher + json evaluator) is configured in lines: 7-19. Required variables and secrets are read from provided run variables or secrets. It will run the git fetcher to get data from GitHub server and then use json-evaluator to evaluate the fetched data. Then the autopilot is used by the check 1 in line 34 which is part of requirement 1.15

 1metadata:
 2  version: v1
 3header:
 4  name: MACMA
 5  version: 1.16.0
 6autopilots:
 7  git-autopilot:
 8    run: |
 9      git-fetcher
10      json-evaluator
11    env:
12      GIT_FETCHER_API_TOKEN: ${{ secrets.GIT_FETCHER_API_TOKEN }}
13      GIT_FETCHER_CONFIG_FILE_PATH: git-fetcher-config.yml
14      GIT_FETCHER_OUTPUT_FILE_PATH: git-fetcher-data.json
15      GIT_FETCHER_SERVER_API_URL: ${{ vars.GIT_FETCHER_SERVER_API_URL }}
16      GIT_FETCHER_SERVER_AUTH_METHOD: token
17      GIT_FETCHER_SERVER_TYPE: github
18      JSON_CONFIG_FILE: json-evaluator-config.yaml
19      JSON_INPUT_FILE: git-fetcher-data.json
20finalize:
21  run: |
22    html-finalizer
23chapters:
24  "1":
25    title: Project management
26    requirements:
27      "1.15":
28        title: Make sure all release required changes are done
29        text: Make sure all release required changes are done
30        checks:
31          "1":
32            title: Check the status of release version Pull requests
33            automation:
34              autopilot: git-autopilot