Getting started with Azure DevOps autopilot

Getting started with Azure DevOps autopilot#

Introduction#

With the ado fetcher and evaluator, you can fetch tickets/work items from an ado project of your organization and subsequently check, whether their properties meet certain conditions. This Tutorial gives an introduction to the Ado fetcher and evaluator and demonstrates how you can configure them.

For this tutorial, we’re using the following example use case:

  • We want to evaluate Epic work items which aren’t finished/done yet.

  • For a priority of 1 and 2, they must neither be overdue, nor unassigned to return a GREEN status.

You can follow along this tutorial in two different ways:

  1. If you have access to an ado project, just use that one.

  2. In case you do not have access to any ado projects, you can upload an additional file and then still successfully run the example. What exactly you’re required to do for that will be explained later. The beginning will be relevant for everyone.

Download resources#

Please go ahead and download the following files. They will be required in the subsequent steps.

Adjusting the config files#

The Ado fetcher and evaluator config file#

Unlike other fetcher/evaluator combinations, the ado-fetcher and ado-evaluator share a common config file. Feel free to have a look at the downloaded ado-fetcher-evaluator-config.yaml file first, to get an overview of it. Subsequently, we we will go over the file’s parts, step by step.

The fetcher requires the query you can find below.

1workItems:
2  query: "SELECT [System.Id], [System.State] FROM WorkItems WHERE [System.TeamProject] = @project AND [System.WorkItemType] = 'Epic' AND ([System.State] = 'To Do' OR [System.State] = 'Doing') AND ([Microsoft.VSTS.Common.Priority] = 1 OR [Microsoft.VSTS.Common.Priority] = 2)"

The query in line two is written in WIQL and is being used to fetch the required work items from the ado project of your choice. (You will set the project that should be used later on in the environment variables). Here, we’re fetching all items which are declared as Epic for their category and are in state To Do or Doing, and have a priority of 1 or 2 assigned to them. All of the downloaded work items are subsequently taken into consideration for evaluation.

See the official WIQL documentation if you want to find out more about writing your own queries.

In the next step, you need to define which properties/fields you require from your tickets in azure for the evaluation process.

3  neededFields:
4    - "assignedTo" #No spaces allowed between words -> lowerCamelCase
5    - "targetDate"

In this case, we require the Assigned To, the State and the Target Date property for our check, so we list them in the required fields. Since State is part of the default list, it doesn’t need to be specified. Beware that you must use camelCase (with an initial lowercase!) to reference the property names. So for example, Target Date (as it’s spelled in the web interface of ado) becomes targetDate.

Next up, we’re going to specify some settings for the evaluator.

 6  evaluate:
 7    settings:
 8      dueDateFieldName: "targetDate"
 9      closedStates:
10        - "Closed"
11        - "Done"

In line 8, you need to state the name of the field that defines the Due Date of a ticket. Also, you need to enter the terms that are used to label work items which should be considered as being closed in the closedStates, depending on the exact terms your tickets use.

Now that we have specified the items that the checks are executed on and the evaluator is set up, we can take care of configuring what the evaluator should actually check for.

12    checks:
13      dataExists: true
14      fields:
15        dueDate: #To check, whether the DueDate is non existant/overdue
16          fieldName: "targetDate"
17          conditions:
18            resolved:
19              - ""

In line 15, you can set a name for the first check that you want to define. Below comes the name of the field which is being evaluated in this check. The property you enter here needs to be part of either the neededFields or the default list. The next keyword follows in line 18, where you can set the condition type. In this case we’re using resolved to check, whether the due date is in the past. Since we don’t want to check for any specific value, we can simply use an empty string as the reference value. If you need more information on that, please see: Evaluator Background Information.

We continue by defining the second condition we need, in order to match our use case’s requirements.

20        assignedTo: #To check, whether the ticket is unassigned or not
21          fieldName: "assignedTo"
22          conditions:
23            illegal:
24              - ""

The second check uses the same logic as before. Here, we can use the condition type illegal so we get a RED status in case the property isn’t defined.

Congratulations, we’re all set with the config file for the ado-fetcher and -evaluator now. Of course, you could also define further checks if you want to. Note that the different checks are always connected with a logical AND. If you want to learn more about realizing more complex conditions, check How to combine two checks with an ‘OR’. However, make sure to finish this tutorial first.

Now that we have configured the fetcher and evaluator, we need to adjust the qg-config.yaml so that they are being used to provide evidence for the desired question. Additionally, we need to provide the required environment variables to the fetcher and evaluator.

Note

If you don’t have access to an ado project of your own, skip the next step and jump to Running the example without Azure DevOps access.

The qg-config.yaml#

7  ADO_API_ORG: team-neutrinos
8  ADO_API_PROJECT: playground

First of all, you need to take care of the two lines 7, 8 above and exchange them with their respected values directly in the file. State the ado organization and the project name you want the items to be fetched from.

16      ADO_API_PERSONAL_ACCESS_TOKEN: ${{ secrets.ADO_API_PERSONAL_ACCESS_TOKEN }}

Next, go ahead and create a personal access token for ADO. To generate a new personal access token, click on the User settings icon in the top right corner of Azure Devops, select Personal access tokens, then select New Token. Give the token Read, write & manage access to work items of the project. You don’t need to grant any other permissions. Add the token as a secret to the Yaku secrets endpoint. If you need help with that, check out: Creating secrets.

38                ADO_CONFIG_FILE_PATH: ./ado-fetcher-evaluator-config.yaml
39                ADO_WORK_ITEMS_JSON_NAME: workItems.json

In case you want to change the filename of the fetcher/evaluator’s config file, you also need to adjust the statement in line 39. In line 40, you can define the name of the file that the fetcher creates, containing all the matching work items from the query.

Running the example#

Now, you can run the example. Upload the config file for the fetcher and evaluator, using the same name as given in line 39 as well as the qg-config.yaml.

Before executing a run, go ahead and create some applicable work items in your ado project to test the config with.

Running the example without Azure DevOps access#

Using the ado-fetcher requires access to ado, using the ado-evaluator does not. So what you can do instead of running the fetcher is uploading the workItems.json file you’ve downloaded in the beginning. It contains a couple of work items in the same format that the ado-fetcher would store them locally after fetching them. By following the subsequent steps, you can tell the ado-evaluator to use that file and it will work as if the ado-fetcher had been run ordinarily.

7   # ADO_API_ORG: team-neutrinos
8   # ADO_API_PROJECT: playground

Comment out line number 7 and 8.

9      autopilots:
10        ado-work-items-autopilot:
11          run: |
13            # ado-work-items-fetcher
14            ado-work-items-evaluator
15      # env:
16          # ADO_API_PERSONAL_ACCESS_TOKEN: ${{ secrets.ADO_API_PERSONAL_ACCESS_TOKEN }}

Comment out line 13, 15 and 16 in the autopilot section.

If you want to, you can have a look at the workItems.json file. It contains example work items, formatted as json objects. Their titles suggest, if and why they should fail or not. Of course, you can also have a look at the targetDate and assignedTo properties to see and understand their actual values.

Now, you can run the example. Upload the config file for the fetcher and evaluator, using the same name as given in line 39 as well as the qg-config.yaml. Additionally upload the workItems.json file.

In the end, your result should like like that:

Screenshot of the qg-full-report.html opened in the browser

Screenshot of the qg-full-report.html, opened in the browser#