Gain better visibility into your development process with our Azure DevOps reporting. Explore our report templates or use our customizable dashboards to monitor and analyze key metrics.

Azure Devops custom reports and metrics dashboards, try one of our templates or create your own in minutes—no SQL required.

This report was inspired by Uber's engineering metrics dashboard, described in detail in [this post from The Pragmatic Engineer](https://newsletter.pragmaticengineer.com/p/uber-eng-productivity).

The five metrics covered in this dashboard include:
- Number of pull requests merged per developer
- Time from PR open to review to merge (Uber's dashboard only shows from open to merge, but we find it helpful to have a more granular breakdown.)
- Code review thoroughness (Uber's dashboard only shows review count per engineer, but we believe the more important number is ratio of pull requests have a review and where the review is thorough.)
- Active coding time per engineer compared to all work time (described as focus time per engineer in Uber's dashboard)
- Time from PR merge to launch (included next to the PR open to review to merge metrics)

Inspired by Uber's Dashboard Report Screen Shot

Inspired by Uber's Dashboard

See how your team performs on Uber's engineering dashboard metrics.

KPIs

Workflow

Overhead


The code/ticket hygiene and traceability report shows you how consistently software engineers write code in version control systems like GitHub, GitLab, Azure DevOps, or Bitbucket that is traceable to tickets or issues and rolled up to epics in systems like Jira or GitHub Issues.

The report first looks at branch usage rate (BUR), which shows how many commits are untraceable because they are made directly to a main/master branch or to a shared integration branch such as develop, staging or qa.

Next, the report looks at pull request usage rate (PRUR), which shows code that is untraceable because it is merged directly (e.g., using the git merge command) to main without using a pull request (called a merge request in GitLab).

The third metric is code/ticket linking rate (CTLR), which measures the amount of time spent on code that does not link to a known ticket. It searches branch names, commit messages, pull request titles, and pull request bodies for ticket links.

Finally, the report displays epic usage rate (EUR), which looks at how much time is spent working on tickets that are nested under a parent epic ticket. Use of epics is important because many project-related reports like project completion or cost capitalization focus on work at an epic level.

The report also shows history charts of code/ticket linking rate and epic usage rate.

Because real life is never 100% perfect, the scores are weighted by how much time developers spend on untraceable code. This helps you focus on improving in areas that will have the biggest impact, with the goal of driving down the percentage of untraceable work.

This report also lets you drill down into teams, people, and specific pull requests to identify the biggest sources of untraceable work.


## Why Is Code/Ticket Hygiene and Traceability Important?

Code/ticket hygiene and traceability is important because it lets you see where people – your most valuable resource – are spending their time.


### Cost Capitalization

Traceability is crucial for accurately allocating developer salary costs to tasks, projects, and initiatives in your ticketing system.

Cost allocation is a requirement if you’re doing R&D or cost capitalization for accounting (which is now mandatory for U.S. companies due to [Section 174](https://www.kbkg.com/tax-insight/kbkg-tax-insight-irs-releases-guidance-on-section-174-re-expenses)). But, even if you’re not capitalizing development costs on your balance sheet, CEOs, CFOs, and boards often want to see where their engineering salary costs are going for management purposes.

Without automated code/ticket traceability, organizations often resort to manual time tracking. Not only does manual time tracking actually waste developers’ time; it is less accurate because people tend not to log separate entries for small tasks and interruptions, which can have a big impact in aggregate.


### Change Control Compliance

Code/ticket traceability is also a critical component of a secure software development process.

If developers write code that makes it into production without having an associated pull request and ticket, then there is no record of whether it was properly reviewed and prioritized according to the organization’s change control policies.

Failure to follow change control guidelines can result in insecure or improperly vetted code reaching customers, which can compromise the organization’s security.

Furthermore, if you undergo a security audit for standards like SOC 2, untraceable code puts you at risk of having an audit exception if discovered by the auditor. The code/ticket traceability report can also be a great piece of evidence to provide to auditors to demonstrate consistent compliance with change control procedures.


## How Does the Code/Ticket Hygiene and Traceability Report Work?

The code/ticket hygiene and traceability works by identifying all of the commits made by each author in your organization. It then associates these commits with merges and pull requests. Finally, it looks at commit messages, branch names, pull request titles, and pull request bodies to find ticket IDs that link back to your ticketing system.

The report then calculates how much time is spent on each commit to provide an overall score showing the percentage of traceable coding time, which is aggregated by team, person, and pull request.


## FAQ


### How does the code/ticket hygiene and traceability report match ticket identifiers?

The code/ticket hygiene and traceability report matches ticket identifiers using a regular expression pattern that looks for valid ticket identifiers. It is case-insensitive and tolerates certain typos.

Additionally, the report verifies that any matches correspond to actual tickets, so, e.g., DEV-99999999 would not count as traceable unless a ticket with that ID actually exists.


### Where does the report look for ticket identifiers?

The report looks for ticket identifier matches in the branch name (also called a “ref”), pull request title, and pull request body. It also looks in commit messages, but commits with ticket identifiers in the commit message can still fail the “use of branches” and “use of pull requests” scorecard checks in this report if they are on a main/master branch or not part of a pull request because they bypass the change control process.


### Does the report handle squash and rebase merges?

Yes, the report properly follows commits through squash and rebase merges using metadata available in pull request records so that they are treated the same way as regular merges. Code written in squashed or rebased commits will be checked for use of pull requests and ticket traceability.


### How does the report handle Git flows with multiple levels of pull requests?

For Git flows with multiple levels of pull requests, like having a side branch that is merged into a feature branch, then into development, and finally into main/master, the report searches for ticket identifiers in all pull requests until the code reaches an integration branch.

In the above example, if the first side branch pull request does not have a ticket identifier, but the PR from the feature branch into development does, then commits on the side branch will be properly counted as linking to a ticket.

However, if the feature branch PR also does not have a ticket, but the PR from development into master mentions some ticket in the body, then the side and feature branch commits will _not_ count as linking to a ticket because they did not have a ticket before reaching development – an integration branch associated with multiple features. In this scenario, mentions of tickets when merging from development to master are disregarded because it’s unclear which specific commits they are associated with.


### What version control systems are supported?

minware currently supports GitHub, GitLab, Bitbucket, and Azure DevOps Git version control systems.


### What ticketing systems are supported?

minware currently supports Jira and GitHub Issues as ticketing systems.


### How are deleted or moved tickets handled?

The report is currently unable to identify code that is linked to a ticket that no longer exists at its original ID because it has been deleted or moved. Deleting tickets is generally an anti-pattern and instead they should be resolved as “won’t fix” or a similar status to preserve traceability.


### How is code from bots or outside contributors treated?

This report excludes commits from bots and users outside of your organization in forked repositories so that they don’t count against the metrics.


Code/Ticket Hygiene and Traceability Report Screenshot

Code/Ticket Hygiene and Traceability

Track how consistently teams follow best practices for version control and ticketing to provide traceability for reporting.

Capitalization

Tickets

The Change Control Compliance report automatically calculates the percentage of time where work is done in compliance with the following change control best practices: use of branches, use of pull requests, and code reviews.

The Change Control Compliance report displays the percentage of time spent spent in the following ways:

  1. On commits in a non-main branch
  2. On branches merged through a pull/merge request
  3. On merged branches that received a code review

You can easily customize the report to filter by team or person to see more detail on how well various groups perform on these metrics.

Adhering to these best practices results in the following benefits:

  1. Shared knowledge across the team
  2. Improved code quality, security, and performance
  3. Auditability of code changes for compliance purposes

This report is useful to team leads, engineering leaders, security teams, and anyone interested in how well the organization complies with change control best practices.

Change Control Compliance Report Screen Shot

Change Control Compliance

View the percentage of time where work is done in compliance with change control best practices.

Compliance

Change Control

The Work Batch Size report automatically calculates information about the amount of development time put into each branch, ticket, and epic.

Development time is calculated by allocating work hours between commits to the commit that follows. So, if you work 9-5 and commit at 1 PM and 5 PM, the 5 PM commit would be counted as one half day of development time.

The report shows scores for large branch rate (LBR) and large ticket rate (LTR), which measure the amount of development effort that goes into large branches and tickets, which is less than 10% for top-performing teams.

Additionally, the report lists the largest pull requests, tickets, and epics based on the total amount of development work so that you can diagnose outliers and fix the root cause of issues related to large work batches.

You can easily customize the report to filter by team or person to see more detail on how well various groups perform on these metrics.

The scorecard metrics use a threshold of 5 dev days for considering a branch of ticket to be “small.” The reasoning is that with a two week sprint, a ticket or branch that takes more than one week of active development time will take up more than half the sprint and should be broken down into smaller tasks.

There is no threshold associated with epics, but you can sort the results to view the largest epics with this report.

Smaller is almost always better when it comes to batch size, and these signals provide important information about batch sizes:



1. A high number on Task Specific Branches indicates that branches have small changesets. This makes it easier to review code, have a working central branch for features/projects, and track which tasks are associated with each branch.
2. A high number on No Omnibus Tickets indicates that tickets are generally small and require less than one week of work. Small tickets are easier to understand and estimate, and have a higher chance of being delivered as expected. 
3. Epic size is important as well. Similarly to small tasks, small epics are easier for everyone to manage.

This report is useful to engineers, team leads, engineering leaders, and product managers to reason about the size of batches of work moving through the system.


Work Batch Sizes

View information about the overly large tickets, branches, and epics, which can impact development efficiency.

Epics

The Context Switching report automatically calculates the percentage of time spent focused on a single ticket (Flow Efficiency) and how well ticket statuses are kept up to date (No Stale Tickets).

Flow Efficiency is important and is worth explaining: if a developer marks a ticket as “In Progress” this morning and sets it to “Done” tomorrow morning but spends half of that time committing against a second ticket, they would have a Flow Efficiency of 50% for the day.

The report displays the percentage of time spent in the following ways:

1. Focused on a single ticket. 
2. On tickets that are marked as In Progress and receive commits at least weekly

You can easily customize the report to filter by team or person to see more detail on how well various groups perform on these metrics.

These signals provide information about the health of the development process:

1. High Flow Efficiency indicates a development process with little context switching so that developers are focused on one thing at a time. 
2. High “No Stale Tickets” indicates that ticket statuses are kept updated and that developers are regularly working on the in-progress items

This report is useful to engineers, team leads, engineering leaders, and anyone interested in the health of the development process.

Report Templates: Azure DevOps

Code/Ticket Hygiene and Traceability

Change Control Compliance

Work Batch Sizes

Context Switching

Inspired by Uber's Dashboard

Didn’t find what you were looking for?

Page Load Failed