Azure DevOps Service is a cloud native Software as a Service (SaaS) from Microsoft that provides organizations with a fully integrated platform for software development. The focus is on built-in functions such as

• Azure Repos (Version Control System)
• Azure Boards (Issue tracking and project management tool)
• Azure Artifacts (Repository Manager - host and share packages)
• Azure Pipelines (Automation - Continuously build, test and deploy)
• Azure Test Plans (Test and ship confidently with an exploratory test toolkit)

to enable a culture of collaboration that eliminates the need for additional tools and integration overhead with third-party systems. Configuration can be performed either via a user interface (web browser) or a REST-API by the authorized personnel. The challenge for administrators, in particular, is to implement repeating activities efficiently, in an automated manner, and with a high degree of quality. Nobody wants to waste their time with unnecessary activities!

So in the further part of the article you will not find screenshots of user interfaces in the browser, but the focus is on the programmatic implementation.

Terraform

Terraform is an amazing infrastructure as Code tool that lets you build, change and version resources (e.g. cloud or on-premise) safely and efficiently. In this section you will not find a deep dive on how to use Terraform (basic knowledge is required). You can find a starting point for learning at Hashicorp.

I just want to address one fundamental concept of Terraform - the Provider.

"Providers are a logical abstraction of an upstream API. They are responsible for understanding API interactions and exposing resources." (Hashicorp definition)

The use of providers significantly extends the functionality of Terraform (analogy e.g. to plug-ins in other programming languages or frameworks). 

The best-known examples include AWS, Azure and Google Cloud Platform for the management of public cloud infrastructure (e.g. virtual machines, networks, traffic routing, databases, identity & access management, etc.).

In addition to the popular use cases, there are other providers, e.g. for interacting with resources (e.g. Kubernetes) or applications or platforms (e.g. Github, Azure Active Directory). Microsoft also provides a Terraform Provider for Azure DevOps for administrative purposes, which I will introduce in more detail in the following sections.

At the beginning we register the provider

Unset

terraform {
  required_version = "~> 1.4"

  required_providers {
    azuredevops = {
	source  = "microsoft/azuredevops"
	version = "~> 0.5"
    }
  }
}

Then we create a PAT and initialize the provider

Unset

provider "azuredevops" {
  org_service_url       = "https://dev.azure.com/<your organisation name>"
  personal_access_token = "<your personal access token>"
}

Please replace the placeholders <your organization name> and <your personal access token> with your values.

Azure DevOps

Project

A project establishes an organizational umbrella for collaboration (e.g. management and progress) in the development of software solutions (e.g. source code). In practice, projects are used to isolate functional teams, for example cross-functional teams in agile organizations, from each other to enable independent lifecycles.

From an administrator's perspective, managing projects simply means using a provider resource

Unset

resource "azuredevops_project" "project" {
  name               = var.name
  description        = var.description
  features           = var.features
  visibility         = var.visibility
  work_item_template = var.template
}

which defines useful default values or values that vary from them depending on requirements. In this example, an agile project is enabled with all features except Azure Testplans. (no additional administrative effort required).

Unset

variable "features" {
  type        = map(string)
  description = "Defines the status (enabled, disabled) of the all project features."
  default     = {
    boards       = "enabled"
    repositories = "enabled"
    pipelines    = "enabled"
    artifacts    = "enabled"
    testplans    = "disabled" # disabled by default because of additional license costs
  }
}

variable "template" {
  type        = string
  description = "Specifies the work item template. Valid values: Agile, Basic, CMMI, Scrum."
  default     = "Agile"
}

Security

In Azure DevOps projects, it is important to define which identities are granted access to resources (RBAC) in order to fulfill their responsibilities (e.g., developer or operator). The baseline for this is usually an existing authorization concept in the organization, which has been implemented by groups and memberships of people in Azure Active Directory. Based on this implementation, it is very easy to apply Azure AD groups to Azure DevOps built-in roles.

Unset

data "azuredevops_group" "project" {
  for_each = var.project

  name       = each.key
  project_id = var.project_id
}

resource "azuredevops_project_permissions" "project" {
  for_each = var.project

  project_id  = var.project_id
  principal   = data.azuredevops_group.project[each.key].id
  permissions = each.value.permissions
}

A suitable configuration is very easy to define, as it consists of a mapping from Built-in role names to Azure AD Group names.

Unset

# Use Azure AD groups to manage built-in group permissions
rbac = {
  Readers      = ["project-guests"]
  Contributors = ["project-developers", "project-supporters"]
}

Another nice functionality is to assign permissions to Azure AD groups at the GIT repository level to map finely granular permissions for the development process.

Unset

data "azuredevops_group" "git" {
  for_each = var.git

  name       = each.key
  project_id = var.project_id
}

resource "azuredevops_git_permissions" "git" {
  for_each = var.git

  project_id  = var.project_id
  principal   = data.azuredevops_group.git[each.key].id
  permissions = each.value.permissions
}

In this simple example, explicit permissions are set for a GIT repository for the Built-in role Contributor. For a complete list of permissions, see the provider resource.

Unset

Contributors = {
  permissions = {
    ForcePush        = "Allow"
    CreateRepository = "Allow"
    DeleteRepository = "Allow"
    RenameRepository = "Allow"
  }
}

Repositories

Within an Azure DevOps project, a repository contains all types of source code (e.g. business applications or infrastructure as code, etc.) in the form of a version control system.

Unset

resource "azuredevops_git_repository" "repo" {
  name           = var.name
  project_id     = var.project_id
  default_branch = var.default_branch

  initialization {
    init_type = "Clean"
  }
}

Furthermore, files can be added at creation time (e.g. CI/CD pipelines, README.md, or similar) to help development teams adapt standard processes without significant effort (more use cases on this can be found below).

Unset

resource "azuredevops_git_repository_file" "initial" {
  for_each = var.files

  branch              = var.default_branch
  content             = each.value.content
  file                = join("/", [each.value.path, each.key])
  repository_id       = azuredevops_git_repository.repo.id
  commit_message      = "Initial commit."
  overwrite_on_create = true
  lifecycle {
    ignore_changes = [
      content
    ]
  }
}

In the following example configuration, a repository named "demo" is created and a file ".gitignore" from a standard template is added in the root directory (git commit).

Unset

repos = {
  demo = {
    files = {
      ".gitignore" = {
        content = data.template_file.tf_gitignore.rendered
        path    = "/"
      }
    }
  }
}

Service Connections

A Service Connection provides a communication proxy between an Azure DevOps project to a remote service (e.g. Azure Resource Manager or Github etc.). This creates a stable API e.g. for Azure Pipelines, which avoids e.g. explicit storage of credentials. Only one provider resource is required for administration; in addition, the principle of least privilege is implemented in the best possible way because the creation and authorization of the service principal can be designed in a fine-grained manner (assuming privileged permissions in Azure AD).

In this example, a service connection is established to the Azure Resource Manager API. The service principal used can either be created earlier or an existing one can be configured for it. The permissions of this account are essential for the functionality of the connection (authorization process).

Unset

resource "azuredevops_serviceendpoint_azurerm" "arm" {
  project_id            = var.project_id
  service_endpoint_name = var.arm.name
  description           = "Managed by terraform"

  credentials {
    serviceprincipalid  = var.arm.client_id
    serviceprincipalkey = var.arm.client_secret
  }

  azurerm_spn_tenantid          = var.arm.tenant_id
  azurerm_subscription_id       = var.arm.subscription_id
  azurerm_subscription_name     = var.arm.subscription_name
  azurerm_management_group_name = var.arm.resource_group_name
}

Pipelines & Agent Pools

Azure Pipelines and Agent Pools together provide a service for automating repeated tasks, e.g. building, testing and deploying applications or managing infrastructure. All of these activities are integral to DevOps in the classic sense and can also be managed with low effort.
A pipeline resource is created by this snippet

Unset

resource "azuredevops_build_definition" "pipeline" {
  project_id = var.project_id
  name       = var.name
  path       = replace(var.path, "/", "\\")

  ci_trigger {
    use_yaml = true
  }

  repository {
    repo_type   = "TfsGit"
    repo_id     = var.repository_id
    branch_name = var.default_branch
    yml_path    = join("/", [var.path, var.file_name])
  }
}

and specified by this configuration. Here also the content of a template is used to create a pipeline in the root directory “/azdo”. This allows pipelines to be provided flexibly.

Unset

pipelines = {
  "azure-pipelines.yaml" = {
    content = data.template_file.pipeline.rendered
    path    = "/azdo"
  }
}

Agent pools can in general also be managed with Terraform, but the implementation e.g. with Azure Virtual Machine Scale Set agents is not fully programmatically possible. For this case, I recommend a workaround using the Azure DevOps Elasticpools REST API  in conjunction with a Lifecycle Terraform resource to orchestrate the HTTP operations within a shell script as follows.

Unset

resource "shell_script" "pool" {
  lifecycle_commands {
    create = "sh ${path.module}/scripts/pool_http_client.sh create"
    update = "sh ${path.module}/scripts/pool_http_client.sh update"
    delete = "sh ${path.module}/scripts/pool_http_client.sh delete"
  }

  environment = {
    POOL_NAME             = var.name
    PROJECT_ID            = var.project_id
    BASE_URL              = local.base_url
    REQUEST_BODY_JSON     = local.pool_create_body
    SERVICE_ENDPOINT_NAME = var.service_endpoint_name
    VM_SCALE_SET_ID       = var.virtual_machine_scale_set_id
  }
}

Important note: The current implementation in this example uses an existing Azure virtual machine scale set that is to be used as an agent pool (incl. network configuration for inbound traffic). Creating this infrastructure is not (yet) part of the solution, but an architectural decision. The focus is on managing Azure DevOps as a platform and reducing dependencies (e.g. Azure infrastructure incl. necessary permissions to manage Azure resources). It is of course technically possible at this point to implement a connector into your own Azure infrastructure.

Short recap

In previous sections, I presented the elementary building blocks for managing Azure DevOps projects. They are based almost entirely on Microsoft's Azure DevOps Terraform Provider and offer administrators a convenient way to perform their daily work with a minimum of time while maintaining high quality. An effort usually only occurs when, differing from the standard, individual requirements for the configuration of Azure DevOps projects have to be mapped (text change in the Terraform configuration). The solution is very smart, controllable (low complexity), automatable and very flexible due to the composition of individual building blocks.

The following diagram shows a possible architecture for implementing a management layer as a standalone platform team that completely takes over the task of Azure Devops administration for a company.

The scaling effect begins when any amount of Azure DevOps projects can be managed in a fully automated manner for the business in a very short period of time, as shown in the following diagram.

You can find the full source code and more examples of the solution presented here on Github.

Use cases

To wrap up, I would like to describe a few real-world use cases as inspiration that can be very easily standardized and implemented.

As a CISO, I want to ensure that an implementation of the Azure DevOps authorization concept can be done effectively and fully without exceptions to comply with existing policies.

As a developer of a Java Spring Boot application, I would like to be provided with a standard DevOps build and deployment process that allows me to run my application based on the technology stack Maven + Cloud native build packs + Docker + Kubernetes so that I can focus on technical development.

As a cloud architect, I want to seamlessly integrate and audit Azure DevOps services into the company's multi-cloud strategy to build a consistent and flexible ecosystem.