Provisioning Azure DevOps Service Connections that let ADO Release Pipelines leverage Azure AD Service Principals for sensitive CI/CD tasks

17 Mar 2023 🔖 beginner azure
💬 EN

We’re almost ready to build an Azure DevOps (“ADO”) Pipeline that can release our tiny web server we asked it to build for us onto our rented 2 webserver hosts (one “nonproduction” and one “production”).

Although we created an Azure Active Directory “(AAD)” service principal that’s authorized to work against our webserver hosts, Azure DevOps doesn’t yet know how to use our Service Principal.

In this article, we’ll set up a cross-reference (known as a “Service Connection”) from ADO project to our AAD Service Principal. Once we’ve done so, we’ll be able to wrap up this series by constructing just a little more ADO Pipeline goodness (auto-updating our live websites every time we update our Git-tracked source code). We’re almost there!

Prerequisites

1. Have an account in Azure.
2. Log your computer’s command line into that Azure account.
3. Please work your way through “Provisioning an Azure resource group” and “Provisioning Azure AD Service Principals that can deploy built webapps” before beginning this exercise. You can skip “Provisioning Azure App Services to host your Hello World webapp” for now if you’re short on time, but you may want to skim what it’s about.
4. Please work your way through my entire exercise “Making Azure DevOps Pipelines build a Hello World webapp from Git-tracked source code changes” before beginning this exercise so that you’ll have set up an Azure DevOps organization and project and familiarized yourself with the Azure DevOps web portal. If you’re short on time, at least make sure you have an organization and project set up.

Provision an ADO Service Connection

There is no Azure PowerShell module capable of manipulating ADO Service Connection resources – only an Azure CLI extension.

With the Azure CLI

In a Windows PowerShell command prompt

az devops service-endpoint azurerm create `
    --organization "https://dev.azure.com/YOUR-ADO-ORG-NAME/" `
    --project "YOUR-ADO-PROJECT-NAME" `
    --name "my-first-service-connection" `
    --azure-rm-service-principal-id $( `
        az ad app list `
            --display-name "my-first-app-reg" `
            --query "[0].appId" `
    ) `
    --azure-rm-tenant-id $( `
        az account show `
            --query "tenantId" `
    ) `
    --azure-rm-subscription-id $( `
        az account show `
            --query "id" `
    ) `
    --azure-rm-subscription-name $( `
        az account show `
            --query "name" `
    )

In a Linux-style command prompt

az devops service-endpoint azurerm create \
    --organization "https://dev.azure.com/YOUR-ADO-ORG-NAME/" \
    --project "YOUR-ADO-PROJECT-NAME" \
    --name "my-first-service-connection" \
    --azure-rm-service-principal-id $( \
        az ad app list \
            --display-name "my-first-app-reg" \
            --query "[0].appId" \
        | xargs \
    ) \
    --azure-rm-tenant-id $( \
        az account show \
            --query "tenantId" \
        | xargs \
    ) \
    --azure-rm-subscription-id $( \
        az account show \
            --query "id" \
        | xargs \
    ) \
    --azure-rm-subscription-name "$( \
        az account show \
            --query "name" \
    )"

With the Azure DevOps web-based portal

1. Visit the “Service connections” page of your Azure DevOps portal at https://dev.azure.com/YOUR-ADO-ORG-NAME/YOUR-ADO-PROJECT-NAME/_settings/adminservices, of course substituting appropriate values for YOUR-ADO-ORG-NAME AND YOUR-ADO-PROJECT-NAME.
2. In the upper right corner, click the “New service connection” button.
3. In the flyout panel at right, click the “Azure Resource Manager” option and then click the “Next” button in the lower-right corner of the flyout panel.
4. In the flyout panel at right, click the “Service principal (manual)” option and then click the flyout panel’s “Next” button.
5. Leave “Environment” set to “Azure Cloud.”
6. Leave “Scope Level” set to “Subscription.”
7. In “Subscription ID,” type the ID of the subscription you rented web hosting resources within. If your command line interface is still logged into the same Azure account as when you rented those resources, the commands are:
   • Azure PowerShell: (Get-AzSubscription).Id
   • Azure CLI: az account show --query "id"
8. In “Subscription Name,” type the name of that same subscription.
   • Azure PowerShell: (Get-AzSubscription).Name
   • Azure CLI: az account show --query "name"
9. In “Service Principal ID,” type the client ID of the AAD Application Registration you created earlier. If you named yours “my-first-app-reg” like I did, the code to fetch it is:
   • Azure PowerShell: (Get-AzAdApplication -DisplayName "my-first-app-reg").AppId
   • Azure CLI: az ad app list --display-name "my-first-app-reg" --query "[0].appId"
10. In “Service principal key,” type the secret associated with the AAD Service Principal your ADO Service Connection is intended to represent.
11. In “Tenant ID,” type the ID of the tenant you rented web hosting resources within. If your command line interface is still logged into the same Azure account as when you rented those resources, the commands are:
    • Azure PowerShell: (Get-AzSubscription).TenantId
    • Azure CLI: az account show --query "tenantId"
12. Click the “Verify” button.
    • If you receive a message that says “Verification Succeeded,” click “Verify and save” at the bottom-right corner of the flyout panel.
    • If you receive a message that says “Verification Failed” and see an error message toward the bottom of the flyout panel indicating “AADSTS700215: Invalid client secret provided,” you may have made a typo in “Service principal key.”
    • If you see a slightly different error at the bottom:
      • Maybe you made a typo in one of the other fields.
        • Are you sure your CLI is logged into the correct Azure account?
        • Accidentally putting an AAD Application Registration object ID or the AAD Service Principal ID instead of the AAD Application Registration’s application/client ID into “Service Principal ID” will tell you:

          “Failed to obtain the Json Web Token(JTW) using service principal client ID. Exception message: AADSTS70016: Application with identifier ‘YOUR-APPLICATION-REGISTRATION-OR-SERVICE-PRINCIPAL-OBJECT-ID-HERE’ was not found in the directory ‘Default Directory’. …”

      • Maybe you forgot to create an Azure RBAC Role Assignment giving your AAD Application Registration + AAD Service Principal pair the ability to work with Azure webhost resources. This often manifests with the following error message:

        Failed to query service connection API: ‘https://management.azure.com/subscriptions/…’. Status Code: ‘Forbidden’, Response from server: ‘{“error”:{“code”:”AuthorizationFailed”,”message”:”The client ‘YOUR-SERVICE-PRINCIPAL-ID-HERE’ with object ID ‘YOUR-SERVICE-PRINCIPAL-ID-HERE’ does not have authorization to perform action ‘Microsoft.Resources/subscriptions/read’ over scope ‘/subscriptions/YOUR-AZURE-SUBSCRIPTION-ID-HERE’ or the scoope is invalid. If access was recently granted, please refresh your credentials.”}}’

        • Unrelated trivia: this error also shows up when you’re trying to build an Azure DevOps pipeline that can write to Microsoft Sharepoint sites and doesn’t do any actual Microsoft Azure work. I’ll cover a workaround for that issue in a future post.
          • (Solution preview: make Azure DevOps happy by creating a custom Azure RBAC role with extremely limited privileges, pick a totally random entirely-unrelated-to-Sharepoint Azure subscription, and use Azure RBAC Role Assignment associate the Sharepoint-authorized AAD Service Principal with your arbitrary subscription using your custom RBAC role.)

ALL DONE – STOP HERE

The rest of this article is edge cases. If everything went well above, you can continue onward in this series.

Enabling an ADO YAML pipeline to authenticate as an ADO Service Connection

Although all Azure DevOps “classic” pipelines in an ADO project can make use of any ADO Service Connection in the project, Azure DevOps “YAML”-based CI/CD pipelines cannot.

However, should you ever find yourself needing to enable a YAML-based ADO pipeline to do sensitive operations like release/deploy “built” CI/CD artifacts onto live webserver hosting resources, here’s how you’d do it:

1. Visit the “Service connections” page of your Azure DevOps portal at https://dev.azure.com/YOUR-ADO-ORG-NAME/YOUR-ADO-PROJECT-NAME/_settings/adminservices, of course substituting appropriate values for YOUR-ADO-ORG-NAME AND YOUR-ADO-PROJECT-NAME.
2. Click the service connection whose secret needs an update.
3. Click the three vertical dots in the upper-right corner (hover-tooltip: “more actions”), to the right of the “edit” button.
4. Click the “Security.”
5. Click the “+” button in the upper-right corner of the panel labeled “Pipeline permissions.”
6. Click the name of the ADO YAML-formatted pipeline you’d like to give permission to act on behalf of the AAD Service Principal your ADO Service Connection represents.
   • If you accidentally clicked the wrong pipeline, you can remove it by clicking the “X” (hover-tooltip: “revoke access”) to the far right side of the pipeline’s name listed “Pipeline permissions.”

Checking if the AAD Service Principal has a secret

With Azure PowerShell

If ( `
    Get-AzAdSpCredential -ObjectId ( `
        Get-AzAdServicePrincipal `
            -ApplicationId ( `
                Get-AzAdApplication `
                    -Filter "DisplayName eq 'my-first-app-reg'" `
                    -First 1 `
            ).AppId `
    ).Id `
) { `
    Write-Host("There is already a secret associated with this AAD Service Principal.  Please ask around what it is."); `
} `
Else { `
    Write-Host("There are no secrets associated with this AAD Service Principal.  Go ahead and create one."); `
}

With the Azure CLI

In a Windows PowerShell command prompt

If ( `
    az ad sp credential list `
        --id $( `
            az ad sp list `
                --query "[?appDisplayName=='my-first-app-reg'].{id:id}[0].id" `
        ) `
        --query "[0].keyId" `
) { `
    Write-Host("There is already a secret associated with this AAD Service Principal.  Please ask around what it is."); `
} `
Else { `
    Write-Host("There are no secrets associated with this AAD Service Principal.  Go ahead and create one."); `
}

In a Linux-style command prompt

if [ \
    $( \
        az ad sp credential list \
            --id $( \
                az ad sp list \
                    --query "[?appDisplayName=='my-first-app-reg'].{id:id}[0].id" \
                | xargs \
            ) \
            --query "[0].keyId"
    ) \
]; \
then
    echo "There is already a secret associated with this AAD Service Principal.  Please ask around what it is."; \
else
    echo "There are no secrets associated with this AAD Service Principal.  Go ahead and create one."; \
fi;

Creating an additional secret for the AAD Service Principal

There is no Azure CLI command for creating an additional secret if one already exists, while leaving the old one intact.

(Although there is a delete-all-and-add-a-new-one command for Azure CLI, covered later in this article.)

With Azure PowerShell

This script will copy the new secret to your computer’s clipboard.

You will not see it displayed in your command-line terminal.

( `
    New-AzAdSpCredential `
        -ObjectId ( `
            Get-AzAdServicePrincipal `
                -ApplicationId ( `
                    Get-AzAdApplication `
                        -Filter "DisplayName eq 'my-first-app-reg'" `
                        -First 1
                ).AppId `
        ).Id `
).SecretText | Set-Clipboard

Deleting all existing secrets and creating a new one for the AAD Service Principal

There is no Azure PowerShell command for resetting an existing secret (although you can certainly delete an existing secret by its “Key ID” and then create a new one in two back-to-back steps).

With the Azure CLI

In a Windows PowerShell command prompt

az ad sp credential reset `
    --id $( `
        az ad sp list `
            --query "[?appDisplayName=='my-first-app-reg'].{id:id}[0].id" `
    ) `
    --query "password" `
| Set-Clipboard

In a Linux-style command prompt

az ad sp credential reset \
    --id $( \
        az ad sp list \
            --query "[?appDisplayName=='my-first-app-reg'].{id:id}[0].id" \
        | xargs \
    ) \
    --query "password" \
| xclip

Updating the secret for an ADO Service Connection

Did you have fun making a mess of your AAD Service Principal’s secret?

There’s a good chance now that you’ll need to edit your Azure DevOps Service Connection with the new secret.

1. Visit the “Service connections” page of your Azure DevOps portal at https://dev.azure.com/YOUR-ADO-ORG-NAME/YOUR-ADO-PROJECT-NAME/_settings/adminservices, of course substituting appropriate values for YOUR-ADO-ORG-NAME AND YOUR-ADO-PROJECT-NAME.
2. Click the service connection whose secret needs an update.
3. Click the “Edit” button in the upper-right corner.
4. In the flyout panel at right, click the “Verify” button.
5. If you receive a message that says “Verification Failed” and see an error message toward the bottom of the flyout panel indicating “AADSTS700215: Invalid client secret provided,” then the old secret this ADO Service Connection has on file no longer reaches your AAD Service Principal properly.
   • Just a bit higher up in the flyout panel, type a new value into “Service principal key.”
   • Scroll back down and click the “Verify” button.
6. If you receive a message that says “Verification Succeeded,” click “Verify and save” at the bottom-right corner of the flyout panel.
   • If you still receive an “AADSTS700215: Invalid client secret provided” error, you may have made a typo.

Posts in this series