Failing Azure DevOps Pipeline builds if unit tests fail

12 Mar 2023 🔖 web development minimum viable build beginner devops git azure
💬 EN

1. We’ve built the world’s tiniest webserver.
2. We’ve added unit tests that can help us predict whether it would run “correctly.”
3. We’ve told Azure DevOps (“ADO”) Pipelines to auto-build a runnable web server for us each time we update Git-tracked source code.
4. How about having Azure DevOps fail the auto-build if our unit tests fail?

Let’s do it!

See the sample codebase on GitHub.

---

Prerequisites

1. Please work your way through all 3 of the exercises mentioned at the top of this article so that you’re familiar with “building” and “running” a webserver from source code, “unit testing” it, and “making a CI/CD build pipeline in Azure DevOps.” (Although in this case, we still won’t be “running” the server, per se.)
2. Have an account in Azure DevOps. You can get one:
   • From Microsoft for free but with a credit card on file.
   • From Microsoft for free without a credit card as a student.
   • From Microsoft without a credit card through a Visual Studio subscription if you have one at work.
   • For 4 hours at a time through an A Cloud Guru subscription (personally, I’d rather get 1 yearly charge from ACG than give Microsoft my credit card and expose myself to getting hacked and running up an “infinity” bill).

---

How the pipeline will behave

The YAML-formatted file “/cicd/my-azure-build-pipeline.yml” (from the sample codebase you just copied) forces any ADO Pipeline running it to walk through all of the same sequential steps as mentioned in the previous article, plus a few additional ones:

1. (Execute only on edits to “main” branch, same as before.)
2. (Borrow a computer from Microsoft, same as before.)
3. (Install Node.js and NPM, same as before.)
4. (Install specific Node modules, same as before.)
5. New: Run unit tests. It does this by executing a “npm run test” command on the agent’s operating system.
6. New: Publish results of unit test execution to the ADO project’s “Test Plans” -> “Runs” dashboard.
7. Conditional only if tests PASS:
   • (Build /dist/, same as before.)
   • (Copy /dist/ to an “artifact staging directory,” same as before.)
   • (Copy the “artifact staging directory” contents to a longer-lived “Artifact” named “MyBuiltWebsite,” same as before.)

Again, after these steps finish executing, it’s safe for Azure DevOps to let the agent (the Linux computer we borrowed from Microsoft) self-destruct.

---

Running a successful build

Follow these steps from the previous article, with a few changes:

1. “Importing my sample codebase into ADO Repos, but…”
   • Only instead of importing a copy of ...03-ado-build.git…
   • Import a copy of:

       https://github.com/kkgthb/web-site-nodejs-04-ado-tests.git
     

2. “Telling Azure Pipelines to build when code changes” -> “Forcing ADO to use our YAML pipeline code.”
3. “Running the pipeline by editing code.”
   • Please only edit README.md for now.
4. “Validating the pipeline ran correctly.”
   • If you look at the job logs, you’ll see that there were more steps involved to execute the pipeline.
   • Logs from the pipeline step that runs unit tests will have output that looks somewhat like this “passing test” output from earlier on your own computer.
   • In the job summary’s 4th column, under “Tests and coverage,” there should also be a clickable link that says “100% passed.” If you click it, you’ll see a green donut chart showing that 2 of 2 tests passed.

Also follow these steps:

1. In the left-hand navigation of Azure DevOps, click on “Test Plans.”
2. Beneath that, click “Runs.” (Note: this particular pages’s dedicated URL will be something like https://dev.azure.com/YOUR-ADO-ORG-NAME/YOUR-ADO-PROJECT-NAME/_testManagement/runs.)
3. Under “Recent test runs” to the right, you should see a test in “completed” state.
4. Double-click it to see a lovely dashboard with green donut and bar charts indicating that 2 of 2 tests passed.
5. Click the “Test results tab to see each of the 2 tests listed separately (they’re also double-clickable, though in this case there’s not really anything in there that adds useful detail).

---

Running a failed build

Repeat the steps from the previous article’s “Running the pipeline by editing code” section.

Only instead of editing a README.md file…

Then repeat the steps from the previous article’s “Validating the pipeline ran correctly” section.

Also follow these steps:

1. In the left-hand navigation of Azure DevOps, click on “Test Plans.”
2. Beneath that, click “Runs.” (Note: this particular pages’s dedicated URL will be something like https://dev.azure.com/YOUR-ADO-ORG-NAME/YOUR-ADO-PROJECT-NAME/_testManagement/runs.)
3. Under “Recent test runs” to the right, you should see a test in “completed” state.
4. Double-click it to see a lovely dashboard with red donut and bar charts indicating that 2 of 2 tests failed.
5. Click the “Test results tab to see each of the 2 tests listed separately (they’re also double-clickable, though in this case there’s not really anything in there that adds useful detail).

---

Extra credit – test-driven development

Repeat the steps from the previous article’s “Running the pipeline by editing code” section.

Only instead of editing a README.md file…

Then repeat the steps from the previous article’s “Validating the pipeline ran correctly” section. Once the pipeline finishesrunning, congratulate yourself that once again, your source code successfully builds into a webserver runtime “artifact” and that all your ADO Test Runs result dashboards are colored green.

---

Credits

• Rupesh Tiwari’s “Publishing Test Results Using JEST in Angular” for teaching me how to reformat JEST output into JUnit format, one of the formats that ADO’s PublishTestResults command can read.

---

Posts in this series