Make automated software test results visible

11 Dec 2025 🔖 web development tips
💬 EN

I’m a big fan of developers proving to QA exactly what happened during automated test execution, by publishing the results into something associated with the Kanban board / work tracking system that the whole team is already using.

Why automate tests? 

Automating the testing of software – including web applications and web sites – increases efficiency.  Automated tests can run quickly and often, catching issues before humans do.  Once written, they can be run as many times as needed – no more repetitive manual testing.  Test automation is an essential “DevOps” best practice for modern management of software lifecycles. 

The goal of automated testing is to catch errors sooner and addressing them before they reach a UX review or QA review. 

This shortens the feedback cycle, leading to higher quality code, a more productive the development cycle, and increased team velocity. 

---

Why automatically publish test results? 

To maximize the benefits of automation, test run results should be published into a shared tracking system where the entire team can access and act on them promptly. 

Benefits include: 

• Transparency:  Developers, QAs, product owners, and other stakeholders can all see the current state of the application at a glance. 
• Faster Feedback:  Issues are caught and triaged quickly, not buried in someone’s local logs or inbox. 
• Accountability:  Test failures are visible to the whole team, encouraging prompt investigation and resolution. 
• Historical Trends:  Tracking results over time helps spot flaky tests, recurring issues, and improvements in quality. 

---

Where should I publish test results? 

If your team is already using the Azure Test Plans service within the Azure DevOps (“ADO”) suite, then its Test Runs sub-service is an excellent destination for publishing the results of your automated software tests. 

Some of its particularly handy features are: 

• Quick-reference charts and graphs. 
• “Create bug” buttons, on failed test result detail pages, that integrate seamlessly with ADO’s Azure Boards work tracking service. 

Note:  Most of this article will refer to various ADO services, because I think Azure Test Plans is super nifty. However, if your team uses other tools, you can still follow the same principles.  See “What if Azure Pipelines doesn’t run our tests?” and “What if we don’t use Azure Test Plans?” toward the bottom of this article. 

---

How can I publish test results into Azure Test Plans? 

If your team is already running your automated software tests as part of CI/CD pipelines in ADO’s Azure Pipelines service, then publishing test results to ADO Test Runs is as simple as adding a “Publish Test Results” step to its existing YAML, “classic build,” or “classic release” Azure Pipeline. 

Just make sure that the step of the CI/CD pipeline that runs the automated software tests generates an output file in a format that Microsoft’s “Publish Test Results” task can read, such as JUnit.  A full list of accepted formats is available in the “Publish Test Results” documentation. 

Running “Publish Test Results” will also automatically add a “Tests” tab to the Azure Pipeline’s run results summary, where developers can quickly see what went wrong. 

---

Can I also publish code coverage into Azure Test Plans?

Yes, though only to the to the Azure Pipeline’s run results summary, not to ADO Test Runs. 

This is relevant for suites of automated tests that also generate details about how much of the application’s source code was executed when running the tests. 

Not all automated test suites do this.  The feature is more commonly found among automation frameworks that specialize in unit tests and integration tests than those that specialize in end-to-end tests. 

However, if your framework generates a report about code coverage, then your team can add a “Code Coverage” tab to the Azure Pipeline’s run results summary by also adding a “Publish Code Coverage” step to the existing Azure Pipeline. 

Just make sure that the step of the CI/CD pipeline that runs the automated software tests generates an output file in the “Cobertura” format, which is what Microsoft’s “Publish Code Coverage” task can read. 

It does not matter in which order you run “Publish Test Results” and “Publish Code Coverage.” 

---

Can one CI/CD pipeline publish results from many test suites? 

Yes.  One Azure Pipelines CI/CD pipeline can include many steps that each run the “Publish Test Results” task. 

Note that there is a “mergeTestResults” setting that will impact how results are displayed in ADO Test Runs and in the pipeline results summary.  For more information, see Microsoft’s “Publish Test Results” documentation. 

---

What if Azure Pipelines doesn’t run my tests? 

If your team uses ADO’s Boards and Test Plans services, but doesn’t run its automated software tests via Azure Pipelines, importing your results into ADO Test Runs might still be possible, but I haven’t looked into exactly how to do it.

---

What if I don’t use Azure Test Plans? 

Although Azure Boards is an excellent work tracking service, and Azure Test Plans is an excellent quality assurance planning service, your team might have reasons that they are using other tools to manage software development lifecycles.

Your existing productivity tools might include a similar tracking system where test results can be automatically published.

For example, there are a lot of popular GitHub Actions actions that transform a JUnit-formatted *.xml file into a pretty HTML-formatted display within a GitHub Actions workflow run’s job summary. I’m not sure if there’re further integrations to publish it into, say, GitHub Issues, but I wouldn’t be suprised.

---

Author’s disclosure

This article was made infinitely better with the editing assistance of a technical writer colleague.