Implementing Continuous Testing in DevOps Pipelines

Co-written with Sudhish Mangalasary

Organizations continue to modernize their software development and delivery practices to minimize the impact of business disruption and stay competitive. Organizations have implemented Agile techniques to accelerate software delivery and are increasingly turning towards DevOps to improve their success and productivity by implementing a continuous delivery model.

Speed of Software Delivery and Quality

Many of them have adopted continuous integration (CI) and continuous delivery (CD) pipelines. However, they have soon realized that there is no longer a choice between the speed of delivery and quality. Organizations need both as they are key differentiating factors. Continuous testing as an integral part of the pipelines can bring back focus on quality in a fast-paced delivery model. This great blog by Marc Hornbeek on “Continuous Testing: The Quest for Quality at Speed” sums it up well.

These five tenets from Hornbeek:

  • Shift Left: Conduct each test as early in the pipeline as possible

form the basis for any team looking to really leverage continuous test to drive quality at speed. Now how does this apply to DevOps pipelines and the automation around that?

Continuous testing ensures to deliver quality into the DevOps pipeline by constantly running manual and automation tests. Continuous testing saves cost, time and resources if implemented appropriately in pipelines, with the added benefit of advancing DevOps pipeline maturity. It provides continuous feedback to stakeholders in every stage of pipelines to take necessary actions or decisions. This will be realizable only if the culture in an organization follows Marc’s five tenets and places a priority on that, a priority that does not wax and wane over time.

However, there are quite a few challenges in implementing Continuous Testing in Pipelines:

  • Best of breed tools: no single solution provides support for a complete toolchain to build CI /CD pipeline

Best practices that can help in implementing and improving quality throughout the development and release cycles are below:

  • Adopt Lean testing: Collaborate with business and other stakeholders to build meaningful and important end-to-end test scenarios.

The ReleaseIQ platform simplifies the creation of an effective continuous test pipeline in three easy steps and is designed to unify teams across the value chain by providing end-to-end visibility and actionable information for all stakeholders in every stage of the pipeline. Our platform provides flexibility to optimize pipelines adhering to best practices while preserving the history of previous runs and pipeline definitions.

Continuous Test Support in the ReleaseIQ Platform

The steps for creating pipelines for continuous testing in ReleaseIQ are:

1. Continuous Test Support in the ReleaseIQ Platform

  • Select tools and provide details for secured ReleaseIQ agent access.
  • Alternatively, plugins can be developed to convert Test results to ReleaseIQ native format as per ReleaseIQ XML schema and deployed to process the Test results.

2. Compose Pipelines

  • Create CI or CI/CD pipelines as needed to embed quality assurance processes in DevOps pipelines for continuous testing and feedback.
  • Set email notifications on test success, failure or both.

3. Monitor & Troubleshoot Pipelines

  • Monitor pipeline status and lead time with personas-based perspective views.
  • Test summary of each Test suite can be viewed by clicking the Testsuite bar. Test start time, duration, test case details, and associated logs are also available.
  • In case of test failures, Troubleshoot with logs available in ReleaseIQ UI that help to resolve issues quickly to avoid delay in releases.
  • File bugs in Jira for issues for tracking them to closure. This can be done directly from ReleaseIQ UI or Jira UI.
  • Once the Manual Test is marked complete the pipeline progresses based on the fault tolerance specified.
  • Heat Map View

The end result of creating these pipelines is a seamless transition from CI/CD to more quality-focused CI/CD delivery model, with no additional effort in providing end-to-end visibility of pipelines and feedback to all stakeholders in an organization.

Want to learn more about our integrations and how to configure, run and gain unified visibility for all your pipelines, watch our recent webinar or view our product brochure.

#prodmgmt and marketing exec/consultant on strategy/execution, author, coach, 41k miles cyclist

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store