How to Make a Quality Test Plan

The audience of the test plan artifact can vary from organisation to organisation. A test plan with an internal project team as a target audience will be detailed compared to the one produced for external regulator’s consumption (you may decide to keep both artifacts similar, however many times in practice, that isn’t the case). So in this paper, I will explain the details of a test plan and how to make the document useful and efficient.

Irrespective of what methodology is followed within each organisation, a test plan detailing how the overall product or service will be tested and delivered is a key artifact. The delivery can be a combination of internal-only applications OR internal and 3rd party vendor-based applications. In either case, it is important to outline how testing will be conducted for internal applications, 3rd party applications, integration between applications, etc…

We can expand this further by considering the example of agile methodology which an organisation may choose to follow – Agile because this is now the most popular way to develop software.

This methodology, as everyone knows, allows you to deliver in small chunks rather than waiting for a big-bang implementation. Refined user stories are a key part of agile methodology.

Let’s jump into test phases e.g. In Sprint testing. After reviewing scope /user stories, try and define the following as a minimum:

 

1. Type of Testing required

  • Functional & integration testing
  • Nonfunctional testing
  • User Acceptance Testing

Consider the below points while describing the type of testing:

    • Scope of testing
    • How new functionality being delivered will be tested
    • data requirements

 

2. Clarity of acceptance criteria

  • Ensure test acceptance criteria are clearly defined
  • Define your test acceptance based on acceptance criteria from user stories
  • Ensure acceptance criteria are reviewed and agreed upon by users

 

3. Test Environment

In many organisations, environments are built by different departments. So, it is vital to define environmental requirements clearly. A meeting to discuss and ensure that your requirements are clear, is always useful:

  • Which systems are required to conduct testing?
  • What data is required to conduct testing?
  • Any other critical activities you expect to be supported (batch runs, monitoring, alerting, etc…)

 

3rd Party product testing

In case your organisation uses 3rd party vendor products, you will probably be getting releases from vendors. In most cases, you won’t have a say in how ‘vendor sprints’ should operate.

In this case, from a test planning point of view, please refer above points to define the plan first.

As a client of receiving organisation, it is very important to ensure that releases you have received are always working as expected. There has to be a way and criteria the releases should meet for further acceptance into the organisation because you don’t want to spend time/effort in deploying releases across different environments and then find out basics do not work.

One way to deal with this is to ensure there is a pre-defined regression pack that can be executed speedily (in hours) which validates key functionality of the release.

Even better, if such test pack can be executed at vendor side on environment config same as yours. You, together with a vendor can decide what is the minimum quality criteria for release acceptance? These upfront steps will save a lot of time/effort and cost.

 

Regression testing

For me, this is one of the most important aspects of testing. It is MUST ensure (by validating) previously working ‘un impacted’ functionality is actually working. Any features/user stories/requirements are tested in isolation across different sprints cannot guarantee if previously working functionality is still going to work and hence it is extremely important to have a regression pack that validates at a minimum key pillar of the service. You should invest to automate such a regression pack and ensure it is run as soon as new code is checked-in or at least on a daily basis.

Defining the above key points is applicable for most of the individual test phases.

After coming out of sprint testing or testing of individual releases from 3rd party vendors, the next step would be to integrate the components and validate the functionality, interfaces end to end.

 

Non-functional testing and Operational Acceptance testing

Non Functional Testing

Many times you are required to validate a product or service from a nonfunctional point of view as soon as key components are sufficiently functionally tested. Typical phases conducted under non-functional testing are:

  • Performance testing
  • Load testing
  • Stress testing
  • Security testing

Additional considerations for performance, load, and stress testing are:

  • Clarity on what needs measuring
  • Ensure load patterns over the time are same as production
  • A clear understanding of user pattern
  • Infrastructure to be measured and performance monitors

Operational Acceptance Testing

Before the end-to-end service goes live, the IT production / Run team is required to ensure they can ‘run’ the service (familiarisation) and act in case there are failures. so it is important to test monitoring and alerting, validate if issue correction procedures are correct, and other support aspects of the service.

Additionally, many times the following is covered in this phase:

  • High availability
  • Failover & Disaster recovery

You may want to consider some additional (to the above) key points while considering integration/user acceptance/end to end service testing.

Data requirements across different components

  • Lot of time is wasted by not thinking upfront
  • What the data requirements for each system is?
  • How does that data need to be in sync across systems?
  • How to validate the setup to ensure it is ready for detailed testing?

Ensure there is enough time spent on this upfront, else every time integration testing is run, there will be a wastage of efforts, delays in conducting actual testing. Additionally, there could be false defects just because of data rather than functionality.

Test Environment and Support requirements

In many organisations, as code moves from ‘lower environments’ to ‘higher environments’, stricter controls are applied. i.e. only certain groups of people are allowed to do certain things. In such cases, make sure there is an upfront request in place for support.

Run book of daily key activities

This is another powerful artifact. All the key activities to be performed, by who, at what time, etc… can be determined in the runbook. This also can become a status reporting mechanism when such a test phase is running.

Testing sign-off

This is a very important part of project delivery so having an upfront defined and agreed criteria always helps. Liaising with various key stakeholders to define such criteria is key. Reminding the project team of these criteria is also essential as many times during go – no go meetings, such criteria get discussed which sometimes results in unnecessary noise which can easily be avoided.

You should always tweak the overall document as per your organisation, project need. Whatever is listed in this paper should be considered as a guide.

 

Written by Samir Patki, Head Of QA at European Central Counterparty N.V.

Privacy Overview
Software Testing News

When you visit any website, it may store or retrieve information on your browser, mostly in the form of cookies. This information might be about you, your preferences or your device and is mostly used to make the site work as you expect it to. The information does not usually directly identify you, but it can give you a more personalized web experience. Because we respect your right to privacy, you can choose not to allow some types of cookies. Click on the different category headings to find out more and change our default settings. However, blocking some types of cookies may impact your experience of the site and the services we are able to offer.

Strictly Necessary Cookies

Strictly Necessary Cookie should be enabled at all times so that we can save your preferences for cookie settings.

Performance Cookies

These cookies allow us to count visits and traffic sources so we can measure and improve the performance of our site. They help us to know which pages are the most and least popular and see how visitors move around the site. All information these cookies collect is aggregated and therefore anonymous. If you do not allow these cookies we will not know when you have visited our site, and will not be able to monitor its performance.

Targeting Cookies

These cookies may be set through our site by our advertising partners. They may be used by those companies to build a profile of your interests and show you relevant adverts on other sites. They do not store directly personal information, but are based on uniquely identifying your browser and internet device. If you do not allow these cookies, you will experience less targeted advertising.