Open webinar "Pairwise Testing Method in Black Box Testing"
Good day to all!
We offer to your attention a detailed description of the open lesson devoted to pair testing. The event took place only a couple of days ago and was timed to the launch of the “QA-specialist” course.
Lecturer - Nina Devayeva - Senior Tester, Team Leader and ISTQB-certified tester, an expert in the direction of Quality Assurance.
At the open lesson we talked about the need for this type of test-design technique, as pairwise testing. We studied case studies in practice and examined in detail the tools available for work.
Before the start of the webinar set the following goals:
A few words about the test design
A test design is a stage in the software testing process at which test cases (test cases) are designed and created in accordance with previously defined quality criteria and testing objectives. Some have difficulty asking them why they need a test design. Meanwhile, the answer lies in one of the principles of testing, namely: exhaustive testing is unattainable . And this is true with the exception of any trivial cases where input data is very small. Moreover, by and large, few budgets will “pull” all possible checks that can cover our product.
Black Box Testing
A well-known method that does not require lengthy explanations. In short, black box testing is functional or non-functional testing that is performed without knowing the internal structure of the component or system. The method is based on working exclusively with the external interfaces of the system under test.
Techniques of test design using the black box method include:
If we talk about pairwise testing, then there are studies that show that most of the bugs occur when a combination of 2 of any parameters. That is why, in conditions of limited resources and with a huge amount of input data, it makes sense to use pairwise testing.
So what is pairwise testing?
Pairwise testing is the development of black box tests, in which test scripts are designed to perform all possible separate combinations of each pair of input parameters. Strictly speaking, pair-wise testing allows us to save a lot of time.
For pairwise testing, algorithms based on the construction of orthogonal arrays or the All-Pairs algorithm are used, which are based on theoretical studies in the field of combinatorial algorithms, discrete mathematics algorithms and, in particular, Latin squares. Let's look at these algorithms in more detail.
Testing using an orthogonal array
Orthogonal array testing is a systematic approach to testing all paired combinations of variables using orthogonal arrays. This approach significantly reduces the number of combinations of variables when checking all paired combinations.
An orthogonal array is essentially a table, where m is the number of rows, n is the number of columns, which corresponds to the number of input parameters, k is the number of options for the values of table elements. The table has the following properties:
For example:
- this is an orthogonal array with four rows and three columns (by the number of variables). The number 2 means that all variables take only two values - 1 and 2.
For example, our application has 3 input parameters, and each binary (takes the value "1" or "2"). Thus, all possible combinations of input data can be represented as:
For clarity, let's assume that we have the Flashlight application, which:
In general, we have three parameters that take binary values.
Now let's see how our sample will look like after transfer to an orthogonal array:
As we see, we have reduced the number of cases from 8 to 4 with 3 different parameters taking binary value. This is a real gain, which will have a positive impact on the budget and on the resources that we use.
All-Pairs Algorithm
The All-Pairs Algorithm is a combinatorial technique that was specially created for pair testing. It is based on the choice of possible combinations of the values of all variables, which contain all possible values for each pair of variables. Based on the definition, the number of combinations will be less than when using orthogonal arrays.
When testing using the All-Pairs algorithm, perform the following steps:
Also, it should be noted that the orthogonal arrays and the All-Pairs Algorithm are used with corrections for the prefetch of the input data.
Tulses for pairwise testing
In order not to waste time on converting all the data, you can and should use tools for pairwise testing. Here are some of them:
Of course, there are Allpairs and VPTag, but talking about them went beyond the last webinar.
Practice and practice again
Now let's try how these tools work in practice. So, try to complete two simple tasks:
On this, perhaps, everything. You can get acquainted with the nuances of pairwise testing by looking at the webinar in full. And, yes, do not miss the Open Day of the QA Specialist course!
We offer to your attention a detailed description of the open lesson devoted to pair testing. The event took place only a couple of days ago and was timed to the launch of the “QA-specialist” course.
Lecturer - Nina Devayeva - Senior Tester, Team Leader and ISTQB-certified tester, an expert in the direction of Quality Assurance.
At the open lesson we talked about the need for this type of test-design technique, as pairwise testing. We studied case studies in practice and examined in detail the tools available for work.
Before the start of the webinar set the following goals:
- find out what pair testing is like;
- find out when it is permissible to use pairwise testing;
- learn how to work with pairwise testing tools.
A few words about the test design
A test design is a stage in the software testing process at which test cases (test cases) are designed and created in accordance with previously defined quality criteria and testing objectives. Some have difficulty asking them why they need a test design. Meanwhile, the answer lies in one of the principles of testing, namely: exhaustive testing is unattainable . And this is true with the exception of any trivial cases where input data is very small. Moreover, by and large, few budgets will “pull” all possible checks that can cover our product.
Black Box Testing
A well-known method that does not require lengthy explanations. In short, black box testing is functional or non-functional testing that is performed without knowing the internal structure of the component or system. The method is based on working exclusively with the external interfaces of the system under test.
Techniques of test design using the black box method include:
- equivalence classes;
- boundary value analysis;
- decision tables;
- state change diagrams;
- pairwise testing.
If we talk about pairwise testing, then there are studies that show that most of the bugs occur when a combination of 2 of any parameters. That is why, in conditions of limited resources and with a huge amount of input data, it makes sense to use pairwise testing.
So what is pairwise testing?
Pairwise testing is the development of black box tests, in which test scripts are designed to perform all possible separate combinations of each pair of input parameters. Strictly speaking, pair-wise testing allows us to save a lot of time.
For pairwise testing, algorithms based on the construction of orthogonal arrays or the All-Pairs algorithm are used, which are based on theoretical studies in the field of combinatorial algorithms, discrete mathematics algorithms and, in particular, Latin squares. Let's look at these algorithms in more detail.
Testing using an orthogonal array
Orthogonal array testing is a systematic approach to testing all paired combinations of variables using orthogonal arrays. This approach significantly reduces the number of combinations of variables when checking all paired combinations.
An orthogonal array is essentially a table, where m is the number of rows, n is the number of columns, which corresponds to the number of input parameters, k is the number of options for the values of table elements. The table has the following properties:
- any two columns of the table contain all combinations of values of these columns;
- if any pair of values of two columns occurs several times, then all possible pairwise combinations of the values of these columns should occur as many times.
For example:
- this is an orthogonal array with four rows and three columns (by the number of variables). The number 2 means that all variables take only two values - 1 and 2.For example, our application has 3 input parameters, and each binary (takes the value "1" or "2"). Thus, all possible combinations of input data can be represented as:
For clarity, let's assume that we have the Flashlight application, which:
- works with iOS and Android;
- has a night and daylight mode;
- allows you to shine constantly or blink in strobe mode.
In general, we have three parameters that take binary values.
Now let's see how our sample will look like after transfer to an orthogonal array:
As we see, we have reduced the number of cases from 8 to 4 with 3 different parameters taking binary value. This is a real gain, which will have a positive impact on the budget and on the resources that we use.
All-Pairs Algorithm
The All-Pairs Algorithm is a combinatorial technique that was specially created for pair testing. It is based on the choice of possible combinations of the values of all variables, which contain all possible values for each pair of variables. Based on the definition, the number of combinations will be less than when using orthogonal arrays.
When testing using the All-Pairs algorithm, perform the following steps:
- similarly as for orthogonal arrays, they define a table of all variables and their values;
- leave in the table only all possible unique combinations of pairs of variable values.
Also, it should be noted that the orthogonal arrays and the All-Pairs Algorithm are used with corrections for the prefetch of the input data.
Tulses for pairwise testing
In order not to waste time on converting all the data, you can and should use tools for pairwise testing. Here are some of them:
- pairwise.teremokgames.com is a site with a clear interface that does not require specific knowledge.
- PICT is a free tool developed by Microsoft for Pairwise Testing. Downloaded from the following link .
Of course, there are Allpairs and VPTag, but talking about them went beyond the last webinar.
Practice and practice again
Now let's try how these tools work in practice. So, try to complete two simple tasks:
- Using pairwiseTool, select the combinations of initial parameters and drop the link to the screenshot of the result in the comment. It takes a conditional site that should open on Win 7, Win 8 and Win 10. Supported browsers are Google Chrome, Opera, Microsoft Edge, Mozilla Firefox, Yandex Browser. Users may or may not use AdBlock.
- Using the PICT program, make a selection of combinations of initial parameters and discard the link to the screenshot of the result in the comments. It is necessary to carry out configuration testing with the following components:
- video cards: GeForce GT 730, GeForce GT 1030, GeForce GTX 1080, GeForce RTX 2070;
- Processors: Intel Core i5, Intel Core i7, AMD Ryzen 7, Intel Core i9;
- Memory: 8GB, 16GB.
On this, perhaps, everything. You can get acquainted with the nuances of pairwise testing by looking at the webinar in full. And, yes, do not miss the Open Day of the QA Specialist course!
Source: https://habr.com/ru/post/437980/