Software Testing and Quality Assurance


Every software development group tests its products, yet delivered software always has defect. Test engineers (at KUDO, Test Engineer == QA Analyst plus QA Engineer) strive to catch them before the product is released but they always creep in and the often reappear. While, software testing is a huge domain, but it can be broadly categorized into two areas: white box testing and black box testing.


White box testing is a software testing method in which the internal structure/design/implementation of the item being tested is known to the tester, mainly applicable to  lower levels of testing: unit testing and integration testing. While, black box testing (also know as functional testing) threats software under test as a black-box without knowing its internal (for clearer explanation, we can re-read Mr. Herman Wahyudi’s post on November 26, 2016: White Box Testing and Black Box Testing).


Adaptive Random Testing (ART)

  • ART is a solution to replace exhaustive testing method, which by using ART, test case can be selected adapted with testing objectives.
  • ART, an enhancement of Random Testing, aims to both randomly select and evenly spread test case. In this article, ART is going to be prepared with sample program with 1000 iterations to find the potential failure area so the testing can be more efficient without more execution time consuming.

ART is for non-point failure pattern – an even spread of random test cases will enhance the fault detection capabilities. When failure causing inputs cluster together into contiguous  regions, an even spread of random test cases will enhance the fault detection capability of the random testing.

Possible approaches of achieving “even spread of test cases”:

Distance → Fixed Size Candidate Set (FSCS) ART, it finds the furthest random input to be the next test case.

Exclusion → Restricted ART, sets an exclusion region around each executed test case, and avoid the next test case falling in any exclusion region.

Dynamic Partition → Bisection ART, partition the input domain and force the next test case to be in an empty partition.

And now, I’d like to ask you to discuss about Distance approach (FSCS ART).


Fixed Size Candidate Set Adaptive Random Testing (FSCS ART)

FSCS ART aims to find the furthest candidate from the executed test cases with fixed size of candidate set. An executed test case is a set of test cases that have been executed but not revealed the failure and the candidate test case is a number of test cases generated randomly as candidates of test cases.


  1. Select a test case randomly
  2. Execute it
  3. If it reveals a failure, stop. Otherwise, add it into executed test cases
  4. Select N test cases of candidate sets randomly
  5. Calculate the distance of each candidate set to executed test cases
  6. Select a candidate set having the largest distance (one way: using max-min criteria)
  7. Go to step-2


Adaptive Random Testing Sample Program

The program below is the project made for Software Testing and Quality Assurance course. This program is written as .cpp file.


#include <bits/stdc++.h>
using namespace std;
int main(){
    int x1,x2,y1,y2;
    int batas = 10;
    x1 = 10; x2 = 60; y1 = -22; y2 = -72;
    int Ex[10000], Ey[10000];
    int Cx[10], Cy[10];
    int index = 0;
    int counter = 0;
    double fMeasure;
    clock_t tStart = clock();
    /* Create File TXT */
    ofstream fileTemp;"ART_ouput.txt");

    fileTemp <<"FSCS-ART EXPERIMENT\n\n";
    /* Iteration as many as batas value (1000) */
    for(int i=0; i<1000; i++){
      /* Program is running */
        printf("Program is running.......");
   //     printf("---------- Round %d ----------\n",i+1);
        fileTemp << "--------------------------------------------------\n";
        fileTemp << "\t\t Round "<< i+1 <<"\n";
        fileTemp << "--------------------------------------------------\n";
        int ct;
        /* Generate Random Value for E1 on Round */
        ct = 0;
     //   printf("\tNilai E1 = (%d,%d)\n", Ex[0],Ey[0]);
        fileTemp << "E-1\t= (" << Ex[0] <<","<<Ey[0]<<")\n";

        /* Iteration before failure is found */
        while(((Ex[ct]<x1 || Ex[ct]>x2) || (Ey[ct]>y1 || Ey[ct]<y2))){
            /* Generate Random C */
                for(int p=0; p<;batas; p++){
                    Cx[p] = rand()%501+(-250);
                    Cy[p] = rand()%501+(-250);
                    //printf("Nilai C-%d : (%d,%d)\n",p+1,Cx[p],Cy[p]);

                /* Kondisi Jika Masih Hanya Ada 1 E */
                int a,b,sisiC[batas], tempC[batas],iMax, maks;
                int *cari;
                if(index == 0){
                    for(int p=0; p<batas; p++){
                        a = Cx[p] - Ex[index];
                        b = Cy[p] - Ey[index];
                        a = pow(a,2);
                        b = pow(b,2);
                        sisiC[p] = sqrt(a+b);
                        tempC[p] = sisiC[p];
                     //   printf("Jarak dari C-%d ke E-%d = %d\n",p,index,sisiC[p]);
                    maks = sisiC[batas-1];
                    cari = find(sisiC,sisiC+batas,maks); // cari bilangan terbesar pada array C
                    iMax = distance(sisiC,cari); //cari index dari bilangan terbesar
                    Ex[ct] = Cx[iMax];
                    Ey[ct] = Cy[iMax];
//                    printf("\tNilai E-%d = (%d,%d)\n",ct+1,Cx[iMax],Cy[iMax]);
                    fileTemp <<"E-"<<ct+1<<"\t= ("<<Cx[iMax]<<","<<Cy[iMax]<<")\n";
                } else{
                    /* Loop sebanyak jumlah C */
                    int A[ct], B[ct], C[ct],T[ct]; //penampung sisi dan jarak setiap C ke semua E
                    for(int c=0; c<batas; c++){
                        /* Loop sebanyak nilai E */
                        for(int d=0; d<ct; d++){
                            A[d] = Cx[c] - Ex[d];
                            B[d] = Cy[c] - Ey[d];
                            a = pow(A[d],2);
                            b = pow(B[d],2);
                            C[d] = sqrt(a+b);
                            T[d] = C[d];
                        sisiC[c] = T[0];
                        tempC[c] = sisiC[c];
                     //   printf("Jarak Minimum untuk C-%d = %d\n",c+1,sisiC[c]);
                    /* Cari Nilai Max dari semua nilai minimum C*/
                    maks = tempC[batas-1];
                    cari = find(sisiC,sisiC+batas,maks); // cari bilangan terbesar pada array C
                    iMax = distance(sisiC,cari); //cari index dari bilangan terbesar
                    Ex[ct] = Cx[iMax];
                    Ey[ct] = Cy[iMax];
  //                  printf("\tNilai E-%d = (%d,%d)\n",ct+1,Cx[iMax],Cy[iMax]);
                    fileTemp <<"E-"<<ct+1<<"\t= ("<<Cx[iMax]<<","<<Cy[iMax]<<")\n";
        } // end of while
   //     printf("\nFailure pada titik E-%d : (%d,%d)---\n",ct+1,Ex[ct],Ey[ct]);
        fileTemp << "\nFailure pada Round "<< i+1<<" berada pada titik E-"<<ct+1<<" : ("<<Ex[ct]<<","<<Ey[ct]<<")\n\n";
        counter += (ct+1);
    } // end of loop batas
    fMeasure = (double)counter/1000;
    fileTemp <<"---------------------------------------------------------------------------------------\n";
    fileTemp << "Conclusion : \n";
    fileTemp << "1. F-Measrure = "<<fMeasure << "\n";
    fileTemp << "2. Execution Time : "<< (double)(clock() - tStart)/CLOCKS_PER_SEC <<" second(s)\n";
    fileTemp <<"---------------------------------------------------------------------------------------";
    printf("\rExecuting program successfully.");
 //   printf("F-Measure = %.2f\n",fMeasure);
    return 0;

                                                           /* credit to: */




You Might Also Like

No Comment

Leave your thought