Category: WHITE BOX Testing

Mutation testing is a kind of testing in which, the application is tested for the code that was modified after fixing a particular bug/defect. It also helps in finding out which code and which strategy of coding can help in developing the functionality effectively. You can refer the Mutation testing Pdf training tutorials after reading this tutorial.

Mutation testing ensure the accuracy of our unit tests.In some cases our unit tests will be successful and won’t detect any bugs or defects in our source code, but that may be not true. In this situations, we will do some alteration in source code (mutating) for making errors and again will run those unit test.Now if our unit tests are perfect then it will catch the defect in mutated code.Otherwise we can understand that our unit tests are not proper. So simply we can say that Mutation Testing is a testing of our tests.

Basics

Mutation testing is a technique by which test examples can be improved. The term mutation refers to the change in the program.

Mutation is an operation by which we make small changes in code of a program. If we mark the original program as P, M is a mutant of that program, and P is a parent of program M. Given the fact that program P must be syntactically accurate, the M program must be as well. It is also possible to make a mutant by adding new code in the original program.

Sorts of mutations

Mutants that have only one change in relation to tested program are called first-order mutants.

Second-order mutants are formed by making two simple changes in a program and also from the first-order mutants, while third-order mutants are formed by making three simple changes in a program and so on.

With exception of first-order mutants, all the others are called higher-order mutants.

So, what is mutation testing?

Mutation testing is by definition a criterion of testing: the tests are either generated by accident until the enough number of mutations are killed, or the tests are designed to purposely kill surviving mutations. In the latter case, testing of mutations can be categorized as a code based technique. For this technique to be effective, a great number of mutants must be produced in a systematic way. If the testing is successful in determining a difference between a program and a mutant, the another one is killed.

Mutation score

Depending on the set of test examples we choose and programs we test, we can calculate mutation score. This score represents a quantified estimation of the quality of the set of test examples. It is a number between 0 and 1. The score 1 means that the set of test examples is adequate, while the score less than 1 represents that this set is inadequate.

Inadequate set of tests can be improved by adding tests examples that can make the mutation result better.

Some more facts about Mutation testing

• Richard Lipton is the man who introduced this testing first.
• Timothy Budd implemented the first Mutation testing tool.
• Mutation testing will be very expensive for large applications without using any tools.
• While mutating the source code, normally small changes on variables and operands are performing.

PDF Tutorials

• Mutation testing pdf tutorial
• Topics in Mutation Testing and Program Perturbation

Security Testing is carried out in order to find out how well the system can protect itself from unauthorized access, hacking – cracking, any code damage etc. which deals with the code of application. This type of testing needs sophisticated testing techniques. After reading this tutorial refer the advanced pdf tutorials about Security testing in software development

In this non-function testing all type of malicious attempts will be simulated against the application to find the loopholes in our application.Security test is a part of the higher level group of tests. It determines the vulnerability of the system.By testing a security, we can protect resources and data from eventual  intruders. The goal is to determine how the system reacts to unauthorized attacks.

The process requires active system analysis because of eventual gaps that can derive from a poor configuration of the system. And from both known and unknown software and hardware differences.

This analysis is done from the position of a potential intruder and it can include active exploitation of security failures.

Security matters found through this testing are presented to the owner of the system. Effective security tests can estimate possible influences on the organization. And to suggest numerous technical and procedural measures lower the risk.

 Security testing of applications

Security testing is a discipline of testing and analyzing applications for security vulnerabilities. It’s a hugely important phase in the application lifecycle no matter if you are building new applications or updating existing ones.

It’s also of the utmost importance to carry out application security testing and if you don’t there will be consequences for business.

How to face the threats

There are many sorts of vulnerabilities. Just as there are many sorts of threats. So the question arise: “How to respond to these threats?“

Because of this variety of threats, it is important to monitor latest trends and methods used by the attackers.

Security tests show that more than a half of all exploits for web applications are actually related to cross-site scripting and SQL injection vulnerabilities.

IT departments from all over the world are under a lot of pressure from their businesses to deliver new applications and services. Therefore, it isn’t rare that security is delivered just as an afterthought at the end of the project.

Because of that, where there is a larger range of security issues it is important to integrate security framework.

PDF tutorials

• Software Security Testing – pdf tutorial

What is Branch Coverage Testing

No software application can be written in a continuous mode of coding, at some point we need to branch out the code in order to perform a particular functionality. Branch coverage testing helps in validating of all the branches in the code and making sure that no branching leads to abnormal behavior of the application.

Test requirements and coverage measure

Branch coverage is best described in terms of test requirements and coverage measure. The test requirements for branch coverage are the branches of the program. In other words, the goal of this coverage is to execute all of the branches in the program. On the other hand, coverage measure is defined accordingly as the number of branches executed by your test cases over the total number of branches in the program. Branches are the outgoing edges from a decision point. Therefore, an if statement or a switch statement.

However, 100% coverage does not provide any guarantee of finding the problems in the code.

The fact that by testing more thoroughly we have more chances of finding a problem in the code. But it doesn’t matter which kind of coverage we utilize and how much coverage we achieve. There’s always a chance that we might miss something.

Branch coverage vs statement coverage

When these two metrics are involved, it’s of a high importance to talk about test criteria subsumption. One test criterion subsumes another criterion when all the tests widths that satisfy those criteria will also satisfy the other one. This is found between branch and statement coverage.

If we identify a test width that achieves 100% branch coverage, the same test width will also achieve, necessarily, 100% statement coverage. That’s what happens in general because branch coverage is a stronger criterion than statement coverage. There is no way to cover all the branches without covering all the statements. It’s not true that any test result satisfies statement coverage will also satisfy branch coverage.

The Conclusion

In general, it’s more expensive to achieve branch coverage than to achieve statement coverage, because achieving  branch coverage requires the generation of a larger number of test cases.

So what this relation means is that branch coverage is stronger than statement coverage, but also more expensive.

Pdf tutorials

• Test data generation for branch coverage testing

In Statement Coverage testing the code is executed in such a manner that every statement of the application is executed at least once. It helps in assuring that all the statements execute without any side effect.This method is also called as line coverage or segment coverage. It is one of the testing coming under white box testing.See the Pdf tutorials about Statement Coverage.

In statement coverage testing we make sure that all of our code blocks are executed. We can also identify which blocks are failed to execute in this method.

Still bugs may be there after executing all the blocks without any failure. Because it wont check with all the conditions in a single block. Its only a basic testing after the complete coding and dynamic analysis. For checking with every conditions we need to to branch and path coverage testing.

What is Statement Coverage

Statement coverage represents a specific coverage criterion of white box testing. And it is characterized by 2 aspects:

1. Which are the best test requirements for the criteria?
2. How you measure the coverage for this criteria?

When it comes to statement coverage these test requirements are all the statements in the program. So, this is the basic and the simplest coverage criteria in the white box testing. And, as you know, white box testing is based on the assumption that if there isn’t a faulty element in the code, you need to execute it in order to find the faulty. And that’s exactly what statement coverage does.

The Ratio

A good measure of how well you exercise the code is the ratio of the number of executed statements. So, all the statements that your test cases execute to the total number of statements in the program.

The higher this number, the better you exercise your code.

Also, another way to look at this coverage criterion is in terms of questions. So, what is the question they were trying to answer when you look at a specific set of test cases and you assess the statement coverage that they achieved? It is whether each statement in the program has been executed. So, the statement coverage is satisfied when all the statements in the program have been executed.

And you can satisfy it to different degrees and the degrees to which it’s satisfied is measured by this value.

How statement coverage is used in practice?

Statement coverage  is the most used kind of coverage criterion in industry. Normally for a company that uses statement coverage the typical coverage target is 80-90%, which means the outcome of the test should be such that 80-90% of the statements are exercised at the end of the testing.

There are several free tools available for conducting statement coverage testing.

Pdf Tutorial

• Efficient Online Computation of Statement Coverage tutorial

Static analysis involves going through the code in order to find out any possible defect in the code. Dynamic analysis involves executing the code and analyzing the output. After reading this tutorial refer the more detailed pdf tutorials about Static & Dynamic Analysis.

Static analysis : This may be the  testing  you are doing most of the time at your coding. While coding there may be a lot of typing errors, syntax error, loop structure, code termination etc etc . This should be fixed by inspecting ( thorough reading ) of your code. You program will run only after clearing all the coding defects by static analysis.
Dynamic analysis : Now you need to check your program output whether it is the desired output or not. This is called as  dynamic  analysis in testing. You will compile the program and check the output, then will do the necessary changes in codes .

Static & Dynamic Analysis

Static analysis is done in a non-runtime environment which is just when the program is not running at all. So, any kind of static analysis tool that is used will look at the code and will look at the run-time behaviors to find any kind of flaws, back door and bad code.

In contrast, dynamic analysis is done just when the program is running. A dynamic test will monitor system memory, function behavior, response time and overall performance of the system.

The strengths and weaknesses of static and dynamic analysis

For static analysis strengths are:

• A more thorough approach and more cost-efficient
• It is able to find future errors that would not be detected in dynamic analysis
• It can point out the exact spot in code where there is an error, so you can easily fix it

However, there is one big weakness of static analysis:

You may think it’s a better method than dynamic analysis, but the dynamic analysis is equally important. Because the static analysis is not going to catch some of the run-time errors  that dynamic analysis would.

The strengths of dynamic analysis are:

• It identifies errors while program is running
• It can analyze code that can’t be accessed

The weaknesses are:

• It takes longer to fix errors because it can’t exactly pinpoint the errors like static analysis can.
• Automated tools that dynamic analysis has don’t do everything, they just do what they are programmed to do, so it can create a false sense of security

Examples of analysis tools

– Findbugs:  It will report any potential issues, warnings will pop up, but not all of them are actually bugs, they are just spot points in the code that can possibly go wrong, so that helps you minimize the errors in your code.

– Veracode:Analysis tool that actually has both static and dynamic analysis. It is an online tool, whereas Findbugs is a tool that you can download.


PDF tutorials

• Combined Static and Dynamic Automated Test Generation
• Program Analysis – testing training material

What is UNIT TESTING

The developer carries out unit testing in order to check if the particular module or unit of code is working fine. The Unit Testing comes at the very basic level as it is carried out as and when the unit of the code is developed or a particular functionality is built.

Unit testing deals with testing a unit as a whole. This would test the interaction of many functions but confine the test within one unit. The exact scope of a unit is left to interpretation. Supporting test code, sometimes called scaffolding, may be necessary to support an individual test. This type of testing is driven by the architecture and implementation teams. This focus is also called black-box testing because only the details of the interface are visible to the test.

Limits that are global to a unit are tested here. In the construction industry, scaffolding is a temporary, easy to assemble and disassemble, frame placed around a building to facilitate the construction of the building. The construction workers first build the scaffolding and then the building. Later the scaffolding is removed, exposing the completed building.

Similarly, in software testing, one particular test may need some supporting software. This software establishes an environment around the test. Only when this environment is established can a correct evaluation of the test take place. The scaffolding software may establish state and values for data structures as well as providing dummy external functions for the test. Different scaffolding software may be needed from one test to another test. Scaffolding software rarely is considered part of the system. Sometimes the scaffolding software becomes larger than the system software being tested. Usually the scaffolding software is not of the same quality as the system software and frequently is quite fragile.

A small change in the test may lead to much larger changes in the scaffolding. Internal and unit testing can be automated with the help of coverage tools. A coverage tool analyzes the source code and generates a test that will execute every alternative thread of execution. It is still up to the programmer to combine this test into meaningful cases to validate the result of each thread of execution. Typically, the coverage tool is used in a slightly different way. First the coverage tool is used to augment the source by placing informational prints after each line of code. Then the testing suite is executed generating an audit trail. This audit trail is analyzed and reports the percent of the total system code executed during the test suite. If the coverage is high and the untested source lines are of low impact to the system’s overall quality, then no more additional tests are required.

PDF Tutorials

• Unit Testing in BlueJ – PDF tutorial
• Quick overview about Unit Testing
• C++ Code unit testing with sample

What is WHITE BOX TESTING

White box testing strategy deals with the internal logic and structure of the code. White box testing is also called as glass, structural, open box or clear box testing. The tests written based on the white box testing strategy incorporate coverage of the code written, branches, paths, statements and internal logic of the code etc. Refer the advantages , Disadvantages, techniques & pdf tutorials about white-box testing

In order to implement white box testing, the tester has to deal with the code and hence is needed to possess knowledge of coding and logic i.e. internal working of the code. White box test also needs the tester to look into the code and find out which unit/statement/chunk of the code is malfunctioning

 

Advantages of White box testing :

• As the knowledge of internal coding structure is prerequisite, it becomes very easy to find out which type of input/data can help in testing the application effectively.
• The other advantage of white box testing is that it helps in optimizing the code
• It helps in removing the extra lines of code, which can bring in hidden defects.

 

Disadvantages of white box testing:

• As knowledge of code and internal structure is a prerequisite, a skilled tester is needed to carry out this type of testing, which increases the cost.
• It is nearly impossible to look into every bit of code to find out hidden errors, which may create problems, resulting in failure of the application.

 

LEVELS / TECHNIQUES OF WHITE-BOX TESTING

1. UNIT TESTING
2. STATIC & DYNAMIC ANALYSIS
3. STATEMENT COVERAGE
4. BRANCH COVERAGE
5. SECURITY TESTING
6. MUTATION TESTING

 
PDF Tutorials

• Black-box Vs White-box testing