Category: BLACK BOX Testing

What is Exploratory Testing

This testing is similar to the ad-hoc testing and is done in order to learn/explore the application. It si shortly known as ET.

Exploratory software testing is a powerful and fun approach to testing. View the pdf tutorials about Exploratory Testing. In some situations, it can be orders of magnitude more productive than scripted testing. At least unconsciously, testers perform exploratory testing at one time or another. Yet it doesn’t get much respect in our field. It can be considered as “Scientific Thinking” at real time.

Exploratory testing is the act of exploring, making discoveries about the software, whether it works or not, the type of job it does and what it doesn’t do. The tester keeps making a new decision based on what are discovered during the course of the test. This is normally thought of as a black box testing technique.

What is exploratory testing?

Exploratory testing was first defined in 1984 by Cem Kaner, as “a style of software testing that emphasizes the personal freedom and responsibility of the individual tester to continually optimize the quality of his/her work by treating test-related learning, test design, test execution, and test result interpretation as mutually supportive activities that run in parallel throughout the project.”

Advantages of exploratory testing

Exploratory testing comes with some advantages which are highlighted below.

• Testers can guide their future testing by making use of deductive reasoning derived from previous results. It is not mandatory that a current series of scripted tests must be completed before advancing to explore a rich environment that is more targeted. When intelligently used, this can help to speed up the process of detecting more bugs.
• Another advantage is that it requires less preparation, it is easy to discover important bugs, and the approach has the tendency to be more intellectually stimulating than scripted tests execution at the time of execution.
• One other advantage is that most bugs are detected by exploratory testing after the initial testing has been done. It simply helps to detect bugs that escaped being seen during the initial testing.
• Exploratory testing is useful in that, when a bug is discovered, it can be used to explore the size, variations, and scope of the bug in order to supply adequate feedback to the developers.
• Exploratory testing is useful in dynamic product environment when the product is at its early stage of development, undergoing changes, or the product is unstable. This is particularly made possible because exploratory testing approach needs less documentation or no documentation at all before testing activities are commenced.
• It provides simultaneous learning. While automated tests assist in executing checks/tests swiftly, humans are able to learn from the tests conducted and come up with different test scenarios.

Disadvantages of Exploratory Testing

Exploratory testing is not without one or two disadvantages. Here are a few of the disadvantages to take note of.

• It is impossible to review in advance tests invented and performed on the fly, and it is also hard to give an idea about the exact tests that have been run.
• When revisited, freestyle exploratory test ideas are not likely to be performed in exactly the same way, making it a disadvantage if it is important that the specific details of the earlier tests are to be repeated.

Conclusion

The importance of exploratory testing cannot be overemphasized. Exploratory testing requires testers with an analytical mindset, who are able to know what to test for and how to analyze the results, even though the tests are carried out by automated software.

PDF Tutorials

• Exploratory Testing Explained
• A Tutorial in Exploratory Testing

What is Ad Hoc Testing

This type of testing is done without any formal Test Plan or Test Case creation. Ad-hoc testing helps in deciding the scope and duration of the various other testing and it also helps testers in learning the application prior starting with any other testing. It is the least formal method of testing. View pdf tutorials about Ad-hoc testing after reading all these details.

One of the best uses of ad hoc testing is for discovery. Reading the requirements or specifications (if they exist) rarely gives you a good sense of how a program actually behaves. Even the user documentation may not capture the “look and feel” of a program. Ad hoc testing can find holes in your test strategy, and can expose relationships between subsystems that would otherwise not be apparent. In this way, it serves as a tool for checking the completeness of your testing. Missing cases can be found and added to your testing arsenal. Finding new tests in this way can also be a sign that you should perform root cause analysis.

Ask yourself or your test team, “What other tests of this class should we be running?” Defects found while doing ad hoc testing are often examples of entire classes of forgotten test cases. Another use for ad hoc testing is to determine the priorities for your other testing activities. In our example program, Panorama may allow the user to sort photographs that are being displayed. If ad hoc testing shows this to work well, the formal testing of this feature might be deferred until the problematic areas are completed. On the other hand, if ad hoc testing of this sorting photograph feature uncovers problems, then the formal testing might receive a higher priority.

 
PDF Tutorials

• Basic introduction about Ad Hoc Testing
• Ad Hoc Software Testing
• How to Measure Ad Hoc Testing

What is Load Testing

The application is tested against heavy loads or inputs such as testing of web sites in order to find out at what point the web-site/application fails or at what point its performance degrades. Load testing operates at a predefined load level, usually the highest load that the system can accept while still functioning properly. View the Load testing tutorials in pdf

Note that load testing does not aim to break the system by overwhelming it, but instead tries to keep the system constantly humming like a well-oiled machine.In the context of load testing, extreme importance should be given of having large data sets available for testing. Bugs simply do not surface unless you deal with very large entities such thousands of users in repositories such as LDAP/NIS/Active Directory; thousands of mail server mailboxes, multi-gigabyte tables in databases, deep file/directory hierarchies on file systems, etc. Testers obviously need automated tools to generate these large data sets, but fortunately any good scripting language worth its salt will do the job.

 
PDF Tutorials

• Oracle Load Testing
• Best Practices for Web Application Load Testing
• Load Testing of Web Sites

What is Stress Testing

The application is tested against heavy load such as complex numerical values, large number of inputs, large number of queries etc. which checks for the stress/load the applications can withstand. Stress testing deals with the quality of the application in the environment. Read the stress testing tutorials in pdf with examples after this basic introduction.

The idea is to create an environment more demanding of the application than the application would experience under normal work loads. This is the hardest and most complex category of testing to accomplish and it requires a joint effort from all teams. A test environment is established with many testing stations. At each station, a script is exercising the system. These scripts are usually based on the regression suite. More and more stations are added, all simultaneous hammering on the system, until the system breaks. The system is repaired and the stress test is repeated until a level of stress is reached that is higher than expected to be present at a customer site.

Race conditions and memory leaks are often found under stress testing. A race condition is a conflict between at least two tests. Each test works correctly when done in isolation. When the two tests are run in parallel, one or both of the tests fail. This is usually due to an incorrectly managed lock. A memory leak happens when a test leaves allocated memory behind and does not correctly return the memory to the memory allocation scheme. The test seems to run correctly, but after being exercised several times, available memory is reduced until the system fails.

 
PDF Reference tutorials with example

• Stress-Testing General Purpose Digital Library Software

What is Functional Testing

In this type of testing, the software is tested for the functional requirements. The tests are written in order to check if the application behaves as expected. Although functional testing is often done toward the end of the development cycle, it can—and should, —be started much earlier. Refer the Pdf tutorials about Functional testing also after reading this introduction. Individual components and processes can be tested early on, even before it’s possible to do functional testing on the entire system.

Functional testing covers how well the system executes the functions it is supposed to execute, including user commands, data manipulation, searches and business processes, user screens, and integrations. Functional testing covers the obvious surface type of functions, as well as the back-end operations (such as security and how upgrades affect the system).

 
PDF Tutorials

• Automating Functional Tests Using Selenium – pdf guide

What is Black Box Testing | Advantages | Disadvantages | Black box testing techniques

What is Black Box Testing

Black Box Testing is testing without knowledge of the internal workings of the item being tested. For example, when black box testing is applied to software engineering, the tester would only know the “legal” inputs and what the expected outputs should be, but not how the program actually arrives at those outputs.

It is because of this that black box testing can be considered testing with respect to the specifications, no other knowledge of the program is necessary. For this reason, the tester and the programmer can be independent of one another, avoiding programmer bias toward his own work. For this testing, test groups are often used, “Test groups are sometimes called professional idiots…people who are good at designing incorrect data.” 1 Also, do to the nature of black box testing, the test planning can begin as soon as the specifications are written. The opposite of this would be glass box testing, where test data are derived from direct examination of the code to be tested. For glass box testing, the test cases cannot be determined until the code has actually been written. Both of these testing techniques have advantages and disadvantages, but when combined, they help to ensure thorough testing of the product.
Synonyms for black-box include: behavioral, functional, opaque-box, and closed-box.

Advantages of Black Box Testing

• more effective on larger units of code than glass box testing
• tester needs no knowledge of implementation, including specific programming languages
• tester and programmer are independent of each other
• tests are done from a user’s point of view
• will help to expose any ambiguities or inconsistencies in the specifications
• test cases can be designed as soon as the specifications are complete

Disadvantages of Black Box Testing

• only a small number of possible inputs can actually be tested, to test every possible input stream would take nearly forever
• without clear and concise specifications, test cases are hard to design
• there may be unnecessary repetition of test inputs if the tester is not informed of test cases the programmer has already tried
• may leave many program paths untested
• cannot be directed toward specific segments of code which may be very complex (and therefore more error prone)
• most testing related research has been directed toward glass box testing

Testing Strategies/Techniques

• black box testing should make use of randomly generated inputs (only a test range should be specified by the tester), to eliminate any guess work by the tester as to the methods of the function
• data outside of the specified input range should be tested to check the robustness of  the program
• boundary cases should be tested (top and bottom of specified range) to make sure the highest and lowest allowable inputs produce proper output
• the number zero should be tested when numerical data is to be input
• stress testing should be performed (try to overload the program with inputs to see where it reaches its maximum capacity), especially with real time systems
• crash testing should be performed to see what it takes to bring the system down
• test monitoring tools should be used whenever possible to track which tests have already been performed and the outputs of these tests to avoid repetition and to aid in the software maintenance
• other functional testing techniques include: transaction testing, syntax testing, domain testing, logic testing, and state testing.
• finite state machine models can be used as a guide to design functional tests
• According to Beizer 2 the following is a general order by which tests should be designed:
  1. Clean tests against requirements.
  2. Additional structural tests for branch coverage, as needed.
  3. Additional tests for data-flow coverage as needed.
  4. Domain tests not covered by the above.
  5. Special techniques as appropriate–syntax, loop, state, etc.
  6. Any dirty tests not covered by the above.

Levels Of Black Box Testing

1. FUNCTIONAL TESTING
2. STRESS TESTING
3. LOAD TESTING
4. AD HOC TESTING
5. EXPLORATORY TESTING
6. USABILITY TESTING
7. SMOKE TESTING
8. RECOVERY TESTING
9. VOLUME TESTING
10. DOMAIN TESTING
11. SCENARIO TESTING
12. REGRESSION TESTING
13. USER ACCEPTANCE TESTING
14. ALPHA TESTING
15. BETA TESTING