Smoke Testing Vs Sanity Testing

Smoke Testing Vs. Sanity Testing

1. “Smoke Testing” is usually done on the nightly/interim build to test its stability. Therefore “Smoke testing” is often called as “Build Verification Testing” too. In contrast, “Sanity Testing” is usually done during the later cycles after thorough regression cycles are over. When multiple cycles of testing are executed, “Sanity Testing” is done towards the Product release phase.
2. “Smoke Testing” is done following a shallow and wide approach where all the basic and major areas are tested without going too deep into the functionality. In contrast, “Sanity Testing” is usually a focused but limited form of regression testing, which follows a deep and narrow approach to test a particular functionality in detail.
3. “Smoke Testing” is done by developers before the build is released or by testers before accepting a build for further testing. On the other hand, “Sanity Testing” is done mostly by the testers.
4. “Smoke Tests” are mostly in form of scripted form (either written test cases or automated test scripts) whereas “Sanity Tests” are mostly non-scripted!
5. “Smoke Testing” can be compared with the normal health check-up of the product whereas the “Sanity Testing” can be compared with some specialized tests to reveal possible problems with a particular functionality of the product!

Example:

Mr. Tester made up his mind to take a new bike and to get rid of the old one. Next weekend, he was seen in the nearest Bike Showroom, talking to the salesman. He was looking for a bike with fair enough pick-up and with good enough fuel consumption. Soon he was able to spot such a model. [At least the bike Company and the Salesman claimed so]. But being a tester himself, how could he take their words for granted without testing it himself! So he wanted to take a test ride and was keen to see the basic features of the bike. He was interested in checking the pick-up, the balance, the control while riding, disk brakes, the digital display etc. However, he was quite satisfied with the result of his test (ride) and brought home the bike.

Now, the poor bike was to go through another series of tests [Poor bike! Must be cursing it’s fate for being sold to a crazy “tester”!] This time Mr. Tester was interested to test the performance of the bike over a period of a couple of months. This time features like the fuel mileage, wear and tear of basic mechanical parts of the bike etc were under the scanner! Fortunately (for the Bike Manufacturer), Mr. Tester was satisfied with his findings and has been riding the bike to office (and elsewhere) till today.”

Moral of the Story: Well, all stories are not necessarily attached with morals. However, this story has certainly got some testing morals.

a) Taking a test ride to test the basic features (functionalities) of the bike can be compared to “Smoke Testing” a product. In the above story, while taking the test ride, Mr. Tester was determining if the basic features of the bike were stable and acceptable. In a typical testing environment, when a build is received for testing, a smoke test is run to determine if the build is stable and can be considered for further testing. Testers usually do a Smoke Testing before accepting the build for further testing. The tester “touches” all areas of the application without getting too deep into the functionality.

b) Testing the bike performance in detail after bringing it home can be compared to “Sanity Testing” a product. Testing those features in detail was not possible in the showroom or while taking test ride. In a typical testing environment, when a new build is received with minor modifications, instead of running a thorough regression test suite, a sanity test is performed so as to determine that the build has actually fixed the issues and no further issue has been introduced by the fixes. Sanity testing is generally a subset of regression testing and a group of test cases are executed that are related with the changes made to the product.