Testing Methods - Principles of Software Development - Lecture Notes, Study notes of Software Engineering

Download Testing Methods - Principles of Software Development - Lecture Notes and more Study notes Software Engineering in PDF only on Docsity!

Integration testing Transitive verification Testing patterns Advanced testing topics Advanced testing topics Wednesday, October 17, 2012 10:02 AM Docsity.com

The process whereby the whole product is built from its parts, testing along the way. Key principle: use the success of past tests to condition the success of future tests. Integration testing bottom-up : start with leaves of module/call diagram, integrate upward. top-down : start with root of module diagram, integrate downward. Two kinds: attempt to localize defects in the smallest testable contexts. rely upon prior tests to assure preconditions required for later tests. Appropriate testing exhibits a "snowball" effect; tests that we have done so far reduce future work. Both kinds: Integration testing Wednesday, November 18, 2009 11:23 AM Docsity.com

provides the environment(s) the unit expects. performs a variety of tests. reports the results. A scaffolding A scaffolding for equivalence partition testing might be 10 times as complex as the unit under test! (I tell my students in implementation courses that the test program is 10 times the length of their program.) Scaffoldings are complex! Scaffoldings Wednesday, November 18, 2009 11:27 AM Docsity.com

A "stub" is a placeholder for a function that has not been written yet. When you first compile a top-level module, one way to compile it is to "stub out" the functions it needs. Stubs Some stubs might do nothing at all, e.g., stubs for a database model. Other stubs might return values that are reasonable but incorrect. Stubs are simple! Stubs Wednesday, November 18, 2009 11:30 AM Docsity.com

Stubs return "conforming" but "incorrect" values. Can test the user interface before one tests the correctness of its data feeds. A natural application of stub-testing: user interfaces Stub testing is typically very obvious and easy. Example of stub testing Wednesday, November 18, 2009 11:36 AM Docsity.com

libraries or reusable code of other kinds. situations in which it is obvious -- at lower levels -- what correct behavior should be. Obvious uses for bottom-up (scaffolded) testing: complex, data-driven user environments. situations in which the top-level code is needed to detect correctness of the lower-level code. Obvious uses for top-down (stubbed) testing: To stub or to scaffold? Wednesday, November 18, 2009 11:39 AM Docsity.com

Builds upon prior successes. Makes best use of the software to test itself (minimizes scaffolding) Is inexpensively repeatable after any change. The best test plan Snowballing in testing Snowballing Wednesday, November 18, 2009 11:44 AM Docsity.com

test once, works everywhere. The dream of testing: portability testing is always needed. even if the application is written to be portable. This is just a dream. An industry joke: Java is a "write once, debug everywhere" language. A possible solution to the quandary: standards bases: define preconditions for the application as a whole within its environment Transitive verification claims Wednesday, November 18, 2009 11:46 AM Docsity.com

Use library functions. Don't open system files. Every android App is constrained to interact with its environment only in specific ways The Android environment is constructed so that -- whether it's running on top of Windows or Linux -- this environment exists and works. A simple example of transitive verification: Android The fact that it works in a linux-based Android means that it works in a Windows- based Android! So, if you write your application in a manner that is entirely compliant with Android best practices, Test once; works everywhere! Application A, Tests T, Environment E. V(E) = assertion that the environment is compliant (e.g., supports Android functions). V(A) = assertion that the app is compliant. T(A,E) = assertion that A passes the tests T in environment E. A simple example of transitive verification Wednesday, October 17, 2012 10:05 AM Docsity.com

if V(E) and V(A) and T(A,E), then if V(E'), we can conclude that T(A,E') Docsity.com

○ an application A to be tested, in compiled form. an environment E where it is to be tested, e.g., RedHat 5 or Ubuntu or.... ○ a test suite T(A,E) that tests that application A works in environment E. ○ an environment verifier V(E) that checks that proper libraries and versions are present in an environment E. ○ an application verifier V(A) that checks that library functions are only called with the proper types of arguments, and that files are only opened in documented and standard locations. ○ LSB components This works on compiled (closed-source) code A! LSB Logic: For an application A and two environments E and E', if V(A) and V(E) and T(A,E), then for all E' with V(E'), there is evidence that T(A,E'). In other words, "test once, (potentially) works everywhere". LSB components Wednesday, November 18, 2009 11:48 AM Docsity.com

names of files that are opened. how system calls are used. which system calls are used. checks: Inside the LSB application validator versions of system calls and libraries and the kernel. where files are located in linux. checks: Inside the LSB environment validator Inside the LSB validator Wednesday, October 17, 2012 5:02 PM Docsity.com

same code for all applications. Really cheap to use. Automatic; no human intervention required. V(A) and V(E) are static verifiers: Expensive to do. Might require human effort, e.g., verifying a user interface. T(A,E) is a dynamic verifier: The big deal of LSB We can skip something expensive (T(A,E')) after doing one expensive thing (T(A,E)) and two cheap things (V(A), V(E)). "Test once, runs everywhere that is verified as a suitable environment." The big deal of LSB Wednesday, October 17, 2012 9:49 AM Docsity.com

"Catches" common environment/configuration mistakes that impair function. coding defects that manifest when changing the architecture of a machine, e.g., SPARC versus X versus X86_64. (this wreaks havoc with uninitialized variables, which have different default values) defects that depend upon files outside the LSB's registered set of files. E.g., program crashes if there is not an /etc/motd! Does not deal with: LSB is an imperfect solution But it is definitely better than nothing! Future imperfect Wednesday, November 18, 2009 11:56 AM Docsity.com