… where we discuss manual and automated testing and present a simplistic approach for the creation of an automated test suite for ABAP legacy code.
What Testing Can Do
Testing cannot be isolated from the development process. Be it waterfall or agile or the V-model to build a bazaar or a cathedral everyone wants high quality code without defects. Tests can
- reveal unknown bugs in the code or
- verify the code will perform according to the specification or.
- help understand how existing code work
Tests validate a specification. This should not come as a surprise to anyone using the Given-When-Then routine for test design.
In a though experiment, let us assume a fictional design surface exist where all processes in the problem domain can be represented. The specification and implementation could then be visualized as diagrams:
Manual Debugging
An ABAP report displays the wrong output. How can I proceed?
- set break-points, step through the code while observing the program state in the debugger until the fault can be consistently reproduced
- i.e. I create a test case and reveal a bug.
- step through the code while observing the program state with the debugger the part of the logic that triggers the wrong behavior is known.
- i.e. I locate a bug.
- identify the error and implement a correction. Then set break-points and step through the code while checking the system state with the debugger to make sure your code is now correct.
- i.e. I verify that code performs according to expectations.
In all steps I used used break-points, the debugger and some expectation about the correct code behavior.
But debugging is a manual process that does not scale. The proposition of this blog is to create test code that will harness productive code and reduce reliance on Debugging for test purposes.
A typical test will run in 1/100th of a second, so they are run all the time, specially after each change. Test failures are then localized in the code that was changed last. We can check it and fix the code (or the test). With a good test coverage and this leads to a new workflow with much less manual testing / debugging.
ABAP Unit Recap
If you understand this diagram, then we are good to go:
If not, check e.g.
- Suketu Dave introducing the fundementals
- James E. McDonough 10 episodes long series on ABAP Unit testing
You could also check the literature:
- TOOS: Binder, Robert; Testing Object-Oriented Sytems: Models, Patterns, and Tools, 2006
- GOOS: Steve Freeman and Nat Pryce: Growing Object-Oriented Software, Guided by Tests, 2010
- WELC: Michael Feathers, Working Effectively with Legacy Code, 2005
- xTP: Gerard Meszaros, xUnit Test Patterns: Refactoring TestCode, 2007
- POOD: Sandi Metz, Practical Object-Oriented Design in Ruby, An Agile Primer, 2007.
But note none of those books use ABAP as a reference.
The Feedback Loop
In Working with ABAP Unit Tests I claimed anyone could try retrofiting test into production code (test last) because writing test code before production code (test first) requires so much more discipline that nobody is doing it.
Since then I learned to appreciate how Test Driven Development could be done in ABAP (cf. openSAP course: while working on legacy code we should gradually extract functionality to an island of happiness were the new code is completely covered by tests). I am now convinced the TDD feedback loop is the most effective way to achieve a good tests coverage at all times.
You have to see the units fail before your fix the production code make them pass. This leads to feedback loop informing the specification from the tests after validating specification by the tests. This guide the software development with changes having an observable impact on the system behavior.
Characterization Testing
- So what is the normal way to add tests to an existing code base? The approach is called characterization testing, as is discussed here.
When the code does not have known errors, the we still identify units of processing, create tests that invoke those unit and write assertions that accurately describe how the code currently works.
In this phase, we aim at creating a test suite with a good coverage of the existing code base. The focus is to achieve a good code coverage to enable refactoring with pervasive design changes in the next phase.
This might already be technically difficult because some operations would be highly tangled (high coupling). Implicit interfaces might have to be introduced without the safety net of a good test suite. But the goal in this phase is simple: write more tests, increase test coverage.
Code Coverage Demo
I have been working on an interpreter for the Scheme language in ABAP based on R7RS small, a specification that is many decades old and seven revisions strong. Get ABAP Scheme from github and install it using abapGit.
Many examples of the expected behavior can directly be converted to tests, so ABAP Scheme has over 600 ABAP unit tests that execute in around 5 seconds, depending on the environment.
In the SAP GUI, open Report ZZ_LISP_IDE and execute the unit tests
Execute_Coverage
Coverage Metrics SE80
Coverage Result SE80
In the ADT for Eclipse, select the Report ZZ_LISP_IDE and run as ABAP unit test.
ADT Enable Coverage
ADT ABAP Unit
ADT Coverage
Both ADT and the SAP GUI editor can display statement coverage if you execute the unit tests with coverage. You can now create ABAP unit test cases that invokes the paths of the code that are still red.
For the By Display in coverage in your the editor, to see with code is still red. If you are not sure about the expected value, that use the values returned in the debugger as expected value.
With this, you can expand your test coverage. Your code paths will be green. But please do not obsess on achieving 100% coverage. All this is preparation for the next phase where you can now safely improve the code without change its external behavior: the refactoring phase.