Motivation
Usually, I try to talk about my hobby projects on this blog, but over the last weeks, I’ve had the pleasure of talking to some younger colleagues about automated testing and I repeated myself often enough that I thought it might warrant being written down somewhere so I can link to it in the future as a conversation starter.
A warning to wary: this article contains code samples in a language called ABAP, SAP’s proprietary language. There is no need to panic. The first parts of this article are language-agnostic. The latter parts are specific to ABAP since that is the language the conversations that sparked me writing this were about. You may still find it interesting, if only to see a language that is a mix between COBOL and Java.
That said, I would like to emphasize that my opinions are my own and not SAP’s.
On Automated Testing
Automated testing is a topic very dear to my heart. The less manual effort is involved in testing the shorter the feedback loop for the developer. Especially in systems which have to run for a long time, good automated tests can help preserve knowledge about the system that might otherwise have been held exclusively by people who have already retired.
They also help new team members get up to speed and make changes without fear of accidentally breaking some tribal knowledge invariant that was never documented anywhere. And even if this tribal knowledge had been documented somewhere, large long-lived systems typically come with documentation formats that changed over the years, lost on some wiki, sometimes in file formats that can no longer be opened (yes, I’ve seen it).
When your best bet at knowing the internals of a system is a 20 year old Powerpoint slide deck, you are truly lost indeed; and yes, I speak from personal experience. Welcome to the world of Enterprise Software, abandon all hope, ye who enter here.
A good automated test suite on the other hand is stable documentation. It does not go out-of-sync with the production code like written documentation tends to do. But most importantly, of course, a good automated test suite ensures that your system is still running as expected after you made changes to it.
One part of a good automated test suite (and the one I write about today) is unit tests.
On Unit Testing
A common misconception I see is that every single function has to be tested explicitly. This is not true and in fact harmful to potential future refactoring efforts, since a lot of tests will have to be changed, if suddenly the functions they were testing are no longer there. Implicit testing is fine and in fact conducive to future refactoring.
In general, you want to test contracts, not specific functions. A module’s (or class or other unit) contract is its public interface; using this contract, we can fix the module’s behavior in place. This helps us ensure a smooth experience for the users of our module.
On Code Coverage
By not testing everything explicitly, I do not mean to say to not aim for 100% code coverage. In fact, I think code coverage is a good tool to get a better understanding about which parts still deserve some testing. As we’ll get into shortly, there is a variety of reasons due to which 100% coverage may not be possible for a given module, but it is still a worthy goal to strive for.
When a certain part of the code is not covered by tests, ask yourself why. Is it just dead code? Is it that the behavior of this particular piece of code is not yet part of the behavior contract of the module? For example, does this method throw an exception when it encounters an error, but up to this point only the happy path has been considered as the module’s contract?
On Mocking
Running your entire code in a test harness sounds great, but what about dependencies? Dependencies are sometimes called code that does not belong to you or that you do not have access to, like an external module, but I like to think of it more in terms of context; when you call a method on an object, you do not care about the internals of that method, you just care about the inputs and outputs.
When you know that the code you call doesn’t do anything costly, you can, of course, just run through it, although then you are conceptually testing more than the current unit. Something costly in this case can mean any number of things: complex calculations that take a long time to compute, file I/O, database reads and writes, just to name a few.
But even when you do know that none of these things happen right now, how can you be sure that somebody doesn’t change the implementation of the code you call and suddenly your unit tests make database connections and delete stuff from various tables? This may seem silly, but in a large enough codebase with an arbitrary number of abstraction layers, it can happen very easily.
This is where mocking comes into play. In its simplest form you use an instance of a class which implements an interface (or has an abstract base class). Instead of programming against the concrete class, you program against the interface. Instead of creating the instance of the concrete class internally, your function takes a reference to an interface as an argument and operates on that. This is called Dependency Inversion.
" Don't do this
CLASS cl_example DEFINITION.
PUBLIC SECTION.
METHODS main
RETURNING VALUE(result) TYPE i.
ENDCLASS.
CLASS cl_example IMPLEMENTATION.
METHOD main.
FINAL(concrete) = NEW cl_concrete( ).
RESULT = concrete->meth( 1 ).
ENDMETHOD.
ENDCLASS.
" Do this, instead, where class cl_concrete implements interface if_abstract
CLASS cl_example DEFINITION FINAL.
PUBLIC SECTION.
METHODS main
IMPORTING abst TYPE REF TO if_abstract
RETURNING VALUE(result) TYPE i.
ENDCLASS.
CLASS cl_example IMPLEMENTATION.
METHOD main.
TRY.
RETURN abst->meth( 1 ).
CATCH lcx_error.
RETURN -1.
ENDTRY.
ENDMETHOD.
ENDCLASS.Then in a unit test of method main you can substitute the concrete class with a test double.
CLASS ltd_concrete_double DEFINITION FOR TESTING.
PUBLIC SECTION.
INTERFACES if_abstract PARTIALLY IMPLEMENTED.
DATA _raise_exception TYPE abap_bool.
ENDCLASS.
CLASS ltd_concrete_double IMPLEMENTATION.
METHOD if_abstract~meth.
IF _raise_exception = abap_true.
RAISE EXCEPTION NEW lcx_error( ).
ENDIF.
RETURN 1.
ENDMETHOD.
ENDCLASS.
CLASS ltc_tests DEFINITION FOR TESTING DURATION SHORT RISK LEVEL HARMLESS.
PRIVATE SECTION.
METHODS happy_path FOR TESTING RAISING cx_root.
METHODS handles_exception FOR TESTING RAISING cx_root.
ENDCLASS.
CLASS ltc_tests IMPLEMENTATION.
METHOD happy_path.
FINAL(cut) = NEW cl_example( ).
FINAL(result) = cut->main( NEW ltd_concrete_double( ) ).
cl_abap_unit_assert=>assert_equals( exp = 1 act = result ).
ENDMETHOD.
METHOD handles_exception.
FINAL(double) = NEW ltd_concrete_double( ).
double->_raise_exception = abap_true.
FINAL(cut) = NEW cl_example( ).
FINAL(result) = cut->main( double ).
cl_abap_unit_assert=>assert_equals( exp = -1 act = result ).
ENDMETHOD.
ENDCLASS.In the actual cl_concrete->meth( ) the situation in which an exception is thrown may be very hard to create; in this way, we can make sure that the code under test works as expected even in difficult to produce scenarios.
Where things get a little more tricky, is when you have static calls in your code.
On Mocking and Isolating Function Modules (or, More Generally, Static Calls)
Function modules and static method calls are commonplace even today in modern ABAP code, often for good reason. While there is work being done to replace them with instance methods wherever possible, for the foreseeable future they are a reality even when writing new code.
So, how can we deal with this? There are two main approaches:
- the test double framework for function modules
- wrapping the function module call inside an instance method of a local class
Let’s look at the latter one first by wrapping the commonly used function module MATERIAL_UNIT_CONVERSION.
CLASS lcx_fm DEFINITION INHERITING FROM cx_static_check.
ENDCLASS.
INTERFACE lif_fm.
METHODS material_unit_conversion
IMPORTING matnr TYPE matnr
qty_in type menge_d
uom_in TYPE meins_d
EXPORTING qty_out TYPE menge_d
buom TYPE meins_d
RAISING lcx_fm.
ENDINTERFACE.
CLASS lcl_fm DEFINITION.
PUBLIC SECTION.
INTERFACES lif_fm.
ENDCLASS.
CLASS lcl_fm IMPLEMENTATION.
METHOD lif_fm~material_unit_conversion.
CALL FUNCTION 'MATERIAL_UNIT_CONVERSION'
EXPORTING
input = qty_in
matnr = matnr
meinh = uom_in
kzmeinh = abap_true
IMPORTING
output = qty_out
meins = buom
EXCEPTIONS
error_message = 1
OTHERS = 2.
IF sy-subrc <> 0.
RAISE EXCEPTION NEW lcx_fm( ).
ENDIF.
ENDMETHOD.
ENDCLASS.Since this is a locally defined interface and we don’t actually want it to be public, we cannot put it into our method signature. We can, however, keep a reference typed to the interface as an instance attribute and set it in the constructor. Then in main we call the instance method which in turn calls the function module.
CLASS cl_example DEFINITION FINAL.
PUBLIC SECTION.
METHODS main
RETURNING VALUE(result) TYPE i.
PRIVATE SECTION.
DATA _fm TYPE REF TO lif_fm.
ENDCLASS.
CLASS cl_example IMPLEMENTATION.
METHOD constructor.
_fm = NEW lcl_fm( ).
ENDMETHOD.
METHOD main.
TRY.
_fm->material_unit_conversion(
EXPORTING
matnr = 'TEST_MATERIAL'
qty_in = CONV #( 10 )
uom_in = 'EA'
IMPORTING
qty_out = DATA(converted_qty)
buom = DATA(base_unit)
).
RETURN CONV #( converted_qty ).
CATCH lcx_fm.
RETURN -1.
ENDTRY.
ENDMETHOD.
ENDCLASS.In the test include of the class we now define a local test helper lth_injector as a local friend of the global class cl_example to inject a test double into our class under test instance.
CLASS lth_injector DEFINITION ABSTRACT FINAL FOR TESTING.
PUBLIC SECTION.
CLASS-METHODS inject
IMPORTING
cut TYPE REF TO cl_example
fm TYPE REF TO lif_fm.
ENDCLASS.
CLASS cl_example DEFINITION LOCAL FRIENDS lth_injector.
CLASS lth_injector IMPLEMENTATION.
METHOD inject.
cut->_fm = fm.
ENDMETHOD.
ENDCLASS.Finally, in our test class we use the setup method that runs before every test execution to set the private attribute of the class under test instance to the test double ltd_fm.
METHOD setup.
cut = NEW #( ).
mock_fm = NEW ltd_fm( ).
lth_injector=>inject(
cut = cut
fm = mock_fm ).
ENDMETHOD.With this we have full control over our test double. We could, for example put in something like this:
CLASS ltd_fm DEFINITION FOR TESTING.
PUBLIC SECTION.
INTERFACES lif_fm PARTIALLY IMPLEMENTED.
DATA fail_conversion_on TYPE meins.
ENDCLASS.
CLASS ltd_fm IMPLEMENTATION.
METHOD lif_fm~material_unit_conversion.
IF uom_in = fail_conversion_on.
RAISE EXCEPTION NEW lcx_fm( ).
ENDIF.
IF uom_in = 'KG' AND qty_in = 20.
qty_out = 1000.
buom = 'EA'.
ELSE.
qty_out = qty_in.
buom = uom_in.
ENDIF.
ENDMETHOD.
ENDCLASS.
CLASS ltc_tests DEFINITION FOR TESTING DURATION SHORT RISK LEVEL HARMLESS.
PRIVATE SECTION.
DATA cut TYPE REF TO cl_example.
DATA mock_fm TYPE REF TO ltd_fm.
METHODS setup.
METHODS happy_path FOR TESTING RAISING cx_root.
METHODS handles_exception FOR TESTING RAISING cx_root.
ENDCLASS.
CLASS ltc_tests IMPLEMENTATION.
METHOD setup.
cut = NEW #( ).
mock_fm = NEW ltd_fm( ).
lth_injector=>inject(
cut = cut
fm = mock_fm ).
ENDMETHOD.
METHOD happy_path.
FINAL(result) = cut->main( ).
cl_abap_unit_assert=>assert_equals( exp = 1 act = result ).
ENDMETHOD.
METHOD handles_exception.
mock_fm->fail_conversion_on = 'EA'.
FINAL(result) = cut->main( ).
cl_abap_unit_assert=>assert_equals( exp = -1 act = result ).
ENDMETHOD.
ENDCLASS.The downside of this approach is that whatever logic is in the local class lcl_fm will not be processed during tests. This is what we want, yes, but it means that we should keep the logic in there to a minimum. A consequence of this is that the local class will also be excluded in the code coverage statistics, so 100% code coverage is not possible in this way.
An alternative to this approach (at least for function modules) is provided by the function module test double framework.
CLASS ltc_tests DEFINITION FOR TESTING DURATION SHORT RISK LEVEL HARMLESS.
PRIVATE SECTION.
CLASS-DATA fm_env TYPE REF TO if_function_test_environment.
CLASS-METHODS class_setup.
DATA cut TYPE REF TO cl_example.
METHODS setup.
METHODS happy_path FOR TESTING RAISING cx_root.
METHODS handles_exception FOR TESTING RAISING cx_root.
ENDCLASS.
CLASS ltc_tests IMPLEMENTATION.
METHOD class_setup.
fm_env = cl_function_test_environment=>create( VALUE #( ( 'MATERIAL_UNIT_CONVERSION' ) ) ).
ENDMETHOD.
METHOD setup.
cut = NEW #( ).
fm_env->clear_doubles( ).
ENDMETHOD.
METHOD happy_path.
DATA(unit_converter) = fm_env->get_double( 'MATERIAL_UNIT_CONVERSION' ).
DATA(input) = unit_converter->create_input_configuration( )->set_importing_parameter( name = 'MEINH' value = 'EA'
)->set_importing_parameter( name = 'INPUT' value = 10 ).
DATA(output) = unit_converter->create_output_configuration( )->set_exporting_parameter( name = 'MEINS' value = 'KG'
)->set_exporting_parameter( name = 'OUTPUT' value = 1 ).
unit_converter->configure_call( )->when( input )->then_set_output( output ).
FINAL(result) = cut->main( ).
cl_abap_unit_assert=>assert_equals( exp = 1 act = result ).
ENDMETHOD.
METHOD handles_exception.
DATA(unit_converter) = fm_env->get_double( 'MATERIAL_UNIT_CONVERSION' ).
DATA(input) = unit_converter->create_input_configuration( )->set_importing_parameter( name = 'MEINH' value = 'EA' ).
unit_converter->configure_call( )->when( input )->then_raise_classic_exception( 'ERROR_MESSAGE' ).
FINAL(result) = cut->main( ).
cl_abap_unit_assert=>assert_equals( exp = -1 act = result ).
ENDMETHOD.
ENDCLASS.With this we save ourselves a lot of effort in terms of setup costs, but we still have the function modules directly in our code; it’s also not quite as flexible as our handmade approach, but it is likely enough for a lot of use cases. Finally, it does not negatively impact our code coverage, because the function module is actually processed.
So, which of these approaches should we prefer? Actually, why not combine them and get the best of both worlds? Keep the static calls in a local class and use the test double framework to handle the mocking. We get the benefit of a nicer interface than most function modules offer, but also keep our code coverage, which is nice if you are in the unfortunate position to have hard requirements on that. This way, we can also handcraft our own test doubles in the future, if, for example, we wanted to look at the parameters we pass to the function modules and have tests for that.
On Mocking Database Tables And CDS Views
Let’s face it, 99% of ABAP programs have one purpose: taking input (either via UI or API), reading and/or writing data from and to a database, and then returning some kind of output.
SQL is baked directly into the language, so where in other languages one might mock the ORM in ABAP we mock the database tables or CDS Views directly via the corresponding test double framework.
There really isn’t much to this, other than being cautious to really mock all tables/views that are used. We wouldn’t want to accidentally delete content from the actual tables, would we? I still think it’s a bit unfortunate, that you have to specify the tables explicitly, though. When would one ever want to interact with the actual database tables in a unit test, after all?
Be that as it may, the process is quite simple. Let’s say, you are building a managed RAP business object R_DocumentTP and want to test it. The business object is built on top of database table document.
define table document {
key client : abap.clnt not null;
key uuid : sysuuid_x16 not null;
document : char10;
item : char4;
status : char1;
last_changed_at : timestampl;
}We want to write a test to make sure that our business object behaves as we expect it to. So, we mock R_DocumentTP and its base dependencies (table document). Then, in our test method create_works we use EML to insert a new record into the database table and commit it. The test double framework catches this and redirects it to the mocks instead. We can then use SQL to select from the table to make sure that the operation worked as expected.
CLASS ltc_basic_operations DEFINITION FOR TESTING DURATION SHORT RISK LEVEL HARMLESS.
PRIVATE SECTION.
CLASS-DATA cds_env TYPE REF TO if_cds_test_environment.
CLASS-METHODS class_setup.
CLASS-METHODS class_teardown.
METHODS setup.
METHODS teardown.
METHODS prepare_doubles.
METHODS create_works FOR TESTING RAISING cx_static_check.
ENDCLASS.
CLASS ltc_basic_operations IMPLEMENTATION.
METHOD class_setup.
cds_env = cl_cds_test_environment=>create_for_multiple_cds(
i_for_entities = VALUE #(
( i_for_entity = 'R_DocumentTP' i_select_base_dependencies = abap_true )
)
).
ENDMETHOD.
METHOD class_teardown.
cds_env->destroy( ).
ENDMETHOD.
METHOD setup.
cds_env->clear_doubles( ).
ENDMETHOD.
METHOD teardown.
ROLLBACK ENTITIES.
ENDMETHOD.
METHOD create_works.
MODIFY ENTITY R_DocumentTP
CREATE AUTO FILL CID FIELDS ( Document Item )
WITH VALUE #(
( Document = 1 Item = '0001' )
)
FAILED FINAL(failed)
REPORTED FINAL(reported).
cl_abap_unit_assert=>assert_initial( failed ).
COMMIT ENTITIES.
cl_abap_unit_assert=>assert_initial( sy-subrc ).
SELECT
FROM document
FIELDS *
INTO TABLE @FINAL(results).
cl_abap_unit_assert=>assert_equals( exp = 1 act = lines( results ) ).
cl_abap_unit_assert=>assert_equals( exp = 1 act = results[ 1 ]-document ).
cl_abap_unit_assert=>assert_equals( exp = '0001' act = results[ 1 ]-item ).
ENDMETHOD.
ENDCLASS.If we wanted, we could also set up some data in the tables beforehand, e.g. in the setup method or at the beginning of our test method.
METHOD prepare_doubles.
DATA mock_document TYPE STANDARD TABLE OF document.
mock_document = VALUE #(
( uuid = 1 document = 1 item = '0001' status = 'A' )
( uuid = 2 document = 2 item = '0001' status = '' )
).
cds_env->insert_test_data( mock_document ).
ENDMETHOD.On Test Seams
A quick note on TEST-SEAMs; personally, I have yet to find a need to use them in new code. As far as I can tell, they were introduced as a way to increase the test coverage for existing code which may not be in a state to actually test it otherwise without major effort. This is not a slight to old code; the old code was clearly good enough to survive until today, otherwise we would not be complaining about it now. It is true though that it can be very challenging if not impossible to get this old code to comply with modern quality metrics.
On The Dangers Of Mocking
The moment you mock something, you state that you understand the whole world of what you mocked. Unfortunately, often it’s the exact opposite; we mock things “just in case” because we don’t know what they do internally; do they commit something? Read from the database or some other persistent store?
Case and point, the first time I mocked MATERIAL_UNIT_CONVERSION, I forgot to set the parameter KZMEINH in the actual code and, obviously, it did not behave as I expected, when running the code in QA. The tests all passed, however. Not a huge misstep, but you can see how similar things can happen very easily and may not be spotted immediately. So keep in mind, mocking does not protect you from having to figure out what the code you are mocking actually does.
On The Necessity Of Mocking
In complex systems, you can’t get around some dependencies between modules. The amount of mocking necessary to be able to test a given module may serve as an indication that there is too much coupling in the system. Nonetheless, it is a necessary step on the road to providing testability, which in turn may allow the system to then be refactored into a more loosely coupled design.