Crash fixes part 3 – Testing crashes

By Hossein Nourikhah On 18 April 2024 In Tutorial

I have previously discussed fixing crashes in 2 parts (segfaults, aborts). Here I discuss testing crashes to avoid creating re-creating regressions.

Why testing crashes?

When you fix a crash, you have to make sure that it does not happen again in the future. The key to achieve such a goal is to write a suitable test. The test should do the exact steps to reproduce the problem on the program in order to detect the known crash before the new code is merged.

This can be done using either UITests, or CppUnitTests. UITests are written in Python. They are easier to write, but they do not run on each and every platform, and they are usually slower. CppUnitTests, on the other hand, are written in C++. They are much faster, and they run on every platform that CI runs to make sure that everything is built and can be run correctly.

An Example Crash Testing

Consider the below issue around footnotes:

tdf#150457 Fix crash on hovering removed footnote reference

This problem was happening when someone created a footnote, deleted the reference, and then hovered the mouse on the removed footnote reference. To reproduce that, one could use keyboard to generate a key sequence that repeats the required steps:

Write something, add footnote, select all the footnotes and remove them, then go back to the text, and hover the footnote reference.

Using keyboard-only is not always enough, but here it was possible. To implement the UITest, you should first find the appropriate place to put the test file, and then write a Python script for that. Here, the test was written in sw/qa/uitest/writer_tests2/deleteFootnotes.py. The UITest test can be found alongside the bug fix, in the class class tdf150457(UITestCase):

tdf#150457 Fix crash on hovering removed footnote reference

If you look into the code, the test_delete_footnotes() function consists of many invocations of postKeyEvent calls, that emulate key input events:

pTextDoc->postKeyEvent(LOK_KEYEVENT_KEYINPUT, 'a', 0);

To insert footnotes, UNO commands are used.

dispatchCommand(mxComponent, ".uno:InsertFootnote", {});

Just doing the same steps would be enough, as if the crash happens with the fix in place, or in a bad commit in the future, the test would fail. This test failure will prevent the same problem in the future.

The nice thing is that it turned out the same test could have been written using C++ and CppUnitTest, which is considered superior.

The new CppUnitTest can be found in the below change:

tdf#150457 Port UITest to CppunitTest

The new test resides in sw/qa/extras/uiwriter/uiwriter3.cxx, and essentially uses postKeyEvent and dispatchCommand as similar functions.

If you look at the current version of the test, you can see that it was simplified in later commits, but the idea is the same: “repeat the same steps that lead to crash in the code”.

Final Words

It is expected that every bug fix is accompanied with a test, to avoid seeing the same problem in the future. If you want to know more about UITests and CppUnitTests, please take a look at this TDF Wiki pages:

Also, keep in mind that looking into the existing tests is the best way to understand how to write a new test. Therefore, make sure that you look into qa/ folders of the modules when you want to add a new test.