LibreOffice Development Blog

In the previous parts of the blog posts series on fixing software crashes, I have written about some crash fixes in LibreOffice around segfaults, aborts, and I discussed how test them. Here I write about fixing assertion failure.

What is Assertion Failure?

Assertion is a mechanism provided to the developers to make sure that things are good in runtime, conforming to what they were assuming. For example, making sure that some pointer is valid, some data elements match expectation, or some similar assumptions that need to be valid in order to get the correct results.

As an example, consider the C++ function basegfx::utils::createAreaGeometryForLineStartEnd(), which creates a geometric representation of an arrow. The code resides here:

basegfx/source/polygon/b2dlinegeometry.cxx:84

This is the line of code which contains assertion:

assert((rCandidate.count() > 1) && "createAreaGeometryForLineStartEnd: Line polygon has too few points");

On top of the actual function implementation, the C++ code asserts many conditions to make sure that they are met. In the above assertion, it checks to make sure that the number of points in the given data structure is more than 1. Otherwise, it leads to an assertion failure.

For various reasons, sometimes these sort of assumption may not be valid. To avoid reaching to incorrect results, it is important to have such assertions in place, to find such issues as soon as possible. If you are developing LibreOffice, you may have already built the code from sources in debug mode. Therefore, you may see software stops working, or simply crashes.

This crash may not happen for the end users, which use the release version of software. Therefore, these type of crashes have lower impact for the end users, and they are usually considered of lower importance compared to the crashes that happen for the end users.

Backtrace

One of the things that can help diagnose the problem is the stack trace, or simply backtrace. The way to obtain a backtrace depends on the platform and/or IDE that you use. If you use an IDE like Qt Creator, Visual Studio, etc., getting a backtrace would be as easy as debugging LibreOffice, making the assert fail, and then copy the backtrace from the UI. To learn more about IDEs, see this Wiki page:

• TDF Wiki: Development/IDE

If you want to use gdb on Linux, you may run LibreOffice with this command line:

$ instdir/program/soffice –backtrace

and then make the assert fail, and you will have the backtrace in gdbtrace.log file. You can learn more int this QA Wiki article:

TDF Wiki: QA/BugReport/Debug_Information

One thing to mention is that if you work on a reported bug regarding to assertion failure, then the actual way to reproduce the issue and make the assertion fail is usually described in the relevant TDF Bugzilla issue. In the meta bug related to assertion failure, you may find some of these issues in the last part of this blog post.

Fixing the Problem

To fix the problem, first you should gain understanding of the assumption, and why it fails. You should be able to answer these questions by reading the code and debugging it:

• What does some assumption mean?
• Why it fails?
• How to fix that, so that it does not fail?

To gain this understanding, you have to look into the places where backtrace points. Backtrace can be complex, containing the whole stack of the function calls across the software, linking to places in the code, but let’s discuss a simplified form.

Consider this bug fix:

tdf#152012 Fix assert fail on opening date picker

The problem was that an assertion failure was happening when opening date picker field in a DOCX file. This is the simplified form of the stack trace:

1  __pthread_kill_implementation           pthread_kill.c:44
2  __pthread_kill_internal                 pthread_kill.c:78
3  __GI___pthread_kill                     pthread_kill.c:89
4  __GI_raise                              raise.c:26
5  __GI_abort                              abort.c:79
6  __assert_fail_base                      assert.c:92
7  __GI___assert_fail                      assert.c:101
8  o3tl::span::operator[]                  span.hxx:83
9  OutputDevice::ImplLayout                text.cxx:1396
10 OutputDevice::DrawTextArray             text.cxx:948
11 Calendar::ImplDraw                      calendar.cxx:71
12 Calendar::Paint                         calendar.cxx:1133

The problem was caused by an out of bound access to a vector of integers, and to fix that, you had to see why this happens, and fix that.

This is the description from the commit title:

Previously, for the 7 days of the week, a 7 letter string smtwtfs (depending on the week start day this can be different, but length is always 7) was sent to this method without providing the length, thus the string length: 7 was used. In this case, positions of the letters were calculated and used from other array named mnDayOfWeekAry[7]. mnDayOfWeekAry[k+1] was used as the position of letter k (k=0..5). In this case, there was 7 letters for 7 days, and only 6 positions provided by the array. This caused assertion failure in span.hxx:83 when trying to access mnDayOfWeekAry[7] via o3tl::span<T>::operator[].

As you can see, a false assumption was creating assertion failure, and the fix was essentially changing the code based on the corrected assumption.

Sometimes, the fix can be easier. As an example, by only checking a pointer to make sure that it is valid, the assertion failure does not happen. Therefore, to fix the problem, you have to carefully study the code and its behavior.

Final Notes

Many of the assertion failure bugs are fixed in daily works of the developers, before causing problems for the end users, which use the “release” version of the software, that do not crash because of the assertion failure. But there are some of these issues remaining.

Do you want to help fix some of the remaining issues? Then, please refer to the list here, read some bug report, and pick one:

• Open bugs dependent on “Crash-Assert”

One of the areas that can help LibreOffice, but may not directly be visible to the users even though it has a big impact is the quality assurance, is automated testing.  Here, I discuss some areas and notes around improving tests for LibreOffice. First, I start with regressions and bug fixes without a test.

Missing Unit Tests

This is the description from the GSoC ideas page:

While there are some automated tests for LibreOffice, there are not nearly enough. Adding more and better tests helps developers who work on the code to be more productive by allowing them to find regressions as early as possible.

To elaborate more, I should say there are many regressions and bugs in general that are fixed, but lack testing. It is important to have tests for those bug fixes, to avoid such problems in the future. You can see a list of those fixed issues that lack tests here:

• The Document Foundation Wiki – CppUnitTest
• The Document Foundation Wiki – Missing Unit Tests

If you want to add some new tests for the bug fixes, first you should read the bug report very carefully to understand what is it about, and try to test the fix yourself. You can either use git revert command to revert the fix, and see the problem in action, or try changing back the fix in the code, if it is not too big.

That is important, because you have to see that the test fails without the fix in place, but succeeds when it is applied. This is an essential step when writing the test.

To know more about unit tests, you may look into these Wiki pages:

• The Document Foundation Wiki – UITests

Porting Existing Test to C++ or Python

Some tests are written in the past, but now have issues because of the way they are written. In this case, porting them can provide improvement.

Tests can be written in multiple languages. At least, C++, Python, Java and BASIC are currently in use for different tests across the LibreOffice code base. Again in the GSoC ideas page, you can read:

There is some support in LibreOffice for automated tests, both at the level of unit tests, and at the level of system tests that drive a full LibreOffice instance. Currently tests can be written in C++, Java, or Python. Various new automated tests should be developed to improve the test coverage.

Tests written exclusively for Java (e.g. the JUnit framework) should be ported to C++ so that they can execute much more rapidly. Similarly, tests that do remote control of an existing LibreOffice instance, should be re-factored to run inside that instance to make debugging much easier.

Almost all JUnitTests and UITests, and also some smoke tests, run as an outside process. To verify, the trick is to remove the soffice(.exe) binary, or to remove its execute permission (on Linux). In this way, out-of-process tests should fail, as they need to run the soffice binary. After a successful build, you can see if this works. For example, try this:

make -d JunitTest_framework_complex

As an example, we have this issue around complex tests that should be ported to Python:

• tdf#97362 – Convert Java complex tests to Python

You may find many examples of the previously ported test in the issue page. Keep in mind that usually old Java tests should be removed in the same patch that provides the CppUnitTest port.

Also, you may find examples of UITests ported to C++ from Python. For example, see this patch:

• tdf#150457 Port UITest to CppunitTest

It ports a previously implemented UITest from Python to C++. The result is a faster test, which is more robust.

Focusing on a Specific Area

If you want to add or port some tests, please focus on a specific area of the code. The LibreOffice code base is huge, and consists of more than 200 modules. It is not easy, and in fact not suggested, to work across different applications and modules. It is much better that you pick a specific application, and even inside that application, like Writer, Calc, Impress, etc., focus on a specific module. In this way, it would be much easier to understand the ideas and the way tests are implemented.

How Hard is it to Write / Port a Test?

One important question is around the complexity of writing or porting tests. The answer is not straightforward, as it depends on the area that the test is written. But one can take a look at what others did in the past. By looking into Gerrit or git log, you can find what other people, including GSoC applicants, were able to achieve during their work.

After writing a few tests, or porting a few tests, it may become easier for you to write more tests. Another thing is that you don’t have to write tests from scratch. There are many tests available in the LibreOffice code base, and you can create your new test based on existing tests and customize it to match the purpose of the issue you are working on.

When working with LibreOffice Impress, “Slide Master” is the place where you can change the templates used for different types of the slides used in your presentation. Here we discuss a possible improvement for the “Slide Master” by making the copy from master slides possible.

Copying the Master Page in Impress

To see the problem and the room for enhancement, open LibreOffice Impress, then choose “View->Master->Slide Master” from the menus. Then, try to copy the master page on the left in the slide sorter. Unfortunately, it is not possible.

Impress slide master

Having this feature is helpful, because different page types have many things in common, and being able to copy/paste helps creating templates much faster.

Impress Code Pointers

Looking into sd/source/core/drawdoc3.cxx, you can see a huge function SdDrawDocument::InsertBookmarkAsPage, which is relevant here. It contains ~600 lines of code. This huge function is in itself a problem. Therefore, to implement the enhancement, on should try to first refactor the function, then add a unit test in sd/qa/unit, find and then separate all the ~6 use cases, and fix the style/name merging.

After the cleanup, the main fix should be implemented. The suggested path to implement this comes from Jean-Francois. He suggest to improve the duplicate() method, which is described in the documentation:

• XDrawPageDuplicator interface, duplicate() method

As described in the above documentation, its role is to duplicate a page:

Creates a duplicate of a DrawPage or MasterPage, including the Shapes on that page and inserts it into the same model.

However, the implementation does not work for master slides, as the macros in the attachment file implies. The solution should add the needed implementation for master slides.

The implementation is inside sd/source/ui/unoidl/unomodel.cxx inside duplicate function:

// XDrawPageDuplicator
uno::Reference< drawing::XDrawPage > SAL_CALL SdXImpressDocument::duplicate( const uno::Reference< drawing::XDrawPage >& xPage )
{

...

}

Final Words

The above issue is filed as tdf#45617. If you like to work on it, just follow the Bugzilla link to see more information.

To implement this feature, first you have to build LibreOffice from the sources. If you have not done that yet, please refer to this guide first:

Getting Started (Video Tutorial)

In the first blog post on LibreOffice build system, gbuild which uses GNU Make, I discussed some of the features of it. Here I discuss more about some gbuild tips and tricks that you may need.

Building a Single Module

In order to build a single module, you need to use its name. For example, to build only “odk”, which contains office development kit, you only have to type:

make odk

On the other hand, there are many other build targets associated with odk. By typing make odk, and then pressing tab, you will see this list, which shows possible targets:

odk odk.buildall odk.perfcheck odk.uicheck odk.all odk.check odk.screenshot odk.unitcheck odk.allbuild odk.checkall odk.showdeliverables odk.allcheck odk.clean odk.slowcheck odk.build odk.coverage odk.subsequentcheck

Each of the above is related to a specific task, in which many of them are common on different modules. Let’s discuss some of them:

make odk -> Builds odk module.

make odk.clean -> Cleans the odk module, removing the generated files.

make odk.check -> Runs test in odk module

make odk.uicheck -> It runs UI tests inside odk module

make odk.perfchek -> Runs performance/callgrind tests inside odk module

make odk.screenshot -> Creates screenshots inside odk module

To get a complete list and detailed description, run make help.

Handling Incomplete Builds

Sometimes because of OS crash or power outage, you may face problems when a build is stopped forcefully. In that case, you may see several zero byte object (*.o) files that exist, and prevent a successful build. In that case, you can find and remove them using this command:

$ rm `find -name *.o -size 0`

After that, you can retry your build without the above problem.

Customizing Build Configuration

The process of creating Makefile starts from configuring LibreOffice for build. This is done by invoking ./autogen.sh. The configuration parameters are read from autogen.input. The build configuration is done via configure.ac, which is an input for GNU autoconf.

There are various steps before the Makefiles are generated. For example, in order to make sure that a library is there when configuring the build, a very small C/C++ file is created, compiled and tested to ensure that the library is ready, and available to use with C/C++ code.

It is also possible to check for some specific version of library, and available functions. As an example, see this patch, which checks for specific version of ZXing library:

• tdf#153328 ZXing::ToSVG() Conditional usage

In the above example, multiple situations are handled:

1) When there is no ZXing library

2) When system ZXing library is used

And also, it is checked that specific version of ZXing is available:

1) When ZXing::ToSVG is not usable

2) When ZXing::ToSVG is usable

Then, the HAVE_ZXING_TOSVG symbolic constant is used in config_host/config_zxing.h.in, which can be used in C++ code.

Knowing More About gbuild

If you are interested in knowing more about gbuild, you can start from my first post on gbuild in this blog. I plan to write more about gbuild, and describe some of the frequently used macros.

Also, you can take a look at the article devoted to gbuild in the TDF Wiki:

• TDF Wiki – Development/Gbuild

In the previous part of the series on C/C++ strings, I described the string literal, plus how and why to use them. Then I introduced the new custom string literals and their benefits: (more…)