LibreOffice Development Blog

I have previously discussed fixing crashes in 2 parts (segfaults, aborts). Here I discuss testing crashes to avoid creating re-creating regressions.

Why testing crashes?

When you fix a crash, you have to make sure that it does not happen again in the future. The key to achieve such a goal is to write a suitable test. The test should do the exact steps to reproduce the problem on the program in order to detect the known crash before the new code is merged.

This can be done using either UITests, or CppUnitTests. UITests are written in Python. They are easier to write, but they do not run on each and every platform, and they are usually slower. CppUnitTests, on the other hand, are written in C++. They are much faster, and they run on every platform that CI runs to make sure that everything is built and can be run correctly.

An Example Crash Testing

Consider the below issue around footnotes:

• tdf#150457 Fix crash on hovering removed footnote reference

This problem was happening when someone created a footnote, deleted the reference, and then hovered the mouse on the removed footnote reference. To reproduce that, one could use keyboard to generate a key sequence that repeats the required steps:

Write something, add footnote, select all the footnotes and remove them, then go back to the text, and hover the footnote reference.

Using keyboard-only is not always enough, but here it was possible. To implement the UITest, you should first find the appropriate place to put the test file, and then write a Python script for that. Here, the test was written in sw/qa/uitest/writer_tests2/deleteFootnotes.py. The UITest test can be found alongside the bug fix, in the class class tdf150457(UITestCase):

• tdf#150457 Fix crash on hovering removed footnote reference

If you look into the code, the test_delete_footnotes() function consists of many invocations of postKeyEvent calls, that emulate key input events:

pTextDoc->postKeyEvent(LOK_KEYEVENT_KEYINPUT, 'a', 0);

To insert footnotes, UNO commands are used.

dispatchCommand(mxComponent, ".uno:InsertFootnote", {});

Just doing the same steps would be enough, as if the crash happens with the fix in place, or in a bad commit in the future, the test would fail. This test failure will prevent the same problem in the future.

The nice thing is that it turned out the same test could have been written using C++ and CppUnitTest, which is considered superior.

The new CppUnitTest can be found in the below change:

• tdf#150457 Port UITest to CppunitTest

The new test resides in sw/qa/extras/uiwriter/uiwriter3.cxx, and essentially uses postKeyEvent and dispatchCommand as similar functions.

If you look at the current version of the test, you can see that it was simplified in later commits, but the idea is the same: “repeat the same steps that lead to crash in the code”.

Final Words

It is expected that every bug fix is accompanied with a test, to avoid seeing the same problem in the future. If you want to know more about UITests and CppUnitTests, please take a look at this TDF Wiki pages:

• The Document Foundation Wiki – CppUnitTest
• The Document Foundation Wiki – Missing Unit Tests
• The Document Foundation Wiki – UITests

Also, keep in mind that looking into the existing tests is the best way to understand how to write a new test. Therefore, make sure that you look into qa/ folders of the modules when you want to add a new test.

If you copy contents from LibreOffice Writer to a plain text editor like gedit or Notepad, you will see that it does a straightforward thing: It copies the text and some basic formatting like converting bullets to ‘•’. For the Writer tables, the conversion is very simple right now: every cell is written in a separate line.

For example, if you have a table like this:

A | B
--|--
C | D

When you copy the table from LibreOffice Writer and paste it into a plain text editor, it will become something like this, which is not always desirable.

A
B
C
D

It is requested that like LibreOffice Calc, or Microsoft Word, and many other programs, the copy/paste mechanism should create a text like this:

A	B
C	D

The columns are separated by <tab>.

This feature request is filed in Bugzilla as tdf#144576:

• tdf#144576 – Copy a table from Writer to plain text editor or as un-formatted text pastes a list instead of matrix

Code pointers for Handling Writer tables

There are many steps in copy/pasting, including the data/format conversion and clipboard format handling. Here, you have to know that the document is converted to plain text via “text” filter.

The plaintext (ASCII) filter is located here in the LibreOffice core source code:

• sw/source/filter/ascii

Therefore, to change the copy/paste output, you have to fix the ASCII filter. That would also provide the benefit that plain text export will be also fixed as requested here.

In this folder, there are a few files:

$ ls sw/source/filter/ascii/
ascatr.cxx parasc.cxx wrtasc.cxx wrtasc.hxx

To change the output, you have to edit this file:

• sw/source/filter/ascii/wrtasc.cxx

In this file, there is a loop dedicated to create the output.

// Output all areas of the pam into the ASC file
do {
    bool bTstFly = true;
    ...
}

Inside this loop, the code iterates over the nodes inside the document structure, and extracts text from them. To check for yourself, add the one line below to the code, build LibreOffice, and then test. You will see that a * is appended before each node.

SwTextNode* pNd = m_pCurrentPam->GetPoint()->GetNode().GetTextNode();
if( pNd )
{
+   Strm().WriteUChar('*');
    ...
}

For example, having this table, with 1 blank paragraph up and down:

A | B
--|--
C | D

You will get this after copy/paste into a plain text editor:

*
*a
*b
*c
*d
*

To fix the bug, you have to differentiate between table cells and other nodes. Then, you should take care of the table columns and print tab between them.

To go further, you can only add star before table cells:

if( pNd )
{
    SwTableNode *pTableNd = pNd->FindTableNode();
+   if (pTableNd)
+   {
+       Strm().WriteUChar('*');
+    }
    ...
}

You can look into how other filters handled tables. For example, inside sw/source/filter/html/htmltab.cxx you will see how table is managed, first cell is tracked and appropriate functions to handle HTML table are called.

For the merged cells, the EasyHacker should first checks the behavior in other software, then design and implement the appropriate behavior.

Final Words

To gain a better understanding of the Writer document model / layout, please see this document:

• OpenOffice Wiki: Writer/Core And Layout

This presentation is also very helpful to gain a good understanding of Writer development:

Introduction to Writer Development – LibreOffice 2023 Conference Workshop, Miklos Vajna

In this blog post, I discuss gbuild for Java tests. The goal is to write a Makefile to compile and run a JUnit test for LibreOffice. You can also refer to part 1 and part 2 for a brif overiew on gbuild, the LibreOffice build system.

Macro Examples from gbuild for Java Tests

In the first post on gbuild, I have mentioned some macro examples including gb_Output_announce which was used to print nice messages like the ones including “[CXX]”. Now let’s explain some more macros related to Java tests.

Consider that you want to compile and run a JUnitTest. To do that, you need to write the test in a Java file, and create a Makefile to run that.

This is an example for running a test defined in Java file sw/qa/complex/indeterminateState/CheckIndeterminateState.java.

$(eval $(call gb_JunitTest_JunitTest,sw_complex))

$(eval $(call gb_JunitTest_add_sourcefiles,sw_complex,\
sw/qa/complex/indeterminateState/CheckIndeterminateState \
))

$(eval $(call gb_JunitTest_use_unoapi_jars,sw_complex))

$(eval $(call gb_JunitTest_add_classes,sw_complex,\
complex.indeterminateState.CheckIndeterminateState \
))

The make file for running this Java test consists of calling multiple macros. It starts with gb_JunitTest_JunitTest macro, which defines the test by its name, sw_complex. This macro is defined in solenv/gbuild/JunitTest.mk. If you grep for define in the same file, you will see this result:

$ grep -w define solenv/gbuild/JunitTest.mk
define gb_JunitTest_JunitTest
define gb_JunitTest_set_defs
define gb_JunitTest_add_classes
define gb_JunitTest_add_class
define gb_JunitTest_add_sourcefile
define gb_JunitTest_add_sourcefiles
define gb_JunitTest_use_jar
define gb_JunitTest_use_jars
define gb_JunitTest_use_jar_classset
define gb_JunitTest_add_classpath
define gb_JunitTest_use_system_jar
define gb_JunitTest_use_system_jars
define gb_JunitTest_use_external
define gb_JunitTest_use_externals
define gb_JunitTest_use_customtarget
define gb_JunitTest_use_customtargets
define gb_JunitTest_use_unoapi_jars
define gb_JunitTest_use_unoapi_test_class
define gb_JunitTest_set_unoapi_test_defaults
define gb_JunitTest_JunitTest

To stick to the macros used in the above example, I describe these macros:

gb_JunitTest_add_sourcefiles: This macro adds a Java source file to the test. It defines the code that adds the sw/qa/complex/indeterminateState/CheckIndeterminateState.java to the test. But please note that you should drop the .java extension:

$(eval $(call gb_JunitTest_add_sourcefiles,sw_complex,\
sw/qa/complex/indeterminateState/CheckIndeterminateState \
))

The other macro gb_JunitTest_use_unoapi_jars, adds the UNO API JAR files to be used with the test.

And in the end, you need to add the test class name using gb_JunitTest_add_classes macro. The class name is visible in the end.

The result can be quite complex, but it works. 🙂

java.exe -Xmx64M -classpath "$W/JavaClassSet/JunitTest/sw_complex;C:/cygwin64/home/user/lode/opt/share/java/junit.jar;$I/program;$W/Jar/OOoRunner.jar;$I/program/classes/libreoffice.jar;$W/Jar/test.jar" -Dorg.openoffice.test.arg.soffice="path:$I/program/soffice" -Dorg.openoffice.test.arg.env=PATH="$PATH" -Dorg.openoffice.test.arg.user=file:///$W/JunitTest/sw_complex/user org.junit.runner.JUnitCore complex.indeterminateState.CheckIndeterminateState

The above is the actual command that runs the test. Please note that if you forget the gb_JunitTest_add_classes macro to define the class name, the test may compile, but it will not run.

As an example, you can see the below patch. This patch fixes the problem of the JUnit test not running:

• Fix sw_complex JUnitTest not running

Final Words

Many macros are available in gbuild, making easier to create Makefiles to compile and run tests, build libraries and executable applications and many other relevant tasks. The best way to find and understand these macros is to look at the Makefiles written by others to the same task. Look for .mk files, and if you want to see the implementation of the macros, look into solenv/gbuild/.

I will write more about gbuild macros in the next series of blog posts on gbuild.

Now that year 2024 has come, I want to briefly discuss the year 2023 around the development blog, and the outlook for 2024 here.

My goal is to help people understand LibreOffice code better, and ultimately get involved in LibreOffice core development to make LibreOffice better for everyone. In 2023, I wrote 23 posts around LibreOffice development in the dev blog (3 of them are unpublished drafts).

At The Document Foundation (TDF), our aim is to improve LibreOffice, the leading free/open source office software that you and many other people around the world use. Our work is community-driven, and we need your help.

LibreOffice conference 2023

Outlook For the New Year

My focus for 2024 in this blog will be:

1. Introducing new EasyHacks
2. Discussing how to fix crashes
3. Explaining LibreOffice architecture
4. Describing user interface creation with VCL
5. Explaining LibreOffice extensions

You can give feedback by writing a comment here, or sending an email to hossein AT libreoffice DOT org.

I provide mentoring support to those who want to start LibreOffice development. You are welcome to contact me if you need help to build LibreOffice and do some EasyHacks via the above email address.

I hope the best for you in the new year 2024.

In my previous blog post on creating LibreOffice extensions with Python, I have discussed how to write a Python code that works with LibreOffice API, and can be run and debugged in an IDE, and packed later in an extension. Now I discuss how to debug the Python code. (more…)