3 Oct 2024

Setting dialogs that are available via UNO Commands

LibreOffice options page provides rich set of settings for everyone who wants to tune LibreOffice to match their needs. But, what if you as a developer, need setting dialogs that are needed elsewhere in the LibreOffice application? Here I discuss some of such use cases, which are handled by defining UNO commands.

Options Page

The code for providing “Tools > Options” is not in a single module, but main part resides in cui module, which contains code which is used across different modules. Looking into cui/source/options/ folder from LibreOffice core source code, you can see various different source files related to the options. The biggest file there is cui/source/options/treeopt.cxx, which is the actual implementation of the tree-based dialog that you see when you open Tools > Options dialog. There are other C++ files that handle .ui files related to options. You can find those UI files in cui/uiconfig/ui/ folder with a name like opt*.ui:

$ ls cui/uiconfig/ui/opt*.ui

These files can be edited and they are used as described in the LibreOffice design blog:

UNO Dispatch Commands

Only some of the dialogs can be opened available via UNO dispatch commands. As an example, you may see “.uno:AdditionsDialog” is used both in cui/source/options/optgdlg.cxx for creating a dialog in Tools > Options (when you click for “more icons”), and also in sfx2/source/appl/appserv.cxx.

You can try running this UNO command in LibreOffice BASIC editor with this code snippet:

Sub Main()
    Set oDispatch = CreateUnoService("com.sun.star.frame.DispatchHelper")
    Dim args(0) As New com.sun.star.beans.PropertyValue
    Set oFrame = StarDesktop.Frames.getByIndex(0)
    oDispatch.executeDispatch(oFrame, ".uno:AdditionsDialog", "", 0, args)
End Sub

The above command is defined specifically to help developers use the “Extensions” dialog, anywhere in LibreOffice UI, from top menus to context menus and toolbars and also in code, in a simple way.

"Extensions

There is another dialog titled “Security Options and Warnings”, which is opened through .uno:OptionsSecurityDialog UNO command. In this way, it can be used easily in other modules of LibreOffice.

SecurityOptionsDialog

SecurityOptionsDialog

Implementing UNO Command

Adding a new UNO command was discussed before, in a separate blog post:

Adding a new UNO command

Adding a new UNO command for an options dialog is basically the same. There can be differences regarding the configurations and the data that is passed between the dialog and the caller.

When you create a dialog box directly like the code snippet below, you have access to the member functions defined for that specific dialog:

IMPL_LINK_NOARG( SwGlossaryDlg, PathHdl, weld::Button&, void )
{
    SvxAbstractDialogFactory* pFact = SvxAbstractDialogFactory::Create();
    ScopedVclPtr<AbstractSvxMultiPathDialog> pDlg(pFact->CreateSvxPathSelectDialog(m_xDialog.get()));
    SvtPathOptions aPathOpt;
    const OUString sGlosPath( aPathOpt.GetAutoTextPath() );
    pDlg->SetPath(sGlosPath);
    if(RET_OK == pDlg->Execute())
    {
        const OUString sTmp(pDlg->GetPath());
        if(sTmp != sGlosPath)
        {
            aPathOpt.SetAutoTextPath( sTmp );
            ::GetGlossaries()->UpdateGlosPath( true );
            Init();
        }
    }
}

As you can see, pDlg->GetPath() is accessible here, and you can use it to pass data. But when you are using UNO commands, those functions will not be available directly. Instead, you may pass values that denote the data that will be read from somewhere else, like the configuration.

For example, consider there are multiple paths that you may want to edit using this UNO command with the same dialog. In this case, you can pass a value that shows the associated path that you are changing. It can be passed as an enumeration, and then you set and/or get the value directly from the configuration.

In this way, the callers in the C++ code will have much easier task to do, as it is only calling the UNO command, and the rest is done in the implementation of the UNO command.

For AdditionsDialog, the calling code is as simple as this in cui/source/options/optgdlg.cxx:

IMPL_STATIC_LINK_NOARG(OfaViewTabPage, OnMoreIconsClick, weld::Button&, void)
{
    css::uno::Sequence<css::beans::PropertyValue> aArgs{ comphelper::makePropertyValue(
u"AdditionsTag"_ustr, u"Icons"_ustr) };
    comphelper::dispatchCommand(u".uno:AdditionsDialog"_ustr, aArgs);
}

Passing Parameters to UNO Commands

UNO dispatch commands can take parameters. As an example, take a look at .uno:NewDoc defined in sfx2/sdi/sfx.sdi:

SfxStringItem NewDoc SID_NEWDOC
(SfxStringItem Region SID_TEMPLATE_REGIONNAME,SfxStringItem Name SID_TEMPLATE_NAME)
[
...
]

As you can see, there are two parameters:

SfxStringItem Region SID_TEMPLATE_REGIONNAME SfxStringItem Name SID_TEMPLATE_NAME

The SfxStringItem is the type, Region and Name are the names of the parameters, SID_TEMPLATE_REGIONNAME and SID_TEMPLATE_NAME are the constants used for passing parameters. The parameter type is not limited to numbers and strings, and it can be any defined class.

To set and get data for this parameters, you may use appropriate Set and Put functions. For example, this gets the parameter if set:

const SfxStringItem *pItem = rSet.GetItemIfSet( SID_TEMPLATE_REGIONNAME, false)

Or, this sets the data:

OUString sVal = u"test"_ustr;

rSet.Put( SfxStringItem( SID_TEMPLATE_REGIONNAME, sVal ) );

Final Notes

To get to know better how to implement more complex UNO dispatch commands, you can refer to the implementation of the existing UNO commands to get idea about how they are implemented. You can see a comprehensive list of UNO commands here:

15 Aug 2024

Extending the UNO API

Various functionalities of the LibreOffice are available through its programming interface, the UNO API. Here I discuss how to extend it.

What is UNO API?

Many functionalities of the LibreOffice is available through UNO API. You can write extensions and external programs that use LibreOffice functionality without the need to change the LibreOffice core source code.

Extensions work seamlessly with the software, and external applications can connect to the LibreOffice process and use it. The ability to do that depends on the UNO API.

On the other hand, some functionalities may not be available through this API. For example, newer features of the decent versions of LibreOffice, or functionalities that are not useful and/or important for external applications. Sometimes, you may want to use such functionalities elsewhere. Then you have to modify the LibreOffice core source code, and expose those functionalities through the API make them available to the external applications.

Let’s refer to the LibreOffice Developer’s Guide, which is mostly around the LibreOffice UNO API. There, you can read:

“The goal of UNO (Universal Network Objects) is to provide an environment for network objects across programming language and platform boundaries. UNO objects run and communicate everywhere.”

As UNO objects should be usable across different languages and platforms, they are described in an abstract meta language called UNOIDL (UNO interface definition language). This is similar to the IDL definitions in many other technologies like CORBA.

Example UNO API: FullScreen

The API that I discuss here, provides functionality to control full screen functionality for top level windows. Stephan, experienced LibreOffice developer, added that API in this commit:

commit af5c4092052c98853b88cf886adb11b4a1532fff

Expose WorkWindow fullscreen mode via new XTopWindow3

...deriving from the existing XTopWindow2. (Exposing this functionality via UNO
is useful e.g. for some embedded LOWA example application.)

The changes in this commit are over these files:

offapi/UnoApi_offapi.mk
offapi/com/sun/star/awt/XTopWindow3.idl
toolkit/inc/awt/vclxtopwindow.hxx
toolkit/source/awt/vclxtopwindow.cxx

First one, offapi/UnoApi_offapi.mk is needed to introduce the IDL file, according to its module, in a proper location. XTopWindow3.idl is added in com/sun/star/awt, which corresponds to com.sun.star.awt module. The other two, vclxtopwindow.hxx and vclxtopwindow.cxx are the implementation of the API in C++.

Let’s look into XTopWindow3.idl:

module com { module sun { module star { module awt {

/** extends XTopWindow with additional functionality

@since LibreOffice 25.2
*/
interface XTopWindow3: XTopWindow2 {
/** controls whether the window is currently shown full screen */
    [attribute] boolean FullScreen;
};

}; }; }; };

As you may see, it contains these important information:

1. It is an interface, called XTopWindow3.

2.It has a boolean attribute, FullScreen.

3. This functionality will be available in LibreOffice 25.2 and later.

4. This interface extends XTopWindow interface. You may find the documentation for XTopWindow in api.libreoffice.org.

More information about XTopWindow interface can be found in XWindow section of the LibreOffice Developer’s Guide, chapter 2.

C++ Implementation

C++ implementation basically consists of two functions to set and get the FullScreen property:

sal_Bool VCLXTopWindow::getFullScreen() {
    SolarMutexGuard g;
    if (auto const win = VCLXContainer::GetAsDynamic()) {
        return win->IsFullScreenMode();
    }
    return false;
}

void VCLXTopWindow::setFullScreen(sal_Bool value) {
    SolarMutexGuard g;
    if (auto const win = VCLXContainer::GetAsDynamic()) {
        return win->ShowFullScreenMode(value);
    }
}

Final Words

Some APIs are much more complex. The one that was discussed here was only an example to show the required things to extend UNO API. You can browse the API documentation in api.libreoffice.org for more complex APIs:

29 Jul 2024

Fuzz testing to maintain LibreOffice code quality

Here I discuss what fuzz testing is, and how LibreOffice developers use it incrementally to maintain LibreOffice code quality.

Maintaining Code Quality

LibreOffice developers use various different methods and tools to maintain LibreOffice code quality. These are some of them:

1. Code review: Every patch from contributors should pass code review on Gerrit, and after conforming to coding standards and conventions, it can become part of the LibreOffice source code.

2. Static code checking: “Coverity Scan” continuously scans LibreOffice source code to find the possible defects. An automated script reports these issues to the LibreOffice developers mailing list so that developers can fix them.

3. Continuous Testing: There are various C++ unit test and Python UI tests in LibreOffice core source code to make sure that the functionalities of the software remain working during the later changes. They are also helpful for making sure that the fixed regressions do not happen again. These test run continuously for each and every Gerrit submission on CI machines via Jenkins.

4. Crash testing: A good way to make sure that LibreOffice works fine is to batch open and convert a huge set of documents. This task is done regularly, and if some failure occurs developers are informed to fix the issue.

5. Crash reporting: LibreOffice uses crash testing to find out about the recurrent crashes, and fix them.

6. Tinderbox Platforms: Using dedicated machines with various different architectures, LibreOffice developers make sure that LibreOffice source code builds and runs without problem on different platforms. Here is the description of tinderbox (TB) from TDF Wiki:

Tinderbox is a script to run un-attended build on multiple repos, for multiple branches and for gerrit patch review system.

LibreOffice tinderboxes status

LibreOffice tinderboxes status

You can see the build status here:

https://tinderbox.libreoffice.org/

7. Fuzz testing: LibreOffice software is checked continuously using Fuzz testing. This is essentially giving various automated inputs to the program to find the possible places in the code where problem occurs. Then, developers will become aware of the those problematic places in the code, and can fix them.

Fuzz Testing LibreOffice

Fuzz testing on LibreOffice source code is active since 2017, and since then there has been various bug fixes for the problems that the fuzz tester reported. You can see more than 1500 of such fixes in the git log until now:

$ git shortlog -s -n --grep=ofz#

Issues Found with Fuzz Testing

This tool can find various different problems. These issues are then filed in a section of Chromium bug tracker, and after ~30 days, they are made public. When developers fix bugs of this kind, they refer to the issue number (for example 321) as ofz#321. A comprehensive list of all issues found is visible here:

Fixing the Issues

Let’s look at one of the fixes. You can find commits related to fuzzing with:

$ git log --grep=ofz

This is a recent fix from Caolán, an experienced LibreOffice developer that provided most of the fixes found through oss-fuzz:

commit d30ecb5fb07f005ebd944e864f0a15678289a4ed
    ofz#69809 Integer-overflow

--- a/filter/source/graphicfilter/icgm/cgm.cxx
+++ b/filter/source/graphicfilter/icgm/cgm.cxx
@@ -227,7 +227,7 @@ double CGM::ImplGetFloat( RealPrecision eRealPrecision, sal_uInt32 nRealSize )
         else
         {
             sal_Int32* pLong = static_cast<sal_Int32*>(pPtr);
-            nRetValue = static_cast(abs( pLong[ nSwitch ] ));
+            nRetValue = fabs(static_cast(pLong[nSwitch]));
             nRetValue *= 65536;
             nVal = static_cast( pLong[ nSwitch ^ 1 ] );
             nVal >>= 16;

As you can see, using abs() first, and then casting to double is changed in this commit to cast to double first, and then using fabs(). The reason of this change lies in the data type of some variables.

pLong is an array of sal_Int32, which is 32 bit signed integer. It can take values from -2,147,483,648 to 2,147,483,647. As you can see, the smallest negative 32-bit signed integer can not be stored in the same 32-bit signed variable if abs() is used to remove sign from that.

As the result is stored in nRetValue, a varible of type double, it is possible to first cast the array item to double, and then use floating point version of absolute function, fabs() over it. In this way, “integer overflow” will not happen anymore.

This patch was one of the smallest examples of what a fix can be. There are many bugs that are more complex, and require more careful examination to provide a fix.

Further Reading

You can read more in the TDF Wiki article around fuzz testing in LibreOffice. Also, OSS-Fuzz documentation is a good place to look into:

25 Jul 2024

LibreOfficeKit for document conversion

In the previous blog post, I provided a brief introduction to LibreOfficeKit API which one can use for accessing LibreOffice functionalities in an external application. Here I discuss in detail how to use LibreOfficeKit for converting an ODT to PDF, or from/to virtually any other format that LibreOffice supports.

LibreOfficeKit C++ Headers

For this example, you only need one header: LibreOfficeKit/LibreOfficeKit.hxx. Include it, and that is enough. This header will be available in the future versions of LibreOffice community. But for now, you may need to download LibreOfficeKit headers folder from LibreOffice core source code, and put it alongside your source code.

C++ Example

The example is relatively short. Here it is:

#include <iostream>
#include <LibreOfficeKit/LibreOfficeKit.hxx>
int main(int argc, char *argv[])
{
    if(argc < 2)
    {
        std::cout << "Usage: lokconvert  \n";
        return 1;
    }
    const char *input = argv[1];
    const char *output = argv[2];

    lok::Office * llo = NULL;
    try
    {
        const char lo_path[] = LOROOT "/program/";
        llo = lok::lok_cpp_init(lo_path);
        if (!llo) {
            std::cerr << "Error: could not initialize LibreOfficeKit\n";
            return 1;
        }

        lok::Document * lodoc = llo->documentLoad(input, NULL /* options */);
        if (!lodoc) {
            std::cerr << "Error: could not load document: " << llo->getError() << "\n";
            return 1;
        }

        if (!lodoc->saveAs(output, "pdf", NULL /* options */))
        {
            std::cerr << "Error: could not export document: " << llo->getError() << "\n";
            return 1;
        }
    }
    catch (const std::exception & e)
    {
        std::cerr << "Error: LibreOfficeKit exception: " << e.what() << "\n";
        return 1;
    }

    std::cerr << "Success!\n";
    return 0;
}

The example is simple. Input and output file names are passed via command line arguments, and are received via argv[1] and argv[2], if argument count, argc, is at least 3. First one is the program name itself, which we don’t use here.

LibreOffice binary folder is needed here, so you should set OO_SDK_URE_BIN_DIR environment variable, or you may even set that in your program itself.

This part of the code is for LibreOfficeKit initialization:

llo = lok::lok_cpp_init(lo_bin_dir);

And after that, it is the time to load the document:

lok::Document* lodoc = llo->documentLoad(input, NULL /* options */);

Having the lodoc pointer is necessary for converting the document using saveAs function:

lodoc->saveAs(output, "pdf", NULL /* options */)

As you can see, relevant code is inside a try/catch block. In case some exception occur, you will see relevant error information on the console. Also, in each step, from initializing LibreOfficeKit, to loading and in the end generating the output, there are error handling mechanisms to make sure that the work goes forward smoothly.

In the end, if everything is successful, you will see this message on the screen:

Success!

Build with cmake

You can use cmake to build this example. Here is an example CMakeLists.txt which also provides LO_ROOT and LO_SDK_ROOT to be used inside C++ file, as an alternative to setting it via an environment variable, or putting it manually in the code.

cmake_minimum_required(VERSION 3.5)

project(lokconv LANGUAGES CXX)

set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON)

add_executable(lokconv main.cpp)

include(GNUInstallDirs)

set(LOVERSION 24.2)

if(WIN32)
    set(LOROOT C:/Progra~1/LibreOffice)
    set(LOSDKROOT ${LOROOT}/sdk)
    add_definitions(-DWIN32)
elseif(APPLE)
    set(LOROOT /Application/LibreOffice.app/Contents/Frameworks)
    set(LOSDKROOT /Application/LibreOffice${LOVERSION}_SDK)
    add_definitions(-DMACOSX)
else()
    set(LOROOT /opt/libreoffice${LOVERSION})
    set(LOSDKROOT ${LOROOT}/sdk)
    add_definitions(-DLINUX)
endif()

add_compile_definitions(LOK_USE_UNSTABLE_API)

target_compile_definitions(lokconv PRIVATE LOROOT="${LOROOT}")

include_directories(lokconv PRIVATE
    ${LOSDKROOT}/include
)

Final Words

There are many nice things that can be done with LibreOfficeKit, but you may have to enable LOK_USE_UNSTABLE_API to be able to use those functionalities. This symbol is visible in the above cmake file. But, in the C++ example I did not use any of such functions. I will discuss about these functionalities in later blog posts.

27 Jun 2024

LibreOfficeKit API in action

If you want to use LibreOffice functionality in your applications, LibreOfficeKit API is one of the good ways to do that. Here I describe how, with some examples. If you want to add the capability of loading, displaying, editing, saving and/or converting LibreOffice/MS Office files to your application, you have come to a good place.

What is LibreOfficeKit?

LibreOfficeKit is the (relatively) new API to access LibreOffice functionalities in C/C++ without the need of older UNO API and its dependencies. It is the underlying technology for LibreOffice Online, and also LibreOffice Viewer for Android.

LibreOffice functionality is usable via the older UNO API, which is several years old and provides many functionalities of LibreOffice through network or sockets. But UNO API is not helpful if you want to have the LibreOffice UI in your application. In comparison, LibreOfficeKit is capable of doing that.

LibreOfficeKit can create tiles from the LibreOffice, as a static library. It lets the user interact with LibreOffice from somewhere else, either different application, or a web browser.

In this way, it becomes much easier to create applications that incorporate the actual functionalities of LibreOffice in them, while having the LibreOffice UI in themselves.

Example: GTK Tiled Viewer

As an example, gtktiledviewer provides a small editor that can load the LibreOffice/MS Office files, display them and the user can actually edit those files.

LibreOffice GTK Tiled Viewer

LibreOffice GTK Tiled Viewer

If you have a working build of LibreOffice, you can run thegtktiledviewer application this way:

bin/run gtktiledviewer --lo-path=$PWD/instdir/program path/to/test.odt

Building such an application will be much easier compared to building and running LibreOffice UNO API applications which require many dependencies.

In order to compile LibreOfficeKit applications, you need LibreOfficeKit headers. These headers will be shipped with the  next version of LibreOffice community distribution, LibreOffice 24.8.

Example Usage: Document conversion

Another interesting example is document conversion. For example, if you want to convert an ODT file to PDF or other formats with LibreOffice, you can use LibreOfficeKit API to do that.

One example code that does that is the lloconv project. You may find it on gitlab:

The conversion code happens inside convert.cc source file, and it is pretty straightforward, although there are some extra code which handles finding LibreOffice binary path, and parsing command line arguments.

The important functions that are used in loading and converting documents can be found in these lines:

Office * llo = lok_cpp_init(lo_path);

And also:

Document * lodoc = llo->documentLoad(input_url.c_str(), options);

And at last:

lodoc->saveAs(output_url.c_str(), format, options)

It  essentially initializes the document vialok_cpp_init, then loads the document using documentLoad function, and at last convert it to the final file format using saveAs() function.

The API for conversion is pretty straightforward, and it is enough to to know the path to the LibreOffice installation path to be able to compile and run the C++ example.

More interesting things are possible with LibreOfficeKit API, and I will write more about them.

30 May 2024

Porting Java tests to C++

In this blog post, I discuss porting Java tests from Java to C++. We have many Java tests, and porting them to C++ is beneficial for LibreOffice. Here I discuss why, and how.

Why Porting Java Tests?

In the past, Java was used extensively in LibreOffice, and many tests where written in Java. They consist of various tests, including JUnit.

Searching for junit in LibreOffice source code gives many results: (Note that this is not the number of tests)

$ git grep -i junit | wc -l

901

Although these tests are useful, they have drawbacks:

1. They are slower than C++ CppUnitTests

2. They are outside process, which is creates them even more slower, and harder to debug

Porting these tests to C++ can make them faster, more reliable, and easier to debug. Let’s see how we can do it.

Start from Examples

The best way to find out how to do porting is to look into the previous examples. This issue is around porting Java API tests to C++:

There are many commits in the above page, which can be an interesting way to learn how to do that. For example, consider this commit:

In the above commit, Jens has ported qadevOOo/tests/java/ifc/sheet/_XSheetCellRanges.java test to C++ test test/source/sheet/xsheetcellranges.cxx.

I can summarize the changes in this way:

1. An equivalent C++ file, in test/ folder

2. Removal of Java file from qadevOOo/.

3. The Makefiles in test/ and qadevOOo/ folders should reflect the removal of Java test, and addition of C++ test.

4. Some Java tests are adjusted to reflect the deletion of the old Java test.

5. Some C++ tests are adjusted to reflect the addition of new C++ test.

Final Words

To be able to port a test, one should be able to understand the old test. Reading the code is the best way that one can achieve this purpose. By looking into similar ports, you can gain a better understanding of what to do. In Bugzilla page for tdf#45904, you may find similar ports.

14 May 2024

Crash fixes, part 4: assertion failure

In the previous parts of the blog posts series on fixing software crashes, I have written about some crash fixes in LibreOffice around segfaults, aborts, and I discussed how test them. Here I write about fixing assertion failure.

What is Assertion Failure?

Assertion is a mechanism provided to the developers to make sure that things are good in runtime, conforming to what they were assuming. For example, making sure that some pointer is valid, some data elements match expectation, or some similar assumptions that need to be valid in order to get the correct results.

As an example, consider the C++ function basegfx::utils::createAreaGeometryForLineStartEnd(), which creates a geometric representation of an arrow. The code resides here:

basegfx/source/polygon/b2dlinegeometry.cxx:84

This is the line of code which contains assertion:

assert((rCandidate.count() > 1) && "createAreaGeometryForLineStartEnd: Line polygon has too few points");

On top of the actual function implementation, the C++ code asserts many conditions to make sure that they are met. In the above assertion, it checks to make sure that the number of points in the given data structure is more than 1. Otherwise, it leads to an assertion failure.

For various reasons, sometimes these sort of assumption may not be valid. To avoid reaching to incorrect results, it is important to have such assertions in place, to find such issues as soon as possible. If you are developing LibreOffice, you may have already built the code from sources in debug mode. Therefore, you may see software stops working, or simply crashes.

This crash may not happen for the end users, which use the release version of software. Therefore, these type of crashes have lower impact for the end users, and they are usually considered of lower importance compared to the crashes that happen for the end users.

Backtrace

One of the things that can help diagnose the problem is the stack trace, or simply backtrace. The way to obtain a backtrace depends on the platform and/or IDE that you use. If you use an IDE like Qt Creator, Visual Studio, etc., getting a backtrace would be as easy as debugging LibreOffice, making the assert fail, and then copy the backtrace from the UI. To learn more about IDEs, see this Wiki page:

If you want to use gdb on Linux, you may run LibreOffice with this command line:

$ instdir/program/soffice –backtrace

and then make the assert fail, and you will have the backtrace in gdbtrace.log file. You can learn more int this QA Wiki article:

TDF Wiki: QA/BugReport/Debug_Information

One thing to mention is that if you work on a reported bug regarding to assertion failure, then the actual way to reproduce the issue and make the assertion fail is usually described in the relevant TDF Bugzilla issue. In the meta bug related to assertion failure, you may find some of these issues in the last part of this blog post.

Fixing the Problem

To fix the problem, first you should gain understanding of the assumption, and why it fails. You should be able to answer these questions by reading the code and debugging it:

  • What does some assumption mean?
  • Why it fails?
  • How to fix that, so that it does not fail?

To gain this understanding, you have to look into the places where backtrace points. Backtrace can be complex, containing the whole stack of the function calls across the software, linking to places in the code, but let’s discuss a simplified form.

Consider this bug fix:

tdf#152012 Fix assert fail on opening date picker

The problem was that an assertion failure was happening when opening date picker field in a DOCX file. This is the simplified form of the stack trace:

1  __pthread_kill_implementation           pthread_kill.c:44
2  __pthread_kill_internal                 pthread_kill.c:78
3  __GI___pthread_kill                     pthread_kill.c:89
4  __GI_raise                              raise.c:26
5  __GI_abort                              abort.c:79
6  __assert_fail_base                      assert.c:92
7  __GI___assert_fail                      assert.c:101
8  o3tl::span::operator[]                  span.hxx:83
9  OutputDevice::ImplLayout                text.cxx:1396
10 OutputDevice::DrawTextArray             text.cxx:948
11 Calendar::ImplDraw                      calendar.cxx:71
12 Calendar::Paint                         calendar.cxx:1133

The problem was caused by an out of bound access to a vector of integers, and to fix that, you had to see why this happens, and fix that.

This is the description from the commit title:

Previously, for the 7 days of the week, a 7 letter string smtwtfs (depending on the week start day this can be different, but length is always 7) was sent to this method without providing the length, thus the string length: 7 was used. In this case, positions of the letters were calculated and used from other array named mnDayOfWeekAry[7]. mnDayOfWeekAry[k+1] was used as the position of letter k (k=0..5). In this case, there was 7 letters for 7 days, and only 6 positions provided by the array. This caused assertion failure in span.hxx:83 when trying to access mnDayOfWeekAry[7] via o3tl::span<T>::operator[].

As you can see, a false assumption was creating assertion failure, and the fix was essentially changing the code based on the corrected assumption.

Sometimes, the fix can be easier. As an example, by only checking a pointer to make sure that it is valid, the assertion failure does not happen. Therefore, to fix the problem, you have to carefully study the code and its behavior.

Final Notes

Many of the assertion failure bugs are fixed in daily works of the developers, before causing problems for the end users, which use the “release” version of the software, that do not crash because of the assertion failure. But there are some of these issues remaining.

Do you want to help fix some of the remaining issues? Then, please refer to the list here, read some bug report, and pick one:

18 Apr 2024

Crash fixes part 3 – Testing crashes

I have previously discussed fixing crashes in 2 parts (segfaults, aborts). Here I discuss testing crashes to avoid creating re-creating regressions.

Why testing crashes?

When you fix a crash, you have to make sure that it does not happen again in the future. The key to achieve such a goal is to write a suitable test. The test should do the exact steps to reproduce the problem on the program in order to detect the known crash before the new code is merged.

This can be done using either UITests, or CppUnitTests. UITests are written in Python. They are easier to write, but they do not run on each and every platform, and they are usually slower. CppUnitTests, on the other hand, are written in C++. They are much faster, and they run on every platform that CI runs to make sure that everything is built and can be run correctly.

An Example Crash Testing

Consider the below issue around footnotes:

This problem was happening when someone created a footnote, deleted the reference, and then hovered the mouse on the removed footnote reference. To reproduce that, one could use keyboard to generate a key sequence that repeats the required steps:

Write something, add footnote, select all the footnotes and remove them, then go back to the text, and hover the footnote reference.

Using keyboard-only is not always enough, but here it was possible. To implement the UITest, you should first find the appropriate place to put the test file, and then write a Python script for that. Here, the test was written in sw/qa/uitest/writer_tests2/deleteFootnotes.py. The UITest test can be found alongside the bug fix, in the class class tdf150457(UITestCase):

If you look into the code, the test_delete_footnotes() function consists of many invocations of postKeyEvent calls, that emulate key input events:

pTextDoc->postKeyEvent(LOK_KEYEVENT_KEYINPUT, 'a', 0);

To insert footnotes, UNO commands are used.

dispatchCommand(mxComponent, ".uno:InsertFootnote", {});

Just doing the same steps would be enough, as if the crash happens with the fix in place, or in a bad commit in the future, the test would fail. This test failure will prevent the same problem in the future.

The nice thing is that it turned out the same test could have been written using C++ and CppUnitTest, which is considered superior.

The new CppUnitTest can be found in the below change:

The new test resides in sw/qa/extras/uiwriter/uiwriter3.cxx, and essentially uses postKeyEvent and dispatchCommand as similar functions.

If you look at the current version of the test, you can see that it was simplified in later commits, but the idea is the same: “repeat the same steps that lead to crash in the code”.

Final Words

It is expected that every bug fix is accompanied with a test, to avoid seeing the same problem in the future. If you want to know more about UITests and CppUnitTests, please take a look at this TDF Wiki pages:

Also, keep in mind that looking into the existing tests is the best way to understand how to write a new test. Therefore, make sure that you look into qa/ folders of the modules when you want to add a new test.

18 Mar 2024

Test improvement – More and better tests for LibreOffice

One of the areas that can help LibreOffice, but may not directly be visible to the users even though it has a big impact is the quality assurance, is automated testing.  Here, I discuss some areas and notes around improving tests for LibreOffice. First, I start with regressions and bug fixes without a test.

Missing Unit Tests

This is the description from the GSoC ideas page:

While there are some automated tests for LibreOffice, there are not nearly enough. Adding more and better tests helps developers who work on the code to be more productive by allowing them to find regressions as early as possible.

To elaborate more, I should say there are many regressions and bugs in general that are fixed, but lack testing. It is important to have tests for those bug fixes, to avoid such problems in the future. You can see a list of those fixed issues that lack tests here:

If you want to add some new tests for the bug fixes, first you should read the bug report very carefully to understand what is it about, and try to test the fix yourself. You can either use git revert command to revert the fix, and see the problem in action, or try changing back the fix in the code, if it is not too big.

That is important, because you have to see that the test fails without the fix in place, but succeeds when it is applied. This is an essential step when writing the test.

To know more about unit tests, you may look into these Wiki pages:

Porting Existing Test to C++ or Python

Some tests are written in the past, but now have issues because of the way they are written. In this case, porting them can provide improvement.

Tests can be written in multiple languages. At least, C++, Python, Java and BASIC are currently in use for different tests across the LibreOffice code base. Again in the GSoC ideas page, you can read:

There is some support in LibreOffice for automated tests, both at the level of unit tests, and at the level of system tests that drive a full LibreOffice instance. Currently tests can be written in C++, Java, or Python. Various new automated tests should be developed to improve the test coverage.

Tests written exclusively for Java (e.g. the JUnit framework) should be ported to C++ so that they can execute much more rapidly. Similarly, tests that do remote control of an existing LibreOffice instance, should be re-factored to run inside that instance to make debugging much easier.

Almost all JUnitTests and UITests, and also some smoke tests, run as an outside process. To verify, the trick is to remove the soffice(.exe) binary, or to remove its execute permission (on Linux). In this way, out-of-process tests should fail, as they need to run the soffice binary. After a successful build, you can see if this works. For example, try this:

make -d JunitTest_framework_complex

As an example, we have this issue around complex tests that should be ported to Python:

You may find many examples of the previously ported test in the issue page. Keep in mind that usually old Java tests should be removed in the same patch that provides the CppUnitTest port.

Also, you may find examples of UITests ported to C++ from Python. For example, see this patch:

It ports a previously implemented UITest from Python to C++. The result is a faster test, which is more robust.

Focusing on a Specific Area

If you want to add or port some tests, please focus on a specific area of the code. The LibreOffice code base is huge, and consists of more than 200 modules. It is not easy, and in fact not suggested, to work across different applications and modules. It is much better that you pick a specific application, and even inside that application, like Writer, Calc, Impress, etc., focus on a specific module. In this way, it would be much easier to understand the ideas and the way tests are implemented.

How Hard is it to Write / Port a Test?

One important question is around the complexity of writing or porting tests. The answer is not straightforward, as it depends on the area that the test is written. But one can take a look at what others did in the past. By looking into Gerrit or git log, you can find what other people, including GSoC applicants, were able to achieve during their work.

After writing a few tests, or porting a few tests, it may become easier for you to write more tests. Another thing is that you don’t have to write tests from scratch. There are many tests available in the LibreOffice code base, and you can create your new test based on existing tests and customize it to match the purpose of the issue you are working on.

29 Feb 2024

Writer tables converted to plain text – difficultyInteresting EasyHack

If you copy contents from LibreOffice Writer to a plain text editor like gedit or Notepad, you will see that it does a straightforward thing: It copies the text and some basic formatting like converting bullets to ‘•’. For the Writer tables, the conversion is very simple right now: every cell is written in a separate line.

For example, if you have a table like this:

A | B
--|--
C | D

When you copy the table from LibreOffice Writer and paste it into a plain text editor, it will become something like this, which is not always desirable.

A
B
C
D

It is requested that like LibreOffice Calc, or Microsoft Word, and many other programs, the copy/paste mechanism should create a text like this:

A	B
C	D

The columns are separated by <tab>.

This feature request is filed in Bugzilla as tdf#144576:

Code pointers for Handling Writer tables

There are many steps in copy/pasting, including the data/format conversion and clipboard format handling. Here, you have to know that the document is converted to plain text via “text” filter.

The plaintext (ASCII) filter is located here in the LibreOffice core source code:

Therefore, to change the copy/paste output, you have to fix the ASCII filter. That would also provide the benefit that plain text export will be also fixed as requested here.

In this folder, there are a few files:

$ ls sw/source/filter/ascii/
ascatr.cxx parasc.cxx wrtasc.cxx wrtasc.hxx

To change the output, you have to edit this file:

In this file, there is a loop dedicated to create the output.

// Output all areas of the pam into the ASC file
do {
    bool bTstFly = true;
    ...
}

Inside this loop, the code iterates over the nodes inside the document structure, and extracts text from them. To check for yourself, add the one line below to the code, build LibreOffice, and then test. You will see that a * is appended before each node.

SwTextNode* pNd = m_pCurrentPam->GetPoint()->GetNode().GetTextNode();
if( pNd )
{
+   Strm().WriteUChar('*');
    ...
}

For example, having this table, with 1 blank paragraph up and down:

A | B
--|--
C | D

You will get this after copy/paste into a plain text editor:

*
*a
*b
*c
*d
*

To fix the bug, you have to differentiate between table cells and other nodes. Then, you should take care of the table columns and print tab between them.

To go further, you can only add star before table cells:

if( pNd )
{
    SwTableNode *pTableNd = pNd->FindTableNode();
+   if (pTableNd)
+   {
+       Strm().WriteUChar('*');
+    }
    ...
}

You can look into how other filters handled tables. For example, inside sw/source/filter/html/htmltab.cxx you will see how table is managed, first cell is tracked and appropriate functions to handle HTML table are called.

For the merged cells, the EasyHacker should first checks the behavior in other software, then design and implement the appropriate behavior.

Final Words

To gain a better understanding of the Writer document model / layout, please see this document:

This presentation is also very helpful to gain a good understanding of Writer development:

Introduction to Writer Development – LibreOffice 2023 Conference Workshop, Miklos Vajna

Please accept YouTube cookies to play this video. By accepting you will be accessing content from YouTube, a service provided by an external third party.

YouTube privacy policy

If you accept this notice, your choice will be saved and the page will refresh.