23 Mar 2023

Internal includes instead of global ones – EasyHack

In every C/C++ source code file, we use header files to put declarations of functions, data types, class, macro and other relevant things inside it. These files have the extension of .h (for C) and .hxx (for C++).

We have different header types: internal and global includes. You can find some of the header files in many places. For example, in LibreOffice we have several modules that many other modules depend on them. Here, we focus on the VCL module. For the global includes, they are placed inside include/ folder, and for internal headers of VCL module, the include files are put inside include/vcl/.

On the other hand, not all the VCL headers are beneficial outside the VCL module. In this case, it is preferred that these include files are put inside the vcl/ folder. Usually, those header files should be inside vcl/inc folder.

As an example, we have the header include/vcl/salnativewidgets.hxx file which have declarations that are not used outside vcl/. In this case, we can put the declarations inside vcl. The suggested place is vcl/inc/nativewidgets.hxx.

Finding Internal Includes

As suggested in the bug report, one trick is to look at the class/function declaration. If it doesn’t have a SAL_DLLPUBLIC attribute decorating it, there is good chance that you can move it to the private headers, and in this case, vcl/inc/ folder.

Other than that, you should use the include syntax <…>. For example, if you have created vcl/inc/nativewidgets.hxx, then the include syntax would be:

#include <nativewidgets.hxx>

This is because the file is not exactly next to the cxx file. You can read more here:

Final Notes

EasyHacks are good starting points for someone who wants to start LibreOffice development. This specific issue for improving header files of VCL is avaiable as tdf#97228 in TDF’s Bugzilla. To understand how to start LibreOffice development, you can refer to our getting involved page in the TDF Wiki, or our video tutorial for getting started with LibreOffice development.

3 Mar 2023

VCL application in its minimal form

LibreOffice uses an internal GUI toolkit, named VCL (Visual Class Library). It creates the GUI widgets for LibreOffice, but it is not generally available for other applications. But there are ways that you can create standalone applications with VCL, at least to learn it better.

If you take a look into the vcl/workben, you will see several workbenches that are actually built during the build process. Previously, we discussed some of them in this post.

Now, I want to discuss the code of the simplest example: minvcl.

It basically creates a simple application with a single window, and that’s all! But even doing that needs several steps. Let’s discuss the code itself.

First come the includes. The include sal/config.h should come with each and every file in LibreOffice, and also VCL. Then, we have several includes from framework, cpppuhelper, comphelper, com/sun/star/lang and com/sun/star/uno, and then language classes from i18nlangtag.

#include <sal/config.h>
#include <framework/desktop.hxx>
#include <cppuhelper/bootstrap.hxx>
#include <comphelper/processfactory.hxx>
#include <com/sun/star/lang/XMultiServiceFactory.hpp>
#include <com/sun/star/uno/XComponentContext.hpp>
#include <i18nlangtag/languagetag.hxx>
#include <i18nlangtag/mslangid.hxx>

After that, comes the includes from VCL. We only use two: the one for a VCL application, and another for a VCL window:

#include <vcl/svapp.hxx>
#include <vcl/wrkwin.hxx>

Keep in mind that you have to provide the link libraries via the make file. In this case, the make file is vcl/Executable_minvcl.mk:

...
$(eval $(call gb_Executable_use_libraries,minvcl,\
    tl \
    sal \
    vcl \
    cppu \
    cppuhelper \
    comphelper \
    i18nlangtag \
    fwk \
))...

And at last, the include file for the main function of the VCL application. Every VCL application is supposed to have SAL_IMPLEMENT_MAIN() instead of main(), which is declared in this file.

#include <sal/main.h>
#include <iostream>

The class TheApplication should fall into an anonymous namespace. This class inherits the VCL application class with the name of Application. As you can see, we keep a VclPtr to a vcl::Window named mpWin that keeps a pointer to our window.

namespace
{
class TheApplication : public Application
{
public:
    virtual int Main();

private:
    VclPtr<vcl::Window> mpWin;
};
}

In the main file, we use a method named Create() to create a window. Then, we set the title using SetText(), and then invoke show() to make the window visible on the screen. This is the main file of the VCL application. It is the entry point for the application.

int TheApplication::Main()
{
    mpWin = VclPtr<WorkWindow>::Create(nullptr, WB_APP | WB_STDWORK);
    mpWin->SetText(u"Minimum VCL application with a window");
    mpWin->Show();
    Execute();
    mpWin.disposeAndClear();
    return 0;
}

We use LibreOffice APIs to create a component context and a service manager. The setProcessServiceFactory() function is used to set the service manager for the process. To understand the service manager, you can refer to the DevGuide:

The LanguageTag::setConfiguredSystemLanguage() function sets the language of the application to the system language. Finally, the framework::getDesktop() function terminates the framework and the DeInitVCL() function frees VCL resources.

SAL_IMPLEMENT_MAIN()
{
    try
    {
        TheApplication aApp;

        auto xContext = cppu::defaultBootstrap_InitialComponentContext();
        css::uno::Reference<css::lang::XMultiServiceFactory> xServiceManager(
        xContext->getServiceManager(), css::uno::UNO_QUERY);
        comphelper::setProcessServiceFactory(xServiceManager);
        LanguageTag::setConfiguredSystemLanguage(MsLangId::getSystemLanguage());
        InitVCL();

        aApp.Main();

        framework::getDesktop(::comphelper::getProcessComponentContext())->terminate();
        DeInitVCL();
        comphelper::setProcessServiceFactory(nullptr);
    }
    catch (...)
    {
        std::cout << "Exception has occurred\n";
        return 1;
    }

    return 0;
}

We should handle the exception in the main function. Here, we just print a simple text to make the user aware that some bad thing has happened.

That’s all! The application is complete now. You can run it via:

./bin/run minvcl

Keep in mind that you have to compile LibreOffice first. 🙂

minvcl

The minvcl window

Final Notes

There are many complex things that you can do with VCL, but this was the simplest example that you can possibly create! I will discuss creating more complex applications that create user interfaces with things like buttons, text and many other UI things. The nice thing about VCL is that you can even create UI from the widgets created from the .ui files designed with Glade. This makes creating GUIs much easier.

I will discuss more complex examples in the next blog posts.

21 Feb 2023

Telemetry required? Ask users first!

In this article, I will discuss the recent problems with compiling LibreOffice using Microsoft Visual Studio, things that I did to debug and find the root cause, the source of problem itself – which is problems in Microsoft’s telemetry – and how I could fix it.

Describing The Problem

Recently, I was encountering a problem when configuring LibreOffice’s source code before compilation. Sometimes, random errors appeared without further details on why. The title: “powershell.exe” was also strange, as I wasn’t using PowerShell directly.

Powershell Error

Powershell Error

At first, I ignored the message, but then it become more error common, and at some point the configuration was aborted. I ignored that for a while, but after a few days, one of the mentees reported a somehow similar problem.

The error was that the UCRT (which is Microsoft Visual Studio C++’s standard C library), was not found. This is an error log:

$ ./autogen.sh
.
.
.
checking for Windows SDK... found Windows SDK 10.0 (/cygdrive/c/PROGRA~2/WI3CF2~1/10)
checking for midl.exe... C:\Program Files (x86)\Windows Kits\10\/Bin/10.0.20348.0/x64/midl.exe
checking for csc.exe... C:\Windows\Microsoft.NET\Framework\v4.0.30319\/csc.exe
checking for al.exe... C:\Program Files (x86)\Microsoft SDKs\Windows\v10.0A\bin\NETFX 4.8 Tools\/al.exe
checking .NET Framework... found: C:/PROGRA~2/WI3CF2~1/NETFXSDK/4.8/
checking whether jumbo sheets are supported... yes
checking whether to enable runtime optimizations... yes
checking for valgrind/valgrind.h... no
checking for sys/sdt.h... no
checking what the C++ library is... configure: error: Could not figure out what C++ library this is
Error running configure at ./autogen.sh line 321.

Checking the Error Logs

The important log that contains the output of the configuration is the config.log file. In this file, I could see these related lines:

...
configure:19511: result: no
configure:20052: checking what the C++ library is
configure:20078: C:/PROGRA~1/MIB055~1/2022/COMMUN~1/VC/Tools/MSVC/1430~1.307/bin/Hostx64/x64/cl.exe -c  -IC:/PROGRA~2/WI3CF2~1/10/Include/ucrt  -IC:/PROGRA~2/WI3CF2~1/10/Include/ucrt -IC:/PROGRA~1/MIB055~1/2022/COMMUN~1/VC/Tools/MSVC/1430~1.307/Include conftest.cpp >&5
conftest.cpp
C:/PROGRA~1/MIB055~1/2022/COMMUN~1/VC/Tools/MSVC/1430~1.307/Include\cstddef(12): fatal error C1083: Cannot open include file: 'stddef.h': No such file or directory
Microsoft (R) C/C++ Optimizing Compiler Version 19.30.30711.2 for x64
Copyright (C) Microsoft Corporation.  All rights reserved.
...

The strange thing was that I could configure that compilation with another Cygwin terminal with slightly different settings. To find the differences, I used the command export to see the values of the environment variables in the two configured terminals, and compare them using diff.

Then, I found that I could evade the problem by setting this environment variable. This was the environment variable from one of the terminals:

export CYGWIN="disable_pcon"
https://cygwin.com/cygwin-ug-net/using-cygwinenv.html

Unfortunately, this was not the case for our mentee who has the same problem. I also knew that this approach may lead to performance degradation.

Looking Further Into the Details

I tried to look further into the details of configure.ac, and debug to understand the root cause of the problem. At first, I changed the version manually in configure.ac, and the configuration actually worked! If you take a look into find_ucrt() function, the relevant part is:

PathFormat "$(win_get_env_from_vsdevcmdbat UniversalCRTSdkDir)"
UCRTSDKDIR=$formatted_path
UCRTVERSION=$(win_get_env_from_vsdevcmdbat UCRTVersion)

Setting the PathFormat and UCRTVERSION to something from a good build fixed the problem: configuration and make went smooth, and finished successfully.

Then, I tried to look into win_get_env_from_vsdevcmdbat() function. As the name implies, it runs the VsDevCmd.bat, and uses the contents of the two environment variables: PathFormat and UCRTVERSION.

This function creates a batch file in the temporary folder, runs it and gets the output, and then removes it. So, removed the removal part, and saved the created batch files.

I was skeptical about the commands that were processing the outputs of the batch files, so I tried to change them a little, but that didn’t help. The nice thing was that each of them were working fine. I ran them several times, but there was no problem! Then I decided to run them exactly one after another, and then I saw that sometimes there was no output.

Finding the Root Cause

At the point, I was almost certain that the problem was from the VSDevCMD.bat itself, but I didn’t know why, and how to fix that. So, I took a look into the script, and guess what: the problem was from the telemetry! If the variable VSCMD_SKIP_SENDTELEMETRY is not set, the command line tries to open a PowerShell script, and send data to Microsoft! That was the source of problem. This is the relevant part of the code:

@REM Send Telemetry if user's VS is opted-in
if "%VSCMD_SKIP_SENDTELEMETRY%"=="" (
    if "%VSCMD_DEBUG%" NEQ "" (
        @echo [DEBUG:%~nx0] Sending telemetry
        powershell.exe -NoProfile -Command "& {Import-Module '%~dp0\Microsoft.VisualStudio.DevShell.dll'; Send-VsDevShellTelemetry -NewInstanceType Cmd;}"
    ) else (
        START "" /B powershell.exe -NoProfile -Command "& {if($PSVersionTable.PSVersion.Major -ge 3){Import-Module '%~dp0\Microsoft.VisualStudio.DevShell.dll'; Send-VsDevShellTelemetry -NewInstanceType Cmd; }}" > NUL
    )
)

To fix that, I used the value 1 for the variable to opt out of telemetry:

set VSCMD_SKIP_SENDTELEMETRY=1

This change is now merged into the LibreOffice code:

So, the problem should be fixed by now.

Best Practices for Doing Telemetry

It took a lot of time to debug and find the root cause of the problem. I think the best way to avoid causing problems for the users of the Visual Studio would be asking for the users’ consent before activating the telemetry.

I agree that there are legitimate or justifiable reasons to do telemetry, but getting the users’ consent is very important before sending data back to the corporate servers.

In LibreOffice, we consider users the top priority, and we are bound to the best practice of: “Telemetry required? Ask users first”, and we ask others to do the same.

5 Feb 2023

QR code improvement – LibreOffice EasyHack

A QR code is a type of 2D barcode that is useful for encoding data, such as a URL, contact information and many other data types. One can scan the code via applications on mobile phones to capture a URL or import contact information.

For example, the code below encodes the address of the LibreOffice website:

Encoding LibreOffice website URL with QR code

Encoding LibreOffice website’s URL in a QR code

Generating QR Codes in LibreOffice

In LibreOffice, it is possible to create different barcodes, either 1D or 2D, via extensions, or LibreOffice’s internal functionality.

QR code dialog

QR code dialog

For QR codes, it is possible to use the internal functionality via Insert > OLE Object > QR and Barcode:

The implementation in LibreOffice is using ZXing QR code library:

If you take a look at the source code in cui/source/dialogs/QrCodeGenDialog.cxx, there is ConvertToSVGFormat() function which creates the SVG output. In the recent versions of ZXing library (we use v1.4 right now), there is a function that does that, ZXing::ToSVG(). The task here is to replace the our function with the one from ZXing library.

Handling Different Library Versions

Unfortunately, only recent versions of ZXing provide ZXing::ToSVG() functions. It is not available in 1.3, but is added in 1.4. Looking into the packaging of the library in different distributions, it becomes obvious that this package version is not available yet in many distributions:

Therefore, a solution should check the ZXing library version, and use the new function if the library version is >= 1.4. In other words, limiting the change to where the new version of the library is present is necessary. This is possible using compiler conditionals. It is important where one configures the LibreOffice compilation to use system provided ZXing library, using --with-system-zxing while invoking autogen.sh.

Final Notes

EasyHacks are good starting points for someone who wants to get started in LibreOffice development. This specific issue for improving QR code is filed as tdf#153328 in TDF’s Bugzilla. To understand how to get started with LibreOffice development, you can refer to our getting involved page in the TDF Wiki, or our video tutorial for getting started with LibreOffice development.

28 Jan 2023

VCL example applications and tools

You may already know that LibreOffice uses its internal GUI toolkit, named VCL (Visual Class Library). This library is responsible for creating the GUI widgets for LibreOffice, providing abstraction from the operating system including the basic rendering. To see more details, you can read its documentation page:

VCL has the nice feature of being able to work with different GUI libraries on different platforms. For example, if you are on Linux, you can use X11, Gtk3/4, Qt, or KDE back-ends for the VCL applications like LibreOffice. You can easily switch between them by using the environment variable SAL_USE_VCLPLUGIN. For example, you can use the old style X11 GUI in this way:

SAL_USE_VCLPLUGIN=gen instdir/program/soffice

Or, by passing it to the LibreOffice program itself:

instdir/program/soffice -env:SAL_USE_VCLPLUGIN=gen

In addition, there are other GUI backends, or plugins which are as follows:

UI Plugin UI Backend
qt5 Qt 5
qt6 Qt 6
kf5 KDE KF5
gtk3 GTK 3
gtk4 GTK 4
gtk3_kde5 GTK 3 KDE 5
win Windows UI (Windows only)
osx macOS X UI (macOS only)

You can read more about the related environment variables here:

The code for the library is available in the vcl/ folder inside core source code. Also, beside the main source code and tests, there is a folder named workbench. Inside this folder, you can find several standalone VCL applications:

VCL Applications and Workbenches

vcldemo: This is a showcase of some of the VCL rendering abilities including the lines, shapes, internationalized text, pictures and more! You can see run this demo by invoking this command, after you could be able to build LibreOffice:

./bin/run vcldemo
vcldemo vcl application

vcldemo

minvcl: This is possibly one of the smallest example application that can be created with VCL. Basically, It creates a window and draws a simple text inside it. Likewise, you can run it by invoking:

./bin/run minvcl
minvcl vcl application

minvcl

Image Tools

icontest: This bitmap image testing is used by some of the developers to test VCL bitmap rendering capabilities. It can be invoked this way:

./bin/run icontest <image>

mtfdemo: This is a small application for displaying and dumping the structure of EMF/EMF+/WMF files, which is a vector image format from Microsoft. This is previously discussed in another blog post here:

Similarly, it can be run in this way:

./bin/run mtfdemo <emf|wmf image>

Font Tools

listfonts: This utility lists the current list of fonts installed in the system, and the features of each font. To display the information on the screen, you can invoke it this way:

./bin/run listfonts --

listglyphs: This utility lists the current glyphs in a font that is already installed on the system. You should provide the font name to it, in this way:

./bin/run listglyphs FreeMono --

VCL Fuzzers and Platform Specific Tools

Beyond the above applications and tools, there are many other platform specific tools that are not available on each and every platform. Also, there are many fuzzers ans testing tools that are helpful for fuzz-testing the library. You can find the complete list of executables using this command:

ls vcl/Executable_*.mk

As can be seen in the output, many of them are fuzzers. To use the fuzzers, first you have to setup the oss-fuzz via ./bin/oss-fuzz-setup.sh script. Then, you may be able to build the fuzzers using ./bin/oss-fuzz-build.sh script. Make sure that you have $OUT, $SRC and $WORK environment variables to set the fuzzer build environment. Be ready for several tarballs and archives downloads at first. You will need curl and svn to continue.

Final Notes

These tools and small applications and their source code are good for testing and getting to understand the VCL library better. As a suggestion, you can start from the minvcl.cxx to understand how to create a GUI application that can work across various platforms with different GUI toolkits. Hopefully, I will discuss the source code of some of these examples in the later blog posts.

13 Jan 2023

String types in LibreOffice C/C++ code – part 1

Strings are very important types of data that are using in LibreOffice. Firstly, they are useful for storing textual data, and is essentially a sequence of characters. As LibreOffice has many modules that depend on various libraries and languages, there are different string types in LibreOffice. Here, we discuss some of them. (more…)

3 Jan 2023

Happy new year 2023!

Happy new year 2023! I wish a great year for you, and the global LibreOffice community.

Here I breifely discuss the year 2022 around the development blog, and the outlook for 2023.

In 2022, I wrote 22 posts around LibreOffice development in this blog. (Some of them are drafts which are not yet published). I hope that my published posts were useful for you to undertsand LibreOffice code better, and to get involved in LibreOffice core development.

At The Document Foundation (TDF), our aim is to improve LibreOffice, the leading free/open source office software that you and many other people around the world use. Our work is community-driven, and we need your help.

Happy new year to the LibreOffice community

Happy new year to the LibreOffice community

These are some the topics I want to discuss in 2023:

  1. Various data types in LibreOffice
  2. LibreOffice build system (gbuild)
  3. New EasyHacks
  4. Fixing crashes
  5. Creating user interfaces with VCL
  6. LibreOffice core development guide
  7. SDK development guide

I will be happy if you provide feedback to me in a comment here, or via email. You can email me using hossein AT libreoffice DOT org.

As you may know, I provide mentoring to those who want to start LibreOffice development. You can contact me if you need help via the above email address.

Have a great time!

21 Dec 2022

UNO Data types in code and API

alsVarious data types are used in LibreOffice code and also in LibreOffice API. Here we discuss some of these data types, which are important when you are working with LibreOffice code and API. (more…)

3 Dec 2022

gbuild: LibreOffice build system – part 1

LibreOffice uses a build system called gbuild which uses GNU Make. Migrating from the old build system to gbuild started in the OpenOffice days, but the migration took a while and a lot of effort, finishing around LibreOffice 4.1. (more…)

17 Nov 2022

Formatting the code in your patch for LibreOffice

Do you want to submit a patch to LibreOffice Gerrit, and you’re wondering if your code will be accepted or not? Other than providing a good solution to resolve a problem (fix a bug, implement a feature or enhancement), you should care about the code conventions, and in particular, code formatting. Suitable code formatting for LibreOffice is what we discuss here. (more…)