Blog

Unpacking the Build Process: Part 2

Sep 20, 2021 - 5 min read - 912 words

Part 2 looks at the core of the build process, turning the source into compiled code. In Serpent OS this is handled by our build tool boulder. It is usually the part of the build that takes the longest, so where speed ups have the most impact. How long it takes is largely down to the performance of your compiler and what compile flags you are building with.

This post follows on from Unpacking the Build Process: Part 1.

Turning Source into Compiled Code

The steps for compiling code are generally quite straight-forward:

Setting up the build (cmake, configure, meson)
Compiling the source (in parallel threads)
Installing the build into a package directory

This will build software compiled against packages installed on your system. It’s a bit more complicated when packaging as we first set up an environment to compile in (Part 1). But even then you have many choices to make and each can have an impact on how long it takes to compile the code. Do you build with Link Time Optimizations (LTO) or Profile Guided Optimizations (PGO), do you build the package for performance or for the smallest size? Then there’s packages that benefit considerably from individual tuning flags (like -fno-semantic-interposition with python). With so many possibilities, boulder helps us utilize them through convenient configuration options.

Unpacking the Build Process: Part 1

Aug 25, 2021 - 4 min read - 705 words

SunnyFlunk

Distro Engineer: Retired

While the build process (or packaging as it’s commonly referred to) is largely hidden to most users, it forms a fundamental and important aspect to the efficiency of development. In Serpent OS this efficiency also extends to users via source based builds for packages you may want to try/use that aren’t available as binaries upstream.

The build process can be thought of in three distinct parts, setting up the build environment, compiling the source and post build analysis plus package creation. Please note that this process hasn’t been finalized in Serpent OS so we will be making further changes to the process where possible.

Setting up the Build Environment

Some key parts to setting up the build environment:

Downloading packages needed as dependencies for the build
Downloading upstream source files used in the build
Fetching and analyzing the latest repository index
Creating a reproducible environment for the build (chroot, container or VM for example)
Extracting and installing packages into the environment
Extracting tarballs for the build (this is frequently incorporated as part of the build process instead)

While the focus of early optimization work has been on build time performance, there’s more overhead to creating packages time than simply compiling code. Now the compiler is in a good place, we can explore the rest of the build process.

A Rolling Boulder Gathers No Moss

Aug 10, 2021 - 4 min read - 622 words

Ikey

Founder: Retired

We actually did it. Super pleased to announce that moss is now capable of installing and removing packages. Granted, super rough, but gotta start somewhere right?

Transactional system roots + installation in moss

OK let’s recap. A moss archive is super weird, and consists of multiple containers, or payloads. We use a strongly typed binary format, per-payload compression (Currently zstd), and don’t store files in a typical archive fashion.

Instead a .stone file (moss archive) has a Content Payload, which is a compressed “megablob” of all the unique files in a given package. The various files contained within that “megablob” are described in an IndexPayload, which simply contains some IDs and offsets, acting much like a lookup table.

That data alone doesn’t actually tell us where files go on the filesystem when installed. For that, we have a specialist Layout Payload, encoding the final layout of the package on disk.

As can be imagined, the weirdness made it quite difficult to install in a trivial fashion.

Databases

Well, persistence really. Thanks to RocksDB and our new moss-db project, we can trivially store information we need from each package we “precache”. Primarily, we store full system states within our new StateDB, which at present is simply a series of package ID selections grouped by a unique 64-bit integer.

Additionally, we remember the layouts within the LayoutDB so that we can eventually recreate said layout on disk.

Moss DB Progress

Aug 3, 2021 - 2 min read - 360 words

Ikey

Founder: Retired

I’ll try to make this update as brief as I can but it’s certainly an important one, so let’s dive right into it. The last few weeks have been rough but work on our package manager has still been happening. Today we’re happy to reveal another element of the equation: moss-db.

Putting moss-db to the test

moss-db is an abstract API providing access to simplistic “Key Value” stores. We had initially used some payload based files as databases but that introduced various hurdles, so we decided to take a more abstract approach to not tie ourselves to any specific implementation of a database.

Our main goal with moss-db is to encapsulate the RocksDB library, providing sane, idiomatic access to a key value store.

High level requirements

At the highest level, we needed something that could store arbitrary keys and values, grouped by some kind of common key (commonly known as “buckets”). We’ve succeeded in that abstraction, which also required us to fork a rocksdb-binding to add the Transform APIs required.

Additionally, we required idiomatic range behaviours for iteration, as well as generic access patterns. To that affect we can now foreach a bucket, pipe it through the awesomely powerful std.algorithm APIs, and automatically encode/decode keys and values through our generic APIs when implementing the mossdbEncode() and mossdbDecode() functions for a specific type.

In a nutshell, this was the old, ugly, hard way:

  /* old, hard way */
  auto nameZ = name.toStringz();
  int age = 100;
  ubyte[int.sizeof] ageEncoded = nativeToBigEndian(ageEncoded);
  db.setDatum(cast(ubyte[]) (nameZ[0 .. strlen(nameZ)]), ageEncoded);

And this is the new, shmexy way:

    db.set("john", 100);
    db.set("user 100", "bobby is my name");

    auto result = db.get!int("john");
    if (result.found)
    {
        writeln(result.value);
    }

    auto result2 = db.get!string("user 100");
    if (result2.found)
    {
        writeln(result2.value);
    }

It’s quite easy to see the new API lends itself robustly to our needs, so that we may implement stateful, strongly typed databases for moss.

Initial Performance Testing

Aug 2, 2021 - 5 min read - 872 words

SunnyFlunk

Distro Engineer: Retired

With further progress on boulder, we can now build native stone packages with some easy tweaks such as profile guided optimizations (PGO) and link time optimizations (LTO). That means we can take a first look at what the performance of the first cut of Serpent OS shows for the future. The tests have been conducted using benchmarking-tools with Serpent OS measured in a chroot on the same host with the same kernel and config.

One of the key focuses for early in the project is on reducing build time. Every feature can either add or subtract from the time it takes to produce a package. With a source/binary hybrid model, users will greatly benefit from the faster builds as well. In terms of what I’ve targeted in these tests is the performance of clang and testing some compiler flag options on cmake.

Clang Shows its Promise

clang has always been a compiler with a big future. The performance credentials have also been improving each release and are now starting to see it perform strongly against its GNU counterpart. It is common to hear that clang is slow and produces less optimized code. I will admit that most distros provide a slow build of clang, but that will not be the case in Serpent OS.

It is important to note that in this comparison the Host distro has pulled in some patches from LLVM-13 that greatly improve the performance of clang. Prior to this, their tests actually took 50% longer for cmake and configure but only 10% longer for compiling. boulder does not yet support patching in builds so the packages are completely vanilla.

Test using clang	Serpent	Host	Difference
`cmake` LLVM	5.89s	10.58s	79.7%
Compile -j4 `llvm-ar`	126.16s	155.32s	23.1%
`configure` gettext	36.64s	63.55s	73.4%

Based on the results during testing, the performance of clang in Serpent OS still has room to improve and was just a quick tuning pass. At stages where I would have expected to be ahead already, the compile performance was only equal (but cmake and configure were still well ahead).