Using Ninja Build to Build Projects Faster

We’re Earthly. We simplify and speed up software building using containerization. It’s a different approach then you’ll find in Ninja and you may want to check it out.

Ninja is a compact build system with a focus on fast incremental builds. It was originally developed by Evan Martin, a Google dev, partly in response to the needs of building large projects such as Google Chrome.

If you’re developing a software system and you require a rebuild every few minutes to test your latest feature or code block, then Ninja will only rebuild what you have just modified or added and nothing else—as opposed to Make, which would rebuild the whole project every single time.

This article will start by explaining build systems in a little more detail. It’ll then introduce Ninja and teach you how to use Ninja to build a simple C++ project.

What Is a Build System?

Software projects are usually composed of many files. And the process of compiling, linking, copying, structuring, testing, or more generally, processing these files into an executable program is called a build.

Build systems fall into two broad categories:

1. Build generator: Software that takes as input a spec file written in a specially designed language (in many cases, Turing Complete programming languages) and generates a build file that tells build tools how to go about building a software package.
2. Build tool: Software that takes in a build file previously generated by a build generator and then builds a software package. Ninja is an example of a build tool.

Next, let’s take a closer look at Ninja and when you may want to use it.

What Is Ninja?

Ninja is a fast build tool that can also be used as a build tool for other build generators. As mentioned, it was originally developed by Evan Martin, a Google dev, as a resource to speed up the building of projects such as Chrome. Since its inception, some notable projects built using Ninja include Chrome, Android, all Meson projects, Swift, and LLVM.

Ninja differs from other build systems in two major ways:

• It’s designed to have its input files generated by a higher-level build system such as CMake or Meson, and
• It’s designed to run builds as fast as possible.

This philosophy even goes into the default arguments, which are designed to provide the best performance with little to no tinkering. For example, it builds things in parallel by default. This motivates developers to ensure that their code can be built in parallel, and any problems can be detected early in the development process.

Next, we’ll take a closer look at the strengths and weaknesses of Ninja as a build system.

Advantages and Limitations of Ninja

The main advantage of Ninja is its speed in incremental builds. It incentivizes developers to write code that can be built in parallel, using defaults that utilize the -jN flag, which causes Ninja to build in parallel. Furthermore, Ninja doesn’t use a background daemon to constantly keep track of things in memory; it always starts its own binary from scratch and works without relying on any state. So developers will always have a realistic idea of build time without fancy background optimizations done by daemons, and it also makes Ninja very portable and simple.

According to benchmarks, Ninja performs as well as Make in a fresh build but outperforms it in an incremental build by what appears to be an exponentially increasing factor.

On the downside, Ninja can’t build projects without a build file, so it must always work with a build generator, such as CMake or Meson, the most popular build generators that work with Ninja. Problems can be introduced depending on the build generator used. For example, Make requires each file to be specified in the build file, making the process of writing these files extremely complicated, time consuming, and prone to errors. So, the key is to choose the right build generator that works well with Ninja.

When Should You Use Ninja?

Ninja works well for large projects with many files that need many incremental builds over a short time. If you’re already using Make, Meson, or CMake to generate build files and using Make to build them, Ninja is a plug-and-play replacement that will, at worst, keep the performance the same or, at best, improve it in an exponential manner in case of incremental builds.

Conversely, Ninja might not be a good choice if you want an end-to-end tool (build generator and tool in one) that has a high-level language to describe relationships between files and is also a build tool. In that case, something like Bazel might be better, but it’s often slower than Ninja and not as portable.

Implementing a Ninja Build

The following sections explain the different ways to install Ninja before going through step-by-step instructions for implementing a Ninja build.

How to Install Ninja

This section explains how to install Ninja on Linux, Mac, and Windows, and how to build it from source.

For a more thorough set of instructions for any specialized installation cases, please see the Ninja GitHub page, a very useful resource. For day-to-day use, the wiki page also includes a list of standard build patterns and build generators that work well with Ninja.

Installing Ninja on Linux

Depending on the Linux flavor, the installation process differs a bit:

• Arch: pacman -S ninja
• Debian/Ubuntu: apt-get install ninja-build
• Fedora: dnf install ninja-build
• Gentoo: emerge dev-util/ninja
• OpenSUSE: zypper in ninja
• Alpine: apk add ninja

Installing Ninja on MacOS

Ninja can be installed using either Homebrew or MacPorts with the following one-liners:

• Homebrew: brew install ninja
• MacPorts: port install ninja

Installing Ninja on Windows

Chocolatey or Scoop can be used to install Ninja with a one-liner on Windows:

• Chocolatey: choco install ninja
• Scoop: scoop install ninja

Installing Ninja via Package Managers

Ninja can also be installed via package managers, which generally provide more convenience when managing multiple other packages in addition to it:

• Conda: conda install -c conda-forge ninja
• Pip: python -m pip install ninja
• Spack: spack install ninja

Building Ninja From Source

Users who don’t want to build Ninja with specialized flags can build it from source with the following instructions.

First, clone and checkout the Ninja repo:

git clone git://github.com/ninja-build/ninja.git && cd ninja
git checkout release

Then, build a basic Ninja binary and a set of files needed to build Ninja:

./configure.py --bootstrap

This will generate the Ninja binary and a build.ninja file that can be used to build Ninja with itself. That is, the basic Ninja binary generated in the previous step can be used to build the particular configuration of Ninja required. A sort of “ninjaception” if you will.

Use the following code to build Ninja:

cmake -Bbuild-cmake
cmake --build build-cmake

The Ninja binary will now be inside the build-cmake directory (though the user can name this directory anything).

The following code will run the unit tests:

./build-cmake/ninja_test

Creating a Project With Ninja

To demonstrate how to use Ninja as well as showcase some of its strengths, this tutorial uses a sample project, which can be found in this GitHub repo.

This tutorial will show you how to create a simple from-scratch project and an incremental project to demonstrate the time-saving features of Ninja.

Prerequisites

The following are prerequisites to follow along:

• CMake is required to build the project. Instructions for CMake installation can be found on their official website.
• For Linux users, CMake can be installed with the single terminal command sudo snap install cmake --classic. It can be installed on macOS with brew install cmake.
• Python is required to generate the sample files used in this tutorial. Ensure Python is installed.

Creating a Project From Scratch

To create a project from scratch, do the following:

1. Clone the repository with git clone https://github.com/AntreasAntoniou/ninja-tutorial.git
2. Navigate to the project directory using cd ninja-tutorial
3. Navigate to the from-scratch project using cd scratch
4. Notice the two files inside this folder: hello_world.cpp and CMakeLists.txt

hello_world.cpp is a simple C++ program that prints “Hello World” to the console:

// C++ program to display "Hello World"

// Header file for input/output functions
#include <iostream>
using namespace std;

// main() function: where the execution of program begins
int main()
{
    // Print "Hello World"
    cout << "Hello World";

    return 0;
}

CMakeLists.txt is a CMake file that describes the project and how to build it:

cmake_minimum_required (VERSION 3.8)

project(HelloWorld CXX)
set(CMAKE_CXX_STANDARD 14)

add_executable(HelloWorld hello_world.cpp)

Now use CMake to generate a build file for Ninja:

cmake -G Ninja

This should generate a build.ninja file in the current directory, along with related configuration files.

The project can now be built with Ninja using the following command:

ninja

The following output should be generated:

[2/2] Linking CXX executable HelloWorld

With that, the Ninja project should be successfully built.

Creating and Building an Incremental Project

Now, go back to the root of the repository and navigate to the incremental project by running:

cd ..
cd incremental

There are three files here: hello_world-template.cpp, CMakeLists-template.txt, and generate_project_files.py.

The generate_project_files.py file is a Python script that generates the C++ project files and the CMake file from the template files.

This script needs to be run twice: once to generate a 1000-file project and a second time to generate a 1001-file project. So the second project will be an incremental build of the first.

Generate the 1000-file project first:

python3 generate_project_files.py --num_files 1000

Now, use CMake to generate a build file for Ninja:

cmake -S sample_project -G Ninja

This should generate a build.ninja file in the current directory, along with related configuration files.

Build the project with Ninja:

ninja

Next, emulate an incremental build by adding one more file to your sample project:

python3 generate_project_files.py --num_files 1001

Use CMake to generate a build file for Ninja:

cmake -S sample_project -G Ninja

This should generate a build.ninja file in the current directory, along with related configuration files.

As before, build the project with Ninja:

ninja

In the second build, Ninja only builds the new file and not the entire project. This can be seen in the terminal output that shows how many files had to be processed, as well as the time taken for the build to complete.

On the local setup (Apple M1 Max, 16-inch) used in this tutorial, the first build took 35 seconds, and the second build took three seconds.

The following is a copy of the terminal output for the second build:

❯ cmake -S sample_project/ -G Ninja
-- The CXX compiler identification is AppleClang 14.0.0.14000029
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: \
  /Library/Developer/CommandLineTools/usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Configuring done
-- Generating done
-- Build files have been written to: \
  /Users/helloworld/ninja-tutorial/incremental

ninja-tutorial/incremental on  main [!+?] via △ v3.24.2 via 🐍 v3.9.13 \
on ☁️  took 4s
❯ ninja
[2000/2000] Linking CXX executable HelloWorld998

ninja-tutorial/incremental on  main [!+?] via 🐍 v3.9.13 on ☁️ took 5m54s
❯ python generate_project_files.py --num_files 1001
Done

ninja-tutorial/incremental on  main [!+?] via 🐍 v3.9.13 on ☁️
❯ cmake -S sample_project/ -G Ninja
-- Configuring done
-- Generating done
-- Build files have been written to: \
  /Users/helloworld/ninja-tutorial/incremental

ninja-tutorial/incremental on  main [!+?] via 🐍 v3.9.13 on ☁️ took 3s
❯ ninja
[2/2] Linking CXX executable HelloWorld1000

As you can see, Ninja is able to build the updated 1001-file project in only three seconds, compared to the 35 seconds it took to build its predecessor project consisting of 1000 files. Because Ninja had already built the 1000-file variant of the same project, it only had to add one more file to the build.

Conclusion

So, we’ve dived deep into build systems and Ninja, covering how to install and use Ninja for your C++ projects. Remember, tools like build systems are your best pals for automating software building, testing, and deployment. Ninja is just one awesome tool that can save you heaps of time.

And if you loved Ninja, you might want to take a peek at Earthly for more build automation improvements.

Happy coding!