Abuild is a system designed to build large software projects or related families of software projects that are divided into a potentially large number of components. It is specifically designed for software projects that are continually evolving and that may span multiple languages and platforms. The basic idea behind abuild is simple: when building a single component (module, unit, etc.) of a software package, the developer should be able to focus on that component exclusively. Abuild requires each component developer to declare, by name, the list of other components on which his or her component depends. It is then abuild's responsibility to provide whatever is needed to the build environment to make other required items visible.

You might want to think of abuild as an object-oriented build system. When working with abuild, the fundamental unit is the build item. A build item is essentially a single collection of code, usually contained within one directory, that is built as a unit. A build item may produce one or more products (libraries, executables, JAR files, etc.) that other build items may want to use. It is the responsibility of each build item to provide information about its products that may be used by other items that depend on it. This information is provided by a build item in its abuild interface. In this way, knowledge about how to use a build item is encapsulated within that build item rather than being spread around throughout the other components of a system.

To implement this core functionality, abuild provides its own system for managing build items as well as the dependencies and relationships among them. It also provides various build rules implemented with underlying tools, specifically GNU Make and Apache Ant accessed using the Groovy programming language, to perform the actual build steps. We refer to these underlying tools as backends. Although the bulk of the functionality and sophistication of abuild comes from its own core capabilities rather than the build rules, the rules have rich functionality as well. Abuild is intended to be your build system. It is not intended, as some other tools are, to wrap around your existing build system. ^[1]

Support for compilation in multiple programming languages and on multiple platforms, including embedded platforms, is central to abuild's design. Abuild is designed to allow build items to be built on multiple platforms simultaneously. An important way in which abuild achieves this functionality is to do all of its work inside of an output directory. When abuild performs the actual build, it always creates an output directory named abuild-platform. When abuild invokes make, it does so in that directory. By actually invoking the backend in the output directory, abuild avoids the situation of temporary files conflicting with each other on multiple simultaneous builds of a given build item on multiple platforms. For ant-based builds (using either the supported Groovy backend or the deprecated xml-based ant backend), each build is given a private ant Project object whose basedir is set to the output directory. Abuild is designed to never create or remove any output files outside of its output directories. This enables abuild's cleanup operation to simply remove all output directories created by any instance of abuild, and also reduces the likelihood of unintentionally mixing generated products with version-controlled sources.

The following list shows the font conventions used throughout this document for the names of different kinds of items.

This section describes what you can expect in terms of abuild version numbers and non-compatible changes.

major.minor.prerelease-or-update

This section describes many of the principles upon which abuild was designed. Understanding this material is not critical to being able to use abuild just to do simple compiles, but knowing these things will help you use abuild better and will provide a context for understanding what it does.

You may always find the latest version of abuild by following the links on abuild's website. To use abuild, the following items must be available on your system:

To build abuild, you must also have version 1.35 or newer of boost. Abuild uses several boost libraries, including regex, thread, system, filesystem, and date_time as well as several header-only libraries such as asio, bind, and function. Abuild is known to buildable by gcc and Microsoft Visual C++ (7.1 or newer), but it should be buildable by any compiler that supports boost 1.35. In order for shared library support to work properly with gcc, gcc must be configured to use the GNU linker. ^[3] Abuild itself contains C++ code and Java code, so all the runtime requirements for both systems are required to build abuild.

In order to build abuild's Java code, which is required if you are doing any Java-based builds, you must have at least version 1.5.7 of Groovy. It is recommended that you have at least version 1.6.0. It is not required that you have Groovy to run abuild because abuild includes an embedded version of the Groovy environment, but a full installation of Groovy is required in order to do the initial bootstrapping build of abuild's Java code. ^[4]

As of abuild version 1.1.0, abuild is known to work with Groovy versions 1.6.7 and 1.7-RC-1, which were the latest available versions at the time of the release. Upgrading abuild's embedded version of Groovy is as simple as just replacing the embeddable Groovy JAR file inside of abuild's lib directory. Just delete the old one and copy the new one in. abuild will automatically find it even though its name will have changed to include the later version number. Ideally, you should also rebuild abuild's java support from source and rerun abuild's test suite just to be sure abuild still works properly with the latest Groovy.

Since abuild determines where it is being run from when it is invoked, a binary distribution of abuild is not tied to a particular installation path. It finds the root of its installation directory by walking up the path from the abuild executable until it finds a directory that contains make/abuild.mk. This makes it easy to have multiple versions of abuild installed simultaneously, and it also makes it easy to create relocatable binary distributions of abuild.

Abuild itself does not require any environment variables to be set, but ant and/or the Java development environment may. If you have the JAVA_HOME and ANT_HOME environment variables set, abuild will honor them when selecting which copy of java to run and where to find the ant JAR files. Otherwise, it will run java and ant from your path to make those determinations. Although abuild is explicitly tested to work without either ANT_HOME or JAVA_HOME set, if any Java builds are being done, abuild will start up a little more quickly if they are set. As many other applications expect these to be set, it is recommended that you set JAVA_HOME and ANT_HOME. When abuild invokes Java for any of the Java-based backends, it will automatically add all the JAR files in $ANT_HOME/lib to the classpath as well as all JAR files in abuild's own lib directory. Abuild includes a copy of Groovy's embeddable JAR in its own lib directory. You can copy additional JAR files into lib as well, but if you do so, just remember that those JAR files will not automatically be available to users whose abuild installations do not include them.

As you begin using abuild, you may find yourself generating a collection of useful utility build items for things like specific third-party libraries, external compilers, documentation generators, or test frameworks. There is a small collection of contributed build items in the abuild-contrib package, which is available at abuild's web site. These may have additional requirements. For details, please see the information about abuild-contrib on the website.

Abuild is self-hosting: it can be built with itself, or for bootstrapping, it can be built with a GNU Makefile that uses abuild's internal GNU Make support. To build abuild's Java code, you also need Groovy, Apache Ant and a Java development environment. Please see the file src/README.build in the source distribution for instructions on building abuild.

If you are creating a binary distribution or installing from source, please see the file src/README.build in the source directory. If you are installing from a pre-built binary distribution, simply extract the binary distribution in any directory. Abuild imposes no requirements on where the directory should be or what it should be called as long as its contents remain in the correct relative locations. You may make a symbolic link to the actual bin/abuild executable from a directory in your path. Abuild will follow this link when attempting to discover the path of its installation directory. You may also add the abuild distribution's bin directory to your path, or invoke abuild by the full path to its executable. ^[5]

To build abuild and use it in a Windows environment for make-based builds, certain pieces of the Cygwin environment are required. ^[6] Note that abuild is able to build with and be built by Visual C++ on Windows. It uses Cygwin only for its development tools. Cygwin is not required to run executables built by abuild in a Windows environment, including abuild itself. However, Cygwin is required to supply make and perl to abuild. The following parts of Cygwin are required:

Perl is required, but appears to be installed by default in recent Cygwin installations.

Note that rebaseall (from the rebase package) may need to be run in order for fork to work from perl with certain modules. (Although abuild itself doesn't call fork from perl, qtest, which is used for abuild's test suite, does.)

Other modules may also be desirable. In particular, libxml2 from the Text section is required in order to run certain parts of abuild's test suite, though the test suite will just issue a warning and skip those tests without failing if it can't find xmllint.

If you intend to use autoconf from Windows and you have Rational Rose installed, you may need to create /usr/bin/hostinfo (inside of the Cygwin environment) as

#!/bin/false

so that ./configure's running of hostinfo doesn't run hostinfo from Rational Rose.

In order to use Visual C++ with abuild, you must have your environment set up to invoke Visual C++ command line tools. This can be achieved by running the shortcut supplied with Visual Studio, or you can create a batch file on your own. The following batch file would enable you to run abuild from a Cygwin environment with the environment set up for running Visual C++ from Visual Studio 7.1 (.NET 2003):

@echo off
call "%VS71COMNTOOLS%"\vsvars32.bat
C:\cygwin\cygwin.bat

Adjust as needed if your Cygwin is installed other than in C:\cygwin or you have a different version of Visual C++ installed.

In order to use qtest with abuild under Windows, the Cygwin version of Perl must be the first perl in your path.

Abuild creates output directories in the source directory, and all generated files are created inside of these abuild-generated directories. All output directories are named abuild-*. It is recommended that you configure hooks or triggers in your version control system to prevent these directories or their contents from being accidentally checked in. It may also be useful to prevent Abuild.backing from being checked in since this file always contains information about the local configuration rather than something that would be CM controlled. If it is your policy to allow these to be checked in, they should be prevented from appearing in shared areas such as the trunk. ^[7]

Abuild imposes few system-based restrictions on how you set it up and use it, but here are a few important things to keep in mind:

Here are a few basic terms you'll need to get started:

For a more complete description of build items, build trees, and build forests, please see Chapter 4, Build Items and Build Trees.

Full details on compiler support and compiler selection are covered in Section 24.1, “Platform Selection”. To get started, on Linux systems, abuild will build with gcc by default. On Windows, if you run abuild from a shell that is appropriately set up to run Microsoft Visual C++ (as by following the command prompt shortcut provided as part of your Visual C++ implementation), abuild will automatically use Visual C++. If you have cygwin installed with gcc and the mingw runtime environment, abuild will attempt to use gcc -mno-cygwin to build as long as you set the MINGW environment variable to 1, though bear in mind that abuild's mingw support is not entirely complete.

The directory cxx-library under doc/example/basic contains a simple C++ library. Our library is called basic-library. It implements the single C++ class called BasicLibrary using the header file BasicLibrary.hh and the source file BasicLibrary.cc. Here are the contents of those files:

#ifndef __BASICLIBRARY_HH__
#define __BASICLIBRARY_HH__

class BasicLibrary
{
  public:
    BasicLibrary(int);
    void hello();

  private:
    int n;
};

#endif // __BASICLIBRARY_HH__

#include "BasicLibrary.hh"
#include <iostream>

BasicLibrary::BasicLibrary(int n) :
    n(n)
{
}

void
BasicLibrary::hello()
{
    std::cout << "Hello.  This is BasicLibrary(" << n << ")." << std::endl;
}

Building this library is quite straightforward. Abuild's build files are generally declarative in nature: they describe what needs to be done rather than how it is done. Building a C or C++ library is a simple matter of creating an Abuild.mk file that describes what the names of the library targets are and what each library's sources are, and then tells abuild to build the targets using the C and C++ rules. Here is this library's Abuild.mk file:

TARGETS_lib := basic-library
SRCS_lib_basic-library := BasicLibrary.cc
RULES := ccxx

The string ccxx as the value of the RULES variable indicates that this is C or C++ code (“c” or “cxx”). In order for abuild to actually build this item, we also need to create an Abuild.conf file for it. The existence of this file is what makes this into a build item. We present the file here:

name: cxx-library
platform-types: native

In this file, the name key is used to specify the name of the build item and the platform-types key is used to help abuild figure out on which platforms it should attempt to build this item. Finally, we want this build item to be able to make the resulting library and header file available to other build items. This is done in its Abuild.interface file:

INCLUDES = .
LIBDIRS = $(ABUILD_OUTPUT_DIR)
LIBS = basic-library

This tells abuild to add the directory containing this file to the include path, the output directory in which the generated targets were created to the library path, and the basic-library library to the list of libraries to be linked with. Notice that the name of the library assigned to the LIBS variable is the same as the value assigned to the TARGETS_lib variable in the Abuild.mk file, and that the abuild-provided variable $(ABUILD_OUTPUT_DIR) is used as the library directory. All relative paths specified in the Abuild.interface file are relative to the directory that contains the Abuild.interface file. They are automatically converted internally by abuild to absolute paths, which helps to keep build items location-independent.

To build this item, you would run the command abuild in the basic/cxx-library directory. Abuild will create an output directory whose name would start with abuild- and be based on the platform or platforms on which abuild was building this item. This is the directory to which the variable $(ABUILD_OUTPUT_DIR) refers in the Abuild.interface file.

There is a lot of capability hiding beneath the surface here and quite a bit of flexibility in the exact way in which this can be done, but this is the basic pattern you will observe for the majority of C and C++ library build items.

The directory basic/cxx-program contains a simple C++ program. This program links against the library created in our previous example. Here is the main body of our program:

#include <BasicLibrary.hh>

int main()
{
    BasicLibrary b(5);
    b.hello();
    return 0;
}

This program includes the BasicLibrary.hh header file from the cxx-library build item. Here is the Abuild.mk for this build item:

TARGETS_bin := cxx-program
SRCS_bin_cxx-program := program.cc
RULES := ccxx

Notice that this is very similar to the Abuild.mk from the library build item. The only real difference is that the TARGETS and SRCS variables contain the word bin instead of lib. This tells abuild that these are executable targets rather than library targets. Notice the conspicuous lack of any references to the library build item or the location of the headers or libraries that it makes available. A principal feature of abuild is that this program build item does not need to know that information. Instead, it merely declares a dependency on the cxx-library build item by name. This is done in its Abuild.conf:

name: cxx-program
platform-types: native
deps: cxx-library

Notice the addition of the deps key in this file. This tells abuild that our program build item depends on the library build item. When abuild sees this, it automatically makes all the information in cxx-library's Abuild.interface available to cxx-program's build, alleviating the need for the cxx-program build item to know the locations of these files. This will also tell abuild that cxx-library must be built before we can build cxx-program.

To build this item, we just run the abuild command as we did for cxx-library. This will automatically build dependency cxx-library before building cxx-program. In this way, you can can start a build from any build item and let abuild automatically take care of building all of its dependencies in the correct order.

The output of running abuild in the cxx-program directory when starting from a clean build is shown below. Your actual output will differ slightly from this. In particular, the output below has the string --topdir-- in place of the path to doc/example, and the string <native> in place of your native platform. ^[8] Notice that abuild builds cxx-library first and then cxx-program:

abuild: build starting
abuild: cxx-library (abuild-<native>): all
make: Entering directory `--topdir--/basic/cxx-library/abuild-<native>'
Compiling ../BasicLibrary.cc as C++
Creating basic-library library
make: Leaving directory `--topdir--/basic/cxx-library/abuild-<native>'
abuild: cxx-program (abuild-<native>): all
make: Entering directory `--topdir--/basic/cxx-program/abuild-<native>'
Compiling ../program.cc as C++
Creating cxx-program executable
make: Leaving directory `--topdir--/basic/cxx-program/abuild-<native>'
abuild: build complete

To remove all of the files that abuild created in any build item's directory, you can run abuild clean in that directory. To clean everything in the build tree, run abuild --clean=all. More details of how to specify what to build and what to clean can be found in Chapter 9, Telling Abuild What to Build.

In our next example, we'll demonstrate how to build a simple Java library. You will find the Java example in basic/java-library. The files here are analogous to those in our C++ library example. First, here is a Java implementation of our BasicLibrary class:

package com.example.basic;

public class BasicLibrary
{
    private int n;

    public BasicLibrary(int n)
    {
        this.n = n;
    }

    public void hello()
    {
        System.out.println("Hello.  This is BasicLibrary(" + n + ").");
    }
}

Next, look at Abuild.conf:

name: java-library
platform-types: java

This is essentially identical to our C++ library except that the platform-types key has the value java instead of the value native. This is always true for Java build items. Next, we'll look at the Abuild.groovy file:

parameters {
    java.jarName = 'java-library.jar'
    abuild.rules = 'java'
}

Java build items have this file instead of Abuild.mk. The contents are very similar. The Abuild.groovy file contains Groovy code that is executed inside a particular context provided by abuild. Most Abuild.groovy files will simply set parameters that describe what will be built. In this file, we set the java.jarName parameter to the name of the JAR file we are creating, and we set the abuild.rules parameter to the value 'java' to indicate that we are using the java rules. For Java build items, we don't explicitly list the source files. Instead abuild automatically finds sources in a source directory which is, by default, src/java. There are many more parameters that can be set, and you have considerable flexibility about how to arrange things and how to get files into your Java archives. Abuild aims to allow you to build by convention, but it gives you the flexibility to do things your own way when you want to. We provide detailed information about the directory structure for Java builds in Section 19.3, “Directory Structure for Java Builds”.

Finally, look at the Abuild.interface file. This file provides information to other build items about what they should add to their classpaths in order to make use of the JAR file created by this build item:

declare java-library.archiveName string = java-library.jar
declare java-library.archivePath filename = \
   $(ABUILD_OUTPUT_DIR)/dist/$(java-library.archiveName)
abuild.classpath = $(java-library.archivePath)
abuild.classpath.manifest = $(java-library.archivePath)

You'll notice here that we are actually setting four different variables. Not all of these are required, but the pattern here is one that you may well wish to adopt, especially if you are working in a Java Enterprise environment. The first statement in the interface file declares a variable called java-library.archiveName as a string and initializes it to the value java-library.jar. This syntax of declaring and initializing an interface variable was introduced into abuild with version 1.1. Here we adopt a convention of using the build item name as the first field of the variable name, and the literal string archiveName as the second field. By including the name of the build item in the name of the interface variable, we reduce the possibility of creating a name clash. By providing a variable to hold the name of the archive provided by this build item, we allow other build items to refer to this JAR file by name without having to know what it is called. The second interface variable, java-library.archivePath, contains the full path to the archive. (Notice that abuild puts the JAR file in the dist subdirectory of the abuild output directory.) This enables other build items to refer to this archive by path without knowing any details beyond this naming convention and the name of the providing build item. Making this type of information available in this way is not necessarily a straight Java “SE” environment, but it can be very useful in a Java “EE” environment where build items that create EAR files may have to reach into other build items to package their artifacts in higher level archives. Experience has shown that adopting a convention like this and following it consistently will pay dividends in the end.

After setting these two build-item-specific variables, we assign to two built-in variables: abuild.classpath, and abuild.classpath.manifest. Most simple JAR-providing build items will do this. Abuild actually provides multiple classpath variables, each of which is intended to be used in a particular way. For a discussion, please see Section 17.5.3, “Interface Variables for Java Items”.

As with the C++ library, it is possible to build this item by running abuild from the basic/java-library directory.

In Java, there is no deep distinction between a “library” and a “program” except that a JAR file that provides a program must have a main method. If a JAR file contains a main method, it can be executed, though it can also be used as a library. Here are the relevant files for the program example:

package com.example.basic;

import com.example.basic.BasicLibrary;

public class BasicProgram
{
    public static void main(String[] args)
    {
        BasicLibrary l = new BasicLibrary(10);
        l.hello();
    }
};

name: java-program
platform-types: java
deps: java-library

parameters {
    java.jarName = 'java-program.jar'
    java.mainClass = 'com.example.basic.BasicProgram'
    java.wrapperName = 'java-program'
    abuild.rules = 'java'
}

A JAR file's manifest file may identify a class that contains a main method. Abuild adds the Main-Class attribute to the manifest file when the java.mainClass parameter is set in the Abuild.groovy. In addition, abuild will create a wrapper script if the java.wrapperName parameter is set. The wrapper script that abuild creates may be useful for casual execution of the Java program for testing purposes, but it is generally not a substitution for having your own deployment mechanism. In particular, the wrapper script references items from your classpath by their paths within the build structure, and additionally, abuild's wrapper scripts are not as portable as the Java code that they help to invoke. ^[9]

Here is the output of running abuild in this directory. As in the C++ program example, the output has been modified slightly: in addition to the --topdir-- substitution, we have also filtered out time stamps and other strings that could potentially differ between platforms:

abuild: build starting
abuild: java-library (abuild-java): all
    [mkdir] Created dir: --topdir--/basic/java-library/abuild-java/classes
    [javac] Compiling 1 source file to --topdir--/basic/java-library/abu\
\ild-java/classes
    [mkdir] Created dir: --topdir--/basic/java-library/abuild-java/dist
      [jar] Building jar: --topdir--/basic/java-library/abuild-java/dist\
\/java-library.jar
abuild: java-program (abuild-java): all
    [mkdir] Created dir: --topdir--/basic/java-program/abuild-java/classes
    [javac] Compiling 1 source file to --topdir--/basic/java-program/abu\
\ild-java/classes
    [mkdir] Created dir: --topdir--/basic/java-program/abuild-java/dist
      [jar] Building jar: --topdir--/basic/java-program/abuild-java/dist\
\/java-program.jar
abuild: build complete

A precise definition of build item would state that a build item is any directory that contains an Abuild.conf. Perhaps a more useful definition would say that a build item is the basic object that participates in abuild's object-oriented view of a software build. A build item provides some service within a build tree. Most build items build some kind of code: most often a library, executable, or Java archive. Build items may provide other kinds of services as well. For example, a build item may implement a code generator, support for a new compiler, or the ability to make use of a third-party software library. In addition, a build item may have certain attributes including a list of dependencies, a list of supported flags, information about what types of platforms the build item may be built on, a list of traits, and other non-dependency relationships to other build items. Each of these concepts is explored in more depth later in the document.

All build items that provide a service are required to have a name. Build item names must be unique within their build tree and all other build trees accessible to their build tree since the build item name is how abuild addresses a build item. Build item names consist of period-separated segments. Each segment may contain mixed case alphanumeric characters, underscores, and dashes. Build item names are case-sensitive.

The primary mechanism for describing build items is the Abuild.conf file. This file consists of colon-separated key/value pairs. A complete description of the Abuild.conf file may be found in Chapter 15, The Abuild.conf File. In the mean time, we will introduce keys as they become relevant to our discussion.

Although every build item has an Abuild.conf file, there are various other files that a build item may have. We defer a complete list and detailed discussion these files for later in the document, but we touch briefly upon a few of the common ones here.

A build tree, as defined before, is a collection of build items arranged hierarchically in the file system. Like build items, build trees have names, and are only referred to from other build trees by name. The root of a build tree is a build item whose Abuild.conf contains the tree-name key. We refer to this item as the tree's root build item.

A build tree is formed as a result of the items it contains holding references to the locations of their children within the file system hierarchy. These locations are named as relative paths in the child-dirs keys of the items' Abuild.conf files. It is customary to have the value of child-dirs contain single path elements (i.e.just a directory without any subdirectories), but this is also not a requirement: child-dirs entries may contain multiple path elements as long as there are no Abuild.conf files in any of the intermediate directories. If a build item's child contains its own tree-name key, that child build item is the root of a separate build tree that is part of the same forest, defined below. Otherwise, the child build item is part of the same tree as its parent.

In addition to containing build items, build trees can contain other attributes. Among these are references to other build trees, a list of supported traits, and a list of plugins. We will discuss these topics later in the document. These attributes are defined using keys in the root build item's Abuild.conf file.

A build forest is a collection of build trees that are connected to each other by virtue of one tree's root build item being referenced as a child of a build item in another tree in the forest. When abuild starts up, it looks for an Abuild.conf in the current directory. It then walks up the file system one directory at a time looking for additional Abuild.conf files. Eventually, it will either find the topmost Abuild.conf file, or it will find an Abuild.conf file that is not listed as a child of the next higher one. Whichever of these cases is found first, the resulting Abuild.conf file is the root of the build forest. The forest then consists of all the trees encountered by following all the child-dirs pointers from the forest root.

Note that, unlike with build items and trees, forests do not have names. Note also that, unlike with trees, there is no explicit marker of the root of a build forest. This is very important as it allows you to extend a forest from above without modifying the forest itself. For a more in-depth discussion, see Chapter 7, Multiple Build Trees.

Note that the hierarchy defined by the layout of build items in the file system is a file system hierarchy and nothing more. It doesn't have to have any bearing at all on the dependency relationships among the build items. That said, it is sensible to organize build items in a manner that relates to the architecture of the system, and this in turn usually has implications about dependencies. Still, it is important to keep in mind that abuild is not file-system driven but rather is dependency driven.

In further describing build items and their attributes, it is useful to classify build items into several types. Most build items serve the purpose of providing code to be compiled. There are a number of special types of build items that serve other purposes. We discuss these here:

Virtually every software development project has some need to integrate with third-party software libraries. In a traditional build system, you might list the include paths, libraries, and library directories right in your Makefile, build.xml, or configuration file for whatever build system you are using. With abuild, the best way to integrate with a third-party library is to use a build item whose sole purpose is to export that library's information using an Abuild.interface file. In the simplest cases, a third-party library build item might be an interface only build item (described above) that just includes the appropriate library directives in a static Abuild.interface file. For example, a build item that provides access to the PCRE (Perl-compatible regular expression) libraries on a Linux distribution that has them installed in the system's standard include path might just include an Abuild.interface with the following contents:

LIBS = pcrecpp pcre

For Java build items, a third-party JAR build item would typically append the path to the JAR file to the abuild.classpath.external interface variable. (For a discussion of the various classpath variables, see Section 17.5.3, “Interface Variables for Java Items”.)

Sometimes, the process may be more involved. For example, on a UNIX system, it is often desirable to use autoconf to determine what interface is required for a particular library. We present an example of using autoconf with abuild in Section 18.3, “Autoconf Example”. Still other libraries may use pkg-config. For those libraries, it may make sense to create a simple set of build rules that automatically generate an Abuild.interface after-build file (also discussed in Section 18.3, “Autoconf Example”) by running the pkg-config command. An example pkg-config build item may be found in the abuild-contrib package available at abuild's web site.

Whichever way you do it for a given package, the idea is that you should always create a build item whose job it is to provide the glue between abuild and the third-party library. Other build items that need to use the third-party library can then just declare a dependency on the build item that provides the third-party library's interface. This simplifies the process of using third-party libraries and makes it possible to create a uniform standard for doing so within any specific abuild build tree. It also alleviates the need to duplicate information about the third-party library throughout your source tree. Whenever you are duplicating knowledge about the path of some entity, you would probably be better off creating a separate build item to encapsulate that knowledge.

Abuild classifies platforms into a three-level hierarchy. The three levels are described by the following terms:

This table (Table 5.1, “Built-in Platforms, Platform Types, and Target Types”) shows the target types along with the built-in platform types and platforms that belong to them.

When a build item is defined with multiple platform types, they must all belong to the same target type. (Since the only target type that has more than one platform type is object-code, this means the target type of a build item with multiple platform types will always be object-code.) Some interface variables are also based on target type. For example, it may be permissible for a java build item to depend on a C++ build item if the C++ build item exports native code or provides an executable code generator, but it would never make sense for a java build item to have an include path or library path in the sense of a C/C++ build item. When one build item depends on another, the platforms on which the two build items are being built come into play. We discuss this in Chapter 24, Cross-Platform Support.

For target type object-code, platform identifiers are of the form os.cpu.toolset.compiler[.option], described below. In all cases, each field of the platform identifier must consist only of lower-case letters, numbers, dash, or underscore. The fields of the platform identifier are as follows:

All of the fields of the platform identifier are made available in separate variables within the interface parsing system. In addition, for object-code build items, the make variable $(CCXX_TOOLCHAIN) is set to the value of the compiler field. Here are some example platform identifiers:

When abuild builds an item, it creates an output directory under that item's directory for every platform on which that item is built. The output directory is of the form abuild-platform-name. Abuild itself and all abuild-supplied rules create files only inside of abuild output directories. ^[13]

When abuild invokes make, it always does so from an output directory. This is true even for platform-independent build items. In this way, even temporary files created by compilers or other build systems will not appear in the build item's local directory. This makes it possible to build a specific item for multiple platforms in parallel without having to be concerned about the separate builds overwriting each other's files.

When abuild builds items using the Groovy backend (and also using the deprecated xml-based ant backend), it performs those builds inside a single Java virtual machine instance. As such, it does not change its working directory to the output directory. (Java does not support changing current directories, and besides, there could be multiple builds going on simultaneously in different threads.) However, each Java-based build has its own private ant Project whose basedir property is set to the output directory. As such, all well-behaved ant tasks will only create files in the output directory.

The sole mechanism for declaring dependencies among build items in abuild is the deps key in a build item's Abuild.conf. Suppose build item A declares build item B as a dependency. The following line would appear in A's Abuild.conf:

deps: B

This declaration causes two things to happen:

We illustrate both of these principles later in this chapter. For an in-depth discussion of build ordering and dependency-aware builds, see Chapter 9, Telling Abuild What to Build. For an in-depth discussion of abuild's interface system, see Chapter 17, The Abuild Interface System.

Another very important point about dependencies in abuild is that they are transitive. In other words, if A depends on B and B depends on C, then A also implicitly depends on C. This means that the conditions above apply to A and C. That is, C is built before A (which it would be anyway since it is built before B and B is built before A), and A sees C's interface in addition to seeing B's interface. ^[14] Assuming that A does not explicitly list C in its deps key, we would call B a direct dependency of A and C an indirect dependency of A. We also say that build item dependencies are inherited when we wish to refer to the fact that build ordering and interface visibility are influenced by both direct and indirect dependencies.

Abuild performs various validations on dependencies. The most important of these is that no cyclic dependencies are permitted. ^[15] In other words, if A depends on B either directly or indirectly, then B cannot depend on A directly or indirectly. There are other dependency validations which are discussed in various places throughout this document.

By default, any build item can depend on any other build item by name. Abuild offers two mechanisms to restrict which items can depend on which other items. One mechanism is through build item name scoping rules, discussed below. The other mechanism is through use of multiple build trees, discussed in Chapter 7, Multiple Build Trees.

Abuild makes no specific commitments about the order in which items will be built except that no item is ever built before its dependencies are built. The exact order in which build items are built, other than that dependencies are built before items that depend on them, should be considered an implementation detail and not relied upon. When abuild is invoked in with multiple threads (using the --jobs option, as discussed in Chapter 13, Command-Line Reference), it may build multiple items in parallel. Even in this mode, abuild will never start building one build item until all of its dependencies have been built successfully.

In this section, we discuss build item name scoping rules. Build item name scoping is one mechanism that can be used to restrict which build items may directly depend on which other build items.

Build item names consist of period-separated segments. The period separator in a build item's name is a namespace scope delimiter that is used to determine which build items may directly refer to which other build items in their Abuild.conf files. It is a useful mechanism for allowing a build item to hide the fact that it is composed of lower-level build items by blocking others from accessing those lower-level items directly.

Each build item belongs to a namespace scope equal to the name of the build item after removing the last period and everything following it. For example, the build item “A.B.C.D” is in the scope called “A.B.C”. We would consider “A.B” and “A” to be ancestor scopes. The build item name itself also defines a scope. In this case, the scope “A.B.C.D” would contain “A.B.C.D.E”. Any build item name scope that starts with “A.B.C.D.” (including the period) would be a descendant scope to “A.B.C.D”. Any build item whose name does not contain a period is considered to belong to the global scope and is accessible by all build items.

One build item is allowed to access another build item by name if the referenced build item belongs to the accessing build item's scope or any of its ancestor scopes. Figure 6.1, “Build Item Scopes”, shows a number of build items arranged by scope. In this figure, each build item defines a scope whose members appear in a gray box at the end of a semicircular arrowhead originating from the defining build item. Each build item in this figure can see the build items that are direct members of the scope that it defines, the build items that are siblings to it in its own scope, and the build items inside of any of its ancestor scopes. You may wish to study the figure while you follow along with the text below.

Figure 6.1. Build Item Scopes

Build items are shown here grouped by scope. Each build item is connected to the scope that it defines.

To illustrate, we will consider item A1.B1.C1. The build item A1.B1.C1 can access the following items for the following reasons:

It cannot access these items:

The item A1.B1.C1 can be accessed by the following items:

It cannot be accessed by these items:

To give a more concrete example, suppose you have a globally accessible build item called networking that was internally divided into private build items networking.src and networking.test. A separate build item called logger would be permitted to declare a dependency on networking but not on networking.src or networking.test. Assuming that it did not create any circular dependencies, networking.test would also be allowed to depend on logger.

Note that these restrictions apply only to explicitly declared dependencies. It is common practice to implement a “public” build item as multiple “private” build items. The public build item itself would not have an Abuild.interface file, but would instead depend on whichever of its own private build items contain interfaces it wants to export. It would, in fact, be a pass-through build item. Because dependencies are inherited, items that depend on the public build item will see the interfaces of those private build items even though they would not be able to depend on them directly. In this way, the public build item becomes a facade for the private build items that actually do the work. For example, the build item networking would most likely not have its own Abuild.interface or Abuild.mk files. Instead, it might depend on networking.src which would have those files. It would probably not depend on networking.test since networking.test doesn't have to be built in order to use networking. ^[16] This means that it would be okay for networking.test to depend on networking since doing so would not create any circular dependencies. Then, any build items that depend on networking indirectly depend on networking.src and would see networking.src's Abuild.interface.

There is nothing that a build item can do to allow itself to declare a direct dependency on another build item that is hidden within another scope: the only way to gain direct access to a build item is to be its ancestor or to be a descendant of its parent. (There are no restrictions on indirect access.) There are times, however, when it is desirable for a build item to allow itself to be seen by build items who would ordinarily not have access to it. This is accomplished by using the visible-to key in Abuild.conf. We defer discussion of this feature until later; see Chapter 25, Build Item Visibility.

Now that the topic of build items and build trees has been explored in somewhat more depth, let's take a look at a simple but complete build tree. The build tree in doc/example/general/reference/common illustrates many of the concepts described above.

The first file to look at is the Abuild.conf belonging to this tree's root build item:

tree-name: common
child-dirs: lib1 lib2 lib3
supported-traits: tester

This is a root build item configuration file, as you can see by the presence of the tree-name key. Notice that it lacks a name key, as is often the case with the root build item. This Abuild.conf contains the names of some child directories and also a build tree attribute: supported-traits, which lists the traits that are allowed in the build tree. We will return to the topic of traits in Section 9.5, “Traits”. In the mean time, we will direct our focus to the child build items.

The first child of the root build item of this tree is in the lib1 directory. We examine its Abuild.conf:

name: common-lib1
child-dirs: src test
deps: common-lib1.src

This build item is called common-lib1. Notice that the name of the build item is not the same as the name of the directory, but it is based on the name of the directory. This is a typical strategy for naming build items. Abuild doesn't care how you name build items as long as they conform to the syntactic restrictions and are unique within a build tree. Coming up with a naming structure that parallels your system's architecture is a good way to help ensure that you do not create conflicting build item names. However, you should avoid creating build item names that slavishly follow your directory structure since doing so will make it needlessly difficult for you to move things around. A major feature of abuild is that nothing cares where a build item is located, so don't set a trap for yourself in which you have to rename a build item when you move it!

This build item does not have any build or interface files. It is a pass-through build item. It declares a single dependency: common-lib1.src, and two child directories: src and test.

Next, look at the common-lib1.src build item's Abuild.conf in the common/lib1/src directory:

name: common-lib1.src
platform-types: native

The first thing to notice is this build item's name. It contains a period and is therefore private to the common-lib1 scope. That means that it is not accessible to build items whose names are not also under that scope. In particular, a build item called common-lib2 would not be able to depend directly on common-lib1.src. It would instead depend on common-lib1 and would inherit the dependency on common-lib1.src indirectly.

This build item doesn't list any child directories and, as such, is a leaf in the file system hierarchy. It also happens not to declare any dependencies, so it is also a leaf in the dependency tree, though one does not imply the other. This build item configuration file contains the platform-types key, as is required for all build items that contain build or interface files. In addition to the Abuild.conf file, we have an Abuild.mk file and an Abuild.interface file:

TARGETS_lib := common-lib1
SRCS_lib_common-lib1 := CommonLib1.cpp
RULES := ccxx

INCLUDES = ../include
LIBDIRS = $(ABUILD_OUTPUT_DIR)
LIBS = common-lib1

There is nothing in these files that is fundamentally different from the basic C++ library example shown in Section 3.4, “Building a C++ Library”. We can observe, however, that the INCLUDES variable in Abuild.interface actually points to ../include rather than the current directory. This simply illustrates that abuild doesn't impose any restrictions on how you might want to lay out your build items, though it is recommended that you pick a consistent way and stick with it for any given build tree. You should also avoid paths that point into other build items. Instead, depend on the other item and put the variable there. As a rule, if you ever have two interface variables or assignments that resolve to the same path, you are probably doing something wrong: a significant feature of abuild is that allows you to encapsulate the location of any given thing in only one place. Instead, figure out who owns a given file or directory and export it from that build item's interface. We will not study the source and header files in this example here, but you are encouraged to go to the doc/example/general/reference/common directory in your abuild source tree or installation directory to study the files further on your own.

Next, look at the test directory. Here is its Abuild.conf:

name: common-lib1.test
platform-types: native
deps: common-lib1
traits: tester -item=common-lib1.src

Notice that it declares a dependency on common-lib1. Since its name is also private to the common-lib1 scope, it would have been okay for it to declare a dependency directly on common-lib1.src. Declaring its dependency on common-lib1 instead means that this test code is guaranteed to see the same interfaces as would be seen by any outside user of common-lib1. This may be appropriate in some cases and not in others, but it demonstrates that it is okay for a build item that is inside of a particular namespace scope to depend on its parent in the namespace hierarchy. This build item also declares a trait, but we will revisit this when we discuss traits later in the document (see Section 9.5, “Traits”).

In addition to the lib1 directory, we also have lib2 and lib3. These are set up analogously to lib1, so we will not inspect every file. We will draw your attention to one file in particular: observe that the common-lib2.src build item in reference/common/lib2/src declares a dependency on common-lib3:

name: common-lib2.src
platform-types: native
deps: common-lib3

We will return to this build tree later to study build sets, traits, and examples of various ways to run builds.

Even when abuild knows about multiple trees, it still won't allow items in one build tree to refer to items in other trees without an explicit instruction to do so. This makes it possible to ensure that items in one tree are not accidentally modified to depend on items in a tree that is supposed to be unrelated. When you want items in one tree to be able to use items in another tree, you declare a tree dependency of one tree on another. This creates a one-way relationship between the two trees such that items in the dependent tree (the one that declares the dependency) can see items in the tree on which it depends, but no visibility is possible in the other direction. To declare a tree dependency, you list the name of the tree dependency in the tree-deps key of the dependent tree's Abuild.conf file. As with item dependencies listed in deps, abuild requires that there are no cycles among tree dependencies.

There is nothing special about a build tree that makes it able to be the target of a tree dependency: any tree can depend on any other tree as long as no dependency cycles are created.

Once you set up another tree as a tree dependency of your tree, all build items defined in the tree named by the tree dependency are available to you (subject to normal scoping rules) as if they were in your local build tree. Since any tree can potentially have a dependency relationship with any other, abuild enforces that none of the build items in any build tree may have the same name as any build item in any tree in the forest. In order to avoid build item name clashes, it's a good idea to pick a naming convention for your build items that includes some kind of tree-based prefix, as we have done with names like common-lib1.

When you declare another tree as a tree dependency of your tree, you declare your dependency on the other tree by mentioning its name in the tree-deps of your tree's root Abuild.conf. In order for this dependency to work, abuild must know where to find the tree. Abuild finds items and trees in the same way: it traverses the build forest from the top down and creates a table mapping names to paths. If the tree your tree depends on is inside of your tree, this poses no problem. But what if it is an external tree that is not inside your tree? In this instance, you must place the external tree somewhere within your overall build area, such as in another subdirectory of the parent of your own tree's root. Then you must create an Abuild.conf file in that common parent directory that knows about the root directories of the two trees. This is illustrated in Figure 7.1, “Top-Level Abuild.conf”.

Figure 7.1. Top-Level Abuild.conf

Tree A declares a tree dependency on tree B. In order for A to find B, an Abuild.conf file that points to both trees' locations must be created in a common ancestor directory. The ovals show the contents of each directory's Abuild.conf files.

The tree named B has an Abuild.conf that that declares no tree dependencies. It is a self-contained tree. However, A's Abuild.conf file mentions B by name. How does A find B? When you start abuild, it walks up the tree to find the highest-level Abuild.conf (or the highest level one not referenced as a child of the next higher Abuild.conf) and traverses downward from there. In this case, the Abuild.conf in A's parent directory knows the locations of both A and B. In this way, abuild has figured out where to find B when A declares the tree dependency. This is illustrated with a concrete example below.

In order for abuild to use multiple trees, it must be able to find the roots of all the trees when it traverses the file system looking for Abuild.conf files. As described earlier, abuild locates the root of the forest by looking up toward the root of the file system for other Abuild.conf files that list previous Abuild.conf directories in their child-dirs key. The parent directory of our previous example contains (see Section 6.4, “Simple Build Tree Example”) the following Abuild.conf file:

child-dirs: common project derived

This is an unnamed build item containing only a child-dirs key. The child-dirs key lists not only the common directory, which is the root of the common tree, but also two other directories: project and derived, each of which we will discuss below. These directories contain additional build tree root build items, thus making them known to any abuild invocation that builds common. It is also okay to create one build tree underneath another named tree. As with build items, having one tree physically located beneath another doesn't have any implications about the dependency relationships among the trees.

We will examine a new build tree that declares the build tree from our previous example as an dependency. This new tree, which we will call the project build tree, can be found at doc/example/general/reference/project. The first file we examine is the new build tree's root build item's Abuild.conf:

tree-name: project
tree-deps: common
child-dirs: main lib

This build item configuration file, in addition to having the tree-name key (indicating that it is a root build item), also has a tree-deps key, whose value is the word common, which is the name of the tree whose items we want to use. Note that, as with build items, abuild never requires you to know the location of a build tree.

Inside the project build tree, the project-lib build item is defined inside the lib directory. It is set up exactly the same way as common-lib1 and the other libraries in the common tree. Here is its Abuild.conf:

name: project-lib
child-dirs: src test
deps: project-lib.src

Now look at project-lib.src's Abuild.conf:

name: project-lib.src
platform-types: native
deps: common-lib1

Notice that it declares a dependency on common-lib1, which is defined in the common tree. This works because abuild automatically makes available to you all the build items in any build trees your depends on.

This build tree also includes a main program, but we will not go through the rest of the files in depth. You are encouraged to study the files on your own. There are also examples of traits in this build tree. We will return to this build tree during our discussion of traits (see Section 9.5, “Traits”).

When you declare another build tree as a tree dependency, you automatically inherit any tree dependencies that that tree declared, so like item dependencies, tree dependencies are transitive. If this were not the case, abuild would not be able to resolve dependencies declared in the other tree if those dependencies were resolved in one of its tree dependencies. To illustrate this, we have a third build tree located in doc/example/general/reference/derived. This build tree is for a second project that is derived from the first project. This build tree declares project as an tree dependency as you can see in its root Abuild.conf file:

tree-name: derived
tree-deps: project
child-dirs: main

For a diagram of the entire general/reference collection of build tress, see Figure 7.2, “Build Trees in general/reference”.

Figure 7.2. Build Trees in general/reference

The derived build tree declares a dependency on the project build tree. The project build tree declares a dependency on the common build tree.

The derived build tree contains a derived-main build item structured identically to the C++ program build items we've seen earlier. Here at the main program's Abuild.conf:

name: derived-main.src
platform-types: native
deps: common-lib2 project-lib
traits: tester -item=derived-main.src

In this file, you can see that derived-main.src depends on project-lib from the project build tree and also common-lib2 which is found in project's dependency, common. We will return to this build tree in the examples at the end of Chapter 9, Telling Abuild What to Build.

As defined in Section 3.2, “Basic Terminology”, the term target refers to a specific build product. In most cases, abuild passes any targets specified on the command line to the backend build system. Abuild provides several standard targets (see Chapter 13, Command-Line Reference). We have already encountered all and clean in earlier examples. It is also possible to add new targets through mechanisms that are covered later in the document. For now, you really only need to know a few things about targets:

Abuild defines two special targets: clean and no-op. These targets are special in two ways: abuild does not allow them to be combined with other targets, and abuild handles them itself without passing them to a backend.

The clean target is used to remove the artifacts that are built by the other targets. Abuild implements the clean target by simply removing all abuild-generated output directories (see Section 5.3, “Output Directories”). When abuild processes the clean target, it ignores any dependency relationships among build items. Since it ignores dependencies and performs the cleanup itself without invoking a backend, running the clean target or cleaning multiple items using a clean set (described below) is very fast.

Note that, starting with version 1.0.3, abuild cleans all build items, not just those with build files. There are several reasons for this:

The no-op target is used primarily for debugging build tree problems. When abuild is invoked with the no-op target, it goes through all the motions of performing a build except that it does not read any Abuild.interface files or invoke any backends. It does, however, perform full validation of Abuild.conf files including dependency and integrity checking. This makes abuild no-op, especially with a build set (described below), very useful for taking a quick look at what items would be built on what platforms and in what order. We make heavy use of the no-op target in the examples at the end of this chapter so that we can illustrate certain aspects of build ordering without being concerned about the actual compilation steps.

We have already seen that, by default, abuild will build all of the build items on which the current item depends (directly or indirectly) in addition to building the current item. Now we generalize on this concept by introducing build sets. A build set is a collection of build items defined by certain criteria. Build sets can be used both to tell abuild which items to build and also to tell it which items to clean. ^[17] When abuild is invoked with no build set specified, its default behavior is to build all of the current item's dependencies as well as the current item. Sometimes, you may wish to assume all the dependencies are up to date and just build the current build item without building any of its dependencies. To do this, you may invoke abuild with the --no-deps option. This will generally only work if all dependencies are up to date. Using --no-deps is most convenient when you are in the midst of the edit/compile/test cycle on a single build item and you want to save the time of checking whether a potentially long chain of dependencies is already up to date. ^[18]

To instruct abuild to build all the items in a specific build set, run abuild --build=set-name (or abuild -b set-name). To instruct abuild to clean all the items in a specific build set, run abuild --clean=set-name (or abuild -c set-name). When building a build set, abuild will also automatically build any items that are direct or indirect dependencies of any items in the build set. However, if you specify any explicit targets on the command line, abuild will not, by default, apply those targets to items that it only added to the build set to satisfy dependencies; it will build those items with the all target instead. This is important as it enables you to add custom targets to a build item without necessarily having those targets be defined for build items it depends on. If you want abuild to build dependencies with explicitly named targets as well, use the --apply-targets-to-deps option. When cleaning with a build set, abuild does not ordinarily also clean the dependencies of the items in the set. To apply the clean target to all the dependencies as well, we also use the --apply-targets-to-deps option. This is a bit subtle, so we present several examples below.

The following build sets are defined:

Ordinarily, when you invoke abuild clean or abuild --clean=set-name, abuild will remove all output directories for any affected build items. You may also restrict abuild to remove only specified output directories. There are two ways to do this. One way is to run abuild clean from inside an output directory. In that case, abuild will remove all the files in the output directory. ^[21] The other way is to use the --clean-platforms option, which may be followed by a shell-style regular expression that is matched against the platform portion of the output directory name. Examples are shown below.

Starting with abuild version 1.0.3, it is possible to list other build items in the build-also key of any named build item's Abuild.conf file. Starting with abuild version 1.1.4, build-also keys can list entire trees and also add options to include tree dependencies or other items at or below the item's directory. When abuild adds any build item to the build set, if that build item has a build-also key, then any build items listed there are also added to the build set. The operation of expanding initial build set membership using the build-also key is applied iteratively until no more build items are added. The principal intended use of this feature is to aid with setting up virtual “top-level” build items. For example, if your system consisted of multiple, independent subsystems and you wanted to build all of them, you could create a build item that lists the main items for each subsystem in a build-also key.

Arguments to build-also may be as follows:

In older versions of abuild, the only way to force building of one build item to build another item was to declare dependencies or tree dependencies. This had several disadvantages, including the following:

Whenever you want building of one build item to result in building of another build item and the first item doesn't need to use anything from the items it causes to be built, it is appropriate to use build-also instead of a dependency.

Starting with abuild version 1.1, it is possible to use the --with-rdeps flag to instruct abuild to expand the build set by adding all reverse dependencies of any build item initially in the build set. When combined with --repeat-expansion, this process is applied iteratively so that all forward and reverse dependencies of every item in the build set will also be in the build set. ^[22] This can be especially useful if you are changing a widely used item and you want to make sure your change didn't break any build items that use your item. For additional details on how the build set is constructed, see Section 33.5, “Construction of the Build Set”..

In abuild, it is possible to assign certain traits to a build item. Traits are a very powerful feature of abuild. This material is somewhat more complicated than anything introduced up to this point, so don't worry if you have to read this section more than once.

Traits are used for two main purposes. Throughout this material, we will refer back to the two purposes. We will also provide clarifying examples later in the chapter.

The first purpose of traits is creation of semantically defined groups of build items. In this case, a trait corresponding to the grouping criteria would be applied to a build item directly. For example, all build items that can be deployed could be assigned the deployable trait.

A second purpose of traits is to create specific relationships among build items. These relationships may or may not correspond to dependencies among build items. These traits may be applied to a build item by itself or in reference to other build items. For example, the tester trait may be applied to a general system test build item by itself and may be applied to every test suite build item with a reference to the specific item being tested.

Traits are used to assist in the construction of build sets. In particular, you can narrow a build set by removing all items that don't have all of a specified list of traits. You can also expand a build set to add any build items that relate to any items already in the set by referring to them through all of a specified list of traits. This makes it possible to say things like “run the deploy target for every build item that has the deployable trait,” or “run the test target for every item that tests my local build item or anything it depends on.”

Since traits are visible in abuild's --dump-data output (see Appendix F, --dump-data Format), they are available to scripts or front ends to abuild. They may also be used for purely informational purposes such as specifying the classification level of a build item or applying a uniform label to all build items that belong to some group. Trait names are subject to the same constraints as build item names: they are case-sensitive and may consist of mixed case alphanumeric characters, numbers, underscores, dashes, and periods. Unlike with build items, the period does not have any semantic meaning when used in a trait name.

Starting with abuild 1.1.6, a build item that uses either the GNU Make backend or the Groovy backend (but not the deprecated xml-based ant backend) may also get access to the list of traits that are declared in its Abuild.conf file. For the GNU Make backend, the variable ABUILD_TRAITS contains a list of traits separated by spaces. For the Groovy backend, the variable abuild.traits contains a groovy list of traits represented as strings. In both cases, any information about referent build items is excluded; only the list of declared traits is provided. Possible uses for this information would include having a custom rule check to make sure a given trait is specified before providing a particular target, having it give an error if a particular trait is not defined, or even having it change behavior on the basis of a trait.

this: potato.test
traits: deployable tester -item=potato.lib -item=potato.bin unclassified

abuild --build=local --only-with-traits deployable deploy

abuild --build=current --related-by-traits=tester test

Although we have described how various options affect which build items are built with which targets, we summarize that information here so that it all appears in one place. Put simply, the default behavior is that abuild applies any explicitly named targets to all build items that directly match the criteria for belonging to the named build set. Any build items that abuild is building just to satisfy dependencies are built with the all target. This behavior is overridden by specifying --apply-targets-to-deps, which causes abuild to build all build items with the explicit targets. The exact rules are described in the list below. These rules apply only when a build set is specified with --build or -b. There are several mutually exclusive cases:

For more detailed information on how the build set is constructed, please see Section 33.5, “Construction of the Build Set”.

Now that we've seen the topics of build sets and traits, we're ready to revisit our previous examples. This time, we will talk about how traits are used in a build tree, and we will demonstrate the results of running abuild with different build sets. We will also make use of the special target no-op which can be useful for debugging your build trees.

abuild: build starting
abuild: common-lib1.src (abuild-<native>): no-op
abuild: common-lib1.test (abuild-<native>): no-op
abuild: common-lib3.src (abuild-<native>): no-op
abuild: common-lib2.src (abuild-<native>): no-op
abuild: common-lib2.test (abuild-<native>): no-op
abuild: common-lib3.test (abuild-<native>): no-op
abuild: build complete

abuild: cleaning common-lib1 in lib1
abuild: cleaning common-lib1.src in lib1/src
abuild: cleaning common-lib1.test in lib1/test
abuild: cleaning common-lib2 in lib2
abuild: cleaning common-lib2.src in lib2/src
abuild: cleaning common-lib2.test in lib2/test
abuild: cleaning common-lib3 in lib3
abuild: cleaning common-lib3.src in lib3/src
abuild: cleaning common-lib3.test in lib3/test

abuild: build starting
abuild: common-lib1.src (abuild-<native>): check
make: Entering directory `--topdir--/general/reference/common/lib1/src/a\
\build-<native>'
Compiling ../CommonLib1.cpp as C++
Creating common-lib1 library
make: Leaving directory `--topdir--/general/reference/common/lib1/src/ab\
\uild-<native>'
abuild: common-lib1.test (abuild-<native>): check
make: Entering directory `--topdir--/general/reference/common/lib1/test/\
\abuild-<native>'
Compiling ../main.cpp as C++
Creating lib1_test executable

*********************************
STARTING TESTS on ---timestamp---
*********************************


Running ../qtest/lib1.test
lib1  1 (test lib1 class)                                      ... PASSED

Overall test suite                                             ... PASSED

TESTS COMPLETE.  Summary:

Total tests: 1
Passes: 1
Failures: 0
Unexpected Passes: 0
Expected Failures: 0
Missing Tests: 0
Extra Tests: 0

make: Leaving directory `--topdir--/general/reference/common/lib1/test/a\
\build-<native>'
abuild: common-lib3.src (abuild-<native>): check
make: Entering directory `--topdir--/general/reference/common/lib3/src/a\
\build-<native>'
Compiling ../CommonLib3.cpp as C++
Creating common-lib3 library
make: Leaving directory `--topdir--/general/reference/common/lib3/src/ab\
\uild-<native>'
abuild: common-lib2.src (abuild-<native>): check
make: Entering directory `--topdir--/general/reference/common/lib2/src/a\
\build-<native>'
Compiling ../CommonLib2.cpp as C++
Creating common-lib2 library
make: Leaving directory `--topdir--/general/reference/common/lib2/src/ab\
\uild-<native>'
abuild: common-lib2.test (abuild-<native>): check
make: Entering directory `--topdir--/general/reference/common/lib2/test/\
\abuild-<native>'
Compiling ../main.cpp as C++
Creating lib2_test executable

*********************************
STARTING TESTS on ---timestamp---
*********************************


Running ../qtest/lib2.test
lib2  1 (test lib2 class)                                      ... PASSED

Overall test suite                                             ... PASSED

TESTS COMPLETE.  Summary:

Total tests: 1
Passes: 1
Failures: 0
Unexpected Passes: 0
Expected Failures: 0
Missing Tests: 0
Extra Tests: 0

make: Leaving directory `--topdir--/general/reference/common/lib2/test/a\
\build-<native>'
abuild: common-lib3.test (abuild-<native>): check
make: Entering directory `--topdir--/general/reference/common/lib3/test/\
\abuild-<native>'
Compiling ../main.cpp as C++
Creating lib3_test executable

*********************************
STARTING TESTS on ---timestamp---
*********************************


Running ../qtest/lib3.test
lib3  1 (test lib3 class)                                      ... PASSED

Overall test suite                                             ... PASSED

TESTS COMPLETE.  Summary:

Total tests: 1
Passes: 1
Failures: 0
Unexpected Passes: 0
Expected Failures: 0
Missing Tests: 0
Extra Tests: 0

make: Leaving directory `--topdir--/general/reference/common/lib3/test/a\
\build-<native>'
abuild: build complete

abuild: build starting
abuild: common-lib1.src (abuild-<native>): no-op
abuild: common-lib3.src (abuild-<native>): no-op
abuild: common-lib2.src (abuild-<native>): no-op
abuild: project-lib.src (abuild-<native>): no-op
abuild: project-main.src (abuild-<native>): no-op
abuild: build complete

abuild: build starting
abuild: common-lib1.src (abuild-<native>): check
abuild: common-lib3.src (abuild-<native>): check
abuild: common-lib2.src (abuild-<native>): check
abuild: project-lib.src (abuild-<native>): check
make: Entering directory `--topdir--/general/reference/project/lib/src/a\
\build-<native>'
Compiling ../ProjectLib.cpp as C++
Creating project-lib library
make: Leaving directory `--topdir--/general/reference/project/lib/src/ab\
\uild-<native>'
abuild: project-main.src (abuild-<native>): check
make: Entering directory `--topdir--/general/reference/project/main/src/\
\abuild-<native>'
Compiling ../main.cpp as C++
Creating main executable

*********************************
STARTING TESTS on ---timestamp---
*********************************


Running ../qtest/main.test
main  1 (testing project-main)                                 ... PASSED

Overall test suite                                             ... PASSED

TESTS COMPLETE.  Summary:

Total tests: 1
Passes: 1
Failures: 0
Unexpected Passes: 0
Expected Failures: 0
Missing Tests: 0
Extra Tests: 0

make: Leaving directory `--topdir--/general/reference/project/main/src/a\
\build-<native>'
abuild: build complete

tree-name: common
child-dirs: lib1 lib2 lib3
supported-traits: tester

name: common-lib1.test
platform-types: native
deps: common-lib1
traits: tester -item=common-lib1.src

abuild: build starting
abuild: common-lib1.src (abuild-<native>): all
abuild: common-lib1.test (abuild-<native>): check
make: Entering directory `--topdir--/general/reference/common/lib1/test/\
\abuild-<native>'

*********************************
STARTING TESTS on ---timestamp---
*********************************


Running ../qtest/lib1.test
lib1  1 (test lib1 class)                                      ... PASSED

Overall test suite                                             ... PASSED

TESTS COMPLETE.  Summary:

Total tests: 1
Passes: 1
Failures: 0
Unexpected Passes: 0
Expected Failures: 0
Missing Tests: 0
Extra Tests: 0

make: Leaving directory `--topdir--/general/reference/common/lib1/test/a\
\build-<native>'
abuild: common-lib3.src (abuild-<native>): all
abuild: common-lib2.src (abuild-<native>): all
abuild: common-lib2.test (abuild-<native>): check
make: Entering directory `--topdir--/general/reference/common/lib2/test/\
\abuild-<native>'

*********************************
STARTING TESTS on ---timestamp---
*********************************


Running ../qtest/lib2.test
lib2  1 (test lib2 class)                                      ... PASSED

Overall test suite                                             ... PASSED

TESTS COMPLETE.  Summary:

Total tests: 1
Passes: 1
Failures: 0
Unexpected Passes: 0
Expected Failures: 0
Missing Tests: 0
Extra Tests: 0

make: Leaving directory `--topdir--/general/reference/common/lib2/test/a\
\build-<native>'
abuild: common-lib3.test (abuild-<native>): check
make: Entering directory `--topdir--/general/reference/common/lib3/test/\
\abuild-<native>'

*********************************
STARTING TESTS on ---timestamp---
*********************************


Running ../qtest/lib3.test
lib3  1 (test lib3 class)                                      ... PASSED

Overall test suite                                             ... PASSED

TESTS COMPLETE.  Summary:

Total tests: 1
Passes: 1
Failures: 0
Unexpected Passes: 0
Expected Failures: 0
Missing Tests: 0
Extra Tests: 0

make: Leaving directory `--topdir--/general/reference/common/lib3/test/a\
\build-<native>'
abuild: project-lib.src (abuild-<native>): all
abuild: project-lib.test (abuild-<native>): check
make: Entering directory `--topdir--/general/reference/project/lib/test/\
\abuild-<native>'
Compiling ../main.cpp as C++
Creating lib_test executable

*********************************
STARTING TESTS on ---timestamp---
*********************************


Running ../qtest/lib.test
lib  1 (test lib class)                                        ... PASSED

Overall test suite                                             ... PASSED

TESTS COMPLETE.  Summary:

Total tests: 1
Passes: 1
Failures: 0
Unexpected Passes: 0
Expected Failures: 0
Missing Tests: 0
Extra Tests: 0

make: Leaving directory `--topdir--/general/reference/project/lib/test/a\
\build-<native>'
abuild: project-main.src (abuild-<native>): check
make: Entering directory `--topdir--/general/reference/project/main/src/\
\abuild-<native>'

*********************************
STARTING TESTS on ---timestamp---
*********************************


Running ../qtest/main.test
main  1 (testing project-main)                                 ... PASSED

Overall test suite                                             ... PASSED

TESTS COMPLETE.  Summary:

Total tests: 1
Passes: 1
Failures: 0
Unexpected Passes: 0
Expected Failures: 0
Missing Tests: 0
Extra Tests: 0

make: Leaving directory `--topdir--/general/reference/project/main/src/a\
\build-<native>'
abuild: build complete

abuild: build starting
abuild: common-lib1.src (abuild-<native>): no-op
abuild: common-lib3.src (abuild-<native>): no-op
abuild: common-lib2.src (abuild-<native>): no-op
abuild: common-lib2.test (abuild-<native>): no-op
abuild: common-lib3.test (abuild-<native>): no-op
abuild: project-lib.src (abuild-<native>): no-op
abuild: project-main.src (abuild-<native>): no-op
abuild: derived-main.src (abuild-<native>): no-op
abuild: build complete

abuild: build starting
abuild: common-lib1.src (abuild-<native>): no-op
abuild: common-lib1.test (abuild-<native>): no-op
abuild: common-lib3.src (abuild-<native>): no-op
abuild: common-lib2.src (abuild-<native>): no-op
abuild: common-lib2.test (abuild-<native>): no-op
abuild: common-lib3.test (abuild-<native>): no-op
abuild: project-lib.src (abuild-<native>): no-op
abuild: project-lib.test (abuild-<native>): no-op
abuild: project-main.src (abuild-<native>): no-op
abuild: derived-main.src (abuild-<native>): no-op
abuild: build complete

abuild: build starting
abuild: common-lib1.src (abuild-<native>): all
abuild: common-lib3.src (abuild-<native>): all
abuild: common-lib2.src (abuild-<native>): all
abuild: project-lib.src (abuild-<native>): all
abuild: derived-main.src (abuild-<native>): check
make: Entering directory `--topdir--/general/reference/derived/main/src/\
\abuild-<native>'
Compiling ../main.cpp as C++
Creating main executable

*********************************
STARTING TESTS on ---timestamp---
*********************************


Running ../qtest/main.test
main  1 (testing derived-main)                                 ... PASSED

Overall test suite                                             ... PASSED

TESTS COMPLETE.  Summary:

Total tests: 1
Passes: 1
Failures: 0
Unexpected Passes: 0
Expected Failures: 0
Missing Tests: 0
Extra Tests: 0

make: Leaving directory `--topdir--/general/reference/derived/main/src/a\
\build-<native>'
abuild: build complete

abuild: cleaning derived-main in main
abuild: cleaning derived-main.src in main/src

abuild: cleaning common-lib1 in ../common/lib1
abuild: cleaning common-lib1.src in ../common/lib1/src
abuild: cleaning common-lib1.test in ../common/lib1/test
abuild: cleaning common-lib2 in ../common/lib2
abuild: cleaning common-lib2.src in ../common/lib2/src
abuild: cleaning common-lib2.test in ../common/lib2/test
abuild: cleaning common-lib3 in ../common/lib3
abuild: cleaning common-lib3.src in ../common/lib3/src
abuild: cleaning common-lib3.test in ../common/lib3/test
abuild: cleaning derived-main in main
abuild: cleaning derived-main.src in main/src
abuild: cleaning project-lib in ../project/lib
abuild: cleaning project-lib.src in ../project/lib/src
abuild: cleaning project-lib.test in ../project/lib/test
abuild: cleaning project-main in ../project/main
abuild: cleaning project-main.src in ../project/main/src

Abuild defines three built-in targets for running test suites: check, test, and test-only. The check and test targets are synonyms. Both targets first ensure that a build item is built (by depending on the all target) and then run the build item's test suites, if any. The test-only target also runs a build item's test suite, but it does not depend on all. This means that it will almost certainly fail when run on a clean build tree. The test-only target is useful for times when you know that a build item is already built and you want to run the test suite on what is there now. One case in which you might want to do this would be if you had just started editing some source files and decided you wanted to rerun the test suite on the existing executables before rebuilding them. Another case in which this could be useful is if you had just built a build tree and then wanted to immediately go back and run all the test suites without having to pay the time penalty of checking to make sure each build is up to date. In this case, you could run abuild --build=all test-only immediately after the build was completed. Such a usage style might be appropriate for autobuilders or other systems that build and test a build tree in a controlled environment.

Abuild is integrated with the QTest test framework. The QTest framework is a perl-based test framework intended to support a design for testability testing mentality. Abuild's own test suite is implemented using QTest. When using either the make or the Groovy backends, if a directory called qtest exists, then the test and check targets will invoke qtest-driver to run qtest-based test suites. If a single file with the .testcov extension exists in the build item directory, abuild will invoke qtest-driver so that it can find the test coverage file and activate test coverage support. Note that abuild runs qtest-driver from the output directory, so the coverage output files as well as qtest.log and qtest-results.xml will appear in that directory. If you wish to have a qtest-based test suite be runnable on multiple platforms simultaneously, it's best to avoid creating temporary files in the qtest directory. If you wish to use the abuild output directory for your temporary files, you can retrieve the full path to this directory by calling the get_start_dir method of the qtest TestDriver object.

In order to use test coverage, you must add source files to the TC_SRCS variable in your Abuild.mk or Abuild.groovy file. Abuild automatically exports this into the environment. If you wish to specify a specific set of tests to run using the TESTS environment variable, you can pass it to abuild on the command line as a variable definition (as in abuild check TESTS=some-test), and abuild will automatically export it to the environment for qtest.

When performing ant-based builds using the Groovy framework, if the java.junitTestsuite property is set to the name of a class, then the test and check targets will attempt to run a JUnit-based test suite. You can also set java.junitBatchIncludes to a pattern that matches a list of class files, in which case JUnit tests will be run from all matching classes. JUnit is not bundled with abuild, so it is the responsibility of the build tree maintainer to supply the required JUnit JARs to abuild. The easiest way to do this is to create a plugin that adds the JUnit JARs to abuild.classpath.external in a plugin.interface file. (For more details on plugins, please see Chapter 29, Enhancing Abuild with Plugins.) You can also copy the JAR file for a suitable version of JUnit into either ant's or abuild's lib directory, as any JAR files in those two locations are automatically added to the classpath.

Adding support for your additional test frameworks is straightforward and can be done by creating a plugin that adds the appropriate code to the appropriate targets. For make-based items, you must make sure that your tests are run by the check, test, and test-only targets. You also must ensure that your check and test targets depend on all and that your test-only target does not depend on all. For Groovy-based items, you must make sure that your tests are run by the test-only target, and abuild will take care of making sure it is run by the test and check targets. For details on plugins, see Chapter 29, Enhancing Abuild with Plugins. For details on writing make rules, see Section 30.2, “Guidelines for Make Rule Authors”. For details on writing rules for the Groovy backend, see Section 30.3, “Guidelines for Groovy Target Authors”.

Backing areas operate at the build forest level. Any build forest can act as a backing area. If abuild needs to reference a build item that is found in the local forest, it will use that copy of the build item. If abuild can't find an item in the local forest, it will use the backing area to resolve that build item. Since abuild never attempts to build or otherwise modify an item in a backing area, backing areas must always be fully built on all platforms for which they will be used as backing areas. (For additional details on platforms, please see Chapter 5, Target Types, Platform Types, and Platforms.)

A build forest may declare multiple backing areas. To specify the location of your backing areas, create a file called Abuild.backing in the root directory of your build forest. As with the Abuild.conf file, the Abuild.backing file consists of colon-separated key/value pairs. The backing-areas key is followed by a space-separated list of paths to your backing areas. Backing area paths may be specified as either absolute or relative paths. The path you declare as a backing area may point anywhere into the forest that you wish to use as the backing area. It doesn't have to point to the root of the forest, and it doesn't have to point to a place in the forest that corresponds to the root of your forest.

When one forest declares another forest as a backing area, we say that the forest backs to its backing area. Creation and maintenance of backing areas is generally a function performed by the people who are in charge of maintaining the overall software baselines. Most developers will just set up their backing areas according to whatever instructions they are given. Having an external tool to create your Abuild.backing file is also reasonable. Note that Abuild.backing files should not generally be controlled in a version control system since they are a property of the developer's work area rather than of the software baseline. If they are controlled, they should generally not be visible outside of the developer's work area.

In this section, we will discuss backing areas from a functionality standpoint. This section presents a somewhat simplified view of how backing areas actually work, but it is good enough to cover the normal cases. To understand the exact mechanism that abuild uses to handle backing areas with enough detail to fully understand the subtleties of how they work, please see Section 33.3, “Traversal Details”.

The purpose of a backing area is to enable a developer to create a partially populated build tree and to fall back to a more complete area for build items that are omitted in the local build tree. A build forest may have any number of backing areas, and backing areas may in turn have additional backing areas. There are a few restrictions, however. As with item and tree dependencies, there may be no cycles among backing area relationships. Additionally, if two unrelated backing areas supply items or trees with the same name, this creates an ambiguity, which abuild will consider an error. ^[25]

When you have one or more backing areas, any reference to a build item or build tree that is not found locally can be resolved in the backing area. What abuild essentially does is to maintain a list of available item and tree names, which it internally maps to locations in the file system. When you using a backing area, abuild uses the backing areas' lookup tables in addition to that from your own forest to resolve items and trees. ^[26] When a build item or tree is defined in a backing area and is also defined in your local forest, your local forest is said to shadow the item or tree. This is not an error. It is a normal case that happens when you are using backing areas. In most cases, your build forest will contain items that either exist now in the backing area or will exist there at some future point. This is because the backing area generally represents a more stable version of whatever project you are working on.

Note that since abuild refers to build items and trees by name and not by path, there are no restrictions about the location of build items in the local forest relative to where they appear in the backing area. This makes it possible for you to reorganize the build items or even the build trees in your local area without having to simultaneously change the backing area. There is only way in which use of backing areas affects how abuild resolves paths: if a directory named in a child-dirs key in some Abuild.conf does not exist and the forest has a backing area, abuild will ignore the non-existence of the child directory. (If you run with --verbose, it will mention that it is ignoring the directory, but otherwise, you won't be bothered with this detail.) This enables you to create sparsely populated build items without having to edit Abuild.conf files of the parents of the directories you have chosen to omit.

If this seems confusing, the best way to think about it is in terms of how this all interacts with a version control system. Typically, there is some master copy of the source code of a project in a version control system. There may be some stable trunk or branch in the version control system that is expected to be self-contained and operational. This is what would typically be checked out into a forest that would be fully built and used by others as a backing area. Then, individual developers would just check out the pieces of the system that they are working on, and set their backing area to point to the stable area. Since their checkouts would be sparse, there may be child directories that don't exist, but it wouldn't matter; once they check in their changes and the stable area from which the backing area is created gets updated, everything should be normal.

One side effect of this is that if you remove the directory containing a build item or tree from your local forest while using a backing area that still contains that item or tree, the thing you removed doesn't really go away from abuild's perspective. Instead, it just “moves” from the local build tree to the backing area. If it is actually your intention to remove the build item so that its name is not known to other build items in your build tree, you can do this by adding the name of the build item to the deleted-items key or the build tree to the deleted-trees key of your Abuild.backing file. This effectively blocks abuild from resolving items or trees with those names from the backing area. Most users will probably never use this feature and don't even need to know it exists, but it can be very useful under certain circumstances. When you tell abuild to ignore a tree in this way, it actually blocks abuild from seeing any items defined in the deleted tree. If you wanted to, you could create a new tree locally with the same name as the deleted tree, and the new tree and the old tree would be completely separate from each other. We present an example that illustrates the use of the deleted-item key in Section 11.5, “Deleted Build Item”.

In plain English, abuild guarantees that if A depends on B and B depends on C, A and B see the same copy of C. To be more precise, abuild checks to make sure that no build item in a backing area references as a dependency or plugin an item that is shadowed in the local forest. (Plugins are covered in Chapter 29, Enhancing Abuild with Plugins.)

We illustrate this in Figure 11.1, “Shadowed Dependency”. Suppose that build items A, B, and C are defined in build tree T2 and that A depends on B and B depends on C. Now suppose you have a local build tree called T1 that has T2 as its backing area, and that you have build items A and C copied locally into T1, but that B is resolved in the backing area.

Figure 11.1. Shadowed Dependency

A in /T1 sees B in /T2 and C in /T1, but B in /T2 sees C in /T2. This means A in /T1 builds with two different copies of C.

If you were to attempt to build A, A would refer to files in B, which comes from a backing area. B would therefore already be built, and it would have been built with the copy of C from the backing area. A, on the other hand, would see C in the local build tree. That means that A is indirectly using two different copies of C. Depending on what changes were made to C in the local build tree, this would likely cause the build of A to be unreproducible at best and completely broken at worst. The situation of B coming from a backing area and depending on C, which is shadowed locally, is what we mean when we say that B has shadowed dependencies. If you attempt to build in this situation, abuild will provide a detailed error message telling you which build items are shadowed and which other build items depend on them. One way to resolve this would be to copy the shadowed build items into your local build tree. In this case, that would mean copying B into T1. Another way to resolve it would be to remove C from your local area and allow that to be resolved in the backing area as well. This solution would obviously only be suitable if you were not working on C anymore.

In this example, we'll demonstrate a task branch. Suppose our task branch makes changes to project but not to common or derived. We can set up a new build forest in which to do our work. We would populate this build forest with whatever parts of project we wanted to modify. We have set up this forest in doc/example/general/task. Additionally, we have set this forest's backing area to ../reference so that it would resolve any missing build items or trees to that location:

backing-areas: ../reference

Note that, although we used a relative path for our backing area in this example, we would ordinarily set our backing area to an absolute path. We use a relative path here only so that the examples can remain independent of the location of doc/example. Since we are not making modifications to any build items in common or derived, we don't have to include those build trees in our task branch. Note that our forest root Abuild.conf still lists common and derived as children, since it is just a copy of the root Abuild.conf from reference:

child-dirs: common project derived

Since this forest has a backing area, abuild ignores the fact that the common and derived directories do not exist. For a diagram of the task branch build trees, see Figure 11.2, “Build Trees in general/task”.

Figure 11.2. Build Trees in general/task

The derived build tree declares a tree dependency on the project build tree. The project build tree declares a tree dependency on the common build tree. Since the common and derived build trees are not shadowed in the task branch, those trees are resolved in the backing area, reference, instead.

As always, for this example to work properly, our backing area must be fully built. If you are following along, to make sure this is the case, you should run abuild --build=all in reference/derived. Next run abuild --build=deptrees no-op in task/project. This generates the following output:

abuild: build starting
abuild: project-lib.src (abuild-<native>): no-op
abuild: project-lib.test (abuild-<native>): no-op
abuild: project-main.src (abuild-<native>): no-op
abuild: build complete

This includes only items in our task branch. No items in our backing area are included because abuild never attempts to build or modify build items in backing areas.

If you study include/ProjectLib.hpp and src/ProjectLib.cpp in task/project/lib in comparison to their counterparts in reference/project/lib, you'll notice that the only change we made in this task branch is the addition of an optional parameter to ProjectLib's constructor. We also updated the test suite to pass a different argument to ProjectLib. This new value comes from a new build item we added: project-lib.extra. To add the new build item, we created task/project/lib/extra/Abuild.conf: and also added the extra directory in task/project/lib/Abuild.conf:

name: project-lib.extra
platform-types: native

name: project-lib
child-dirs: src test extra
deps: project-lib.src

We didn't modify anything under task/project/main at all, but we included it in our task branch so we could run its test suite. Remember that abuild won't try to build the copy of project-main there, and even if it did, that copy of project-main would not see our local copy of project-lib: it would see the copy in its own local build tree, which we have shadowed. This is an example of a shadowed dependency as described in Section 11.3, “Integrity Checks”. This is the output we see when running abuild --build=deptrees check from task/project:

abuild: build starting
abuild: project-lib.src (abuild-<native>): check
make: Entering directory `--topdir--/general/task/project/lib/src/abuild\
\-<native>'
Compiling ../ProjectLib.cpp as C++
Creating project-lib library
make: Leaving directory `--topdir--/general/task/project/lib/src/abuild-\
\<native>'
abuild: project-lib.test (abuild-<native>): check
make: Entering directory `--topdir--/general/task/project/lib/test/abuil\
\d-<native>'
Compiling ../main.cpp as C++
Creating lib_test executable

*********************************
STARTING TESTS on ---timestamp---
*********************************


Running ../qtest/lib.test
lib  1 (test lib class)                                        ... PASSED

Overall test suite                                             ... PASSED

TESTS COMPLETE.  Summary:

Total tests: 1
Passes: 1
Failures: 0
Unexpected Passes: 0
Expected Failures: 0
Missing Tests: 0
Extra Tests: 0

make: Leaving directory `--topdir--/general/task/project/lib/test/abuild\
\-<native>'
abuild: project-main.src (abuild-<native>): check
make: Entering directory `--topdir--/general/task/project/main/src/abuil\
\d-<native>'
Compiling ../main.cpp as C++
Creating main executable

*********************************
STARTING TESTS on ---timestamp---
*********************************


Running ../qtest/main.test
main  1 (testing project-main)                                 ... PASSED

Overall test suite                                             ... PASSED

TESTS COMPLETE.  Summary:

Total tests: 1
Passes: 1
Failures: 0
Unexpected Passes: 0
Expected Failures: 0
Missing Tests: 0
Extra Tests: 0

make: Leaving directory `--topdir--/general/task/project/main/src/abuild\
\-<native>'
abuild: build complete

As with the no-op build, we only see output relating to local build items, not to build items in our backing areas as they are assumed to be already built.

Here we present a new forest located under doc/example/general/user. This forest backs to the task forest from the previous example. We will use this forest to illustrate the use of the deleted-item key in the Abuild.backing.

Suppose we have a user who is working on changes that are related in some way to the task branch. We want to create a user branch that backs to the task branch. Our user branch contains all three trees: common, project, and derived. We will ignore derived for the moment and focus only common and project. For a diagram of the user build trees, see Figure 11.3, “Build Trees in general/user”.

Figure 11.3. Build Trees in general/user

The user forest backs to the task forest. All trees are present, so they all resolve locally.

Observe that common contains only the lib1 directory and that project contains only the lib directory. We make a gratuitous change to a source file in common-lib1.src just as another example of shadowing a build item from our backing area.

In project, we have made changes to project-lib to make use of private interfaces, which we discuss in Chapter 23, Interface Flags and will ignore for the moment. We have also deleted the new build item project-lib.extra that we added in the task branch. To delete the build item, we removed the extra directory from project/lib and from the child-dirs key in project/lib/Abuild.conf:

name: project-lib
child-dirs: src test
deps: project-lib.src

That in itself was not sufficient since, even though the extra directory is no longer present in the child-dirs key of project-lib's Abuild.conf, we would just inherit project-lib.extra from our backing area. To really delete the build item, we also had to add a deleted-item key in user/Abuild.backing:

backing-areas: ../task
deleted-items: project-lib.extra

This has effectively prevented abuild from looking for project-lib.extra in the backing area. If any build item in the local tree references project-lib.extra, an error will be reported because abuild now considers that to be an unknown build item.

Although we don't present any examples of using deleted-tree, it works in a very similar fashion. Ordinarily, any build tree you do not have locally will be inherited from the backing area. If your intention is to change the code so that it no longer uses a particular tree, and you want to make sure that that tree is not available at all in your local area, you can delete it using deleted-tree. However, if you simply remove it from all your tree-deps directives, there is no risk of your using its items by accident. As such, most people will probably never need to use the deleted-tree feature.

When running abuild, the basic invocation syntax is as follows:

abuild [options] [definitions] [targets]

Options, definitions, and targets may appear in any order. Any argument that starts with a dash (“-”) is treated as an option. Any option of the form VAR=value is considered to be a definition. Anything else is considered to be a target.

Arguments of the form VAR=value are variable or parameter definitions. Variables defined in this way are made available to all backends. These can be used to override the value of interface variables or variables set in backend build files.

For the make backend, these variable definitions are just passed along to make.

For the Groovy backend, these variables are stored in a manner such that abuild.resolve will give them precedence over normal parameters or interface variables. They are also defined as properties in the ant project.

For the deprecated xml-based ant framework, these definitions are made available as ant properties that are defined prior to reading any generated or user-provided files.

These options print information and exit without building anything.

These options change some aspect of how abuild starts or runs.

These options change the type of output that abuild generates.

These options tell abuild what to build and what targets to apply to items being built.

Abuild's backends define several targets that are available for use from the command line, so you can rely on these targets being defined. ^[28]

Every build item must contain Abuild.conf. The Abuild.conf file is a simple text file consisting of colon-separated key/value pairs. Blank lines and lines that start with # are ignored. Long lines may be continued to the next line by ending them with a backslash character (\). Certain keys are permitted for some kinds of build items and not for others. For a discussion of different types of build items, please see Section 4.5, “Special Types of Build Items”.

The following keys are supported in Abuild.conf:

Note that the child-dirs keys is the only key that deals with paths rather than names.

This section contains a prose description of the interface system's functionality and presents the basic syntax of Abuild.Interface without providing all of the details. This material provides the basis for understanding how the interface functionality works. In the next section, we go over the details.

The Abuild.interface file has a fairly simple syntax that supports variable declarations, variable assignments, and conditionals. Interface files are rigorously validated. Any errors detected in an interface file are considered build failures which, as such, will prevent abuild from attempting to build the item with the incorrect interface and any items that depend on it. Most Abuild.interface files will just set existing variables to provide specific information about that item's include and library information, classpath information, or whatever other standard information may be needed depending upon the type of item it is. For casual users, a full understanding of this material is not essential, but for anyone trying to debug interface issues or create support within abuild for more complex cases, it will be important to understand how abuild reads Abuild.interface files.

The basic purpose of Abuild.interface is to set variables that are ultimately used by a build item to access its dependencies. The basic model is that an item effectively reads the Abuild.interface files of all its dependencies in dependency order. (This is not exactly what happens. For the full story, see Section 33.7, “Implementation of the Abuild Interface System”.) As each file is read, it adds information to the lists of include paths, libraries, library directories, compiler flags, classpath, etc. All variables referenced by Abuild.interface are global variables, even if they are declared inside the body of a conditional, much as is the case with shell scripts or makefiles. Although this is not literally what happens, the best way to think about how abuild reads interface files is to imagine that, for each build item, all of the interface files for its dependencies along with its own interface file are concatenated in dependency order and that the results of that concatenation are processed from top to bottom, skipping over any blocks inside of false conditional statements.

Once abuild parses the Abuild.interface files of all of a build item's dependencies and that of the build item itself, the names and values of the resulting variables are passed to the backends by writing them to the abuild dynamic output file, which is called .ab-dynamic.mk for make-based builds and .ab-dynamic.groovy for Groovy/ant-based builds. The dynamic output file is created in the output directory. Although users running abuild don't even have to know this file exists, peeking at it is a useful way to see the results of parsing all the Abuild.interface files in a build item's dependency chain.

The Abuild.interface file contains the following items:

Similar to make or shell script syntax, each statement is terminated by the end of the line. Whitespace characters (spaces or tabs) are used to separate words. A backslash (\) as the last character of the line may be used to continue long statements onto the next line of the file, in which case the newline is treated as a word delimiter like any other whitespace. ^[29] Any line that starts with a # character optionally preceded by whitespace is ignored entirely. Comment lines have no effect on line continuation. In other words, if line one ends with a continuation character and line two is a comment, line one is continued on line three. This makes it possible to embed comments in multiline lists of values. In this example, the value of ODDS would be one three:

ODDS = \
  one \
# odd numbers only, please
  # two \
  three

Characters that have special meanings (space, comma, equal, etc.) may be quoted by preceding them by a backslash. For consistency, a backslash followed by any character is treated as that character. This way, the semantics of backslash quoting won't change if additional special characters are added in the future.

All variables must be declared, though most Abuild.interface files will be assigning to variables that have already been declared in other interface files. There are no variable scoping rules: all variables are global, even if declared inside a conditional block. Variable names may contain alphanumeric characters, dash, underscore, and period. By convention, make-based rules use all uppercase letters in variable names. This convention also has the advantage of avoiding potential conflict with reserved statements. Java-based rules typically use lower-case period-separated properties. Ultimately abuild interface variables become make variables or ant properties and keys in parameter tables for Groovy, which is the basis for these conventions. Note, however, that variables of both naming styles may be used by either backend, and some of abuild's predefined interface variables that are available to both make and Groovy/ant are of the all upper-case variety.

Once declared, a variable may be assigned to or referenced. A variable is referenced by enclosing its name with parentheses and preceding it by a dollar sign (as in $(VARIABLE)), much like with standard make syntax, except that there is no special case for single-character variable names. Other than using the backslash character to quote single characters, there is no quoting syntax: the single and double quote characters are treated as ordinary characters with no special meanings.

Environment variables may be referenced using the syntax $(ENV:VARIABLE). Unlike many other systems which treat undefined environment variables as the empty string, abuild will trigger an error condition if the environment variable does not exist unless a default value is provided. A default value can be provided using the syntax $(ENV:VARIABLE:default-value). The default-value portion of the string may not contain spaces, tabs, or parentheses. ^[30] Although it can sometimes be useful to have abuild interface files initialize interface variables from the environment, this feature should be used sparingly as it is possible to make a build become overly dependent on the environment in this way. (Even without this feature, there are other ways to fall into this trap that are even worse.) Note that environment variables are not abuild variables. They are expanded as strings and can be used in the interface file wherever ordinary strings can be used.

In addition, starting in version 1.1.1, abuild can access command-line parameters of the form VAR=val from interface files. This works identically to environment variables. Parameter references are of the form $(PARAM:PARAMETER) or $(PARAM:PARMETER:default-value). As with environment variable references, accessing an unspecified parameter without a default is an error, and parameter expansions are treated as strings by the interface parser. This feature should also be used sparingly as it can create plenty of opportunity for unpredictable builds. The main valid use case for accessing parameters from an interface file would be to allow special debugging changes that allow modifying build behavior from the command-line for particular circumstances. Keep in mind that changing parameters on the command line has no impact on dependencies, so gratuitous and careless use of this feature can lead to unreproducible builds. That said, this feature does not make abuild inherently less safe since it has always been possible to access parameters and the environment directly from make code.

Variables may contain single scalar values or they may contain lists of values of one of the three supported types: boolean, string, or filename.

Boolean variables are simple true/false values. The values 1 and true are interpreted interchangeably as true, and the values 0 and false are interpreted interchangeably as false. Regardless of whether the word or numeric value is used to assign to boolean variables, the normalized values of 0 and 1 are passed to the backend build system. (For simplicity and consistency, this is true even for the Groovy backend, which could handle actual boolean values instead.) String variables just contain arbitrary text. It is possible to embed spaces in string variables by quoting them with a backslash, but keep in mind that not all backends handle spaces in single-word variable values cleanly. For example, dealing with embedded spaces in variable names in GNU Make is impractical since it uses space as a word delimiter and offers no specific quoting mechanisms. The values of filename variables are interpreted to be path names. Path names may be specified with either forward slashes or backslashes on any platform. Relative paths (those that do not start with a path separator character or, on Windows, also a drive letter) are interpreted as relative to the file in which they are assigned, not the file in which they are referenced as is the case with make. This means that build items can export information about their local files using relative paths without having to use any special variables that point to their own local directories. Although this is different from how make works, it is the only sensible semantic for files that are referenced from multiple locations, and it is one of the most important and useful features of the abuild interface system.

List variables may contain multiple space-separated words. Assignments to list variables may span multiple lines by using a trailing backslash to indicate continuation to the next line. Each element of a list must be the same type. Lists can be made of any of the supported scalar types. (Lists of boolean values are supported, though they are essentially useless.) List variables must be declared as either append or prepend, depending upon whether successive assignments are appended or prepended to the value of the list. This is described in more depth when we discuss variable assignment below.

Scalar variables may be assigned to in one of three ways: normal, override, and fallback. A normal assignment to a scalar variable fails if the variable already has a value. An override assignment initializes a previously uninitialized variable and replaces any previously assigned value. A fallback assignment sets the value of the variable only if it has not previously been initialized. Uninitialized variables are passed to the backend as empty strings. It is legal to initialize a string variable to the empty string, and doing this is distinct from not initializing it.

List variables work differently from anything you're likely to have encountered in other environments, but they offer functionality that is particularly useful when building software. List variables may be assigned to multiple times. The value in each individual assignment may contain zero or more words. Depending on whether the variable was declared as append or prepend, the values are appended to or prepended to the list in the order in which they appear in the specific assignment. An example is provided below.

Scalar and list variables can both be reset using the reset statement. This resets the variable back to its initial state, which is uninitialized for scalars and empty for lists.

Any variable assignment statement can be made conditional upon the presence of a given interface flag. Interface flags are introduced in Chapter 23, Interface Flags, and the details of how to use them in interface files are discussed later in this chapter.

Abuild supports nested conditionals, each of which may contain an if clause, zero or more elseif clauses, and an optional else clause. The abuild interface syntax supports no relational operators: all conditionals are expressed in terms of function calls, the details of which are provided below.

In addition to supporting variables and conditionals, it is possible to specify that certain variables are relevant only to build items of a specific target type. A target type restriction applies until the next target-type directive or until the end of the current file and all the files it loads as after-build files. By default, declarations in an Abuild.interface file apply to all target types. The vast majority of interface files will not have to include any target type restrictions.

It is possible for a build item to contain interface information that is intended for items that depend on it but not intended for the item itself. Typical uses cases would include when some of this information is a product of the build or when a build item needs to modify interface information provided by a dependency after it has finished using the information itself. To support this, an Abuild.interface file may specify additional interface files that are not to be read until after the item is built. The values in any such files are not available to the build item itself, but they are available to any items that depend on the build item that exports this interface. Such files may be dynamically generated (such as with autoconf; see Section 18.3, “Autoconf Example”), or they may be hand-generated files that are just intended not to apply to the build of the current build item (see Section 27.1, “Opaque Wrapper Example”).

By default, once a variable is declared and assigned to in a build item's Abuild.interface, the declaration and assignments are automatically visible to all build items that depend on the item that made the declaration or assignment. In this sense, abuild variables are said to be recursive. It is also possible to declare a variable as non-recursive, in which case assignments to the variable are only visible in the item itself and in items that depend directly on the item that makes the assignment. Declarations inherit normally. ^[31]

It is also possible to declare an interface variable as local. When a variable is declared as local, the declaration and assignment are not visible to any other build items. This can be useful for providing values only to the current build item or for using variables to hold temporary values within the Abuild.interface file and any after-build files that it may explicitly reference.

In this section, we provide the syntactic details for each of the capabilities described in the previous section. There are some aspects of how Abuild.interface files are interpreted that are different from other systems you have likely encountered. If you are already familiar with the basics of how these files work, this section can serve as a quick reference.

A single Abuild.interface conditional must appear in parentheses after an if or elseif statement. The conditional may be a simple boolean variable reference, or it may be a call to any of the provided conditional functions, each of which returns a boolean value. Conditional functions may be nested as needed. Any boolean argument described below may a be function call or a simple boolean variable reference, thus allowing function calls to nest. The following functions are defined:

Abuild maintains a single variable symbol table. All variables are global, and all variables are visible to interface code of any item regardless of target type. Variables may be declared to apply to a specific target type. By default, they apply to all target types. When interface variables are passed to the backend, only variables declared in either the special target type all or in the item's own target type are made available.

In general, end users will not have to be concerned about which target types a variable applies to. A build item could, in principle, assign to both INCLUDES and abuild.classpath without having to care that only object-code items will see INCLUDES and only java items will see abuild.classpath.

Before abuild reads any Abuild.interface files, it provides certain predefined variables. We divide them into categories based on target type.

The variables mentioned here, along with any additional variables that are declared in Abuild.interface files, are made available to the backends in the form of identically named make variables or Groovy framework definitions and ant properties.

Although most Abuild.interface files are reasonably simple and have easily understandable consequences, there will inevitably be situations in which some interface variable has a value that you don't understand. For example, you might see an assignment in one Abuild.interface file that appears to have no effect, or you may wonder which of a very long list of dependencies was responsible for a particular variable assignment or declaration.

Starting with abuild version 1.0.3, you can have abuild dump everything it knows about a build item's interface variables into an XML file. Do this by passing the --dump-interfaces flag to any abuild command that builds something. Doing so will cause abuild to create interface dump files for every build item including those that don't build anything and even those that have no Abuild.interface files themselves.

For build items that do not have build files, abuild creates a file called .ab-interface-dump.xml in the output directory for every platform on which that build item exists. This file contains information about all interface variables that are known to that item. For build items that have build files, abuild creates two files: .ab-interface-dump.before-build.xml and .ab-interface-dump.after-build.xml. If a build has no Abuild.interface or the item's Abuild.interface has no after-build files, the two files are identical and are analogous to .ab-interface-dump.xml files of build items that don't have build files. Otherwise, the .ab-interface-dump.before-build.xml file reflects the interface as seen by the build item itself (before any after-build files are loaded), and the .ab-interface-dump.after-build.xml shows what interface this build item provides to items that depend on it.

Note that the interface dump files contain not just a list of variables with their values but a complete list of everything abuild knows about each variable. This includes its type, where it was declared, every assignment that was made to it, every reset of every variable, etc. When you reference an interface variable, abuild computes the value on the fly, sometimes influenced by interface flags that may be in effect. To get maximum benefit from the information in the interface dump files, you must understand how this works. For those details, please refer to Section 33.7, “Implementation of the Abuild Interface System”. The format of the interface dump file is described in Appendix G, --dump-interfaces Format.

The Abuild.mk file is read by GNU Make and is a GNU Make fragment. It therefore has GNU Make syntax. The Abuild.mk file is intended to contain only variable settings. It contains no make rules or include directives. Abuild automatically includes your Abuild.mk file at the appropriate time and in the appropriate context.

The most important line in Abuild.mk is the setting of the RULES variable. Its purpose is to tell abuild which rule set should be used to generate targets from sources. Most of the remaining variables that are set are dependent upon which rules are being used. It is always possible to use abuild's help system to get detailed rule-specific help about what variables you are expected to define in your Abuild.mk for a specific set of rules. Run abuild --help help for additional information. Abuild provides some built-in rules. Additional rules may provided my plugins or items that you depend on. You can always run abuild --help rules list to get a list of rules that are available to your build item.

In rare instances, it may be necessary to create local rules for a specific build item. Examples may include one-off, special-purpose code generators that are specific to a particular build item. To use local rules, place a list of files that contain definitions of your rules in the LOCAL_RULES variable. Files listed there are resolved relative to the Abuild.mk. They may contain any valid GNU Make code. If you have written the same local rule in more than one or two places, you are probably doing something wrong and should be using build-item-specific rules (Chapter 22, Build Item Rules and Automatically Generated Code ) or plugins (Chapter 29, Enhancing Abuild with Plugins) instead.

Please note that local rules are run from the context of the output directory—you must keep this in mind when using relative paths from your local rules. The make variable SRCDIR is always set to a relative path to the directory that contains the Abuild.mk file. Also, local rules should avoid creating files outside of the output directory since these files will not be removed by the clean target.

The following sections describe the make-based rule sets provided by abuild.

This example demonstrates how to use autoconf and also shows one use of the after-build statement within Abuild.interface. In this example, we create a stub library that replaces functionality from an external library if that library is not available. Our example is somewhat contrived, but it demonstrates the core functionality and patterns required to do this. Our example resides in doc/example/general/user/derived/world-peace.

Notice that the Abuild.conf in the world-peace directory itself defines a pass-through build item (see Section 4.5, “Special Types of Build Items”) that depends on the world-peace.stub build item:

name: world-peace
child-dirs: autoconf stub
deps: world-peace.stub

The world-peace.stub build item, in turn, depends on the world-peace.autoconf build item:

name: world-peace.stub
platform-types: native
deps: world-peace.autoconf

The world-peace.autoconf build item's Abuild.interface file adds its output directory to the INCLUDES variable since this where the autoconf-generated header file will go. Then it declares autoconf.interface in its output directory as an after-build file using the after-build statement:

# $(ABUILD_OUTPUT_DIR) contains the autoconf-generated header.
INCLUDES = $(ABUILD_OUTPUT_DIR)

after-build $(ABUILD_OUTPUT_DIR)/autoconf.interface

This means that the autoconf.interface file won't be included when this build item is built but will be included when other build items that depend on this one are built. This is important since the file won't actually exist yet when this build item is being built from a clean state.

Next, look at the autoconf/Abuild.mk file:

AUTOFILES := autoconf.interface
AUTOCONFIGH := world-peace-config.h
RULES := autoconf

Here, we set the variables that the autoconf rules require. The AUTOFILES variable is set to the value autoconf.interface, which is the same as the file name used as the after-build file in the Abuild.interface file. Additionally, we set the variable AUTOCONFIGH to the name of the header file that we will be generating.

Here is the autoconf/configure.ac file:

AC_PREREQ(2.59)
AC_INIT(world-peace,1.0)
AC_CONFIG_HEADERS([world-peace-config.h])
AC_CONFIG_FILES([autoconf.interface])

AC_PROG_CXX
AC_LANG(C++)
AC_SUBST(HAVE_PRINTF)
AC_CHECK_FUNCS(printf, [HAVE_PRINTF=1], [HAVE_PRINTF=0])
AC_SUBST(HAVE_CREATE_WORLD_PEACE)
AC_CHECK_FUNCS(create_world_peace, [HAVE_CREATE_WORLD_PEACE=1],
                                   [HAVE_CREATE_WORLD_PEACE=0])

AC_OUTPUT

This contains normal autoconf macros. There are two important things to notice here. The first is the AC_CONFIG_FILES macro, which tells autoconf to generate the autoconf.interface file from autoconf.interface.in. The second is the AC_CONFIG_HEADERS call, which takes name of the file set as the value of the AUTOCONFIGH variable in Abuild.mk. The header file template is generated automatically using autoheader. The need to duplicate this information is unfortunate, and this may be improved in a future version of abuild. Note that the autoconf macros don't have any knowledge of the abuild output directory. This works because we actually run autoconf inside the output directory with copies of the input files.

Use of AC_CONFIG_HEADERS and AUTOCONFIGH are optional. If you omit one, you should omit both. If you decide to use an autoconf-generated header, you should be aware of the possibility that you may have duplicated preprocessor symbols defined by different autoconf-based build items. There are several ways to avoid this. One way would be to create your own header file template and generate it using