Abuild is a system designed to build large software projects or related families of software projects that are divided into a potentially large number of components. It is specifically designed for software projects that are continually evolving and that may span multiple languages and platforms. The basic idea behind abuild is simple: when building a single component (module, unit, etc.) of a software package, the developer should be able to focus on that component exclusively. Abuild requires each component developer to declare, by name, the list of other components on which his or her component depends. It is then abuild's responsibility to provide whatever is needed to the build environment to make other required items visible.

You might want to think of abuild as an object-oriented build system. When working with abuild, the fundamental unit is the build item. A build item is essentially a single collection of code, usually contained within one directory, that is built as a unit. A build item may produce one or more products (libraries, executables, JAR files, etc.) that other build items may want to use. It is the responsibility of each build item to provide information about its products that may be used by other items that depend on it. This information is provided by a build item in its abuild interface. In this way, knowledge about how to use a build item is encapsulated within that build item rather than being spread around throughout the other components of a system.

To implement this core functionality, abuild provides its own system for managing build items as well as the dependencies and relationships among them. It also provides various build rules implemented with underlying tools, specifically GNU Make and Apache Ant, to perform the actual build steps. We refer to these underlying tools as backends. Although the bulk of the functionality and sophistication of abuild comes from its own core capabilities rather than the build rules, the rules have rich functionality as well. Abuild is intended to be your build system. It is not intended, as some other tools are, to wrap around your existing build system. ^[1]

Support for compilation in multiple programming languages and on multiple platforms, including embedded platforms, is central to abuild's design. Abuild is designed to allow build items to be built on multiple platforms simultaneously. An important way in which abuild achieves this functionality is to do all of its work inside of an output directory. When abuild performs the actual build, it always creates an output directory named abuild-platform and invokes the backend in that directory. By actually invoking the backend in that directory, abuild avoids the situation of temporary files conflicting with each other on multiple simultaneous builds of a given build item on multiple platforms. Abuild is designed to never create or remove any output files outside of its output directories. This enables abuild's cleanup operation to simply remove all output directories created by any instance of abuild, and also reduces the likelihood of unintentionally mixing generated products with version-controlled sources.

The following list shows the font conventions used throughout this document for the names of different kinds of items.

This section describes what you can expect in terms of abuild version numbers and non-compatible changes.

      major.minor[patch-or-pre]

This section describes many of the principles upon which abuild was designed. Understanding this material is not critical to being able to use abuild just to do simple compiles, but knowing these things will help you use abuild better and will provide a context for understanding what it does.

You may always find the latest version of abuild by following the links on abuild's website. To use abuild, the following items must be available on your system:

To build abuild, you must also have version 1.33.1 or newer of the boost regex and thread libraries. ^[2] Abuild is known to buildable by gcc, xlc, Microsoft Visual C++ (7.1 or newer), and mingw, ^[3] and it should be buildable by on any system and with any compiler that supports the boost thread and regular expression libraries. In order for shared library support to work properly with gcc, gcc must be configured to use the GNU linker. ^[4] Abuild itself contains C++ code and Java code, so all the runtime requirements for both systems are required to build abuild.

Since abuild determines where it is being run from when it is invoked, a binary distribution of abuild is not tied to a particular installation path. It finds the root of its installation directory by walking up the path from the abuild executable until it finds a directory that contains make/abuild.mk. This makes it easy to have multiple versions of abuild installed simultaneously, and it also makes it easy to create relocatable binary distributions of abuild.

Abuild itself does not require any environment variables to be set, but ant and/or the Java development environment may. If you have the ANT_HOME environment variable set, abuild will honor it when selecting which copy of ant to run. Otherwise, it will run ant from your path.

As you begin using abuild, you may find yourself generating a collection of useful utility build items for things like specific third-party libraries, external compilers, documentation generators, or test frameworks. There is a small collection of contributed build items in the abuild-contrib package, which is available at abuild's web site. These may have additional requirements. For details, please see the information about abuild-contrib on the website.

Abuild is self-hosting: it can be built with itself, or for bootstrapping, it can be built with a GNU Makefile that uses abuild's internal GNU Make support. To build abuild's Java code, you also need Apache Ant and a Java development environment. Please see the file src/README.build in the source distribution for instructions on building abuild.

If you are creating a binary distribution or installing from source, please see the file src/README.build in the source directory. If you are installing from a pre-built binary distribution, simply extract the binary distribution in any directory. Abuild imposes no requirements on where the directory should be or what it should be called as long as its contents remain in the correct relative locations. You may make a symbolic link to the actual bin/abuild executable from a directory in your path. Abuild will follow this link when attempting to discover the path of its installation directory. You may also add the abuild distribution's bin directory to your path, or invoke abuild by the full path to its executable.

To build abuild and use it in a Windows environment for make-based builds, certain pieces of the Cygwin environment are required. ^[5] Note that abuild is able to build with and be built by Visual C++ on Windows. It uses Cygwin only for its development tools. Cygwin is not required to run executables built by abuild in a Windows environment, including abuild itself. However, Cygwin is required to supply make and perl to abuild. The following parts of Cygwin are required:

Perl is required, but appears to be installed by default in recent Cygwin installations.

Note that rebaseall (from the rebase package) may need to be run in order for fork to work from perl with certain modules. (Although abuild itself doesn't call fork from perl, qtest, which is used for abuild's test suite, does.)

Other modules may also be desirable. In particular, libxml2 from the Text section is required in order to run certain parts of abuild's test suite, though the test suite will just issue a warning and skip those tests without failing if it can't find xmllint.

If you intend to use autoconf from Windows and you have Rational Rose installed, you may need to create /usr/bin/hostinfo (inside of the Cygwin environment) as

#!/bin/false

so that ./configure's running of hostinfo doesn't run hostinfo from Rational Rose.

In order to use Visual C++ with abuild, you must have your environment set up to invoke Visual C++ command line tools. This can be achieved by running the shortcut supplied with Visual Studio, or you can create a batch file on your own. The following batch file would enable you to run abuild from a Cygwin environment with the environment set up for running Visual C++ from Visual Studio 7.1 (.NET 2003):

@echo off
call "%VS71COMNTOOLS%"\vsvars32.bat
C:\cygwin\cygwin.bat

Adjust as needed if your Cygwin is installed other than in C:\cygwin or you have a different version of Visual C++ installed.

In order to use qtest with abuild under Windows, the Cygwin version of Perl must be the first perl in your path.

Abuild creates output directories in the source directory, and all generated files are created inside of these abuild-generated directories. All output directories are named abuild-*. It is recommended that you configure hooks or triggers in your version control system to prevent these directories or their contents from being accidentally checked in. It may also be useful to prevent Abuild.backing from being checked in since this file always contains information about the local configuration rather than something that would be CM controlled. If it is your policy to allow these to be checked in, they should be prevented from appearing in shared areas such as the trunk. ^[6]

Abuild imposes few system-based restrictions on how you set it up and use it, but here are a few important things to keep in mind:

Here are a few basic terms you'll need to get started:

For a more complete description of build items and build trees, please see Chapter 4, Build Items and Build Trees.

Full details on compiler support and compiler selection are covered in Section 21.1, “Platform Selection”. To get started, on Linux systems, abuild will build with gcc by default. On Windows, if you run abuild from a shell that is appropriately set up to run Microsoft Visual C++ (such as by running %VS71COMNTOOLS%\vsvars32.bat, or similar for your compiler version), abuild will automatically use Visual C++. If you have cygwin installed with gcc and the mingw runtime environment, abuild will attempt to use gcc -mno-cygwin to build as long as you set the MINGW environment variable to 1.

The directory cxx-library under doc/example/basic contains a simple C++ library. Our library is called basic-library. It implements the single C++ class called BasicLibrary using the header file BasicLibrary.hh and the source file BasicLibrary.cc. Here are the contents of those files:

#ifndef __BASICLIBRARY_HH__
#define __BASICLIBRARY_HH__

class BasicLibrary
{
  public:
    BasicLibrary(int);
    void hello();

  private:
    int n;
};

#endif // __BASICLIBRARY_HH__

#include "BasicLibrary.hh"
#include <iostream>

BasicLibrary::BasicLibrary(int n) :
    n(n)
{
}

void
BasicLibrary::hello()
{
    std::cout << "Hello.  This is BasicLibrary(" << n << ")." << std::endl;
}

Building this library is quite straightforward. Abuild's build files are generally declarative in nature: they describe what needs to be done rather than how it is done. Building a C or C++ library is a simple matter of creating an Abuild.mk file that describes what the names of the library targets are and what each library's sources are, and then tells abuild to build the targets using the C and C++ rules. Here is this library's Abuild.mk file:

TARGETS_lib := basic-library
SRCS_lib_basic-library := BasicLibrary.cc
RULES := ccxx

The string ccxx as the value of the RULES variable indicates that this is C or C++ code (“c” or “cxx”). In order for abuild to actually build this item, we also need to create an Abuild.conf file for it. The existence of this file is what makes this into a build item. We present the file here:

this: cxx-library
platform-types: native
parent-dir: ..

In this file, the this key is used to specify the name of the build item, the parent-dir key is used to connect this build item into the build tree, and the platform-types key is used to help abuild figure out on which platforms it should attempt to build this item. Finally, we want this build item to be able to make the resulting library and header file available to other build items. This is done in its Abuild.interface file:

INCLUDES = .
LIBDIRS = $(ABUILD_OUTPUT_DIR)
LIBS = basic-library

This tells abuild to add the directory containing this file to the include path, the output directory in which the generated targets were created to the library path, and the basic-library library to the list of libraries to be linked with. Notice that the name of the library assigned to the LIBS variable is the same as the value assigned to the TARGETS_lib variable in the Abuild.mk file, and that the abuild-provided variable $(ABUILD_OUTPUT_DIR) is used as the library directory.

To build this item, you would run the command abuild in the basic/cxx-library directory. Abuild would create an output directory whose name would start with abuild- and be based on the platform or platforms on which abuild was building this item. This is the directory to which the variable $(ABUILD_OUTPUT_DIR) refers in the Abuild.interface file.

There is a lot of capability hiding beneath the surface here and quite a bit of flexibility in the exact way in which this can be done, but this is the basic pattern you will observe for the majority of C and C++ library build items.

The directory basic/cxx-program contains a simple C++ program. This program links against the library created in our previous example. Here is the main body of our program:

#include <BasicLibrary.hh>

int main()
{
    BasicLibrary b(5);
    b.hello();
    return 0;
}

This program includes the BasicLibrary.hh header file from the cxx-library build item. Here is the Abuild.mk for this build item:

TARGETS_bin := cxx-program
SRCS_bin_cxx-program := program.cc
RULES := ccxx

Notice that this is very similar to the Abuild.mk from the library build item. The only real difference is that the TARGETS and SRCS variables contain the word bin instead of lib. This tells abuild that these are executable targets rather than library targets. Notice the conspicuous lack of any references to the library build item or the location of the headers or libraries that it makes available. A principle feature of abuild is that this program build item does not need to know that information. Instead, it merely declares a dependency on the cxx-library build item by name. This is done in its Abuild.conf:

this: cxx-program
platform-types: native
parent-dir: ..
deps: cxx-library

Notice the addition of the deps key in this file. This tells abuild that our program build item depends on the library build item. When abuild sees this, it automatically makes all the information in cxx-library's Abuild.interface available to cxx-program's build, alleviating the need for the cxx-program build item to know the locations of these files. This will also tell abuild that cxx-library must be built before we can build cxx-program.

To build this item, we could just run the abuild command as we did for cxx-library, but this would only work properly if cxx-library were already built. A better way to build this item would to go to the cxx-program directory and run abuild --with-deps. The --with-deps option to abuild tells abuild to build all build items that this item depends on before building this item itself. Using the --with-deps command line option, you can start a build from any build item and let abuild automatically take care of building all of its dependencies in the correct order. (For a complete list of all of abuild's command line options, please see Chapter 11, Command-Line Reference.)

The output of running abuild --with-deps in the cxx-program directory when starting from a clean build is shown below. Your actual output will differ slightly from this. In particular, the output below has the string --topdir-- in place of the path to doc/example, and the string <native> in place of your native platform. ^[7] Notice that abuild builds cxx-library first and then cxx-program:

abuild: cxx-library (abuild-<native>): all
make: Entering directory `--topdir--/basic/cxx-library/abuild-<native>'
Compiling ../BasicLibrary.cc as C++
Creating basic-library library
make: Leaving directory `--topdir--/basic/cxx-library/abuild-<native>'
abuild: cxx-program (abuild-<native>): all
make: Entering directory `--topdir--/basic/cxx-program/abuild-<native>'
Compiling ../program.cc as C++
Creating cxx-program executable
make: Leaving directory `--topdir--/basic/cxx-program/abuild-<native>'

To remove all of the files that abuild created in any build item's directory, you can run abuild clean in that directory. To clean everything in the build tree, run abuild --clean=all. More details of how to specify what to build and what to clean can be found in Chapter 8, Telling Abuild What to Build.

In our next example, we'll demonstrate how to build a simple Java library. Abuild has two different ways to build Java code: a property-driven method, and a build.xml-driven method. The property-driven method of building Java code is truer to abuild's philosophy of using declarative build files but has proven to be inadequate for some of the more complex Java build problems that arise in an enterprise environment. It is hoped that a future version of abuild will provide a more complete answer to the Java build problem, perhaps by integrating with a Java build engine other than Apache Ant. In the mean time, we will demonstrate the property-driven approach to doing Java builds with abuild using this library example. For an example of the build.xml-based approach, see Section 19.3, “Build.xml-driven Java Example”.

You will find the Java example in basic/java-library. The files here are analogous to those in our C++ library example. First, here is a Java implementation of our BasicLibrary class:

package com.example.basic;

public class BasicLibrary
{
    private int n;

    public BasicLibrary(int n)
    {
        this.n = n;
    }

    public void hello()
    {
        System.out.println("Hello.  This is BasicLibrary(" + n + ").");
    }
}

Next, look at Abuild.conf:

this: java-library
platform-types: java
parent-dir: ..

This is essentially identical to our C++ library except that the platform-types key has the value java instead of the value native. This is always true for Java build items. Next, we'll look at the Abuild-ant.properties file:

abuild.jar-name = java-library.jar

Property-driven Java build items have this file instead of Abuild.mk. The only thing we have to set here is the property abuild.jar-name. For Java build items, we don't explicitly list the source files. Instead abuild automatically finds sources in the src/java directory. There are more properties that can be set, and there are other ways to get files into the JAR file. We provide detailed information about the directory structure for property-driven Java builds in Section 16.2, “Directory Structure For Java Builds”. Finally, look at the Abuild.interface file. This file provides information to other build items about what they should add to their classpaths in order to make use of the JAR file created by this build item:

abuild.classpath = $(ABUILD_OUTPUT_DIR)/dist/java-library.jar

As with the C++ library, it is possible to build this item by running abuild from the basic/java-library directory. Notice that abuild puts the JAR file in the dist subdirectory of the abuild output directory.

In Java, there is no deep distinction between a “library” and a “program” except that a JAR file that provides a program must have a main method. If a JAR file contains a main method, it can be executed, though it can also be used as a library. A JAR file can also contain a manifest file that identifies a class that contains a main method. Starting with version 1.0.1, abuild adds the Main-Class attribute to the manifest file when the abuild.main-class property is set in the Abuild-ant.properties.

Here are the relevant files for the program example:

package com.example.basic;

import com.example.basic.BasicLibrary;

public class BasicProgram
{
    public static void main(String[] args)
    {
        BasicLibrary l = new BasicLibrary(10);
        l.hello();
    }
};

this: java-program
platform-types: java
parent-dir: ..
deps: java-library

abuild.jar-name = java-program.jar
abuild.main-class = com.example.basic.BasicProgram
abuild.wrapper-name = java-program

Note the addition of the abuild.main-class and abuild.wrapper-name properties. If these are both set, abuild will create a wrapper executable (script file and/or batch file as appropriate) so that you can run the resulting program with the correct classpath. The wrapper programs are placed directly in the abuild output directory.

Here is the output of running abuild --with-deps in this directory. As in the C++ program example, the output has been modified slightly: in addition to the --topdir-- substitution, we have also filtered out time stamps and other strings that could potentially differ between platforms:

abuild: java-library (abuild-java): all
Buildfile: --abuild.xml--

init:
    [mkdir] Created dir: --topdir--/basic/java-library/abuild-java/empty
    [mkdir] Created dir: --topdir--/basic/java-library/abuild-java/classes
    [mkdir] Created dir: --topdir--/basic/java-library/abuild-java/dist
    [mkdir] Created dir: --topdir--/basic/java-library/abuild-java/src/java
    [mkdir] Created dir: --topdir--/basic/java-library/abuild-java/src/r\
\esources
    [mkdir] Created dir: --topdir--/basic/java-library/abuild-java/src/web
    [mkdir] Created dir: --topdir--/basic/java-library/abuild-java/src/conf

-do-compile:
    [javac] Compiling 1 source file to --topdir--/basic/java-library/abu\
\ild-java/classes

-jar-without-main-class:
      [jar] Building jar: --topdir--/basic/java-library/abuild-java/dist\
\/java-library.jar

BUILD SUCCESSFUL
Total time: <time>
abuild: java-program (abuild-java): all
Buildfile: --abuild.xml--

init:
    [mkdir] Created dir: --topdir--/basic/java-program/abuild-java/empty
    [mkdir] Created dir: --topdir--/basic/java-program/abuild-java/classes
    [mkdir] Created dir: --topdir--/basic/java-program/abuild-java/dist
    [mkdir] Created dir: --topdir--/basic/java-program/abuild-java/src/java
    [mkdir] Created dir: --topdir--/basic/java-program/abuild-java/src/r\
\esources
    [mkdir] Created dir: --topdir--/basic/java-program/abuild-java/src/web
    [mkdir] Created dir: --topdir--/basic/java-program/abuild-java/src/conf

-do-compile:
    [javac] Compiling 1 source file to --topdir--/basic/java-program/abu\
\ild-java/classes

-jar-with-main-class:
      [jar] Building jar: --topdir--/basic/java-program/abuild-java/dist\
\/java-program.jar

BUILD SUCCESSFUL
Total time: <time>

A precise definition of build item would state that a build item is any directory that contains an Abuild.conf. Perhaps a more useful definition would say that a build item is the basic object that participates in abuild's object-oriented view of a software build. A build item provides some service within a build tree. Most build items build some kind of code: usually a library, executable, or Java archive. Build items may provide other kinds of services as well. For example, a build item may implement a code generator, support for a new compiler, or the ability to make use of a third-party software library. In addition, a build item may have certain attributes including a list of dependencies, a list of supported flags, information about what types of platforms the build item may be built on, a list of traits, and other non-dependency relationships to other build items. Each of these concepts is explored in more depth later in the document.

All build items that provide a service are required to have a name. Build item names must be unique within their build tree and all other build trees accessible to their build tree since the build item name is how abuild addresses a build item. Build item names consist of period-separated segments. Each segment may contain mixed case alphanumeric characters, underscores, and dashes. Build item names are case-sensitive.

The primary mechanism for describing build items is the Abuild.conf file. This file consists of colon-separated key/value pairs. A complete description of the Abuild.conf file may be found in Chapter 13, The Abuild.conf File. In the mean time, we will introduce keys as they become relevant to our discussion.

Although every build item has an Abuild.conf file, there are various other files that a build item may have. We defer a complete list and detailed discussion these files for later in the document, but we touch briefly upon a few of the common ones here.

A build tree, as defined before, is a collection of build items arranged hierarchically in the file system. A build tree is formed as a result of the items it contains holding references to the locations of their parents and children within the file system hierarchy. These locations are named as relative paths in the parent-dir and child-dirs keys of the items' Abuild.conf files. Ordinarily, the value of the parent-dir key is “..”, but it doesn't have to be. It is also customary to have the value of child-dirs contain single path elements (i.e.just a file without a directory), but this is also not a hard restriction. When abuild starts up, it looks for an Abuild.conf in the current directory and looks for its parent-dir key. Using that, it locates the parent build item's Abuild.conf file and reads it for its parent-dir key. This process is repeated until abuild discovers a build item that does not have a parent-dir key. This build item is known as the root build item of the build tree.

Note that the hierarchy defined by the layout of build items in the file system is a file system hierarchy and nothing more. It doesn't have to have any bearing at all on the dependency relationships among the build items. ^[8] That said, it is sensible to organize build items in a manner that relates to the architecture of the system, and this in turn usually has implications about dependencies. Still, it is important to keep in mind that abuild is not file-system driven but rather is dependency driven.

In addition to containing build items, build trees can contain other attributes. Among these are references to other build trees, a list of supported traits, and a list of plugins. We will discuss these topics later in the document. Most of a build tree's attributes appear in the root build item's Abuild.conf file.

In further describing build items and their attributes, it is useful to classify build items into several types. Most build items serve the purpose of providing code to be compiled. There are a number of special types of build items that serve other purposes. We discuss these here:

Virtually every software development project has some need to integrate with third-party software libraries. In a traditional build system, you might list the include paths, libraries, and library directories right in your Makefile or configuration file for whatever build system you are using. With abuild, the best way to integrate with a third-party library is to use a build item whose sole purpose is to export that library's information using an Abuild.interface file. In the simplest cases, a third-party library build item might be an interface only build item (described above) that just includes the appropriate library directives in a static Abuild.interface file. For example, a build item that provides access to the PCRE (Perl-compatible regular expression) libraries on a Linux distribution that has them installed in the system's standard include path might just include an Abuild.interface with the following contents:

LIBS = pcrecpp pcre

For Java build items, a third-party JAR build item would typically append the path to the JAR file to the abuild.classpath.external interface variable. Sometimes, the process may be more involved. For example, on a UNIX system, it is often desirable to use autoconf to determine what interface is required for a particular library. We present an example of using autoconf with abuild in Section 15.3, “Autoconf Example”. Still other libraries may use pkg-config. For those libraries, it may make sense to create a simple set of build rules that automatically generate an Abuild.interface after-build file (also discussed in Section 15.3, “Autoconf Example”) by running the pkg-config command. ^[10]

Whichever way you do it for a given package, the idea is that you should always create a build item whose job it is to provide the glue between abuild and the third-party library. Other build items that need to use the third-party library can then just declare a dependency on the build item that provides the third-party library's interface. This simplifies the process of using third-party libraries and makes it possible to create a uniform standard for doing so within any specific abuild build tree. It also alleviates the need to duplicate information about the third-party library throughout your source tree.

Abuild classifies platforms into a three-level hierarchy. The three levels are described by the following terms: