Gentoo Linux Jam Guide

1. Introduction

What is Jam?

Jam is a software build tool. It is an alternative to GNU make. Jam was written by Christopher Seiwald of Perforce Software, and has been released as open source.

Jam includes a case sensitive interpreted procedural language to define the rules and actions processed by Jam. Jam uses these rules (from a Jamfile or a group of Jamfiles) to define how Jam should build a set of targets, which can be object files, libraries, or executables, or other files.

Jam's language features include functions, procedures, local variables, standard conditional and iterative constructs, lists, and target-specific variables. Jam processes dependencies rapidly, and its automatic generation of C & C++ header dependencies eliminates the necessity of declaring header or object files in the Jamfile.

Jam's operation consists of separate start-up, parsing, binding, and updating phases. During the start-up phase Jam obtains system information from environment variables, which is stored in Jam variables of the same name. Parsing involves setting variable values and dependencies based on user defined rules and rule invokations. In the binding phase Jam uses dependence information to bind variables to target files or other filesystem locations, and checks modification times before commencing the updating phase in which actions are executed for targets which require updating.

Since shell commands executed during the updating phase will always preceed the processing of rules during the parsing phase, a Jamfile cannot ordinarily set a variable value based on filesystem data. However, Jam can invoke itself recursively to cope with that unusual case.

A table which briefly summarizes the standard Jam documentation is available here .

Prerequisites

While no prior knowledge of Jam is assumed, a basic understanding of the C++ language and the GNU toolchain, and a working knowledge of the bash shell is recommended. XML is used as well as C++ to illustrate Jam usage, but detailed knowledge of these languages is not necessary in order to understand the examples. However, it is advisable that the reader have a prior knowledge of the relative advantages of the various build tools.

2. Installing Jam

First of all Jam needs to be installed on your system. For Gentoo Linux systems, the following is sufficient:

# emerge dev-util/ftjam

The following commands enable environment variables for optimized compilation to be visible to Jam on Gentoo systems. These commands can go into your ~/.bash_profile.

source /etc/make.conf
export CFLAGS="${CFLAGS}"
export CXXFLAGS="${CXXFLAGS}"

Jam can be built on the following operating systems: AIX, BSD/386 1.0, COHERENT/386, DGUX 5.4, FreeBSD, HPUX 9.0, IRIX 5.0, Linux, NEXTSTEP 3.2, OSF/1, PTX V2.1.0, Solaris 2, SunOS4.1, Ultrix 4.2, BeOS, Windows NT, and OS/2.

On Linux the following commands are sufficient to build Jam:

$ wget ftp://ftp.perforce.com/pub/jam/jam-2.5.tar
$ tar xf jam-2.5.tar
$ cd jam-2.5
$ make
$ cp bin.linuxx86/jam /usr/local/bin

Notes on building for other operating systems can be found in the README file.

3. Jam Language

Jam's data type

Jam variables are lists which can contain zero or more items. The items can be iterated with for or accessed by index.

servers = mail sql web ;

Echo "The server names are:" $(servers) ;

for i in $(servers) {

        Echo "  " server $(i) ;
}

Echo "Server 2 is" $(servers[2]) ;

When a list is declated the items are separated by whitespace. An item which includes whitespace is declared by enclosing it in double quotes. The following two list declarations are equivalent.

servers = "mail" sql web ;

servers =
    mail
    sql
    web
;

Variable expansion

Jam variables can be expanded by referencing other variables. The expression for a value obtained by reference is the name of the referenced variable enclosed within the $( and ) characters. The expansion of a variable which contains references to more than one other variable results in the set of all combinations of each of the items of the lists which comprise each referenced variable.

Jam provides several predefined variables. One of these, called JAMUNAME contains data about the system than Jam is running on.

servers = mail sql web ;

ip_mail = 192.168.0.2 ;
ip_sql = 192.168.0.4 ;
ip_web = 192.168.0.4 ;

host = $(JAMUNAME[4]) ;

Echo "The ip address of" $(host) "is" $(ip_$(host)) ;
Echo "The ip addresses are " $(ip_$(servers)) ;

Conditionals

Jam can test variables for the existance of values by making the variable itself the expression which is tested by if. Jam uses the C operators !=, !, &&, and !! for inequality, inversion, and, and or respectively. Jam uses the operator = to test for equality, and along with <, >, <=, and >= the operator applies to lists rather than to singular values.

4. Hello World

Here a small set of files is created in order to test the build process.

Creating a source file

#include <iostream>

using namespace std;

int main (int argc, char *argv[]) {

	cout << "hello world" << endl;
}

Creating a Jamfile

C++ = g++ ;
LINK = $(C++) ;

Main hello : hello.cc ;

Making, testing, and removing the executable

$ jam
...found 11 target(s)...
...updating 2 target(s)...
Link hello 
Chmod1 hello 
...updated 2 target(s)...

$ ./hello
hello world

You can remove the object and executable files by typing jam clean.

$ jam clean
...found 1 target(s)...
...updating 1 target(s)...
Clean clean 
...updated 1 target(s)...

5. Linking a user-built object file and library

Creating the header file

#ifndef _MYTOOLS_
#define _MYTOOLS_

#include <string>

namespace std {

	void whatever (string s);
}

#endif

Creating the source file

#include <iostream>
#include "mytools.h"

using namespace std;

void std::whatever (string s) {

	cout << "whatever [" << s << ']' << endl;
}

Modifying hello.cc

The new header file needs to be included, and the procedure whatever needs to be called by main.

#include <iostream>
#include "mytools.h"

using namespace std;

int main (int argc, char *argv[]) {

	cout << "hello world" << endl;
	whatever ("test");
}

Linking an object file

The file mytools.cc needs to be added to the main dependency list.

C++ = g++ ;
LINK = $(C++) ;

Main hello : hello.cc mytools.cc ;

Linking multiple executables

Using the Main rule more than once in a Jamfile can result in repeated dependencies. When a Jamfile is used to build multiple executables, the Objects and MainfromObjects rules should be used instead of Main.

C++ = g++ ;
LINK = $(C++) ;

Objects hello.cc mytools.cc ;
MainFromObjects hello : hello.o mytools.o ;

Linking a library

The Library rule is used to build the mytools library, and the LinkLibraries rule is used to add the library to the target program.

C++ = g++ ;
LINK = $(C++) ;

Main hello : hello.cc ;
Library mytools : mytools.cc ;
LinkLibraries hello : mytools ;

6. Linking a pre-compiled library

Modifying the source file

The header file math.h should be included along with the other headers. To test the library the arctangent of 1 is calculated.

#include <iostream>
#include <math.h>
#include "mytools.h"

using namespace std;

void whatever (string s) {

	cout << "whatever [" << s << ']' << endl;
	cout << "atan (1.0) = " << atan (1.0) << endl;
}

Modifying the Jamfile

The math library libm needs to be added to library list.

C++ = g++ ;
LINK = $(C++) ;

LINKLIBS += -lm ;

Main hello : hello.cc mytools.cc ;

The += operator causes the identifiers following the operator to be added to the list of identifiers which was previously bound to the variable, instead of replacing them as in the case of the = operator.

Examining the build command

Run Jam with the -an flags to see how the math library is added to the build command.

$ jam -an
...found 29 target(s)...
...updating 3 target(s)...
C++ hello.o 

g++ -c -o hello.o -O   hello.cc

C++ mytools.o 

g++ -c -o mytools.o -O   mytools.cc

Link hello 

g++  -o hello  hello.o mytools.o  -lm 

Chmod1 hello 

chmod 711 hello

...updated 3 target(s)...

7. Target specific variables and user defined actions

Target specific variables

Target specific variables enable the attachment of diverse information to specific build targets.

LINKLIBS = -lm ;

In this example the math library will be linked for all targets built by the Jamfile.

LINKLIBS on hello = -lm ;

In this second example the math library will only be linked on the hello target.

Target specific variables can be referenced from within Jam by using the on keyword.

on hello Echo "The libraries for target hello are $(LINKLIBS)" ;

Here the identifer hello which follows the on keyword specifies that variable references will be specific to that target. A target specific variable cannot be assigned a value using this syntax, but non-specific variables can still be referenced.

Another variable which has a predefined function is LINKFLAGS, which causes flags to be added to the start of the linker argument list. Also, multiple targets can be specified for target-specific examples.

LINKFLAGS on hello myproject test = -pg ;

In this example the flag which activates profiling for g++ will be set for all three targets.

Enabling profiling in g++ requires that the -pg flag is set in both the compile and link phases if the two phases are executed separately. Unfortunately the C++FLAGS variable does not work as expected for the compile phase in stock Jam. An alternative method of setting C++ compile flags is the use of the ObjectC++Flags rule.

ObjectC++Flags hello : -pg ;

Target specific variables are not limited to variables which have a predefined meaning within Jam. An arbitrary variable may be attached to a target and then referenced by the on keyword (as mentioned earliear in this section), or by user defined actions which act on that target.

User defined actions

User defined actions enable the specification of the shell commands which produce build targets. A target may be a file, which typically will be an object file or an executable built by a compiler. Pseudotargets may also be defined in Jam. Jam does not check any file status (i.e. the most recent time of modification of the file) that is assocated with a pseudotarget as occurs with an ordinary target. Two predefined pseudotargets are clean and all. The psuedotarget all is built when no target was explicitly named when Jam was executed from the command shell. In the following example the pseudotarget prof controls the generation of profile data for hello.

actions Profile {
echo "The target is $(1)"
gprof -b hello $(PROFILE_DATA)
}

PROFILE_DATA on prof = gmon.out ;
NOTFILE prof ;
ALWAYS prof ;

Profile prof ;

Here a set of actions are defined for the identifier Profile. NOTFILE identifies prof as a pseudotarget, and ALWAYS is used for targets that have no dependencies to trigger the build process. The last line of the example code associates the actions with the target.

Jam actions are associated with the binding of file targets to variables. Binding only ever binds a variable to a single file once when Jam is run, and uses the last value that was assigned to that variable during the parsing phase of Jam's execution. The first argument of the action is bound to the target (or targets) of the action, and the second argument is bound to any dependencies.

Binding is the main behaviour which sets a build system like Jam apart from a general purpose procedural language. The binding process is what enables target dependencies to conditionally trigger build actions.

8. Rules

Overview

Rules are primarily used to define the dependecies between targets, and can also be used for factoring common build patterns in the Jamfile so as to increase readability and to decrease opportunities for bugs to occur. Rules take zero to nine arguments (each a list) and may return a list. Rules may declare local variables and may be invoked recursively.

rule hash key : source : target  {

    local result ;

    if $(source) {
        if $(key) = $(source[1]) {
            result = "\"$(target[1])\"" ;
            return $(result) ;
        } else {
            return [ hash $(key) : $(source[2-]) : $(target[2-]) ] ;
        }
    }
}

colours = red green blue ;
letters = alpha beta gamma ;

colour = blue ;
result = [ hash $(colour) : $(colours) : $(letters) ] ;

Echo The letter for $(colour) is $(result) ;

9. Other languages

Overview

Jam is not exclusively for C and C++ programs. It can be used to build executables for any language, so long as the appropriate rules and actions are defined.

Example

This new rule defines build dependencies for a set of actions and ensures that the output file will be removed when jam clean is invoked. The first two arguments will only be bound to targets if a set of actions with the same name as the rule exists.

rule MakeHtml {

	Depends $(1) : $(2) ;
	Depends all : $(1) ;
	Clean clean : $(1) ;
}

Descriptive names for arguments can be used in a rule.

rule MakeHtml html : xml_xsl {

	Depends $(html) : $(xml_xsl) ;
	Depends all : $(html) ;
	Clean clean : $(html) ;
}

The target for a set of actions is the first argument for that set of actions, which is referenced as $(1). An action can only have a single input argument, referenced as $(2), but this argument may consist of a multiple file targets. The actions have reduced whitespace so that the build command appears in Jam's output the same way as Jam's built-in build commands do.

actions MakeHtml {
Xalan -v $(2) >$(1) ;
}

MakeHtml jamref.html : jamref.xml jamref.xsl ;

Depends all : jamref.html ;

A rule that references the dependencies must be invoked before Jam will attempt to build anything. Making the all pseudotarget dependant upon a specific target causes that target to be built automatically when Jam is invoked without any arguments. Is this example the dependency has been repeated; once within the rule and again after the rule is invoked.

Arguments to Jam rules take the form of parameter lists. The rule MakeHtml takes two arguments: the input list, and the output list. The colon is the delimiter which separates the two lists.

Improvements

A rule that invokes MakeHtml can make the main invocation more succinct:

rule Html {

	local h = $(1).html ;
	local m = $(1).xml ;
	local s = $(1).xsl ;

	MakeHtml $(h) : $(m) $(s) ;
}

Html jamref ;

The declaration of local variables in the previous example was unnecessary; a shorter form of the rule is:

rule Html { MakeHtml $(1).html : $(1).xml $(1).xsl ; }

Parameters can be passed to the stylesheet by adding an extra argument to the Html rule: a name list which refers to globally declared data. The list is expanded by the rule to conform with the syntax required by Xalan and the resulting parameter list is passed to the action as a target specific variable.

rule MakeHtml {

	Depends $(1) : $(2) ;
	Clean clean : $(1) ;
}

actions MakeHtml { 
Xalan -v $(parms) $(2) >$(1) ; 
} 
 
rule Html { 
 
	local target = $(1).html ;
        local names = $(2) ; 
        local i value ; 
 
        for i in $(names) { 
                value = $($(1)_$(i)) ; 
                parms on $(target) += -p $(i) "\"'$(value)'\"" ; 
        } 

        MakeHtml $(target) : $(1).xml $(1).xsl ;
} 
 
jamref_date = "1 Jan 2007" ; 
jamref_revision = 1.1 ; 
 
Html jamref : date revision ;

Depends all : jamref.html ;

The files used in this example are available as jamref.xml, jamref.xsl, and jamref.html.

10. Directories

Overview

Two rules, SubDir and SubInclude, are used to manage subdirectories.

A SubDir rule identifies the directory which contains a Jamfile. The first argument in a SubDir rule is the name of the variable which refers to the the directory root (in this guide this variable is named JAMR). The other arguments for the SubDir rule are not variable names, but are instead literal tokens which identify the directories in the path from the root.

At the root of the tree containing the Jamfiles there should be a Jamrules file, which contains any default rules for the Jamfiles. A side effect of the SubDir rule is that the Jamrules file is read. The Jamrules file is only ever read once, at the first occurrence of the SubDir rule. The SubDir rule should precede any rules which refer to the contents of the directory.

A SubInclude rule causes another Jamfile to be read in. Like the SubDir rule, the first argument is the name of a variable, but the other arguments are literal directory names. Typically the Jamfile in the root directory (where the Jamrules file is) will contain a SubInclude rule for each subdirectory one level down. Multiple levels of subdirectories may be managed by writing SubInclude rules for a particular directory in the Jamfile of the parent directory, rather that putting all of the SubInclude rules in the Jamfile located in the root directory.

An environment variable may be used to define the root of the directory tree which contains Jamfiles. The SubDir rule will compute a value for this variable if it is not already defined. The SubDir rule sets $(SUBDIR) to the full path name of the directory identifies by the rule.

Example

export JAMR=~/jam

An example directory structure may be set up with the following commands:

$ mkdir $JAMR
$ cd $JAMR
$ mkdir main tools

The files mytools.h & mytools.cc should go into the tools directory. The file hello.c should go into the main directory.

The Jamfile in the JAMR directory should be as follows:

SubDir JAMR ;
SubInclude JAMR tools ;
SubInclude JAMR main ;

The Jamrules file belongs in the JAMR directory. Here are the rules for variables which are commonly referenced by the individual Jamfiles. The Jam language assigns values to strings in a fashion similar to that of the bash shell. Here the string variable TOOLS is set to the path of the tools subdirectory.

Echo "Parsing Jamrules" ;

C++ = g++ ;
LINK = $(C++) ;
TOOLS = $(JAMR)/tools ;

The Jamfile in the tools subdirectory uses the Objects rule to specify that only the compilation of the object file is desired.

SubDir JAMR tools ;

Objects mytools.cc ;

The Jamfile in the main directory refers to the tools directory for a dependency. The SubDirHdrs rule modifies the list of directories that the compiler will search for header files, and must include the tools subdirectory in this case.

SubDir JAMR main ;

SubDirHdrs $(TOOLS) ;

Main hello : hello.cc $(TOOLS)/mytools.cc ;

11. Debugging

Pre-empive strategies

One of the advantages of Jam is its terse syntax, which enables production patterns to be written without a slew of extra punctuation characters. The downside of this feature is that Jam cannot readily distinguish between an badly formed expression and a list with items which necessarily contain semicolons or colons. This means that typos may not be reliably identified by Jam as such, and therefore measures should be taken when writing a Jamfile to assist in debugging, particulaly when multiple files are included.

There are several coding techniques which can make life easier when it comes to tracking down errors. The techniques of modularisation is particularly relevant since build systems do not usually have the benefit of top-down specification of function. Debugging rules which involve readable descriptions of the variables that they output are also beneficial.

Modularisation can be effected by minimising the number of expressions that do not exist within rules, and by using local variables rather than global ones wherever possible. When a rule sets a global variable there should ideally be a corresponding debugging rule which prints a description of the variable and its value. These rules can be commented out or renamed as non-functional variants in order to avoid being overwhelmed by debugging output when a Jamfile is run.

Another modularisation technique is the use of milestone targets. Milestone targets are simple file targets which are updated when a set of dependencies have been met. They enable a large build to be broken up into a set of chunks which can be independently tested, and they also enable dependencies to be filtered out of Jam's debug output in order to focus on a particular set of targets.

The multiple target trap

Jam's behaviour is not intuitive when more than one build target is produced from a single set of actions. This is because the dependency graph is constructed from the targets of actions and not from the actions themselves. If there are two or more build targets from a set of actions, and those targets are not dependant upon the same set of source targets, then the actions may be executed when all of the dependencies of the source targets have not been met, since the actions will be executed based on the dependencies of any of the build targets, not the dependencies of all of them.

This trap can be avoided by ensuring that actions which update multiple targets only ever have a single source target, and by ensuring that all of the build targets are dependant upon it. The source target could have a set of actions which simply write a debugging tag to a file, and would be dependant upon all of the source targets of the original action.

12. Recursive Invocation

Overview

Recursive invocation of Jam is a technique which enables Jam to set variable values based on the execution of arbitrary commands.

Example

actions GetDistroPath {
source /etc/make.conf
export DISTDIR
jam make_distro
}

actions MakeDistro {
tar czf $(DISTDIR)/demo.tgz $(JAMR)
}

GetDistroPath distro ;
NOTFILE distro ;
ALWAYS distro ;

MakeDistro make_distro ;
NOTFILE make_distro ;
ALWAYS make_distro ;

In this example calling Jam with the pseudotarget distro results in Jam invoking itself recursively, with the environment variable DISTDIR set to the value defined in make.conf.