10. The Build Configuration System

XEmacs makes extensive use of the external features provided by the system it is running on. Determining which features are present and where they are located is the responsibility of the build configuration system.

10.1 The version.sh Script

The `version.sh' script is a snippet of Bourne shell script which sets version variables. By convention, these variables are given descriptive names, all in lower case ASCII letters, with words separated by underscores (`_', ASCII 0x5F). They are converted to C preprocessor macro definitions and added to `src/config.h' by `configure'. Thus each must have a corresponding `#undef' in `src/config.h.in'. Each macro's name is the same as the shell variable's, converted to all uppercase. Finally, the macros are used to initialize Lisp variables defined in `src/emacs.c'. These Lisp variables have the same name as the shell variables and preprocessor macros, except that they obey the Lisp conventions that Lisp variable names are all lowercase with words separated by hyphens (`-', ASCII 0x2D), while the C implementations are the same as the shell variable with the letter `V' (ASCII 0x56) prepended.

The file is updated by various release engineers and their scripts. Other developers should have no need to edit this file. The main exception would be to add a branch tag and possibly other information to `xemacs_extra_name' to describe informal releases from a private branch. In particular, `xemacs_release_date' and the `emacs_*_version' variables should refer to the most recent release in the parent branch, so "private branch" maintainers should not update them. If the branch is significant and long-lasting, you might enjoy assigning your own codenames. (Of course, if you have no intent of merging your changes to the mainline, you can do what you want with any of the variables. But in that case you should change the name of the program, as well, in version strings and the like.)

Regarding the syntax of the file, it is simply a sequence of shell variable assignments. So the only thing that you can rely on is that the shebang (the shell's interpreter comment, #!/bin/sh) will occupy the first line of the file. You should not count on order or other comments being preserved. On the other hand, some maintainers' tools do depend on the order, so as much as possible your tools should preserve the order of assignments.

Here is a table of the currently defined variables and their meanings (as of February 2005):

`#!/bin/sh'

The shebang, making this an executable script on Unix.

`emacs_is_beta'

Set to `t' when the release is a beta test release, otherwise null.

`emacs_major_version'

`emacs_minor_version'

`emacs_beta_version'

Strings containing decimal numbers representing the components of the version of the source tree. The name `emacs_beta_version' is a relic of the time when XEmacs had a two component version for public releases. Since XEmacs 21.1, both the beta series and the stable series have three-component version numbers, and `emacs_beta_version' holds the lowest-order component of the stable series as well as the beta series.

`xemacs_codename'

An optional string containing a codename for the release. Recent maintainers have chosen humorous themes for their codenames, and typically the names are used in alphabetical order.

`emacs_kit_version'

An optional string used for special branches. (This should be deprecated in favor of xemacs_extra_name.)

`infodock_major_version'

`infodock_minor_version'

`infodock_build_version'

Strings containing decimal numbers representing the components of the version of the Infodock applied to the source tree. (The Infodock project has been in hibernation since XEmacs 21.1.9 or so; these variables are unused in current XEmacsen.)

`xemacs_extra_name'

A string containing arbitrary additional information. If length is positive, it is automatically added to the version string after the codename.

`xemacs_release_date'

A string containing the date of the latest release in the series in ISO 8601 format. The time zone should not be present, it is defined to be UTC. Time is optional. Not currently used in the version string.

10.2 Adding Configurable Features

Adding a configurable feature requires at the very least adding an option to the `configure' script and a macro definition to `src/config.h.in' (see section 10.3 The configure Script), and often changes to Makefile precursors (see section 10.4 The Makefile Precursors).

Be prepared for the feature to be absent (even if you think that is always present for a particular OS release) and work with --with-site-prefixes to handle libraries and headers that are in unusual locations. There is no end to the strange ways in which systems can be configured and XEmacs is expected to cope with anything thrown at it.

10.3 The configure Script

At the heart of the XEmacs build configuration system is the `configure' script. This beast is maintained using the Autoconf system, which is a truly terrifying monstrosity based on a fundamentally flawed programming model (extensive use of macros), with an unpleasant implementation (GNU `m4'), used to string together a large set of ad hoc tests, to implement a configuration language with conventions that are unimportant in simple cases and counterintuitive when things get complicated. If that doesn't scare you off, Welcome! I think you're ready to become a configure hacker! (But be prepared for things to go downhill from here.)

Unless you plan to develop autoconf macros, much of this complexity can be removed by following the following rule:

Always quote (ie surround with `[]') every argument to every macro. This includes macros that appear in the argument list of other macros.

If this rule is followed and the macro produces incorrect results then the macro is buggy.

`configure' is, of course, is written in POSIX shell language, and autogenerated from a precursor. As of March 2005 on the mainline the precursor is called `configure.ac' and it is built using autoconf 2.59. Prior to that, and with XEmacs 21.4 and earlier it was called `configure.in' and build using autoconf 2.13. autoconf 2.5X is not completely backward compatible with autoconf 2.13 so the XEmacs Project chose to stick with the devil it knew for the stable line of XEmacs 21.4 releases.

One reason for worrying about the level of compatibility is the fact that XEmacs uses a lot of homebrew code, including `m4' macros, to implement special features in its `configure' script. Here are some of the important features:

• Selectively enabling debugging, error checking, and tracing.
• Specifying options by either --with or --enable.
• Complex options, which are set-valued (i.e., unordered; ordered lists of options, for example "take the first available from the list," are neither used currently nor given special support).

Where these are implemented as `m4' macros, the prefix `XE_' is used to identify them as XEmacs features in the `configure' precursor code. Here is a list of prototypes of the convenience macros provided for performing common operations:

USAGE_ERROR(string)

Prints a usage error and dies.

PRINT_VAR(var var ...)

Prints the name and value of the list of shell variables.

XE_ADD_OBJS(foo.o)

Appends the argument to the variable extra_objs. This variable goes to make up part of the link command line. If the command line argument --verbose is supplied a message is printed out.

XE_APPEND(value, varname)

Append the value (separated by a space) to the shell variable varname. varname should not be prefixed with a `$'. If the command line argument --verbose is supplied a message is printed out.

XE_PREPEND(value, varname)

Prepend the value (separated by a space) to the shell variable varname. varname should not be prefixed with a `$'. If the command line argument --verbose is supplied a message is printed out.

XE_DIE(message)

Used for situations that can't lead to a successful build, such as missing include files or conflicts between requested features.

XE_CHECK_FEATURE_DEPENDENCY(feature1, feature2)

--with-feature1 requires that --with-feature2 be also set and will die if the latter is not specified.

XE_STRIP_4TH_COMPONENT(var)

Converts the four part system name (eg i986-pc-linux-gnu) in var to a three part names (eg i986-pc-linux).

CANONICALIZE_PATH(varname)

Strips automount brokenness from the path in varname.

XE_PROTECT_LINKER_FLAGS(shell_var)

Wrap the command line arguments in shell_var with suitable incantations to ensure that the compiler front end passes them to the linker. Currently the magic is added only for gcc.

COLON_TO_SPACE(path)

Converts a colon separated list of paths into a space separated list of paths.

XE_ADD_RUNPATH_DIR(directory)

Internal function used by XE_COMPUTE_RUNPATH.

XE_COMPUTE_RUNPATH()

Calculate the appropriate dynamic library run path for XEmacs and the value to the shell variable ld_switch_run.

XE_SPACE(var, words)

Append to var a space separated list of words.

XE_SHLIB_STUFF

Generate the appropriate shared library support black magic. This is implemented in the file `aclocal.m4'.

XEmacs merged option support

Autoconf 2.59 divides the `configure' options into those that specify features (`--enable') and those that specify external libraries (`--with'). Many XEmacs options to not fall neatly into either of these categories and so as a matter of policy all options can be specified by either method.

These merged options are declared with the XE_MERGED_ARG macro. The arguments to the option are the same as AC_ARG_WITH and AC_ARG_ENABLE and code that worked with either of these macros will worked unchanged with XE_MERGED_ARG. The option value is stored in both with_FEATURE and enable_FEATURE shell variables.

XE_MERGED_ARG(package, help-string, action-if-true, action-if-false)

Declare an option that can be selected by either `--enable' or `--with'. The value of the option is stored in both with_FEATURE and enable_FEATURE.

XE_HELP_SUBSECTION(heading)

Add a subsection heading to the help output of merged arguments.

XEmacs keyword option support

A keyword option is one that accepts one of a number of pre-defined values (if support for sets of values is needed, x1see "complex options" below). For example, `--with-mail-locking=flock'.

Keyword options are defined with an expanded form of `XE_MERGED_ARG' called `XE_KEYWORD_ARG', which takes 5 parameters. The first 4 parameters are the same as original macro with the exception that all of these four parameters are required. The action-if-true code is run after the argument list has been parsed.

The 5th parameter is a list of supported keywords. The whole list must be quoted but the individual keywords should not. Here is how the `mail-locking' flag is defined:

XE_KEYWORD_ARG([mail-locking],
	AC_HELP_STRING([--with-mail-locking],[Specify the locking to be
                        used by movemail to prevent concurrent updates
                        of mail spool files. Valid types are `lockf',
                        `flock', `dot', `locking' or `mmdf'.]),
        [],
        [],
        [lockf,flock,file,locking,mmdf,pop])

(Note that the help string will be reformatted by `autoconf' so that all whitespace is first compressed to a single space, then folded to appear in the right-hand column as above. Thus the help string may appear differently when ./configure --help is invoked.)

If the option value is a not a valid keyword then an error message is generated, otherwise the value is left untouched.

This support is implemented via the following `GNU m4' macros. Macros labeled internal are not expected to be used by `configure.ac' programmers; they are part of the implementation of higher-level features.

XE_KEYWORD_ARG(package, help-string, action-if-true, action-if-false, [keyword1, keyword2, ....])

Expanded version of XE_MERGED_ARG for keyword options. All the parameters are required. The last argument is a comma-separated list of supported keywords, `m4'-quoted with `[]'.

XE_PARSE_KEYWORD_OPTION(prefix, cmdline-flag)

Internal macro to parse the option values. If an undeclared option is found then an error is generated.

XE_KEYWORD(keyword)

Internal macro to convert the keyword list into the various forms needed by XE_PARSE_KEYWORD_OPTION.

XEmacs complex option support

A complex option is one that takes a number of related values, as a set. For example, we might use --with-sound=native,nas to play sounds using the native libraries and via NAS.

Complex options are defined with an expanded form of `XE_MERGED_ARG' called `XE_COMPLEX_ARG', taking 5 parameters. The first 4 parameters are the same as original macro with the exception that all four parameters are required. The action-if-true code is run after the argument list has been parsed.

The 5th parameter is a list of XE_COMPLEX_OPTION macro calls that define the valid components and their default values. The list must be quoted but the individual macro calls should not. Here is how the `sound' flag is defined:

XE_COMPLEX_ARG([sound],
	AC_HELP_STRING([--enable-sound],[Compile with sound support.
                        Valid types are `native', `nas' and `esd'.
                        Prefix a type with 'no' to disable.
                        The first type can be `none' or `all'.  `none' means
                        `nonative,nonas,noesd'.  `all' means `native,nas,esd'.
                        Later options override earlier ones for the same TYPE.
                        The default is to autodetect all sound support except 
                        for ESD which defaults to off.]),
	[],
	[enable_sound_nas=""],
	[XE_COMPLEX_OPTION([native],[""]),
	 XE_COMPLEX_OPTION([nas],[""]),
	 XE_COMPLEX_OPTION([esd],[no])])

Each option is interpreted as a separate feature to be enabled or disabled. In keeping with `XE_MERGED_ARG', option values are stored in the variables with_package_component and enable_feature_component (e.g. with_sound_native and enable_sound_native).

The user of configure specifies the configuration by providing a list of components. The special components `all' and `none' may occur first in the list, setting the defaults for all components to `yes' or `no' respectively. Prefixing a component with `no' (eg `nofoo') disables the component.

In `configure.ac', default values of option values may be `yes' which means that the option must be used and an error must occur if there is a configuration problems (such as a missing library) or `no' which means that the option must not be used. The default value can also be the null string `""', usually meaning that `configure' will attempt to find support for the feature on the system, and will enable the configuration if it is available. Sometimes the null string means that configure's default is system-dependent. (This usage is not consistent, and depends on the implementation of the feature detector rather than the argument parser.) Users cannot specify the null string for an individual component from the command line.

There are two possible uses in XEmacs for this kind of facility. One is exemplified by sound: there are alternative protocols (native, ESD, NAS) and each is supported by a corresponding library. The other is a single library which may or may not be supported by multiple components of XEmacs, as exemplified by Xft. This latter usage may be more common during development of a feature. Perhaps specialized APIs should be provided, see comment on XE_COMPLEX_OPTION_HELP_STRING below.

Examples

Given a complex option --with-quux with three values: `foo', `bar' and `baz' that have defaults of `yes', `no' and "" the corresponding shell variables under different conditions are:

$ configure
with_quux_foo=yes
with_quux_bar=no
with_quux_baz=""

$ configure --with-quux
with_quux_foo=yes
with_quux_bar=no
with_quux_baz=""

$ configure --without-quux
with_quux_foo=no
with_quux_bar=no
with_quux_baz=no

$ configure --with-quux=bar
with_quux_foo=yes
with_quux_bar=yes
with_quux_baz=no

$ configure --with-quux=all
with_quux_foo=yes
with_quux_bar=yes
with_quux_baz=yes

$ configure --with-quux=none,bar
with_quux_foo=no
with_quux_bar=yes
with_quux_baz=no

$ configure --with-quux=all,nofoo
with_quux_foo=no
with_quux_bar=yes
with_quux_baz=yes

XE_COMPLEX_OPTION(option, yesno)

Declare a complex option and its default value. The value must be either `yes' or `no' or the null string `""'. The null string means "maybe", whose semantics are determined by the implementation of the option, not by the parser. Typical semantics are "use the library if found in the usual places" or "default is platform-dependent".

XE_COMPLEX_OPTION_HELP_STRING(flag, long, short, components, libraries)

Format a boilerplate help string for complex options.

This was originally written for the Xft option, and doesn't read so well for options based on alternative libraries like sound. Hackers beware: the API may be enhanced to deal with this in the future.

XE_COMPLEX_ARG(PACKAGE, HELP-STRING, ACTION-IF-TRUE, ACTION-IF-FALSE, [XE_COMPLEX_OPTION(a,yes), ....])

Extended version of XE_MERGED_ARG for complex options. All the parameters are required.

XE_EXPAND_COMPLEX_OPTION(prefix, component, yesno)

Internal macro create the option's shell variable containing the default value and to note the values in an option list.

XE_EXPAND_COMPLEX_OPTIONS(prefix, option_list)

Internal macro which recursively expands an option list.

XE_INIT_COMPLEX_OPTION(prefix, option_list)

Internal macro to initialise the complex option shell variables.

Variables of the form `prefix_option' contain the default value for that option. `prefix_types' contains a space-separated list of all the options and `prefix_default' contains a comma-separated list of all the default values.

XE_PARSE_COMPLEX_OPTION(prefix, cmdline-flag)

Internal macro to parse the option values. If an undeclared option is found then an error is generated.

10.4 The Makefile Precursors

As with other autoconf based programs, XEmacs's Makefiles are not written, they are generated. The configure program uses Makefile precursors, or templates, to generate the actual Makefiles. Unlike other programs this is a multistage process. The developer changes the file `Makefile.in.in', then configure first generates an intermediate file `Makefile.in', and finally produces a portable Makefile called `Makefile', and a Makefile optimized for GNU make called `GNUmakefile'.

`Makefile.in.in' is run through the C preprocessor as part of this process. This means that common files can be included and conditional construction of the Makefile can occur. When `GNUmakefile' is being produced USE_GNU_MAKE is defined.

Comments in `Makefile.in.in' must start with `##' to avoid confusing the preprocessor.