Bye autotools hello Scons

Building C++ projects with Scons

 26 March 2014  |   10 August 2018
  C++ python build ecosystem

Remember this?

That’s not so bad, as long as you have the right compiler and linker flags configured, depending on the target OS. The real problem, however, is trying to figure out how to alter something if you didn’t write the Makefile yourself. Or if you in fact did write it, but it was some time ago. Two days. No, four hours.

The problem

Try to study the autoconf and automake flow diagram, explained on Wikipedia: the GNU build system. Headache coming up? Suppose we would like to use these … uhm, “thingies”, for a simple C++ project.

First, let me define simple:

Onward, to the Makefile creation station! This is a sample file, from the Google Test Makefile:

GTEST_DIR = ..

USER_DIR = ../samples

CPPFLAGS += -isystem $(GTEST_DIR)/include
CXXFLAGS += -g -Wall -Wextra -pthread

TESTS = sample1_unittest
GTEST_HEADERS = $(GTEST_DIR)/include/gtest/*.h \
                $(GTEST_DIR)/include/gtest/internal/*.h

all : $(TESTS)

clean :
    rm -f $(TESTS) gtest.a gtest_main.a *.o

GTEST_SRCS_ = $(GTEST_DIR)/src/*.cc $(GTEST_DIR)/src/*.h $(GTEST_HEADERS)
gtest-all.o : $(GTEST_SRCS_)
    $(CXX) $(CPPFLAGS) -I$(GTEST_DIR) $(CXXFLAGS) -c \
            $(GTEST_DIR)/src/gtest-all.cc

gtest_main.o : $(GTEST_SRCS_)
    $(CXX) $(CPPFLAGS) -I$(GTEST_DIR) $(CXXFLAGS) -c \
            $(GTEST_DIR)/src/gtest_main.cc

gtest.a : gtest-all.o
    $(AR) $(ARFLAGS) [email protected] $^

gtest_main.a : gtest-all.o gtest_main.o
    $(AR) $(ARFLAGS) [email protected] $^

sample1.o : $(USER_DIR)/sample1.cc $(USER_DIR)/sample1.h $(GTEST_HEADERS)
    $(CXX) $(CPPFLAGS) $(CXXFLAGS) -c $(USER_DIR)/sample1.cc

sample1_unittest.o : $(USER_DIR)/sample1_unittest.cc \
                     $(USER_DIR)/sample1.h $(GTEST_HEADERS)
    $(CXX) $(CPPFLAGS) $(CXXFLAGS) -c $(USER_DIR)/sample1_unittest.cc

sample1_unittest : sample1.o sample1_unittest.o gtest_main.a
    $(CXX) $(CPPFLAGS) $(CXXFLAGS) -lpthread $^ -o [email protected]

This first builds the gtest_main.a binary, to be able to link that with our test after the source (sample1.o) has been is built. The syntax is clumsy, simple files require me to have a deep knowledge how flags and linking work, and I don’t want to specify everything in one block.

As esr in his blog post Scons is full of win today said, it’s a maintenance nightmare. What to do?

There are a few alternatives which aim to cover everything autotools does, such as QMake from Trolltech or CMake (that actually generates Makefiles. You’re not helping, CMake!). Or, one could go for Scons.

build your software, better.

Scons starts with a single SConstruct file, which acts as the makefile. You can bootstrap the default build target using the scons command. (cleaning with scons --clean). The big deal here is that the contents of that file is simply python (2.7, I know)!

Want to write a utility function to gather all your cpp files? Fine, go ahead, def mystuff(): (you do know this already exists, right? Use Glob()) Want to unit test these, and include them? Done. Want to split up everything per source directory? Use SConscript files and include these from within your root SConstruct using SConscript('file', 'envVarToExport').

This is my blueprint construct file:

env = Environment(CXX = 'g++')

gtest = env.SConscript('lib/gtest/SConscript', 'env')
src = env.SConscript('src/SConscript', 'env')
out = env.SConscript('test/SConscript', 'env gtest src')

# output is an array with path to built binaries. We only built one file - run it (includes gtest_main).
test = Command( target = "testoutput",
                source = str(out[0]),
                action = str(out[0]) )
AlwaysBuild(test)

Things to note:

This is the conscript which builds google test:

Import('env')
env = env.Clone(CPPPATH = './:./include')

env.Append(CXXFLAGS = ['-g', '-Wall', '-Wextra', '-pthread'])
gtest = env.Library(target = 'gtest', source = ['src/gtest-all.cc', 'src/gtest_main.cc'])

Return('gtest')

Things to note:

Source:

Import('env')
env = env.Clone(CPPPATH = './')
src = env.Library(target = 'wizards', source = Glob('*.cc'))

Return('src')

Things to note:

And finally, test, linking both source and google test:

Import('env', 'gtest', 'src')
env = env.Clone()

env.Append(LIBPATH = ['#lib/gtest', '#src'])
env.Append(LIBS = [gtest, src])
out = env.Program(target = 'wizards_unittests', source = Glob('*.cc'))

Return('out')

Things to note:

So where does that leave us? Yes there’s still “syntax” to be learned, even if you’re a seasoned python developer; you need to know which function to use for what, that’s what the excellent scons doc is for. I know it made my life a lot easier while trying to do something simple and this is only the tip of the iceberg. Scons is relatively popular according to Stack Overflow, the documentation is excellent and if all else fails you can write your own garbage in a full-fledged dynamic language.

The only really irritating bit is the python 2.7 dependency, so don’t forget to use virtualenv.

In practice

The section below contains practical snippets used by myself in some point in the past, commented in Dutch. Feel free to grab a piece.

  1. SCons Wiki Frontpage
  2. Single HTML Manpage
  3. SCons Construction Variables om bvb de compiler te specifiëren.

Separating SConstruct and SConscript

Why? http://www.scons.org/doc/2.1.0/HTML/scons-user/c3356.html

Defining a build output, duplicating source files, etc. For example, in a SConstruct file:

SConscript('SConscript', variant_dir######'build', duplicate0)

This is an example file to build and run some GTest in SConscript:

def Glob( pattern ###### '*.*', dir  '.' ):
    import os, fnmatch
    files = []
    for file in os.listdir( Dir(dir).srcnode().abspath ):
        if fnmatch.fnmatch(file, pattern) :
            files.append( os.path.join( dir, file ) )
    return files

# construction variables: http://www.scons.org/doc/0.96.90/HTML/scons-user/a3061.html
env ###### Environment(CXX  'g++',
                                    CPPPATH = '../:./include')

# add to library search path env.Append(LIBPATH = ['/usr/local/lib/'])
# add to libraries link path env.Append(LIBS = ['SDL_image','GL'])

env.Append(CPPFLAGS = ['-isystem ./include'])
env.Append(CXXFLAGS = ['-g', '-Wall', '-Wextra', '-pthread'])

env.SharedLibrary(target ###### 'gtest_main.dll', source  ['../src/gtest-all.cc'])

# after that, we should link with gtest_main

Domain Driven Design in C

Who says imperative languages don't do DDD?  3 August 2018

Python Class structure basics

A crash course in Python classes  1 October 2013

 Top