cson  AmalgamationBuild

ACHTUNG: THIS PAGE IS NOW MAINTAINED IN THE NEW WIKI: http://whiki.wanderinghorse.net/wikis/cson/?page=AmalgamationBuild

Amalgamation Build

The build process supports compounding all of the cson implementation files into a bundle containing only a single .c and .h, the so-called "amalgamation build."

The primary intention is to simplify inclusion of cson into arbitrary source trees.

The secondary intention is performance. The developers of the sqlite project have determined that such an amalgamation can improve performance of the code by a few ticks (i think they were averaging 5% or so) because compilers can (reportedly) typically perform more optimizations if all code is in the same compilation unit.

To create the amalgamation build do one of the following from the top source directory:

~> make amal
# or:
~> ./createAmalgamation.sh

The output will be several files named cson_amalgamation_SOMETHING.{c,h}, which can be compiled into library form or included directly in a client application. The "SOMETHING" part is one of:

  • core contains just the core cson library, without the cson_session or cson_cgi parts.
  • session requires the core amalgamation and adds the cson_session parts, including a full copy of the underlying cpdo database access library. This also includes cson_cpdo (because of how the dependencies work out).
  • cgi requires the session amalgamation and adds the cson_cgi.

The amalgamation builds each build off of the next one up in the list. Thus to build a CGI-capable bundle you need the 3 different "core", "session", and "cgi" files. The amalgamation builds do not contain the application code implemented in the test/demo applications (i.e. files with their own main() routines).

In addition to the above files, there is an "all-inclusive" (almost) file pair named cson_amalgamation.{c,h}. Those contain everything the above files contain, packaged as a single-file source bundle.

FIXME: the cson_sessmgr_whio_ht and cson_sessmgr_whio_epfs parts cannot currently be included in the amalgamation because of internal collisions of code used by the underlying cpdo and whio libraries. They both use their own copies (with renamed public APIs) of a custom printf implementation, and the symbols internal to that code collide when included in this amalgamation build.

Client-side Requirements for the Session/CGI Parts

When compiling in the session code (or the CGI bits, which use the session code), which uses an underlying database access library, clients must define the following macros when compiling the amalgamation if they want to enable the appropriate database back-end:

  • CSON_ENABLE_CPDO=1 must be set if cpdo support is to be enabled, plus we need additional macros for the specific database back-ends:
  • sqlite3: CPDO_ENABLE_SQLITE3=1
  • MySQL v5: CPDO_ENABLE_MYSQL5=1

The cpdo bits are only used internally (in the C code), but are exposed to the client via the header file because we need that for cson_cpdo to be able to work. However, only CSON_ENABLE_CPDO needs to be set from client code if they whish to use cson_cpdo, otherwise clients of the compiled amalgamation build do not need to set the above macros. However, all client code may (depending on how you end up building this code) need to link to the appropriate back-end database libraries, e.g. libsqlite3 or libmysql_client. The linker arguments for doing so are (at least for MySQL) quite platform- and environment-dependent. See the cpdo wiki for full details about how to determine the proper link arguments.

Build Examples

The files are quite big, but a large percentage of the header file is documentation (about 85%, last time i checked). The C files contains only a small amount of comments/documentation (about 25% last i checked, and most of that comes from API docs for internal APIs).

Compiling the amalgamation is quite fast. Using the code from 20110421 on a dual-core 3.2 GHz 64-bit Ubuntu Linux system:

~> ls -la cson_amalgamation_*.[ch]
-rw-r--r-- 1 stephan stephan  78320 Apr 21 22:42 cson_amalgamation_cgi.c
-rw-r--r-- 1 stephan stephan  31459 Apr 21 22:42 cson_amalgamation_cgi.h
-rw-r--r-- 1 stephan stephan 149901 Apr 21 22:42 cson_amalgamation_core.c
-rw-r--r-- 1 stephan stephan  65802 Apr 21 22:42 cson_amalgamation_core.h
-rw-r--r-- 1 stephan stephan 206307 Apr 21 22:42 cson_amalgamation_session.c
-rw-r--r-- 1 stephan stephan 132628 Apr 21 22:42 cson_amalgamation_session.h

# Core library:
~> time gcc -std=c89 -c -pedantic -Wall cson_amalgamation_core.c

real	0m0.286s
user	0m0.260s
sys	0m0.010s

# Core + sessions
# Most of this code is the underlying database access library.
~> time gcc -std=c89 -c -pedantic -Wall \
   -DCSON_ENABLE_CPDO=1 \
   -DCPDO_ENABLE_SQLITE3=1 \
   -DCPDO_ENABLE_MYSQL5=1 \
   cson_amalgamation_session.c

real	0m0.556s
user	0m0.490s
sys	0m0.030s

# Core + sessions + CGI:
~> time gcc -c -std=c89 -pedantic -Wall \
   -DCSON_ENABLE_CPDO=1 \
   -DCPDO_ENABLE_SQLITE3=1 \
   -DCPDO_ENABLE_MYSQL5=1 \
   cson_amalgamation_cgi.c

real	0m0.171s
user	0m0.140s
sys	0m0.010s

# Same thing using the all-inclusive files:
~> time gcc -c -std=c89 -pedantic -Wall \
   -DCSON_ENABLE_CPDO=1 \
   -DCPDO_ENABLE_SQLITE3=1 \
   -DCPDO_ENABLE_MYSQL5=1 \
   cson_amalgamation.c

real	0m0.926s
user	0m0.850s
sys	0m0.050s

Linking client applications to those builds is trivial. Here's an example taken from the cson source tree:

gcc -o app \
    cson_amalgamation.o \
    cgi-test.o \
    $(mysql_config --libs) \
    -lsqlite3 -ldl -lpthread

When linking with MySQL it might not be possible to create a static client app, depending on the platform. i was able to build a static application only by building a custom libsqlite3 with extension-loading support disabled (which needs libdl, which can't work together with static linking). MySQL uses other libraries which, on Linux, apparently cannot work when statically linked.

An example of building a statically-linked binary:

~> gcc -static -o app \
    cson_amalgamation.o \
    cgi-test.o \
    -L. -lsqlite3 -lpthread
# libpthread is needed by sqlite3 unless it is built with SQLITE_THREADSAFE=0.

The application binary was over 2MB (1.4MB after stripping).