Building eXtremeDB Libraries on Unix-Linux

As explained in the Building and running SDK Samples page, makefiles are provided in the eXtremeDB root directory and all sample (project) directories. If it is necessary to rebuild the eXtremeDB libraries, the build system is a process of running the GNU make utility with a combination of cascade makefiles, project files and possible additional third-party products as described in the following sections.

The instructions in the sections below use the following terms:

Build systems entry-points

The build system supports the following entry points:

all

Build all binaries

 
    make … all …
     

clean

Clean the tree. This command removes only intermediate files leaving the final products and build-logs intact.

 
    make … clean …
     

distclean

Total cleanup. This command removes all the intermediate files and directories; the final files and the build logs will be purged out of the directory tree.

 
    make … distclean …
     

 

Also, the following old-fashion targets (from eXtremeDB releases prior to version 8.0) are left for compatibility:

host

Builds only the host-side executables.

target

Builds only the target-side utilities and libraries.

tools

This is the logical combination host + target

samples

Builds the samples only.

 

Make systems tips

Following are some tips for using the build system:

For MacOS systems:

Make sure your JAVA_HOME environment variable has a valid value. The typical way to initialize the variable is in the ~/.profile. Add following line into the file:

     
    export JAVA_HOME=$(/usr/libexec/java_home)
     

Make sure your user has enough resources available. Increase the "number of processes per user" limit using:

 
    sudo sysctl -w kern.maxprocperuid=4096
     

Templates

Following are some templates for files of the build system, ready for copy-paste. You need to make a unique name for your project and substitute <project-name> with this name.

Sub-directory or Project-directory level makefile

Do not forget to change <offset-path-to-the-root-directory> with your offset path.

 
    MCO_ROOT=$(abspath <offset-path-to-the-root-directory>)
    include $(MCO_ROOT)/include/settings.makefile
    include $(if $(findstring yes,$(MCO_QUICK_BUILD))$(findstring on,$(MCO_QUICK_BUILD)),,
        $(MCO_ROOT)/)makefile.cascade
    include $(MCO_ROOT)/include/rules.makefile
     

 

Sub-directory level makefile.cascade

Substitute <directory1>…<directoryN> with the names of your directories. If some of the directories are optional put "-" before the include directive.

 
    include $(call mco_makefile_file, \
        <directory1>/makefile.cascade \
        ...
        <directoryN>/makefile.cascade \
    )
     

 

Project-directory level makefile.cascade

Do not forget to change <project-name> with the unique name of your project. If you have several projects in the same directory and some of them are optional (i.e. the existence of the project depends on features included into the package), then "-" may be specified in front of the include directive.

 
    MCO_FILE_DIR:=$(dir $(lastword $(MAKEFILE_LIST)))
    include $(MCO_FILE_DIR)makefile.project.<project-name-1>
    ...
    include $(MCO_FILE_DIR)makefile.project.<project-name-N>
     

 

Project-directory level makefile.project.<project-name> for a sample program

Do not forget to change <project-name> with the unique name of your project.

 
    MCO_PRJ_LIST += <project-name>
    PRJ_TARGET_EXECUTABLE_DPTR += <project-name>
    PRJ_TARGET_EXECUTABLE_OFFS += <project-name>
    PRJ_<project-name>_TAGS:=<project-name> sample core
    PRJ_<project-name>_DIR:=$(call mco_project_dir)
    PRJ_<project-name>_SRC:=$(call mco_project_file,main.c) $(MCO_SAMPLES_COMMON_DIR)/common.c
    PRJ_<project-name>_SCHEMAS:=$(call mco_project_file,schema.mco)
    PRJ_<project-name>_DATABASES:=<database-name>
    PRJ_<project-name>_RUNTIME:=YES
    PRJ_<project-name>_MEMDEV:=PRIVATE
    PRJ_<project-name>_STORAGE:=TRANSIENT
    PRJ_<project-name>_SYNC:=AUTO
    PRJ_<project-name>_TRANSMGR:=MURSIW
     

 

Project-directory level makefile.project.<project-name> for a library

Do not forget to change <project-name> with the unique name of your project.

 
    MCO_PRJ_LIST += <project-name>
    PRJ_TARGET_LIBRARY_DPTR_STATIC  += <project-name>
    PRJ_TARGET_LIBRARY_DPTR_DYNAMIC += <project-name>
    PRJ_TARGET_LIBRARY_OFFS_STATIC  += <project-name>
    PRJ_TARGET_LIBRARY_OFFS_DYNAMIC += <project-name>
    PRJ_<project-name>_TAGS:=<project-name> target core
    PRJ_<project-name>_DIR :=$(call mco_project_dir)
    PRJ_<project-name>_SRC :=$(call mco_project_file, mcosrc1.c … mcosrcN.cpp)
     

 

Issues and diagnostics

In most cases, build issues come from the incompatibility of third-party products installed in the build system and eXtremeDB requirements. Sometimes the versions of the products are different from those required. Sometimes the build system lacks the development headers of the products.

Usually, the process of issue detecting is specific for every case, but a good approach is to:

1. Use MCO_EXPLAIN=python,openssl to see why and how the build script decided to use the third-party components:

 
    $ make qqq MCO_EXPLAIN=openssl,python
    Analyzing build environment...
    MCO_FORCE_PYTHON=auto
    MCO_BUILD_PYTHON=/usr/bin/python
    [X] - OpenSSL 1.1.1 exists on the build system
    [ ] - Forced to use MCO_OPENSSL_INCLUDE_PATH env. variable
    [ ] - Forced to use build system OpenSSL installation
    [ ] - Forced to build and use own OpenSSL installation
    [ ] - auto-detection enabled, MCO_OPENSSL_INCLUDE_PATH env. variable is 
        set and development headers are present
    [ ] - auto-detection enabled, the build system match to the target system 
        and there are usable development headers in the local OpenSSL installation
    [X] - auto-detection enabled, build and use own OpenSSL installation if all above failed
    [ ] - Invalid value of MCO_FORCE_OPENSSL detected, build and use own OpenSSL installation
    MCO_TARGET_OPENSSL_INCLUDE=/home/x/t1219/eXtremeDB/target/sal/net/openssl-Linux-x86_64/include
 
    [X] Python does exist on the build system
    [X] Target and build-system arch. signatures are the same [+] or Build system is x86_64 
        and the target is x86 [+] (python is runnable and good for the target usage)
    [X] DBus width match (otherwise not good for target usage)
    [X] Build python is 2.7.x (otherwise not good for target/cross usage)
    [ ] Dev. headers are present (otherwise not good for target usage)
    BUILD_PYTHON =/usr/bin/python
    CROSS_PYTHON =/home/x/t1219/eXtremeDB/target/python/Python-2.7.14-Linux-x86_64/bin/python2.7
    TARGET_PYTHON=/home/x/t1219/eXtremeDB/target/python/Python-2.7.14-Linux-x86_64/bin/python2.7
 
    Loading cascade make system...
    Setting up the build procedure...
    Processing...
    make: *** No rule to make target 'qqq'.  Stop.
     

 

2. Run make with explanations enabled and gather it's output log and internal database:

 
    $ make MCO_EXPLAIN=openssl,python all -p>issue-log 2>&1
     

 

3. For investigation of any compilation, it is good to disable parallel building and enable detailed logging (no -j make option!)

 
    make MCO_BUILD_LOG=full all
     

 

The make utility

The eXtremeDB build system utilizes the GNU make utility to build the product's source code into a set of executables and libraries used during development of end-user applications. To do its job, the make utility maintains products and source files dependency information in the form of a dependency tree where the top-level node (target in make's terminology) that depends on several sub-targets, then sub-sub-targets and so on down to the terminal nodes which are the source files. The make utility is able to track source file modifications using the file systems timestamps and file "exist" functions. Thus, make is able to track changes in the dependency chains and update all affected targets that depend on the modified source file.

The build system includes two sub-systems: one is an array of project files that describe each of the eXtremeDB components as projects, defines the properties of the project and its dependencies; the other is a set of cascade make files that delivers the project information to the make utility in order to maintain the full dependency tree of all components, source code, headers, and eXtremeDB utilities. This full dependency tree gives the make utility the ability to trace and build (and re-build if necessary) any part of eXtremeDB. The build operations can be run as a series of build actions one-step-at-a-time or in parallel to utilize the full multi-CPU power of the build system.

The cascade part of the build system

The cascade makefile system is a set of makefile.cascade files located in all major directories of the product. The cascade makefile defines the hierarchy of directories and includes other downstream makefile.cascade files and project makefiles.

The build system's project files

Each component of the product is described by a project makefile for the build system. The project makefile defines what the component is, how it is to be built, it's source code files, flags, and properties. There are several types of project in the product: executables (for the host and the target separately), libraries (static and dynamic), JAVA class and jar files, Python modules and third-party modules used in the product.

 

Implementation of the build system

The product build system is a set of interconnected makefiles. Among all these build systems files, there are some that are special: the top-level makefile, the include/settings.makefile and the include/rules.makefile.

The top-level makefile initiates the build process. First, it includes the include/settings.makefile. This file contains all start-up settings for the build system and the required auto-detection heuristic. A user can override the settings externally or put necessary tunings into the include/settings.makefile. Then, the top-level makefile includes the top-level cascade file which runs (using the cascade inclusion technique) through the all the product's source tree and gathers other cascade and project makefiles. This step delivers the project properties of all the components and samples into the make system. Finally, the top-level makefile includes the include/rules.makefile. This file instructs the make utility how to build all the product files and all necessary intermediate files. This includes the dependency information that instructs the make utility what depends on what. Also, it generates the top-level target for the make command that initiates the build process.

All other sub-directory makefiles (those with no extension) are necessary to be able to initiate the build procedure for this sub-directory and downstream directories. Internally the sub-directory makefiles define the current and top-level directory, and include the top-level makefile. This gives the build system the ability to check and, if necessary, rebuild not only the files in the current directory but even those in another parallel (from the source tree's point of view) directories in order to maintain correctness of the build process when one or several of the components was/were modified.

The auto-detection heuristic implemented in the include/settings.makefile does the following:

1. Detects the build platforms environment: Hardware architecture and OS.

2. Sets the build utilities (such as echo, copy, make, etc.) used in the build process. (This setting depends on the build system.)

3. Sets the build system's compiler family. The same compiler family is set for both host and target systems if no override is given in form of command line options HOST_COMPILER_FAMILY and TARGET_COMPILER_FAMILY (see section Make system overrides below).

4. Using the compiler family information, the script detects the hardware and OS definition for host and target systems.

5. Defines the default setting for eXtremeDB components such as: default memory drivers, synchronization primitives wrapper, file system drivers, etc.

6. Defines the libraries and feature switches for the command generation for compilation and linking, depending on the project properties (for example the libraries that need to be linked into a program to support SQL functionality; or what needs to be given to the source code to make it aware of "Sequences" support).

7. Finally, the script checks the existence of third-party products in the system and decides if it is possible to use the existing installations or if it is required to build its own. (See the detail below on the test and decision-making procedure.)

The dependency-generation code is located in the include/rules.makefile. This code makes the dependency tree for the make utility using the project properties gathered by the cascade sub-system. Each of the projects has a unique name and the set of properties associated with that name. All project properties are defined using make variables, such as PRJ_<project-name>_XXXX:=YYYY, where XXXX is a property identifier and YYYY is a value. Also, there are several special variables named "lists" that contain the project names as references.

The project needs to be linked into the MCO_PRJ_LIST list (i.e. its name needs to be added there as MCO_PRJ_LIST += <project-name>) and at least into one of the following lists:

A project can be added to several suitable PRJ* lists (eg. C/C++ projects are good for libraries and executables, Java-projects - for Classes and Jars). For example, the mcovtmem project is included into the PRJ_TARGET_LIBRARY_DPTR_STATIC and PRJ_TARGET_LIBRARY_DPTR_DYNAMIC lists only, while the mcovtdsk library is present in all four PRJ_TARGET_LIBRARY_* lists. The build system sets up a special build procedure for every project based on these lists.

The build script maintains global scope make targets, such as build all and clean all, using the MCO_PRJ_LIST list. The build all action is run from the command line make all, and produces all of the product binary files. The clean all action is run from the command line make clean, and removes all of the intermediate files but leaves the final binaries and build logs. The make distclean command line cleans everything. Also, it is possible to build each of the binary files (executables, the libraries, classes, and jars) separately using a command line like make <binary-location-and-name> (for example make /home/user/eXtremeDB/target/bin/08-benchmarks-perf).

Note that the build system uses full file names to avoid "process current directory"-related disambiguate issues for parallel building.

The build system includes a mechanism that allows the user to include or exclude selective components of eXtremeDB into or out of the build process. The filter is tag-based. Each of the project files defines a set of tags for each of the components and the MCO_ENABLE / MCO_DISABLE controls (see section Make system controls below) let the user determine the way to enumerate which of the target components need to be enabled or disabled. This mechanism not only excludes the named components from the build process, but also makes all of the other components aware of the status of the enabled / disabled components, and updates them accordingly. For example, MCO_DISABLE=openssl will exclude OpenSSL from the build procedure as well as the references to OpenSSL from the eXtremeDB crypto-driver and OpenSSL-specific examples.

Third-party products in the eXtremeDB build system

eXtremeDBuses some third-party software components (such as LUA, Java, Python, and OpenSSL) in its code. The use of these components is pretty version-sensitive: eXtremeDB code relies on the components' API and build-procedures. This is why the build script needs to make sure that the pre-installed version of the components located on the build system is suitable (i.e. that the version matches eXtremeDB requirements, the required hardware is supported, the development headers are present and so on). To achieve this, there are special precautions implemented in the build script:

Note: it is possible to override the auto-detection for Python and OpenSSL using the MCO_FORCE_OPENSSL and MCO_FORCE_PYTHON controls. (See section Make system overrides below.)

Structure of the makefiles

This section explains the internal structure of different kinds of makefiles used in the build system.

root-level and sub-directory makefiles

The root-level makefile defines the location of the eXtremeDB installation as an absolute path in the MCO_ROOT variable, includes the required parts of the build system and activates the cascade sub-system and the build process:

 
    MCO_ROOT=$(abspath .)
    include $(MCO_ROOT)/include/settings.makefile
    include $(if $(findstring yes,$(MCO_QUICK_BUILD))
        $(findstring on,$(MCO_QUICK_BUILD)),,$(MCO_ROOT)/)makefile.cascade
    include $(MCO_ROOT)/include/rules.makefile
     

The only difference for a sub-directory makefile is in the way to define MCO_ROOT. It converts an offset path to the root directory to its absolute representation:

 
    MCO_ROOT=$(abspath ..)
    include $(MCO_ROOT)/include/settings.makefile
    include $(if $(findstring yes,$(MCO_QUICK_BUILD))
        $(findstring on,$(MCO_QUICK_BUILD)),,$(MCO_ROOT)/)makefile.cascade
    include $(MCO_ROOT)/include/rules.makefile
     

In most cases, it is enough to copy an existing sub-directory makefile into a new directory and adjust the MCO_ROOT variable to produce a new sub-directory makefile.

Note that the makefiles have some logic that work according to the MCO_QUICK_BUILD control. (See section Make system controls below for details.)

makefile.cascade

Typical sub-directory makefile.cascade files include all cascade files from the sub-directories below its level:

 

 
    -include $(call mco_makefile_file, \
        crypt/makefile.cascade \
        mcobackup/makefile.cascade \
        ...
        python/makefile.cascade \
        feedhandler/makefile.cascade \
    )
     

The project-level makefile.cascade file includes the project files and delivers the project properties into the make utility:

 
    MCO_FILE_DIR:=$(dir $(lastword $(MAKEFILE_LIST)))
    include $(MCO_FILE_DIR)makefile.project.mcolib
 

The simplest way to create a new sub-directory or project makefile.cascade file is to copy an existing one and adjust the inclusion of the downstream cascade and project files.

Note that very few cascade files contain logic that includes downstream projects depending on settings from the include/settings.makefile. For example

 
    target/sal/sync/makefile.cascade: include $(call mco_makefile_file,
            
        $(addprefix makefile.project.mcos,$(MCO_TARGET_SYNC_IMPLEMENTATIONS)))
            
         

Note that some cascade files have a dash ("-") in front of the "include" directive. This construct of the make utility language defines optional inclusion, which is good for cases when a sub-level makefile.cascade needs to be included (may be or may be not present) in the source tree depending on the features included in a package.

makefile.project.XXXX

A typical project makefile defines the name of the project, its properties (such as source code files, databases, eXtremeDB features used etc.) and ties the project to appropriate lists to cause the build system to generate build procedures for the project.

Following is a sample of a library-project makefile:

 
    MCO_PRJ_LIST += mcosql
    PRJ_TARGET_LIBRARY_DPTR_STATIC  += mcosql
    PRJ_TARGET_LIBRARY_DPTR_DYNAMIC += mcosql
    PRJ_TARGET_LIBRARY_OFFS_STATIC  += mcosql
    PRJ_TARGET_LIBRARY_OFFS_DYNAMIC += mcosql
    PRJ_mcosql_TAGS:=target sql malloc
    PRJ_mcosql_DIR :=$(call mco_project_dir)
    PRJ_mcosql_SRC :=$(call mco_project_file, dbapi.cpp compiler.cpp 
        sqlc.cpp operators.cpp optimizer.cpp nodes.cpp schema.cpp 
        sql.cpp utils.cpp exceptions.cpp stub.cpp value.cpp hash.cpp 
        multisql.cpp mcoapi.cpp mcosql.cpp mcoseq.cpp mcoapic.cpp)
         

Following is a sample of an executable-project makefile:

 
    MCO_PRJ_LIST += hasimple-master
    PRJ_TARGET_EXECUTABLE_DPTR +=
    PRJ_TARGET_EXECUTABLE_OFFS += hasimple-master
    PRJ_hasimple-master_TAGS:=sample ha
    PRJ_hasimple-master_DIR:=$(call mco_project_dir)
    PRJ_hasimple-master_SRC:=$(call mco_project_file, master.c) 
        $(MCO_SAMPLES_COMMON_DIR)/common.c $(MCO_SAMPLES_COMMON_DIR)/commonha.c
    PRJ_hasimple-master_SCHEMAS:=$(call mco_project_file,schema.mco)
    PRJ_hasimple-master_DATABASES:=simplemstdb
    PRJ_hasimple-master_RUNTIME:=YES
    PRJ_hasimple-master_MEMDEV:=PRIVATE
    PRJ_hasimple-master_STORAGE:=TRANSIENT
    PRJ_hasimple-master_SYNC:=AUTO
    PRJ_hasimple-master_TRANSMGR:=MURSIW
    PRJ_hasimple-master_F_HA:=TCP
     

Note that the project files add the projects to the corresponding build system lists and the defined project properties. For the meaning of the "properties" see section Project makefile flags below.

Third-party makefiles

The make system wraps the third-party make systems with special project files. A project file is "special" depending on the details of how the third-party product build process needs to be handled. These files are pretty complicated. For reference see: target/sal/net/makefile.project.openssl or target/python/makefile.project.python. The build procedure, products, build-logs and intermediate files are handled via the PRJ_CUSTOM list. (See section Project makefile flags below for details.)

 

Control switches and overrides

Following is the full list of available control switches, overrides and project properties.

Project makefile flags

Build lists

The following lists are common for all project makefiles. A project needs to be included into MCO_PRJ_LIST and into at least one of other PRJ_-lists depending on the meaning of the project. The project can be included in as many PRJ_-lists as required provided the project's code is suitable for it.

Note that the "+=" operator is used for the inclusion.

MCO_PRJ_LIST

A project needs to be included into MCO_PRJ_LIST to make the build system aware of it. This list is used to generate global-scope operation code for the build all and clean all operations.

Example:

 
    MCO_PRJ_LIST += <project-name>
     

 

PRJ_HOST_EXECUTABLE

This list contains all the projects that generate host-side executable files including mcocomp, sql2mco eXtremeDB host utilities.

Example:

 
    PRJ_HOST_EXECUTABLE += <project-name>
 

 

PRJ_TARGET_EXECUTABLE_DPTR

PRJ_TARGET_EXECUTABLE_OFFS

These lists include target-side eXtremeDB executables such as the regunlock utility and xSQL. The project needs to be put in the DPTR-list or OFFS-list depending on its meaning, or possibly into both of them.

Example:

 
    PRJ_TARGET_EXECUTABLE_DPTR += <project-name>
    PRJ_TARGET_EXECUTABLE_OFFS += <project-name>
     

 

PRJ_TARGET_LIBRARY_DPTR_DYNAMIC

PRJ_TARGET_LIBRARY_OFFS_DYNAMIC

PRJ_TARGET_LIBRARY_DPTR_STATIC

PRJ_TARGET_LIBRARY_OFFS_STATIC

These lists include all target-side eXtremeDB library-projects. If a library is meant to be built as a static code container, it needs to be added to the STATIC-list, and to DYNAMIC-list for dynamic-loaded modules. The DPTR-lists and OFFS-lists separate DPTR and OFFS versions of the libraries. In most cases a library project would be included into all four of the lists but in some cases only a dynamic version is desired (for example feed handlers modules), or only the OFFS version makes sense (eg. the mcovtdsk library).

Example:

 
    PRJ_TARGET_LIBRARY_DPTR_DYNAMIC += <project-name>
    PRJ_TARGET_LIBRARY_OFFS_DYNAMIC += <project-name>
    PRJ_TARGET_LIBRARY_DPTR_STATIC += <project-name>
    PRJ_TARGET_LIBRARY_OFFS_STATIC += <project-name>
     

 

PRJ_TARGET_CLASS

This list defines projects meant to be Java-classes. These projects are processed by the Java compiler into separate class-files.

Example:

 
    PRJ_TARGET_CLASS += <project-name>
     

 

PRJ_TARGET_JAR

The projects from this list produce Java code library JAR files. These projects are compiled using the Java compiler and are then put together with the jar utility.

Example:

 
    PRJ_TARGET_JAR += <project-name>
 

 

PRJ_TARGET_ANALYZE_ONLY

This is a quite special list. The build system processes the projects on the list, generates build properties out of the project properties, but does not generate code that produces the project products. The generated build properties may be used for some projects that need a special procedure to build its products.

Example:

     
    PRJ_TARGET_ANALYZE_ONLY += <project-name>
     

 

PRJ_CUSTOM

This is another special project list. The projects from this list need to be built using a special custom procedure defined in the project's makefile.

Other PRJ-flags are required to be set for projects such as PRJ_<project-name>_PRODUCTS, PRJ_<project-name>_PRODUCT_DIRS, PRJ_<project-name>_INTERMEDIATE_DIRS and PRJ_<project-name>_BUILD_LOGS. In most cases, the project needs to add its build procedure to the PRJ_GENERATE_CUSTOM_RULES list.

Example:

 
    PRJ_CUSTOM += <project-name>
    PRJ_GENERATE_CUSTOM_RULES += <project-name-build-proc>
    PRJ_<project-name>_PRODUCTS         :=$(addprefix $(MCO_PRODUCT_ROOT)/target/bin/<dir>,
        <project-name>.so <project-name>_debug.so)
    PRJ_<project-name>_PRODUCT_DIRS     :=$(MCO_PRODUCT_ROOT)/target/bin/<dir>
    PRJ_<project-name>_INTERMEDIATE_DIRS:=$(subst $(MCO_ROOT),$(MCO_BUILD_ROOT),
        $(PRJ_<project-name>_DIR)/build-$(MCO_TARGET_ARCH_SFX))
     

 

Common project flags

The following flags define basic project properties and need to be included into every project file. Certainly, every project has to have a unique <project-name>; like "00-ddl" for instance.

Note that the ":=" operator is used for value assignment. Other kinds of assignment mean value expansion for the make utility and need to be used very carefully due to the asynchronous nature of the build procedure. There is no way to predict the order of the values assignment.

 

PRJ_<project-name>_DIR:=$(call mco_project_dir)

This list defines the location of the project as an absolute path. All projects initialize this property via $(call mco_project_dir). This construct calculates the path out of the makefile.project-file location.

Example:

 
    PRJ_00-ddl_DIR:=$(call mco_project_dir)
     

 

PRJ_<project-name>_TAGS

This list assigns tag(s) to the project (as a space-separated list). Using the tags it is possible to define logical groups within the project (by a feature, os-specific, etc). The build system is able to enable/disable the groups by user request or automatically.

Example:

 
    PRJ_00-ddl_TAGS:=sample core
     

 

PRJ_<project-name>_SRC

This list defines a set of project source code files (as a space-divided list). It may include C/C++ files and headers and Java files. Each of the files must be specified by its name and absolute filesystem path. The build system construct $(call mco_project_file, <file1.c> <file2.cpp> ... <fileN.h>) helps with the file location.

Example:

 
    PRJ_00-ddl_SRC:=$(call mco_project_file,oidsize.h main.c) $(MCO_SAMPLES_COMMON_DIR)/common.c
     

 

Project-dependent flags

The following flags are project-dependent and may or may not be specified depending on the meaning of the project.

 

PRJ_<project-name>_RUNTIME

Enables or disables the eXtremeDB runtime for the project.

Possible values:

Example:

 
    PRJ_00-ddl_RUNTIME:=YES
     

 

PRJ_<project-name>_STORAGE

Defines the type of database storage for the project

Possible values:

Example:

 
    PRJ_00-ddl_STORAGE:=TRANSIENT
     

 

PRJ_<project-name>_MEMDEV

Defines the memory device used for the project as the database memory storage

Possible values:

Example:

 
    PRJ_00-ddl_MEMDEV:=PRIVATE
     

 

PRJ_<project-name>_STORAGEDEV

Defines the persistent storage driver for the project. This works for persistent and hybrid database based projects.

Possible values:

Example:

 
    PRJ_02-open-disk_file_STORAGEDEV:=AUTO
     

 

PRJ_<project-name>_SYNC

Defines the synchronization implementation for the project. In the most cases, AUTO will serve the project well.

Possible values:

Example:

 
    PRJ_00-ddl_SYNC:=AUTO
     

 

PRJ_<project-name>_TRANSMGR

Defines the transaction manager for the project.

Possible values:

Example:

 
    PRJ_00-ddl_TRANSMGR:=MURSIW
     

 

PRJ_<project-name>_DATABASES

PRJ_<project-name>_SCHEMAS

These two flags define the project databases (names) and the schema files. There can be several database names/schemas divided by spaces in the list. The number of items in the lists must match. The schema files must be represented by the file names with absolute path ( $(call mco_project_file,<schema.mco> helps here).

Possible values:

Example:

 
    PRJ_16-uda-udameta_SCHEMAS:=$(call mco_project_file,
        uda1.mco uda2.mco uda3.mco uda4.mco uda5.mco)
    PRJ_16-uda-udameta_DATABASES:=uda1 uda2 uda3 uda4 uda5
     

 

PRJ_<project-name>_F_BACKUP

Indicates that the project uses Inline Backup feature.

Possible values:

Example:

 
    PRJ_18-backup-online_F_BACKUP:=YES
     

 

PRJ_<project-name>_F_CLUSTER

Enables the Cluster feature for the project.

Possible values:

Example:

 
    PRJ_cluster_attach_F_CLUSTER:=TCP
     

Note: do not forget to put PRJ_<project-name>_TRANSMGR:=MVCC for Cluster-enabled projects.

 

PRJ_<project-name>_F_CPP

Instructs the build system to generate the C++ interface for the project's databases (equivalent to setting the -hpp flag for mcocomp).

Possible values:

Example:

 
    PRJ_20-languages-cpp_smartptr_F_CPP:=YES
     

 

PRJ_<project-name>_F_CRT

Defines whether to use the C Runtime Library CRT wrappers for the project.

Possible values:

Example:

 
    PRJ_fh_F_CRT:=YES
     

 

PRJ_<project-name>_F_CRYPT

Enables cryptography support for the project.

Possible values:

Example:

 
    PRJ_02-open-security-disk_cipher_F_CRYPT:=YES
     

 

PRJ_<project-name>_F_DBCALC

Enables the DBCALC feature for the project

Possible values:

Example:

 
    PRJ_17-statistics-dbcalc-conv_F_DBCALC:=YES
     

 

PRJ_<project-name>_F_FHLIB

Enables FeedHandler support for the project

Possible values:

Example:

 
    PRJ_fh_F_FHLIB:=YES
     

 

PRJ_<project-name>_F_HA

Enables High Availability feature support for the project

Possible values:

Note: there can be multiple space-divided values for this setting.

Example:

 
    PRJ_hasimple-master_F_HA:=TCP
    PRJ_hamultichan-master_F_HA:=TCP UDP
     

 

PRJ_<project-name>_F_HV

Enables HttpView feature support for the project

Possible values:

Example:

 
    PRJ_22-httpview-sql_F_HV:=YES
     

 

PRJ_<project-name>_F_IOT

Enables IoT feature support

Possible values:

Example:

 
    PRJ_iot_simple_dev_F_IOT:=YES
     

 

PRJ_<project-name>_F_LOG

Enable the Transaction Logging feature for the project

Possible values:

Example:

 
    PRJ_tlogbasic_F_LOG:=YES
     

 

PRJ_<project-name>_F_NATIVE

Force native fields ordering for the mcocomp utility (equivalent to setting the -nosort option).

Possible values:

Example:

 
    PRJ_createdb_sniffer_F_NATIVE:=YES
     

 

PRJ_<project-name>_F_ODBC

Enable ODBC support for the project

Possible values:

Example:

 
    PRJ_simpleodbc_F_ODBC:=YES
     

 

PRJ_<project-name>_F_PERFMON

Enable the Performance Monitor feature for the project

Possible values:

Example:

 
    PRJ_25-perfmon_F_PERFMON:=YES
     

 

PRJ_<project-name>_F_REST

Enable REST interface feature support (for Web Services) for the project

Possible values:

Example:

 
    PRJ_24-rest-mt_F_REST:=YES
     

 

PRJ_<project-name>_F_SEQUENCES

Enable Sequence calculations support for the project

Possible values:

Example:

 
    PRJ_sequences-basic_F_SEQUENCES:=RLE
     

 

PRJ_<project-name>_F_SERI

Enable serialization for the project

Possible values:

Example:

 
    PRJ_18-backup-bse-conv_F_SERI:=YES
     

 

PRJ_<project-name>_F_SMARTPTR

Enable generation of the SmartPtr interface for the C++ interface to the projects database (equivalent to setting the -smartptr option for mcocomp)

Possible values:

Example:

 
    PRJ_20-languages-cpp_smartptr_F_SMARTPTR:=YES
     

 

PRJ_<project-name>_F_SQL

Enable the SQL feature support for the project.

Possible values:

Example:

 
    PRJ_api-sql-01-c-api-mursimul_F_SQL:=LOCAL
    PRJ_tpcodbc_F_SQL:=REMOTE
    PRJ_api-sql-10-rsql-client-standalone_F_SQL:=REMOTE_STANDALONE
     

 

PRJ_<project-name>_F_UDA

Enables UDA database interface feature for the project

Possible values:

Example:

 
    PRJ_16-uda-udaops_F_UDA:=YES
     

 

PRJ_<project-name>_F_XML

Enable the XML database serialization feature for the project (equivalent to setting the -x option for mcocomp)

Possible values:

Example:

 
    PRJ_16-uda-udaopen_F_UDA:=YES
     

 

Overrides

The following flags override default build system behavior on the project level.

 

PRJ_<project-name>_DEFINES

Adds special defines for the project in the form <define-name>[=<define-value>]. The build script will add the necessary compiler declaration directives automatically.

Example:

 
    PRJ_mcoews_DEFINES:=MCOEWS_BUILD_RUNTIME MCO_HTTP_OVERRIDE_CONFIG 
        MCO_HTTP_CGI MCO_SECURITY_NONE MCO_HTTP_CONNECTION_USER_DATA
         

 

PRJ_<project-name>_INCLUDE

Additional include locations for the project as directory paths. The build script will add the necessary compiler include path declaration directives.

Example:

 
    PRJ_mcocomplib_INCLUDE:=$(MCO_ROOT)/host/mcocomp
     

 

PRJ_<project-name>_DEBUG

Force the project to be linked with the debug version of the eXtremeDB libraries.

Possible values:

Example:

 
    PRJ_fhdbg_DEBUG:=YES
     

 

PRJ_<project-name>_F_USE_DYNAMIC_LIBRARIES

Force the project to be linked with the dynamic version of the eXtremeDB libraries.

Possible values:

Example:

 
    PRJ_fh_F_USE_DYNAMIC_LIBRARIES:=YES
     

 

PRJ_<project-name>_ADDITIONAL_LIBRARIES

Add special libraries to the linkage of the project. The script will add all necessary library directives for the compiler.

Examples:

 
    PRJ_mcoluaload_ADDITIONAL_LIBRARIES:=luajit
     

 

PRJ_<project-name>_SKIP_AUTO_BUILD

Exclude this project from the auto-build sequence. The project build procedure needs to be initiated by another project (specify the product of the project as a required file for another project).

Possible values:

Examples:

 
    PRJ_openssl_SKIP_AUTO_BUILD  :=YES
     

 

PRJ_<project-name>_BUILD_SFX

This defines a custom suffix for the products intermediate directory. Good for projects with custom build procedure.

 

PRJ_<project-name>_RESULT_NAME_BASE

Overrides the name of the product of the project. The build script will use this value instead of <project-name> during product file name generation. The script will add all necessary prefixes and suffixes according to the projects product nature.

Example:

 
    PRJ_mcocomplib_RESULT_NAME_BASE:=mcocomp
     

 

Custom-build flags

The following flags are good for custom-build projects only.

 

PRJ_<project-name>_PRODUCTS

The list of the products of the project.

 

PRJ_<project-name>_PRODUCT_DIRS

The list of directories where the products of the project will be created (if the location is not the default bin directory).

 

PRJ_<project-name>_BUILD_LOGS

The list of build logs for the project.

 

PRJ_<project-name>_INTERMEDIATE_DIRS

The list of directories where the intermediate files of the project will be created during the build procedure.

 

PRJ_<project-name>_MAKEFUNC

Defines the name of the custom build procedure that generates the build target for the project.

 

Make system controls

The following flags are defined to control the make system build process.

 

MCO_BUILD_ROOT

Defines the directory where the script will put all the build activities and intermediate files. It must be an absolute path to the directory. The default is the same directory as MCO_ROOT. Using this control it is possible to build eXtremeDB binaries out of the source tree.

 

MCO_PRODUCT_ROOT

Defines the directory where the build script will create products of the build process. It must be an absolute path to the directory. The default is MCO_BUILD_ROOT. Using this control it is possible to separate the final products from the intermediate files and the source code tree.

 

MCO_QUICK_BUILD

Values: on or yes. Disables the detailed dependency tracking mechanism but makes the build process faster overall. When the override is active, the script starts the cascade import not from the MCO_ROOT but from the current directory, so the build procedure will cover only the current and all downstream directories.

 

MCO_BUILD_LOG

Defines how to show the build process log. There are three settings: on, off and full. The default setting is on and it displays only brief progress information. The off setting makes the build script quiet (see the result code and the build logs for results). The full setting prints all of the output immediately and does not maintain build logs. For the on and off settings the build scrip puts all the commands and the output into the build log files located in the projects intermediate directories. Usually the build log is named the same as the project's product name with the ".build-log" extension. Also the build script produces separate build logs for object files naming them as <object-file-name>.o.build-log.

Note that it is highly not recommended to use the full setting for a parallel build. The asynchronous nature of the process makes a mess in the output. It is very tricky to find the actual error message in this build log.

 

MCO_EXTRA_CHECKS

The on setting enables extra checks (equivalent to -Wall and -pedantic options) for gcc-based builds.

 

MCO_ENABLE_EVAL_BINARIES

The yes setting enables producing the evaluation version of eXtremeDB binaries in bin-directories with the ".eval" suffix.

 

MCO_DISABLE

This control switch allows the user to exclude groups of or specific projects / features out of the build process. The value of this control is a comma-separated or space-separated list of project tags that need to be disabled. Disabling a tag results not only in excluding the specified projects from the build process, but also disabling compilation defines indicating the presence of disabled features.

Example:

 
    MCO_DISABLE:=sequences
    MCO_DISABLE:=java python lua
    MCO_DISABLE:=net,fsystem
     

 

MCO_ENABLE

This control switch enables projects / features that are normally disabled.

Example:

 
    MCO_ENABLE :=wchar 
    MCO_ENABLE :=rtree
     

 

Additional switches

The following switch settings have the effect of setting other build system controls.

 

SHOW_BUILD_LOG=on

Uses MCO_BUILD_LOG=full.

 

x64=on|off

The build system detects the target databus width automatically (with the target compiler's help). It is necessary to specify x64=off to make the build script produce 32-bit code.

 

MCO_SKIP_PERFMON=yes

Skips the Performance Monitor (perfmon) build.

 

MCO_SKIP_SEQ_LIB=yes

Disables sequences support.

 

MCO_SKIP_LUA_UDF=yes

Disables UDF for Lua support.

 

MCO_SKIP_XSQL_BIN=yes

Skips the xSQL build.

 

JAVA=off

Skips Java-related code.

 

PYTHON=off

Skips Python-related code.

 

LUA=off

Skips Lua-related code.

 

FEEDHANDLER=off

Skip FeedHandler code.

 

Make system overrides

The following switches override the gcc/g++ compiler name for GCC-based toolchains. This makes it possible to use alternative compilers.

MCO_BUILD_GCC_OVERRIDE=<gcc-executable-name>

MCO_BUILD_GXX_OVERRIDE=<g++-executable-name>

MCO_HOST_GCC_OVERRIDE=<gcc-executable-name>

MCO_HOST_GXX_OVERRIDE=<g++-executable-name>

MCO_TARGET_GCC_OVERRIDE=<gcc-executable-name>

MCO_TARGET_GXX_OVERRIDE=<g++-executable-name>

Example:

 
    MCO_TARGET_GCC_OVERRIDE=gcc456
    MCO_TARGET_GXX_OVERRIDE=g++456
     

 

MCO_FORCE_OPENSSL

Overrides auto-detection of the OpenSSL package.

Possible values:

 

MCO_FORCE_PYTHON

Overrides auto-detection of the Python package.

Possible values:

 

HOST_COMPILER_FAMILY

TARGET_COMPILER_FAMILY

Overrides the auto-detected host/target compiler family. The auto-detection is based on the host/target platform OS.

Example:

 
    TARGET_COMPILER_FAMILY=gcc
     

 

HOST_ARCH

TARGET_ARCH

Overrides the auto-detection of the host/target hardware architecture. The auto-detection is based on the host/target compilers information.

Example:

 
    TARGET_ARCH=aarch64
     

 

HOST_DBUS_WIDTH

TARGET_DBUS_WIDTH

Overrides auto-detection of the host/target databus width. The auto-detection is based on the host/target compilers and architecture information.

Possible values are 16, 32 or 64.

Example:

 
    TARGET_DBUS_WIDTH=32
     

HOST_FLAGS

TARGET_FLAGS

Unconditionally adds arguments to the compiler/linker command line for HOST / TARGET code compilation. (Arguments are added exactly as they appear - no checks are applied.)

Example:

 
    TARGET_FLAGS="--sysroot=/opt/toolchains/arm-unknown-linux/"
     

 

Test Suite makefile.test compatibility notes

The build system includes compatibility support for the Test Suite run.py script. Keep in mind that the makefile.test files are not good for parallel processing. Do not try to use make -j for them.

There are two cases of possible usage of the build system from run.py:

Method 1. Run makefile.test directly as a project makefile.

Example of the makefile:

 
    MCO_ROOT=$(abspath ../../..)
    include $(MCO_ROOT)/include/settings.makefile
    TEST_TAGS        = disk in-memory callback threads
    TEST_COMMENT     = Allocation callback test. It checks callback 
        notification about exceded in-mamory or disk database space
    TEST_TIMEOUT     = 300
    MCO_ROOT        ?= ../../..
    PRJ_TYPE         = APP
    PRJ_SHOW_BUILD   = YES
    PRJ_SIDE         = TARGET
    PRJ_SOURCES      = main.c
    PRJ_SCHEMAS      = schema.mco
    PRJ_INTERFACES   =
    PRJ_DBNAMES      = allocclbkdb
    PRJ_STORAGE     ?= MEMORY
    PRJ_MEMDEV      ?= CONV
    PRJ_STORAGEDEV   = AUTO
    PRJ_SYNC         = AUTO
    PRJ_TRANSMGR    ?= MURSIW
    PRJ_DIRECTPTR   ?= NO
    PRJ_HA          ?= NONE
    PRJ_RPC          = NONE
    PRJ_F_CPP        = NO
    PRJ_F_SQL        = NO
    PRJ_F_LOG        = NO
    PRJ_F_DBCALC     = NO
    PRJ_F_UDA        = NO
    PRJ_F_XML        = NO
    PRJ_F_SERI       = NO
    PRJ_F_HV         = NO
    PRJ_F_CLUSTER    = NO
    PRJ_RMFILES      =
    PRJ_TITLE        = allocclbk_$(PRJ_SUFFIX)
    THIS_MAKEFILE := $(lastword $(MAKEFILE_LIST))
    # include $(MCO_ROOT)/include/header.mak
    include $(MCO_ROOT)/include/rules.makefile
     

The build system detects this kind of makefile.test files by the presence of PTR_TYPE = APP and TEST_NAME. For this kind of makefile.test the build system creates a virtual project makefile using PRJ_TITLE as the project name (<project-name> mentioned above) and sets all of the build properties as PRJ_$(PRJ_TITLE)_XXXX where XXXX is the property name. Project tags come from the TEST_TAGS variable. The eXtremeDB runtime is always enabled. The final executable name is set as the value of PRJ_TITLE because the value includes PRJ_SUFFIX already. PRJ_DIRECTPTR controls the inclusion into the PRJ_TARGET_EXECUTABLE_-list. All other build properties are set according to the rules of the old build system.

 

Method 2. Run the normal build systems makefile from under makefile.test control.

Example of the makefile.test:

 
    TEST_NAME        = 08-benchmarks-perf-conv
    TEST_TAGS        = samples fast perf
    TEST_COMMENT     = 08-benchmarks perf sample
    TEST_TIMEOUT     = 300
    TEST_STORAGE     = MEMORY
     
    MCO_ROOT        ?= ../../../../..
     
    all:
        $(MAKE) -C . $(MAKEFLAGS) $(MAKECMDGOALS) PRJ_TITLE=$(TEST_NAME)_$(PRJ_SUFFIX)
 
    clean:
        $(MAKE) -C . $(MAKEFLAGS) $(MAKECMDGOALS)
         

The build system detects this kind of makefile.test by the presence of the PRJ_TYPE and TEST_NAME variables. As the makefile.test actually runs a typical makefile there is no need to create a virtual project. All the build properties come from the makefile and the script cares about the PRJ_STORAGE, PRJ_MEMDEV, PRJ_TRANSMGR, PRJ_DIRECTPTR, PRJ_SUFFIX and TEST_TAGS overrides only. PRJ_DIRECTPTR controls inclusion into the PRJ_TARGET_EXECUTABLE_-lists. Project tags comes from TEST_TAGS. The final executable name PRJ_<project-name>_RESULT_NAME_BASE is constructed from <project-name>_$(PRJ_SUFFIX). The values of the PRJ_STORAGE, PRJ_MEMDEV, PRJ_TRANSMGR controls override normal values of the project build settings PRJ_<project-name>_STORAGE, PRJ_<project-name>_F_MCOCOMP_FORCE, PRJ_<project-name>_STORAGEDEV, PRJ_<project-name>_MEMDEV, PRJ_<project-name>_TRANSMGS and PRJ_<project-name>_F_SEQUENCES according to the rules described above (see section Common project flags above).