EasyBuild¶

NHR@KIT uses EasyBuild to manage the software stack on HoreKa 2. EasyBuild lets the HPC operations team provide a broad range of software consistently across platforms, and respond quickly when users request additional packages. For most use cases, the modules provided by the HPC operations team are all you need. See Software Modules for how to find and load available software.

EasyBuild can also be used by users themselves to build and maintain their own private software stack on top of the system-provided modules. This page gives a quick introduction for users who want to do exactly that.

Concepts¶

Toolchain¶

A toolchain is a collection of modules used together to build software: typically one compiler suite, zero or one MPI implementation, and one or more libraries (e.g. math libraries).

Toolchains can be hierarchical — for example, foss is an extension of gompi. The name is usually a combination of the starting letters of the included components. A complete list is available on the EasyBuild website.

Toolchains are released once or twice a year. Versions were traditionally named <year>a / <year>b; the EasyBuild community is now switching to a dot-number scheme (e.g. 2026.1). HoreKa 2 offers GCC and Intel compiler toolchains in multiple versions for x86_64, and all installed toolchains remain available for the lifetime of the system.

There is a special system toolchain that uses tools from the underlying OS. It is used for basic low-level modules and should not be used for scientific software.

EasyBlock¶

EasyBlocks describe how software is built at a lower level. They abstract the complexity of the build process so that EasyConfig files only need to set a few variables and dependencies. Generic EasyBlocks like ConfigureMake, CMakeMake and SCons cover most build systems; more complex application-specific EasyBlocks also exist. In most cases it is not necessary to write your own EasyBlock.

EasyConfig files (.eb)¶

EasyConfig files describe the build process in plain text. Each file:

• uses an EasyBlock (if none is specified, EasyBuild looks for one matching the software name — see example)
• uses exactly one toolchain; since toolchains are hierarchical, use the one with the fewest dependencies
• contains metadata: name, version, homepage, description
• specifies source URLs, optional local patches, and optional checksums (verified before the build starts)
• lists build and runtime dependencies
• can set variables to influence any build step, e.g. preconfigopts = 'autoreconf && '
• can skip steps, e.g. skipsteps = ['configure', 'check']

Example using a generic EasyBlock¶

easyblock = 'ConfigureMake'
name = 'zlib'
version = '1.3.1'
homepage = 'https://www.zlib.net/'
description = """zlib is designed to be a free, general-purpose, legally unencumbered
        -- that is, not covered by any patents -- lossless data-compression
        library for use on virtually any computer hardware and operating system."""
toolchain = {'name': 'GCCcore', 'version': '14.3.0'}
toolchainopts = {'pic': True}
source_urls = ['https://zlib.net/fossils']
sources = [SOURCELOWER_TAR_GZ]
checksums = ['9a93b2b7dfdac77ceba5a558a580e74667dd6fede4585b91eefb60f03b72df23']
# use same binutils version that was used when building GCC toolchain
builddependencies = [('binutils', '2.44', '', SYSTEM)]
sanity_check_paths = {
  'files': ['include/zconf.h', 'include/zlib.h',
      'lib/libz.a', 'lib/libz.%s' % SHLIB_EXT],
}
moduleclass = 'lib'

name = 'bzip2'
version = '1.0.8'
homepage = 'https://sourceware.org/bzip2'
description = """
bzip2 is a freely available, patent free, high-quality data compressor. It
typically compresses files to within 10% to 15% of the best available
techniques (the PPM family of statistical compressors), whilst being around
twice as fast at compression and six times faster at decompression.
"""
toolchain = {'name': 'GCCcore', 'version': '14.3.0'}
toolchainopts = {'pic': True}
source_urls = ['https://sourceware.org/pub/%(name)s/']
sources = [SOURCE_TAR_GZ]
patches = ['bzip2-%(version)s-pkgconfig.patch']
checksums = [
'ab5a03176ee106d3f0fa90e381da478ddae405918153cca248e682cd0c4a2269', # bzip2-1.0.8.tar.gz
'9299e8ee4d014ea973777b6ea90661fe329dfa991f822add4c763ea9ddb9aab1', # bzip2-1.0.8-pkgconfig.patch
]
builddependencies = [
  ('binutils', '2.44'),
]
moduleclass = 'tools'

Build steps¶

EasyBuild executes the following steps in order: fetch, ready, extract, patch, prepare, configure, build, test, install, extensions, postiter, postproc, sanitycheck, cleanup, module, permissions, package, testcases. Every step can be influenced via the EasyConfig or EasyBlock. In most cases this is not necessary — but defining a sanity check is always a good idea.

How EasyBuild handles dependencies¶

EasyBuild is implemented in Python and uses .eb files to describe builds — in contrast to Spack, which uses pure Python. EasyBuild aims to be as independent of the underlying OS as possible, so many low-level packages are bundled as modules. As a result, loading a module will automatically load all its lower-level dependencies, and module list output can be quite long.

$ module load foss/2025b
$ module list

Currently Loaded Modules:
  1) GCCcore/14.3.0                      13) libfabric/2.1.0-GCCcore-14.3.0
  2) zlib/1.3.1-GCCcore-14.3.0           14) PMIx/5.0.8-GCCcore-14.3.0
  3) binutils/2.44-GCCcore-14.3.0        15) PRRTE/3.0.11-GCCcore-14.3.0
  4) GCC/14.3.0                          16) UCC/1.4.4-GCCcore-14.3.0
  5) numactl/2.0.19-GCCcore-14.3.0       17) OpenMPI/5.0.8-GCC-14.3.0
  6) XZ/5.8.1-GCCcore-14.3.0             18) OpenBLAS/0.3.30-GCC-14.3.0
  7) libxml2/2.14.3-GCCcore-14.3.0       19) FlexiBLAS/3.4.5-GCC-14.3.0
  8) libpciaccess/0.18.1-GCCcore-14.3.0  20) FFTW/3.3.10-GCC-14.3.0
  9) hwloc/2.12.1-GCCcore-14.3.0         21) gompi/2025b
 10) OpenSSL/3                           22) FFTW.MPI/3.3.10-gompi-2025b
 11) libevent/2.1.12-GCCcore-14.3.0      23) ScaLAPACK/2.2.2-gompi-2025b-fb
 12) UCX/1.19.0-GCCcore-14.3.0           24) foss/2025b

Build individual EasyBuild packages as a user¶

The HoreKa 2 EasyBuild module tree can be used as a base to build additional software in your $HOME or a workspace directory.

For the following example we assume:

• The software will be installed in your $HOME directory in the sub-directory eb
• You have already cloned the easyconfigs github repository to ~/eb/easybuild-easyconfigs

EasyBuild configuration file¶

The configuration file must be placed in your home at ~/.config/easybuild/config.cfg with the following content:

[basic]
robot=<path_to_your_home>/eb/easybuild-easyconfigs/easybuild/easyconfigs
[config]
modules-tool: Lmod
prefix=<path_to_your_home>/eb
buildpath=/scratch/build
[override]
allow-modules-tool-mismatch=True
# optional — uncomment and adjust as needed:
# group-writable-installdir=True
# cuda-compute-capabilities=8.0,9.0
# amdgcn-capabilities=gfx90a
# accept-eula-for=CUDA,cuDNN

Key options:

• robot: tell EasyBuild where to look for EasyConfig files and build all dependencies automatically
• modules-tool: define what flavour of modules to use when creating the module files — use Lmod for Lua modules
• prefix: root path where software and module files will be installed
• buildpath: where to build the software; using /scratch avoids load on the parallel file system
• group-writable-installdir: make the whole group have write permissions on the installed files
• cuda-compute-capabilities: CUDA compute capabilities to target — see the mapping here
• amdgcn-capabilities: AMD GPU targets — see the mapping here
• accept-eula-for: pre-accept EULAs for specific packages, enabling unattended installs

This is only a subset of options. For more details check the documentation.

Add local/private module path¶

mkdir -p ~/eb/modules/all
module use ~/eb/modules/all
module load EasyBuild

Find or create an EasyConfig file¶

Search your easybuild-easyconfigs clone for a matching .eb file:

ls ~/eb/easybuild-easyconfigs/easybuild/easyconfigs/*/* | grep eb$

If there is no match, check open pull requests. If there is still no match, adapt an existing .eb file or create one from scratch — see Create an EasyConfig file from scratch.

Build your software with EasyBuild¶

module load EasyBuild
eb cowsay-3.04.eb

Use your software¶

module load cowsay
cowsay "Hello EasyBuild"
_________________
< Hello EasyBuild >
-----------------
        \   ^__^
        \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||

Provide an updated version of software already in EasyBuild¶

The version you are looking for may be included in the git repo already. If not, check if there is a version in one of the pull requests.

Building software from pull requests¶

EasyBuild provides a convenient way to build software directly from a pull request, by specifying the pull request number, e.g.

eb --from-pr=23304

eb --from-pr=26394,25509,25657,26027,26028,26025,26108 --ignore-test-failure

eb BLIS/BLIS-2.0-GCC-15.2.0.eb --include-easyblocks-from-pr=4034

Updating EasyConfigs manually¶

If an EasyBuild file for a new version is not available yet, start with updating an older .eb file and update version, toolchain and dependencies. Finding the correct versions of all the dependencies can be a lengthy process. EasyBuild provides a number of options that may help with updating (use with care — they modify the build spec automatically):

If you are building the latest version for the latest toolchain you may need to create EasyConfig files for most/some/all dependencies as well, since they may not be available for the latest toolchain version yet.

If modules with many sub-packages/modules are involved (e.g. Python, Rust or R bundle), this can be rather a lot of work.

Make sure to update the checksums using eb --inject-checksums.

Finally, run eb with the new EasyConfig and fix any problem.

Create an EasyConfig file from scratch¶

1. Identify the build system the software uses (e.g. autotools, CMake).
2. Find a matching generic EasyBlock — e.g. ConfigureMake for autotools, CMakeMake for CMake.
3. Find an existing EasyConfig using the same EasyBlock as a starting point.
4. Update: name, version, toolchain, homepage, description, sources, (build) dependencies, and module class.
5. Build with eb and fix any problems iteratively.

Install software with GPU support¶

For CUDA-based software, it is usually sufficient to add CUDA as a dependency. Set cuda-compute-capabilities and/or amdgcn-capabilities in your config file — the EasyBlock will then handle the rest. The AMD (amdgcn) integration may still require some manual compiler option tuning.

For testing, both options can also be passed on the command line:

eb --cuda-compute-capabilities=10.0 CUDA_Blackwell_software.eb

Some packages only build for a single CUDA compute capability — specify the matching option in that case.

Known problems¶

Test steps may hang or take a long time¶

Some software packages have extended test suites that take a long time or hang. Use --skip-test-step to skip the test step entirely.

For packages with known intermittent test failures, use --ignore-test-failure — the test step still runs, but failures are ignored and the build completes successfully.

Patching software¶

If a build fails — especially for a new toolchain — it may be necessary to provide a patch. EasyBuild uses standard unified diff patches (e.g. generated with diff -uNr). Add them to the EasyConfig file in the patches section:

patches = [
    ('OpenMPI-5.0.7_build-with-internal-cuda-header.patch', 1),
    'OpenMPI-5.0.7_fix_gpfs_compatibility.patch',
]

Sources are not publicly available¶

Some software packages (e.g. commercial software) are not provided as an online download. The sources need to be made available for EasyBuild to access — place them in the current directory or the matching /software/easybuild/source directory.

Software is not/no longer compatible with the compiler in the target toolchain¶

Some versions of software require an older or newer compiler than is available in your preferred toolchain. CDO is one example: its latest versions only build against compilers supporting recent C++ standards, and Intel compilers are known to implement only subsets of new standards and may fail. Check the required compiler version for the software before choosing a toolchain.

Intel compiler won't compile on AMD¶

At compile time the Intel compiler checks the local processor for the best optimisation flags. On an AMD processor it will print something like:

Please verify that both the operating system and the processor support Intel(R) X87,
CMOV, MMX, SSE, SSE2, SSE3, SSSE3, SSE4_1, SSE4_2, MOVBE,   POPCNT, AVX, F16C, FMA,
BMI, LZCNT, AVX2, AVX512F, AVX512DQ, ADX, AVX512CD,   AVX512BW, AVX512VL, AVX512VBMI,
AVX512_VPOPCNTDQ, AVX512_BITALG, AVX512_VBMI2, AVX512_VNNI and SHSTK instructions.

eb HDF5-1.14.5-iimpi-2024a.eb --optarch=-march=skylake-avx512

Since most HoreKa 2 nodes are not Intel-based, --optarch is required whenever building with the Intel compiler.

Advanced topics¶

Use Slurm to build software¶

EasyBuild's --job option submits builds as Slurm jobs. Each dependency not yet installed gets its own job, allowing many packages to build in parallel. If a package requires a GPU to build, the job must target a GPU partition.

Set the following environment variables to enable Slurm as the job backend:

export EASYBUILD_JOB_BACKEND='Slurm'
export SBATCH_PARTITION='gpu-h100'

Then submit a build like this:

eb VTK-9.5.2-foss-2025b.eb --job --job-cores 10 --job-max-walltime 8 --trace

More details regarding jobs can be found in the EasyBuild job documentation or in the EESSI workshop material.

Build a complete software tree¶

EasyBuild provides the concept of EasyStack. An EasyStack describes a collection of software packages. These EasyStacks can be installed with one command. After calculating the dependencies, EasyBuild will build all the software specified in the EasyStack in the correct order. Naturally, this will take a long time. If one of the software packages fails to build, only the remaining packages, which do not depend on the broken one will be built.

EasyStack files are yaml files, where all the EasyConfig files are listed. Each entry can be customized to add local options or optimizations, see easystack files documentation for details.

The file hk2-easystack.yaml looks like this:

easyconfigs:
  - ANTLR-2.7.7-GCCcore-13.3.0.eb
  - ATK-2.38.0-GCCcore-13.2.0.eb
  - Autoconf-2.71-GCCcore-13.2.0.eb
  - Autoconf-2.72-GCCcore-13.3.0.eb
  ...
  - openfoam-v2206-foss-2022a.eb:
    options:
      parallel: "1"
...

To build an EasyStack use the --easystack option like this:

eb --easystack hk2-easystack.yaml

EasyStacks can be used to describe a complete software tree for a cluster. EasyStack can also be combined with a batch system. This makes the rebuild of a complete software tree convenient and fast Of course, not all of the software will build on the first try. Re-running eb --easystack .. will only build the remaining software packages.

Further Information¶

• EasyBuild documentation
• Official EasyBuild tutorial
• Lumi tutorials