Philosophy

The ports collection is maintained according to a set of principles designed to uphold quality, keep the ports maintainable, and to enable giving back improvements to the upstream projects. Ports are usually named after their primary program or library if relevant. Libraries should always be prefixed with lib. The port should be ideally split if it happens to contain both a major program and a major library. Upstream projects should consider this operating system to be yet another unknown operating system implementing the standard interfaces. Explicit support for this operating system should generally not be upstreamed and should instead be maintained in the patches. This policy is to avoid burdening upstreams with maintaining support that is subject to change. Simply registering the operating system as existing is exempted, but preferably the software should be patched to compile for any unknown operating system that has the needed interfaces. Patches should ideally be of upstreamable quality, although it may not be reasonably practical to make a general solution the upstream could accept. Non-trivial patches should be prefaced with a comment explaining the rationale for the patch. Third party software should be selected into the ports collection with care, ideally because they're widely considered high quality and stable. Low quality software with poor code quality should preferably not be ported, instead a better replacement should be standardized on. The build system is expected to follow the relevant conventions, such as the GNU coding conventions for ./configure and Makefile scripts and the de-facto conventional behavior. Deficiencies are fixed via patches instead of being worked around. Ports with their own custom build system implementation should be improved if they don't implement the interface correctly. Generated files are preferably patched instead of their input files due to the difficulty regenerating them. For instance configure should be patched directly instead of configure.ac and likewise Makefile.in instead of Makefile.am. Ports must not bundle their dependencies as it's problematic to silently have multiple stale versions of the same library. If a port bundles a dependency unless it's installed, then the dependency must be made mandatory. Ports must both compile natively and cross-compile. Ports must assume the best about the host operating system when cross-compiling and unable to test for bugs. It must be possible to reasonably boostrap the latest ports collection from the latest stable release.

Metadata

A port named example can be built along with the operating system by creating the /src/ports/example/example.port file with the meta information documented in port(5). Start the port by defining how to get the latest stable version of the upstream release:

NAME=example 
BUILD_LIBRARIES='libfoo libbar? libqux?' 
VERSION=1.2.3 
DISTNAME=$NAME-$VERSION 
COMPRESSION=tar.gz 
ARCHIVE=$DISTNAME.$COMPRESSION 
SHA256SUM=b8d67e37894726413f0d180b2c5a8369b02d4a2961bb50d2a8ab6f553f430976 
UPSTREAM_SITE=https://example.com/pub/example 
UPSTREAM_ARCHIVE=$ARCHIVE 
BUILD_SYSTEM=configure 
LICENSE=example 
DEVELOPMENT=true

The upstream website usually contains the needed information. Guides such as Beyond Linux from Scratch and the packaging metadata for other systems may contain useful information on how to port and patch the software. UPSTREAM_SITE is the upstream primary download directory without a trailing slash and should download via a secure connection insofar possible. SHA256SUM is the sha256sum(1) of the upstream release, which should be verified with other reputable packaging systems. It can be left as the wrong value, and the download will fail and mention which hash it found instead. BUILD_LIBRARIES is the build time library dependencies (space delimited with a ‘?’ suffix for optional dependencies). It can be left empty until known. The dependencies can likely be found in the documentation, running ‘./configure --help’, or noticing what the software looks for while being built. The above example defines a mandatory build time dependency on libfoo and optional build time dependencies on libbar and libqux. Mandatory dependencies need to be ported first. BUILD_SYSTEM is set here to a standard configure script. LICENSE can be set to the abbreviation of the software's license. DEVELOPMENT is set to true to switch the port into development mode, where the port's working directory has writable files that can be patched during the build. The example.patch, example.execpatch, and example.rmpatch patch files are regenerated when the port builds successfully. If the upstream release works out of the box without any patches, then the port is already completed and the development mode can be skipped. The port's working directory will never be deleted automatically when it is in development mode. The DEVELOPMENT variable must be removed before submitting the finished port.

Building

The port can be finished by repeatedly trying to build it and iteratively fixing any issues that occur per development(7):

cd /src 
make clean-sysroot 
make PACKAGES='example!'

The PACKAGES environment variable builds only the mentioned ports and the ‘!’ suffix includes the port's mandatory dependencies as well. The top level makefile uses a sysroot when building the ports which can be used to determine which dependencies are mandatory. The mandatory ports can be found by starting with an empty list of build libraries and a clean sysroot, and then trying to build the port and adding the dependencies whose absense is causing the build to fail. The higher level tix-port(8) program manages the build by downloading the source code, patching it, building the port, and installing the binary package. The lower level tix-build(8) program will abort with an interactive menu if the port fails to build. See the section below for how to troubleshoot common situations. The binary package repository/$HOST/example.tix.tar.xz is cached when built and the repository/$HOST/example.version file is used to rebuild the binary package when the version changes. The files should be manually removed to force a rebuild when the port has been modified while in the development mode.

rm -f repository/*/example.tix.tar.xz # delete stale binary package 
rm -f repository/*/example.version # optional 
make clean-repository # or: remove all binary packages

The port's patch files are regenerated whenever the port builds successfully. It's recommended to add them to a work-in-progress git commit when working to keep track of the progress.

Finishing

The port can be finished by testing it thoroughly and reviewing the details. Test the port works with all the optional dependencies:

make clean-sysroot 
make PACKAGES='example!!'

The ‘!!’ suffix includes the port's optional dependencies as well. Read all the configure and makefile output and look for anything where it looks like a wrong conclusion was made or an optional dependency was not used. Review the size of the binary package and whether it installs any large unnecessary files:

du -h repository/*/example.tix.tar.xz 
tar -t -f repository/*/example.tix.tar.xz

Review the .patch, .execpatch and .rmpatch files (if any) and clean them up as needed by updating the port's source code and forcing a rebuild to regenerate the patches. Consider installing a cross-compiler for a second architecture and testing the port works everywhere per cross-development(7). Comments about the port should go in comments at the end of the .port file using shell-style comments. Non-trivial patches and aspects should be documented this way. Finally remove the DEVELOPMENT variable and submit the port for review.

Upgrading ports

Ports can be upgraded by switching the port back in development mode with a updated version number and checksum.

make available-ports # Search for newly available versions of ports 
make upgrade-ports PACKAGES=example # Update example port 
make PACKAGES='example!!' # Find the new sha256sum 
# Verify the new sha256sum is authentic. 
make PACKAGES='example!!' # Any old patches may fail to apply. 
find ports/example -name '*.rej' -o -name '*.orig' 
make PACKAGES='example!!' # Regenerate patches on a successful build. 
sed -E '/^DEVELOPMENT=true$/d' ports/example/example.port 
make PACKAGES='example!!' # Final build and testing.

The available-ports top-level makefile target can be used to search for ports with newly available versions, while upgrade-ports updates VERSION to the latest available version and switches the port into development mode. The PACKAGES environment variable can be set to a subset of ports to upgrade. The build will fail and report the sha256sum observed upstream and the SHA256SUM variable must be manually updated once it has been verified as as authentic. The old patch will be reapplied and the build will fail if any hunks could not be applied. In that case, the .rej files contains the rejected hunks and the merge conflicts need to be resolved. Delete any .rej and .orig files afterwards and continue the build. The patch files with be regenerated on a successful build as usual. Finally perform the quality assurance and testing of new ports.

config.sub

The config.sub file parses the build/host/target triplet and is duplicated into all software using autoconf. If the port ships a version older than 2015-08-20, configure will fail to parse the operating system:

checking host system type... Invalid configuration `x86_64-sortix': system `sortix' not recognized

Fix the situation by adding an entry for the operating system to the config.sub file which is usually in the build-aux subdirectory:

             | -aos* | -aros* | -cloudabi* | -sortix* \

Configure

If the configure script fails, then autoconf configure scripts produce a config.log file which contain useful diagnostic information, such as which programs were compiled to check for a particular feature and what happened when they were compiled, linked, and executed. This information can be used to locate the relevant logic in the configure script.

Non-verbose make

Some ports default to a non-verbose make mode that doesn't show the commands being run. Ports should show the commands invoked by make, which can done with an environment variable assignment V=1 which can be made permanent using MAKE_VARS=V=1.

Patching

The build may fail or warn for various reasons which needs to be patched, or the port may need extra support for the operating system. The sources become writable once the port is in development mode and can simply be edited to fix the problem. The patch files are regenerated whenever the port builds successfully. Non-trivial patches should contain a “// PATCH:” comment explaining why the patch had to be made and make it clear whether the patch fixes a problem in the upstream release or is specific to the operating system. Patches may be read by many people, including upstream developers, and the context helps them understand if they want the patch too. Ports should ideally continue to work on good operating systems after being patched. If the port uses an obsolete function, the port should be patched to unconditionally use the modern replacement instead. If the patch is specific to the operating system, the patch should be guarded with the operating system preprocessor macro.

// PATCH: Use foo instead because bar doesn't work yet. 
#ifdef __sortix__ 
	foo(); 
#else 
	bar(); 
#endif

If the port tries to use a system header that might be added in the future and it doesn't have a macro for whether it exists, then __has_include extension can be used:

#if __has_include(<foo.h>) 
#include <foo.h> 
#endif

Configuration files

Configuration files in /etc belongs to the system administrator and must not be installed by ports, as local changes will otherwise be undone whenever the port is upgraded. Ports shipping default configuration files should instead install them in /etc/default and search this directory as a fallback if the system administrator has not made their own file. Example configuration files can be installed in /etc/example.

Portability issues

The port might not be portable and requires patching. Often the port uses a non-standard interface when a standard interface is available and should be used instead. Other times the operating system is missing functionality which should ideally be implemented before proceeding with the port, or perhaps the absence can be worked around. Common portability problems and their resolution are documented in portability(7).

libtool .la files

Libraries using libtool may install /lib/*.la files which are unnecessary and contain absolute paths which cause cross-compilation to fail. Ports should never install such files and instead rely on pkg-config(1) for locating dependencies. The tix-eradicate-libtool-la(8) program can be used to remove any installed .la files in the DESTDIR as a post-install script:

POST_INSTALL=tix-eradicate-libtool-la

Post-install command

Ports may install files with an unconventional directory layout or may install unnecessary large files. Ideally the makefile should be patched to install correctly in the first place, but if doing so is impractical, then a post-install command can be used to fix up the installation in the DESTDIR before the binary package is created. An executable example.post-install script can be placed next to the example.port files:

POST_INSTALL=../example.post-install

The script must fail if any errors occur and protect against being accidentally invoked without the appropriate environment variables being set. E.g.:

#!/bin/sh 
set -e 
[ -n "$TIX_INSTALL_DIR" ] 
mv "$TIX_INSTALL_DIR$PREFIX/share/foo" "$TIX_INSTALL_DIR$PREFIX/share/bar"

Splitting into multiple ports

Upstream releases might have multiple parts, such as a program that also comes with a library, which should be split into multiple ports whenever reasonable. The port whose name matches the upstream release should be the source package, and the other ports can use the SOURCE_PORT variable to reuse its source code. The SUBDIR variable can be used to build inside a subdirectory, if the port has already been logically organized. CONFIGURE_ARGS variable can be used to pass options to configure to request the subset of functionality. MAKE_BUILD_TARGET and MAKE_INSTALL_TARGET can be used to make a subset of the port.

Gnulib

Gnulib is a GNU portability layer that has three purposes:

• Providing fallback implementations of missing standard library interfaces.
• Replacing buggy implementations of standard library interfaces.
• Sharing common utility functions between projects.

Unfortunately the replacement of standard library functionality is problematic:

• The replacement implementations tend to be non-portable and one is supposed to modify the source code to rely on private standard library details that may be subject to change.
• Gnulib is highly forked by design and adding support for new operating systems needs to be done upstream and it may take years for the support to reach new downstream releases.
• Gnulib has assumed the worst in the past when cross-compiling, assuming unknown operating systems are buggy, injecting a non-portable replacement that doesn't compile, even when the standard library function is correct and could just have been used.
• Replacing standard library functions can hide bugs that would otherwise have found and fixed.
• Gnulib is not organized into the three categories and it's non-trivial to find out whether any interface has been replaced that shouldn't have been.
• There is no way to satisfy gnulib by correctly implementing the standard library without contributing explicit support upstream for new systems and committing to private implementation details.

Meanwhile the shared utility functions means gnulib is tightly integrated into the software and cannot be disabled. Consequently a lot of software using gnulib does not cross-compile to new operating systems and the compilation fails asking for standard library implementation details. Gnulib can be effectively disabled by copying the autoconf cache environment variables set at the top of the patched configure script in an existing port with gnulib. Each variable is used to say the standard library does not have a particular bug. Modern versions of gnulib has improved to assume the best when cross-compiling but it may still be an obstacle for some ports.

Custom configure script

Configure scripts generated by autoconf are useful because they have a consistent interface. Ports with a hand-written configure script can fail if they fail to implement the autoconf configure interface correctly. The best solution is modify the configure script to implement the missing parts of the interface. Custom configure scripts sometimes fail to implement the --prefix and --exec-prefix options correctly, failing to discern between an unset option and the empty value, which matters for the prefix where the default prefix (/usr/local) is different from the empty prefix (the root directory). Custom configure scripts sometimes fail to implement cross-compilation using the --build, --host, and --target options to locate the compiler and cross-compiler.

Makefile

Ports might not have a configure script and only a makefile:

BUILD_SYSTEM=makefile

The build environment is communicated to the makefile using conventional environment variables, however loose makefiles vary wildly and likely needs to be patched to support the variables used by tix-build(8). In some cases, it may be preferable to write a new makefile from scratch.

Running cross-compiled program

Ports may contain logic errors and attempt to execute a cross-compiled program during the build, which fails because the program can't run on the current system. If the program is supposed to check whether a feature works at runtime, then the right solution is to simply assume it's correct when cross-compiling or to have a suitable runtime fallback. If the program is part of the compilation process, then it should be compiled with the build machine's compiler instead, which can be located using the CC_FOR_BUILD environment variable instead of CC before falling back to a standard program. Likewise each toolchain variable has a counterpart for the build machine instead of the host machine, such as CFLAGS_FOR_BUILD and CPPFLAGS_FOR_BUILD.

pkg-config

Ports are supposed to locate locate dependencies for the host machine using the PKG_CONFIG environment variable, and if unset, then using the --host option to locate a cross-pkg-config, and finally falling back on invoking pkg-config(1) directly . Dependencies for the build machine should likewise be located using the PKG_CONFIG_FOR_BUILD environment variable. Ports failing to use this search order may fail to cross-compile as they accidentally use dependencies from the build machine when cross-compiling to the host machine. The easiest fix is to use a shell parameter expansion:

${PKG_CONFIG:-pkg-config} 
${PKG_CONFIG_FOR_BUILD:-${PKG_CONFIG:-pkg-config}}

foo-config instead of pkg-config

Ports are supposed to use pkg-config(1) as above for dependencies as it's a general solution with cross-compilation support, but some ports install their own foo-config program in the PATH. These programs are inherently unable to support cross-compilation, as they provide answers about the build machine, and the host machine's bin directory cannot be executed on the current machine. Ports installing foo-config programs must be patched to not install them. pkg-config(1) configuration files should be installed instead. Ports invoking foo-config programs, even as a fallback, must be patched to unconditionally use pkg-config(1) instead.

Bootstrap

Ports may require the same version of the port to be installed on the build machine when cross-compiling. Such ports can cross-compiled with a bootstrap phase:

USE_BOOTSTRAP=true

This configuration builds the port in two phases, first building it for the build machine and installing it into a temporary directory, which is in the PATH while the port is cross-compiled during the second phase.

DESTDIR

The makefile install target must support the DESTDIR environment variable as a secondary temporary prefix during the installation. The makefile may need to be patched to inherit the variable from the environment and to use it during the installation phase. The build will erroneously attempt to install onto the root directory of the build machine if the environment variable isn't respected.

make distclean

The upstream release is to supposed to be distclean, i.e. not contain any files that are recreated during the compilation, and the distclean makefile target is supposed to properly clean up. If it isn't, then the patch may contain spurious hunks adding, removing, or modifying generated files which may be large. The makefile distclean target should be patched to delete any temporary files produced during the build to avoid spurious files in the patch.

Dynamic linking

Dynamic linking is not currently implemented in the standard library and the compiler toolchain does not support dynamic linking. Ports with libraries should check whether the toolchain supports dynamic linking and otherwise fall back on static linking. Ports may be difficult if they use shared libraries for modules. It may be possible to link the modules statically instead using a supported mechanism or with additional patching. If there is no simple solution, it may be possible to statically link the modules and implement a fake dlopen that iterates a table of entry points for each module.

Other problems

Portability issues are very varied and ports often don't properly implement the conventional interface. portability(7) lists common differences in the standard library. port(5) documents the advanced features useful for certain situations. Ports may fail at runtime instead of during compilation. Resolving these issues can require troubleshooting, debugging, research, and seeking help from the operating system developers and the upstream. Missing operating system functionality may need to be implemented.