bisdn / bisdn-linux

BISDN Linux build system

This repository holds the main repo manifest for setting up the build system for BISDN Linux, based on Yocto.

The build process takes around 3-4 hours with 8 CPU cores and 8 GiB RAM. A single build requires ~70 GiB of disk space. Adding additional CPU cores speeds up the build time significantly.

General requirements

• Disk space and RAM

We recommend a minimum of 8 GiB RAM and 150 GiB free disk space for building a single full image.

• A functional Yocto build environment

This can be achieved by either installing and configuring everything as documented in the section Installing your own build environment, or by starting from the prebuilt Yocto environment in a docker container, as described below.

Using the docker-yocto-builder

To ease the process of "getting started building BISDN Linux with Yocto", we have compiled all dependencies needed by Yocto itself and BISDN Linux specifically, into a docker container. This container can be pulled directly from the GitHub container registry ghcr.io here. Please make sure you pull the 'ci' version of the container linked above (or switch to the user 'builder' in the 'base' version). To continue with the steps documented in the section Bootstrap build system you have to directly exec into the container, while ideally also mounting a local path on your file system (e.g. a new folder in your local /tmp) to the default TMPDIR directly inside of the container. To ensure that this /tmp directory will be writable by the builder user inside of the container, please make sure that you either have the same UID (1000) as the builder user inside of the container, or create the folder beforehand with more permissive access rights. This can be achieved by running:

# create a folder to be mounted as /tmp inside of the container with very
# permissive access rights
mkdir /tmp/builder-tmp && chmod 0777 /tmp/builder-tmp
# start the docker container and mount the folder previously created as /tmp
docker run -ti -v /tmp/builder-tmp:/tmp ghcr.io/bisdn/docker-yocto-builder-ci:main bash

If you are planning to overwrite the TMPDIR in the conf/local.conf (TMPDIR = "/tmp/${MACHINE}" by default) as mentioned in the section Configure target machine and cache directory, you should also consider to mount your local file system path to this new path inside of your container.

After you have successfully exec'd into the container, you are ready to bootstrap the build system.

Installing your own build environment

• repo tool

Some distros include repo, so you might be able to install from there (Ubuntu 20.04 however does not provide those packages).

# Debian/Ubuntu.
sudo apt-get install repo

# Gentoo.
sudo emerge dev-vcs/repo

You can install it manually as well as it's a single script.

# install
mkdir ~/bin
PATH=~/bin:$PATH
curl https://storage.googleapis.com/git-repo-downloads/repo > ~/bin/repo
chmod a+x ~/bin/repo

The repo tool requires an executable named python in your path and building open network linux is currently still depending on python2. To suffice this dependency, we recommend installing python2 with any of the default methods fitting for your environment. On Ubuntu 20.04 you can install python2 directly via apt.

sudo apt-get install python

See the official documentation for further details.

• install your OS specific build essentials for Yocto

Some Yocto packages may require additional utilities to be present on the build host.

• BISDN-Linux specific build requirements

Additionally libelf with development headers and PyYAML are needed. On Ubuntu (20.04 and newer) this can be installed by running.

sudo apt-get install libelf-dev python-yaml

For information on installing these requirements on a different OS please visit the respective project page.

Bootstrap build system

The commands needed to bootstrap and initialise your build system are shown below. Before starting to copy paste those, please make sure to set $BRANCHNAME in your environment to an existing branch in the bisdn/bisdn-linux repo from which you want to start your build (e.g. export BRANCHNAME=v4.8.0 or export BRANCHNAME=master)

# init repo
mkdir -p ~/workspace/poky-bisdn-linux
cd !$
repo init -b ${BRANCHNAME} -u https://github.com/bisdn/bisdn-linux.git

# sync repos
repo sync

# init build system
. poky/oe-init-build-env poky/build-bisdn-linux/

Configure target machine and cache directory

Edit conf/local.conf and set MACHINE to your desired target. Cached files and built artifacts will be stored in /tmp by default. Change the variables SSTATE_DIR, TMPDIR, DL_DIR if you wish to use a different location. See the Yocto Project Reference Manual: Variables Glossary for a complete overview of variable definitions.

Build image

Chose one of the available image types to build

• minimal: bare minimum of packages for booting the system. Does not include any closed source packages outside of required firmware files.

• full: include the full BISDN Linux system, including baseboxd and OF-DPA.

NOTE: When trying to build a release prior to 4.6.1, please read the Known issues when building older versions section before starting.

# build the yocto artifacts
bitbake <minimal|full>

The finished image can be found at ${TMPDIR}/deploy/images/${MACHINE}/onie-bisdn-full-${MACHINE}.bin.

If you used the docker-yocto-builder container as build environment, you should be able to also find the image in the output path you directly mounted into the container (e.g. /tmp/builder-tmp/deploy/images/${MACHINE}/onie-bisdn-full-${MACHINE}.bin on your host system).

Install image

Please refer to our BISDN Linux docs on how to install the resulting image.

Known issues when building older versions

• The fix for git CVE-2022-24765 directly affects the build of BISDN Linux prior to version 4.6.1, and will cause it to fail with an error message like:

  ERROR: python3-oslo.i18n-3.17.0+gitAUTOINC+f2729cd36f-r0 do_install: 'python3 setup.py install --root=/tmp/accton-as4610/work/cortexa9-vfp-poky-linux-gnueabi/python3-oslo.i18n/3.17.0+gitAUTOINC+f2729cd36f-r0/image     --prefix=/usr     --install-lib=/usr/lib/python3.8/site-packages     --install-data=/usr/share' execution failed.
  ERROR: python3-oslo.i18n-3.17.0+gitAUTOINC+f2729cd36f-r0 do_install: Execution of '/tmp/accton-as4610/work/cortexa9-vfp-poky-linux-gnueabi/python3-oslo.i18n/3.17.0+gitAUTOINC+f2729cd36f-r0/temp/run.do_install.2518342' failed with exit code 1
  ERROR: Logfile of failure stored in: /tmp/accton-as4610/work/cortexa9-vfp-poky-linux-gnueabi/python3-oslo.i18n/3.17.0+gitAUTOINC+f2729cd36f-r0/temp/log.do_install.2518342
  Log data follows:
  | DEBUG: Executing python function extend_recipe_sysroot
  | NOTE: Direct dependencies are ['virtual:native:/home/ubuntu/workspace/poky-bisdn-linux/poky/meta/recipes-devtools/python/python3-pbr_5.4.4.bb:do_populate_sysroot', '/home/ubuntu/workspace/poky-bisdn-linux/poky/meta/recipes-core/glibc/glibc_2.31.bb:do_populate_sysroot', '/home/ubuntu/workspace/poky-bisdn-linux/poky/meta/recipes-devtools/python/python3_3.8.13.bb:do_populate_sysroot', '/home/ubuntu/workspace/poky-bisdn-linux/poky/meta/recipes-devtools/quilt/quilt-native_0.66.bb:do_populate_sysroot', '/home/ubuntu/workspace/poky-bisdn-linux/poky/meta/recipes-devtools/python/python3-pbr_5.4.4.bb:do_populate_sysroot', '/home/ubuntu/workspace/poky-bisdn-linux/poky/meta/recipes-devtools/python/python3-pip_20.0.2.bb:do_populate_sysroot', 'virtual:native:/home/ubuntu/workspace/poky-bisdn-linux/poky/meta/recipes-devtools/pseudo/pseudo_git.bb:do_populate_sysroot', '/home/ubuntu/workspace/poky-bisdn-linux/poky/meta/recipes-devtools/gcc/gcc-cross_9.3.bb:do_populate_sysroot', 'virtual:native:/home/ubuntu/workspace/poky-bisdn-linux/poky/meta/recipes-devtools/python/python3_3.8.13.bb:do_populate_sysroot', '/home/ubuntu/workspace/poky-bisdn-linux/poky/meta/recipes-devtools/gcc/gcc-runtime_9.3.bb:do_populate_sysroot', 'virtual:native:/home/ubuntu/workspace/poky-bisdn-linux/poky/meta/recipes-devtools/patch/patch_2.7.6.bb:do_populate_sysroot', 'virtual:native:/home/ubuntu/workspace/poky-bisdn-linux/poky/meta/recipes-devtools/python/python3-setuptools_45.2.0.bb:do_populate_sysroot']
  | NOTE: Installed into sysroot: []
  | NOTE: Skipping as already exists in sysroot: ['python3-pbr-native', 'glibc', 'python3', 'quilt-native', 'python3-pbr', 'python3-pip', 'pseudo-native', 'gcc-cross-arm', 'python3-native', 'gcc-runtime', 'patch-native', 'python3-setuptools-native', 'python3-pip-native', 'linux-libc-headers', 'xz', 'libtirpc', 'libxcrypt', 'autoconf-archive', 'gdbm', 'openssl', 'libnsl2', 'readline', 'bzip2', 'opkg-utils', 'sqlite3', 'zlib', 'libffi', 'util-linux', 'libmpc-native', 'binutils-cross-arm', 'autoconf-native', 'zlib-native', 'flex-native', 'automake-native', 'libtool-native', 'gnu-config-native', 'gmp-native', 'texinfo-dummy-native', 'xz-native', 'mpfr-native', 'openssl-native', 'bzip2-native', 'sqlite3-native', 'libffi-native', 'pkgconfig-native', 'util-linux-native', 'libnsl2-native', 'libtirpc-native', 'gdbm-native', 'readline-native', 'libgcc', 'attr-native', 'unzip-native', 'ncurses', 'libcap-ng', 'bash-completion', 'm4-native', 'gettext-minimal-native', 'libpcre2-native', 'ncurses-native', 'libcap-ng-native']
  | DEBUG: Python function extend_recipe_sysroot finished
  | DEBUG: Executing shell function do_install
  | ERROR:root:Error parsing
  | Traceback (most recent call last):
  |   File "/tmp/accton-as4610/work/cortexa9-vfp-poky-linux-gnueabi/python3-oslo.i18n/3.17.0+gitAUTOINC+f2729cd36f-r0/recipe-sysroot-native/usr/lib/python3.8/site-packages/pbr/core.py", line 96, in pbr
  |     attrs = util.cfg_to_args(path, dist.script_args)
  |   File "/tmp/accton-as4610/work/cortexa9-vfp-poky-linux-gnueabi/python3-oslo.i18n/3.17.0+gitAUTOINC+f2729cd36f-r0/recipe-sysroot-native/usr/lib/python3.8/site-packages/pbr/util.py", line 271, in cfg_to_args
  |     pbr.hooks.setup_hook(config)
  |   File "/tmp/accton-as4610/work/cortexa9-vfp-poky-linux-gnueabi/python3-oslo.i18n/3.17.0+gitAUTOINC+f2729cd36f-r0/recipe-sysroot-native/usr/lib/python3.8/site-packages/pbr/hooks/__init__.py", line 25, in setup_hook
  |     metadata_config.run()
  |   File "/tmp/accton-as4610/work/cortexa9-vfp-poky-linux-gnueabi/python3-oslo.i18n/3.17.0+gitAUTOINC+f2729cd36f-r0/recipe-sysroot-native/usr/lib/python3.8/site-packages/pbr/hooks/base.py", line 27, in run
  |     self.hook()
  |   File "/tmp/accton-as4610/work/cortexa9-vfp-poky-linux-gnueabi/python3-oslo.i18n/3.17.0+gitAUTOINC+f2729cd36f-r0/recipe-sysroot-native/usr/lib/python3.8/site-packages/pbr/hooks/metadata.py", line 25, in hook
  |     self.config['version'] = packaging.get_version(
  |   File "/tmp/accton-as4610/work/cortexa9-vfp-poky-linux-gnueabi/python3-oslo.i18n/3.17.0+gitAUTOINC+f2729cd36f-r0/recipe-sysroot-native/usr/lib/python3.8/site-packages/pbr/packaging.py", line 870, in get_version
  |     raise Exception("Versioning for this project requires either an sdist"
  | Exception: Versioning for this project requires either an sdist tarball, or access to an upstream git repository. It's also possible that there is a mismatch between the package name in setup.cfg and the argument given to pbr.version.VersionInfo. Project name oslo.i18n was given, but was not able to be found.
  | error in setup command: Error parsing /tmp/accton-as4610/work/cortexa9-vfp-poky-linux-gnueabi/python3-oslo.i18n/3.17.0+gitAUTOINC+f2729cd36f-r0/git/setup.cfg: Exception: Versioning for this project requires either an sdist tarball, or access to an upstream git repository. It's also possible that there is a mismatch between the package name in setup.cfg and the argument given to pbr.version.VersionInfo. Project name oslo.i18n was given, but was not able to be found.
  | ERROR: 'python3 setup.py install --root=/tmp/accton-as4610/work/cortexa9-vfp-poky-linux-gnueabi/python3-oslo.i18n/3.17.0+gitAUTOINC+f2729cd36f-r0/image     --prefix=/usr     --install-lib=/usr/lib/python3.8/site-packages     --install-data=/usr/share' execution failed.
  | WARNING: exit code 1 from a shell command.
  | ERROR: Execution of '/tmp/accton-as4610/work/cortexa9-vfp-poky-linux-gnueabi/python3-oslo.i18n/3.17.0+gitAUTOINC+f2729cd36f-r0/temp/run.do_install.2518342' failed with exit code 1
  ERROR: Task (/home/ubuntu/workspace/poky-bisdn-linux/poky/meta-cloud-services/meta-openstack/recipes-devtools/python/python3-oslo.i18n_git.bb:do_install) failed with exit code '1'
  NOTE: Tasks Summary: Attempted 4488 tasks of which 8 didn't need to be rerun and 1 failed.
  
  Summary: 1 task failed:
    /home/ubuntu/workspace/poky-bisdn-linux/poky/meta-cloud-services/meta-openstack/recipes-devtools/python/python3-oslo.i18n_git.bb:do_install
  Summary: There were 25 WARNING messages shown.
  Summary: There were 2 ERROR messages shown, returning a non-zero exit code.
  

  To avoid this you can either try building from version 4.6.1 (or newer), or disable the git safeguard and manually mark all directories as safe to use from by running:

  git config --global safe.directory '*'
  

  WARNING This will effectively reintroduce the vulnerability of CVE-2022-24765, please read the advisory carefully to make sure you understand the risk.

bisdn-linux's Issues

Switch was recognized as AS4610-30 instead of AS4610-54 in bisdn linux 5.1

Hi,

with bisdn linux 5.1 my switch was recognized as AS4610-30 instead of AS4610-54. This caused the interface not to come up. I changed /etc/onl/platform from "arm-accton-as4610-30-r0" to "arm-accton-as4610-54-r0" manually. That resolved the issue.

onlpdump -o
Product Name: 4610-54T-O-AC-B
Part Number: F0PEC4654004Z
MAC Range: 55
Manufacturer: Accton
Manufacture Date: 03/17/2019 04:52:33
Vendor: Edgecore
Platform Name: arm-accton_as4610_30-r0
Label Revision: R01A
Country Code: CN
Diag Version: 001.9
ONIE Version: 2016.05.00.04

Best regards,

GM

Switch to Yocto 5.0 (LTS)

Yocto 5.0 (Scarthgap) is planned to be the next LTS release, and is planned for April 24. To keep up to date, we should work on switching to it once it nears the release.

Update OpenNetworkLinux base to latest

Surprisingly PRs are getting merged into ONL again, so let's update our recipe to the new version, which allows us dropping accepted patches.

Cleanup and merge PoE drivers

There's currently a lot of duplication between the two PoE drivers for AS4610 and AS4630. Fix this by moving common code into a library function, reducing code duplication and making improvments and fixing things esier.

Add SNMP Polling for switch interfaces

If you install the net-snmp-server package on the switch you can poll general switch information but the switch port TAP interfaces does not pass bandwidth usage info over from the Switch ASIC. I am using a Accton 4610-54p for reference. Would there be a way of adding a SNMP extend to poll switch port usage info?

Add automated option to create a simple L2 learning switch

To allow users to easily test the default switch functionality directly after installing the switch via ONIE and without the hassle of manually adding any configuration files, it might be useful to build in a simple option to make the switch a simple L2 learning one. To keep the amount of configuration small and easy for later removal (in case the user decides to go with something handmade and more complicated), we should probably only genrate a minimal bridge configuration (.netdev and .network) and only one wildcard matching port configuration (.network with Name=port*) to bring all ports up and attach them to the bridge. To clearly mark all files as autogenerated, we could use a specifc pre- or suffix for all files (e.g. *_bisdn-autogen*), which could also be used later to remove them in a scripted way. The easiest way to add this functionality i can think of right now, would be to write a simple script (e.g. bisdn-init-l2-learning-config), make it executable and add it to the same folder as our other scripts (onie-bisdn-upgrade and so on). To allow easy cleanup, we could add an option to the script (e.g. --clean) to remove the files. To show the user which files are generated before they are actually placed in the folder, another option like --dry-run might be useful as well.

Enable (optional) CVE reporting for the build

Yocto allows CVE checking during the build, so let's make use of that to get a better understanding of security issues affecting BISDN Linux.

• Update Kernel recipes with fixed CVEs.
• Check our recipes for missing CPE IDs
• Add fixed CVEs for the changelog

Improve switch_tcpdump

We recently had a problem with an ACL table rule that was left by switch_tcpdump after it was interrupted by the user.

There are several other issues with the script that should be fixed. For instance, the script should default to --stdout instead of insisting that either --stdout or --filePath be passed as arguments. Also, additional tcpdump options should be supported.

Stacked VLAN support (QinQ)

Currently stacked VLANs are not configurable and not working properly, see https://docs.bisdn.de/network_configuration/vlan_bridging_qinq.html#stacked-vlan-support for details.

For proper expected behavior, we need to update/enhance baseboxd to handle stacked VLAN devices/bridges.

[ ] support stacked VLAN interfaces, e.g. port1.2.3
[ ] support VLAN interfaces as bridge members, to automatically en-/decapsulate traffic with an extra tag on that port
[ ] support stacked vlan aware bridges?

replace repo with kas

There is a repo like utility for managing bitbake projects, might be interesting to use as an alternative: https://github.com/siemens/kas.

Especially since it offers setting up a build environment and building, so could replace several open coded steps in the build process.

AS4610: Install fails with previous ONL install

I'm seeing the following on as AS4610 switch that had a prior ONL install:

ONIE: Executing installer: http://repo.bisdn.de/pub/onie/accton-as4610/onie-bisdn-accton-as4610-v4.8.1.bin
Image build date: 2022-10-05
Verifying image checksum ... OK.
Preparing image archive ... OK.
DEBUG: onl_platform=arm-accton-as4610-54-r0
DEBUG: onl_baseplatform=arm-accton-as4610-54
BISDN Linux Installer
DEBUG: onie_partition_type=gpt
DEBUG: firmware=u-boot
removing partition 1 (Linux)
removing partition 2 (Linux)
removing partition 3 (Linux)
removing partition 4 (Linux)
Creating new BISDN Linux gpt partition /dev/sda5 ...
Could not create partition 5 from 34 to 12582945
Unable to set partition 5's name to 'BISDN-Linux'!
Error encountered; not saving changes.
Error: Unable to create partition 5 on /dev/sda
Failure: Unable to install image: http://repo.bisdn.de/pub/onie/accton-as4610/onie-bisdn-accton-as4610-v4.8.1.bin

The partition layout is:

Number  Start (sector)    End (sector)  Size       Code  Name
   1            2048          262143   127.0 MiB   8300  Linux filesystem
   2          262144          524287   128.0 MiB   8300  Linux filesystem
   3          524288         2621439   1024.0 MiB  8300  Linux filesystem
   4         2621440        15720447   6.2 GiB     8300  Linux filesystem

Switch to Linux 6.6 (LTS)

Now that then ext LTS kernel has been announced, we should switch to it.

• Add support for 6.6
• Switch to 6.6

Update libnl to 3.8

libnl 3.8 was recently released, so lets update it.

bisdn linux 5.1: package python3-appdirs missing in repository

The package python3-appdirs is missing in the bisdn linux 5.1 repository although some other python packages depend on it. The consequence is that the latter cannot be installed.

• • nothing provides python3-appdirs needed by python3-twisted-core-22.2.0-r0.cortexa9-vfp

Best regards,

GM

allow fans of delta ag7648 to be controlled

currently the fans are only exposed through onlp and configured through raw i2c calls.

The fan controller seems to be a max6639 chip, for which an in-kernel driver exists. The source code comment says max6620, but the register macros names and contents match the max6639 driver's, not the max6620.

Loading the driver to the appropriate register should allow configuring the fans without much effort.

According to the platform i2c library the chips are a at I²C bus 3, addresses 0x2a and 0x29.

Provide downloadable packages for various use-cases

Currently any downloadable extra package is more of a side effect of Yocto recipes providing multiple packages when only one is requested to be installed.

To provide support for different use cases, we should actually provide a curated list of additional packages to install.

Update FRR to 9.0

Our FRR version is getting out of date, so let's investigate if we can update to 9.0 (latest).

• Import FRR recipe for 9.0 from meta-openembedded master (if it exists)
• If not, create one ourselves and upstream it
• Ensure all tests work as expected

import custom_led.bin binaries from SONiC

To reduce the need for embedded custom_led.bin binaries in OF-DPA, add support for importing these from the SONiC project, and add them for the Edgecore platforms we support.

There is no binary for the Celestica Questone 2A available, so that one still needs one embedded.

Allow password-less sudo for basebox user

Currently, we write the line basebox ALL=(ALL) ALL into /etc/sudoers.d/basebox [1].

This is not sufficient if we want to use the basebox user with password-less sudo. Even if added the user to the sudo group and gave members of the sudo group password-less sudo, the file specifically for the basebox user would take precedence. Do we want to change the current default in BISDN Linux?

Also, having basebox and basebox.default lying around in /etc/sudoers.d/ (presumably upgrade artifacts) is confusing; we should consider cleaning that up.

[1] https://github.com/bisdn/meta-switch/blob/main/recipes-core/basebox-user/basebox-user.bb

docker broken in main on arm

Docker is broken in for arm due to go linker changes.

• identify core issue
• submit fix
• fix accepted for master
• backport fix
• fix accepted for kirkstone

let the installer use all available space

The installer currently uses a fixed partition size of 6 GiB. But depending on the machine, there might a larger SDD included, so it would be nice to be able to make use of it.

So let the installer instead span the BISDN Linux partion the whole available space instead.

Make adding new platform support easier

The most difficult part for ONL supported platforms is making the code compile with GCC 12 and Linux kernel 6.1. To reduce the amount of work required for extending BISDN Linux support, it would be nice to have those already present as ready to use, and only requiring the "soft" conversions which can be tested on device, i.e. updating the platform init script, OF-DPA configuration and installer.

• Split platform libraries out of onlp package
• enable additional platforms build as installable packages

4.8.1 -> 4.8.2 Upgrade fails

Existing installation found!
creating backup of existing configuration
rmdir: '/tmp/tmp.vXru01/etc/systemd': No such file or directory
Warning: The kernel is still using the old partition table.
The new table will be used at the next reboot.
The operation has completed successfully.
Error: Partition(s) 3 on /dev/sda have been written, but we have been unable to inform the kernel of the change, probably because it/they are in use.  As a result, the old partition(s) will remain in use.  You should reboot now before making further changes.
Failure: Unable to install image: http://repo.bisdn.de/pub/onie/accton-as4610/onie-bisdn-accton-as4610-v4.8.2.bin

It does work the second time, but of course this is problematic for unattended upgrade.

proper MSTP support via in-kernel MSTI support

Linux recently (in 5.18) added support for defining MSTIs in the kernel as vlan groups for STP (skipping the FID part).

Using that, we can simplify the mstp support, and do not need a stg group per vlan anymore.

To be able to do that, we will require changes in various components:

• kernel: notify about msti stp changes. Currently these are kept in-kernel, for switchdev only. To allow baseboxd to pick this up we need to have them send out via netlink.
• libnl: pick up and expose msti stp changes. As above.
• libnl: pick up expose global vlan options. For baseboxd to know which VIDs are members of which MSTIs, we need to be notified about their membership (changes).
• baseboxd: pick up and configure MSTs based on the received libnl changes. This will also need a way of knowing if we are in PVSTP or MST mode, as the port STP meaning changes.
• mstpd: implement setting MSTI memberships for vlans for mst enabled bridges. Will require using similar code like the current PVSTP implementation, as you cannot set port states for non-existing MSTIs, and you can only assign existing vlans to MSTIs.

Additionally, we will also want

• iproute2: add mst support (apply the patches from https://github.com/wkz/iproute2/tree/mst)
• systemd-networkd: allow creating bridges in mst enabled mode The mst mode can only be changed when there are no vlans defined on a bridge, so when using systemd-networkd we need to create them in this mode before configuring any vlans.