jnielsendotnet / isboot Goto Github PK
View Code? Open in Web Editor NEWFreeBSD kernel module to support booting from iSCSI via IBFT
FreeBSD kernel module to support booting from iSCSI via IBFT
The iSCSI boot driver for FreeBSD This driver provides an iSCSI boot feature for FreeBSD. It can be used with a bootable initiator offered by many server BIOSes, NIC firmwares or iPXE/gPXE. It connects to the iSCSI target specified in the iSCSI Boot Firmware Table (iBFT) automatically, making it simple to run a diskless machine. Your boot firmware will create and populate the iBFT, connect to the specified iSCSI target and attempt to boot from the target volume (like any other supported disk). Any normal bootloader (such as FreeBSD's gptboot) can be used. The bootloader should load the kernel and modules (including isboot) and begin execution. During boot, the isboot driver configures the NIC and creates a new connection to the iSCSI target. The disk is then available to the operating system as normal, and can be used to mount the root volume. Installation: 1. extract the archive: # tar zxvf isboot-x.x.x.tar.gz 2. compile the module: # cd isboot-x.x.x/src # make 3. install the compiled module to the kernel directory: # make install 4. edit /boot/loader.conf, and add the following line: isboot_load="YES" Note: If you want use isboot with VIMAGE kernel, add CFLAGS+= -DVIMAGE to Makefile. The boot device may change after installation. It may be necessary to edit /etc/fstab in single-user mode. The iBFT code supports two loader tunables: hw.ibft.acpi_table: Defaults to 1. If you have multiple iSCSI boot firmwares you may need to set this to a different value (such as 2). hw.ibft.verbose: Defaults to 0. Set it to 1 to enable verbose iBFT logs. Similarly, there is a loader tunable for isboot debug output: net.isboot.debug: Defaults to 0. Set it to 1 for high-level messages, 2 for all informational messages and 3 for full trace output. After boot you can see the boot device information via sysctl(8). For example: # sysctl net.isboot net.isboot.device: da5 net.isboot.nic: cxl0 net.isboot.version: 0.2.15 # sysctl hw.ibft hw.ibft.verbose: 1 hw.ibft.acpi_table: 1 hw.ibft.nic_gateway: 0.0.0.0 hw.ibft.nic_prefix: 24 hw.ibft.target_lun: 0 hw.ibft.target_port: 3260 hw.ibft.target_address: 192.168.104.10 hw.ibft.target_name: iqn.2002-05.net.jnielsen:max-iscsi hw.ibft.initiator_address: 192.168.104.9 hw.ibft.initiator_name: iqn.2020-06.com.chelsio.boot:0007432FFBD0 # camcontrol inquiry da5 pass8: <JNC iSCSI 0001> Fixed Direct Access SPC-5 SCSI device pass8: Serial Number 42 pass8: 300.000MB/s transfers Supported OS versions: o FreeBSD 12.4 o FreeBSD 13.2 o FreeBSD 14.x Project website: https://github.com/jnielsendotnet/isboot The original author's blog (in Japanese) had some additional information: https://web.archive.org/web/20160810012908/http://shell.peach.ne.jp/aoyama/ Authors: Daisuke Aoyama <[email protected]> Michael Zoon <[email protected]> John Nielsen <[email protected]>
In dmesg I see:
Boot NIC: cxl0
Configure IPv4 by NIC0
Attempting to login to iSCSI target and scan all LUNs.
connect error
connect failed
cxl0: link state changed to UP
Then the iSCSI login subsequently succeeds on a retry (with no further messages) and things work as normal.
As per https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=227313 ctld does not handle 'login retry' only 'login'.
The attached patch makes it try both.
Renaming the boot interface is useful to apply specific network configuration to the iBFT interface and to prevent it from being configured by other services (e.g. DHCP).
This is similar to the behavior of the dracut ifname=
cmdline argument.
Here is an rc.d script I use which does this:
#!/bin/sh
#
# $FreeBSD$
#
# PROVIDE: isbootifname
# BEFORE: dhclient netif
# KEYWORD: nojail
. /etc/rc.subr
name="isbootifname"
rcvar="isbootifname_enable"
start_cmd="${name}_start"
stop_cmd="${name}_stop"
load_rc_config $name
: ${isbootifname_enable:=yes}
isbootifname_start()
{
_isboot_nic="$(sysctl -bi net.isboot.nic)"
if [ -n "$_isboot_nic" ]; then
echo "${name}: Renaming interface ${_isboot_nic} to bootnet0"
ifconfig "${_isboot_nic}" name "bootnet0"
fi
}
isbootifname_stop()
{
_isboot_nic="$(sysctl -bi net.isboot.nic)"
if [ -n "$_isboot_nic" ]; then
echo "${name}: Renaming interface bootnet0 back to ${_isboot_nic}"
ifconfig "bootnet0" name "${_isboot_nic}"
fi
}
run_rc_command "$1"
It might be useful to include this in the documentation.
E.g. Then you can disable automatic DHCP on that interface, regardless of what the original interface name is.
sysrc ifconfig_bootnet0="NOAUTO"
Another possibility is to add a new loader tunable (e.g. hw.ibft.ifname=bootnet0
) that internally uses something equivalent to SIOCSIFNAME
. net.isboot.nic
would also probably need to be set to match this, to take into account any scripts already reading that sysctl.
isboot 0.2.13 worked (or at least compiled) for FreeBSD 11, 0.2.14 does not.
non-verbose output from isboot should be grouped together as much as possible and prepended with 'isboot0:' so it's easy to scan for (and so the eventual disk is attached to a previously-mentioned device)
or at all. Need to look in to why.
Is this code from "peach" from about 10 years ago? There was an installer of FreeBSD 9.x that could install over iscsi. I'd like to see this become available again. I work with ipxe a lot.
I'm trying to get iSCSI boot working via an offboard Mellanox card. After solving two other hurdles I came across (see the two other issues I just opened ;) ), I'm stuck with a seemingly infinite loop of Root mount waiting for: CAM
messages.
As noted in one of my other issues, I've changed the subsystem of isboot
to SI_SUB_ROOT_CONF-1
, to make it load after the mellanox module. Could that be related to the issue I'm having?
Here's the relevant part of my boot console (I can post the entire thing if you think it will help).
Load isboot
iSCSI boot driver version 0.2.15-alpha
iBFT instance 4 (ACPI_SIG_IBFT) not found
iBFT instance 3 (ACPI_SIG_IBFT) not found
iBFT instance 2 (ACPI_SIG_IBFT) not found
iBFT instance 1 (ACPI_SIG_IBFT) not found
iBFT instance 0 (ACPI_SIG_IBFT) not found
iBFT instance 4 (IBFT_SIGNATURE) not found
iBFT instance 3 (IBFT_SIGNATURE) not found
iBFT instance 2 (IBFT_SIGNATURE) found
found iBFT via ACPI
iBFT: length=609, revision=1, checksum=0x7f
iBFT: oemid='FENSYS', oemtableid='iPXE'
iBFT: sum = 0x0
iBFT: CS: length=18, index=0, flags=0x0
iBFT: CS: initiator=80, nic0=160, target0=384, nic1=0, target1=0
iBFT: IS: length=74, index=0, flags=0x3
IS: Initiator name: iqn.2010-04.org.ipxe:00000000-0000-0000-0000-3cecef70f2f4
iBFT: NIC0: length=102, index=0, flags=0x3
NIC0: IP address: 10.0.0.2
NIC0: Prefix: 24
NIC0: Origin: 1
NIC0: VLAN: 0
NIC0: MAC address: 00:02:c9:52:77:88
NIC0: PCI Bus/Dev/Func: 6300 (63/0/0)
iBFT: TGT0: length=54, index=0, flags=0x3
TGT0: Target IP address: 10.0.0.1
TGT0: Target Port: 3260
TGT0: Target LUN: 0
TGT0: CHAP type: 0
TGT0: NIC index: 0
TGT0: Target name: [REDACTED]
Boot NIC: mlxen0
Configure IPv4 by NIC0
isboot start, thread id=186a0
kproc_start
Attempting to login to iSCSI target and scan all LUNs.
isboot kproc start, thread id=18ab2
isboot iscsi start, thread id=18ab2
main loop, thread id=18ab2
initialize session, thread id=18ab2
Initiator: iqn.2010-04.org.ipxe:00000000-0000-0000-0000-3cecef70f2f4
Target: [REDACTED]
Target IP=10.0.0.1, Port=3260, LUN=0
strdup(iqn.2010-04.org.ipxe:00000000-0000-0000-0000-3cecef70f2f4)57
strdup([REDACTED])38
strdup(10.0.0.1)8
strdup(None)4
strdup(None,CRC32C)11
strdup(None,CRC32C)11
isboot_connect
open socket
try connect...(18ab2)
connect error
connect failed
boot retry (59)
isboot_connect
open socket
try connect...(18ab2)
connect error
connect failed
boot retry (58)
<6>mlx4_en: mlxen0: Link Up
mlxen0: link state changed to UP
<6>mlx4_en: mlxen0: Link Down
mlxen0: link state changed to DOWN
<6>mlx4_en: mlxen0: Link Up
mlxen0: link state changed to UP
isboot_connect
open socket
try connect...(18ab2)
wait connect...
old so=0, new so=0xfffff82069dfdb10
isboot_do_login
login start
xmit PDU
recv PDU
update option
KEY=[TargetAlias], VAL=[REDACTED]
KEY=[TargetPortalGroupTag], VAL=[1]
KEY=[HeaderDigest], VAL=[None]
free[None,CRC32C]
strdup(None)4
KEY=[DataDigest], VAL=[None]
free[None,CRC32C]
strdup(None)4
KEY=[DefaultTime2Wait], VAL=[2]
KEY=[DefaultTime2Retain], VAL=[0]
KEY=[ErrorRecoveryLevel], VAL=[0]
KEY=[MaxConnections], VAL=[1]
KEY=[InitialR2T], VAL=[Yes]
KEY=[ImmediateData], VAL=[Yes]
KEY=[MaxBurstLength], VAL=[1048576]
KEY=[MaxOutstandingR2T], VAL=[1]
KEY=[DataPDUInOrder], VAL=[Yes]
KEY=[DataSequenceInOrder], VAL=[Yes]
KEY=[FirstBurstLength], VAL=[262144]
KEY=[MaxRecvDataSegmentLength], VAL=[262144]
rsp login
free PDU
login end
cam attach
isboot action 4
isboot action 4 done
cam attach end
cam rescan
isboot action 4
isboot action 4 done
isboot action 4
isboot action 4 done
isboot action 4
isboot action 4 done
isboot action 4
isboot action 4 done
isboot action 15
isboot action 15 done
isboot action 16
isboot action 16 done
isboot action 4
isboot action 4 done
isboot action 901
isboot scsi io
add ccb
isboot action 4
isboot action 4 done
isboot action 4
isboot action 4 done
isboot_free_mbufext
Trying to mount root from ufs:/dev/gpt/[REDACTED] [rw,noatime]...
Root mount waiting for: CAM
Root mount waiting for: CAM
Root mount waiting for: CAM
Root mount waiting for: CAM
Root mount waiting for: CAM
Root mount waiting for: CAM
Root mount waiting for: CAM
Root mount waiting for: CAM
Root mount waiting for: CAM
Root mount waiting for: CAM
Root mount waiting for: CAM
Root mount waiting for: CAM
Root mount waiting for: CAM
Root mount waiting for: CAM
Root mount waiting for: CAM
I have a Supermicro motherboard which supports iSCSI boot via the onboard NICs (which are 1G), and I am trying to get iSCSI boot working via a PCIe NIC (specifically a 10G mellanox card). I am using iPXE to boot and create the iBFT. Unfortunately, what actually happens is that there's a garbage iBFT, related to the onboard NIC iSCSI boot capability, that isboot
finds.
But, I managed to figure out that the iBFT created by iPXE is actually present, it's just not the first one that AcpiGetTable
finds. I was able to work around the issue by using the following dirty hack:
--- a/src/ibft.c
+++ b/src/ibft.c
@@ -545,12 +545,30 @@ ibft_acpi_lookup(void)
/*ACPI_IBFT_HEADER *ibft_hdr, *end;*/
ACPI_STATUS status;
- status = AcpiGetTable(ACPI_SIG_IBFT, 1, (ACPI_TABLE_HEADER **)&ibft);
- if (ACPI_FAILURE(status)) {
- status = AcpiGetTable(IBFT_SIGNATURE, 1, (ACPI_TABLE_HEADER **)&ibft);
- if (ACPI_FAILURE(status))
- return (NULL);
+
+ for (int i = 4; i >= 0; i--) {
+ status = AcpiGetTable(ACPI_SIG_IBFT, i, (ACPI_TABLE_HEADER **)&ibft);
+ if (!ACPI_FAILURE(status)) {
+ printf("iBFT instance %i (ACPI_SIG_IBFT) found\n", i);
+ goto found;
+ } else {
+ printf("iBFT instance %i (ACPI_SIG_IBFT) not found\n", i);
+ }
}
+
+ for (int i = 4; i >= 0; i--) {
+ status = AcpiGetTable(IBFT_SIGNATURE, i, (ACPI_TABLE_HEADER **)&ibft);
+ if (!ACPI_FAILURE(status)) {
+ printf("iBFT instance %i (IBFT_SIGNATURE) found\n", i);
+ goto found;
+ } else {
+ printf("iBFT instance %i (IBFT_SIGNATURE) not found\n", i);
+ }
+ }
+
+ return NULL;
+
+ found:
return (uint8_t *)ibft;
}
But obviously that isn't a great solution, it just will find the last iBFT table present, provided there are 4 or fewer.
On my system, the output is:
Load isboot
iSCSI boot driver version 0.2.15-alpha
iBFT instance 4 (ACPI_SIG_IBFT) not found
iBFT instance 3 (ACPI_SIG_IBFT) not found
iBFT instance 2 (ACPI_SIG_IBFT) not found
iBFT instance 1 (ACPI_SIG_IBFT) not found
iBFT instance 0 (ACPI_SIG_IBFT) not found
iBFT instance 4 (IBFT_SIGNATURE) not found
iBFT instance 3 (IBFT_SIGNATURE) not found
iBFT instance 2 (IBFT_SIGNATURE) found
I'm quite new to FreeBSD kernel hacking - is there a clean way to add a parameter, settable via loader.conf
, that would allow the user to specify the Instance
argument that gets passed to AcpiGetTable
? It would be nice if we could expose some way for the user to work around janky motherboard firmware like this...
I am trying to get iSCSI boot working with an outboard Mellanox Connectx-2 NIC, and one issue I ran in to is that the mellanox driver module mlx4en
is loaded after this module, meaning isboot
will fail to find the NIC specified in the iBFT when it tries to look it up by MAC address.
The mlx4en
module seems to use the linux compatibility kpi, and it's assigned to the subsystem SI_SUB_OFED_PREINIT
, which is defined to be SI_SUB_ROOT_CONF-2
. isboot
is assigned to subsystem SI_SUB_OFED_PREINIT
, which is a few entries earlier.
I was able to work around this particular issue by changing
-DECLARE_MODULE(isboot, mod_data, SI_SUB_PROTO_END, SI_ORDER_ANY);
+DECLARE_MODULE(isboot, mod_data, SI_SUB_ROOT_CONF-1, SI_ORDER_ANY);
in isboot.c
. (though I still haven't gotten iSCSI boot working completely)
I'm quite new to FreeBSD kernel hacking, and I'm not sure if this workaround will have undesirable side-effects. Do you have any thoughts on the matter?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.