bryteise / ister Goto Github PK

Need to update functions: setup_mounts, get_source_image.

Also should test against cdn.download.clearlinux.org uri for both dns and curl.

Originally reported by @zyuan2x in clearcontainers/runtime#869

The following error is encountered when installing Clear Linux to a BXT_GP board:

fdisk: cannot open /dev/mmcblk0rpmb: Input/output error

@bryteise I thought you resolved this issue in 92305eb

Update default bundles removing os-clr-on-clr (https://github.com/bryteise/ister/blob/master/ister_gui.py#L1981) with user-basic (https://github.com/clearlinux/clr-bundles/blob/master/bundles/user-basic)

Due to some assumptions made by ister_gui.py and ister.py regarding device partition naming schemes, Ister will fail to install on NVME devices.

Following the auto install route the install will report success, but will fail to boot and will give an EFI shell.

Following the manual install route one can partition the NVME drive via the baked-in cgdisk tool, but the actual install step (ister.py) will fail with No partition found on /dev/nvmep.

Related Screencaps

courtesy of Bun K. Tan

cgdisk identifies device just fine

Need to verify the partition prefix is p

Are we just assuming it is "p" at this point?

This indicates we are detecting a p prefix

Manual install fails

Manual install failure logs

On line 8, 'partition': '1p1' does not look right.

When opted in, the bundle selector screen displays the telemetrics bundle in the required bundle list. When the user selects 'Previous' on the bundle selector screen and then returns to it by pressing 'Next' an additional entry is added for the telemetrics bundle.

(the below screenshot is from experiments with palette)

validate_postnonchroot_template is limiting what can run to only script files that can be found on the filesystem. this is painful when trying to performing some custom commands after a pxe boot install. I'm
forced to change the initrd to hack in some script.. would be much cleaner and direct if the shell commands could be included in the ister.json.

it would be helpful if validate_postnonchroot_template did no checks at all or another option like PostNonChrootRaw was added.

running raw commands would be nice.

"PostNonChroot": ["bash -c 'mkdir -p $0/usr/local/maintenance; cd $0/usr/local/maintenance; curl https://xxxxxx.com/linux/postinstall.tar.gz | tar -xz'"]

This came as a request via clearlinux/clr-bundles#13

In some boards (e.g. UDOO X86) equipped with EMMC, the block devices can be the following:

/dev/mmcblk0
/dev/mmcblk0boot0
/dev/mmcblk0boot1
/dev/mmcblk0p1
/dev/mmcblk0p2
/dev/mmcblk0p3
/dev/mmcblk0rpmb

The fdisk -l will fail on /dev/mmcblk0rpmb:

udoox86 ~ $ sudo fdisk -l /dev/mmcblk0rpmb
fdisk: cannot open /dev/mmcblk0rpmb: Illegal seek

More info about mmcblk0rpmb can be found at LWN.

The installer (e.g. used in Clear Linux) fail with no way to recover in case those devices are scanned.

I have a patch to skip it. Would you prefer filtering it at get_list_of_disks() or in get_disk_info()?

I build one local kernel and want to replace the origin default one (kernel version: 4.14.7-497).

the content of my local kernel:
├── config-4.15.0-rc4
├── lib
│   └── modules
│   └── 4.15.0-rc4
├── System.map-4.15.0-rc4
└── vmlinuz-4.15.0-rc4

My work steps:

1. copy vmlinuz-4.15.0-rc4 to /lib/kernel
2. delete the org.clearlinux.native.4.14.7-497 and rename vmlinuz-4.15.0-rc4 to org.clearlinux.native.4.15.0-rc4
3. delete default-native
4. create new link "default-native" and link with new kernel org.clearlinux.native.4.15.0-rc4
5. delete origin kernel modules (4.14.7-497.native) in /lib/modules
6. copy new kernel modules 4.15.0-rc4 to /lib/modules and rename it with 4.15.0-rc4.native
7. reboot the machine
8. check the kernel version with command "uname -a"

I see the kernel version can't changes
How to change the kernel version and load the new kernel and new kernel modules?

Allow ister to do package installation. This will be used as a post install process at first but may eventually be used in place of a source image to do installation. Will only work with package manager based systems (supporting zypper first).

Will need to extend the template and decide how to handle package installation failures relative to the entire installation process's success.

In the "Choose target device for installation" step, ister shows details including sizes for existing partitions, but does not show unpartitioned free space, or any information beyond device name for unpartitioned disks. Given a system with multiple unpartitioned disks (e.g. one SSD, multiple large HDDs, it may be necessary to drop to a shell to determine the desired drive name (e.g. lsblk -lo NAME,TYPE,MOUNTPOINT,SIZE,MODEL,SERIAL,SUBSYSTEMS,HCTL,STATE).

Even though the warning message says an invalid bundle is being skipped, the bundle name still shows up when you do "swupd bundle-list" for the created image.

Example:
clear@clr-da50329d227d4062bb2170a8b8162bd2 ~/mixer $ sudo -E mixer build image
Password:
Reading configuration
Validating configuration
Creating virtual disk
Creating partitions
Mapping loop device
Creating file systems
Setting up mount points
Starting swupd. May take several minutes
Installing 4 bundles (and dependencies)...
.
....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
Verifying version 10
Warning: Bundle "BADBUNDLE" is invalid, skipping it...
Downloading packs...

STEPS TO REPRO:

1. Add an invalid bundle name to config json file.
2. Create an image
3. Boot up the image
4. swupd bundle-list and you can see the invalid bundle name

The netifaces library pulls in the world when it comes to dependencies and balloons our installer image size. Considering that netifaces is minimally used to determine interface list and IP it should be possible to implement the functionality ourselves and save us the image size.

Hello @bryteise

Is it a good idea to be able to install ister as a CLI utility via pip? (pip install https://some-url)

I can submit a PR for that but I expect a lot of filesystem refac.
Regards

Towards the end of building an image with mixer, I get a small non-fatal error which I think might be the output of ister. The referenced path looks a bit weird.

$ sudo mixer build image
...
Inspected 8386 files
  8379 files were missing
    8379 of 8379 missing files were replaced
    0 of 8379 missing files were not replaced
Calling post-update helper scripts.
sh: /tmp/ister-latest-6uuayzoc///usr/bin/update-helper: No such file or directory
Fix successful
Cleaning up
Successful installation

Thanks.

process_kernel_cmdline strips off any extra = signs in the kernel command line, thus I am unable to use foreman or any other sort of token based installer as many use an = sign for the token value.
The specific issue is this line where it is stripping off everything past the second = sign.
https://github.com/bryteise/ister/blob/master/ister.py#L1478

For foreman each time a system is booted to be provisioned, the url for ister.conf is dynamically generated. Ex

isterconf=http://$foreman_url/unattended/provision?token=c399245f-0c4b-4cda-b561-911d0f4bb760

When this url is hit from ister.py everything past token is stripped, thus does not work.

Thanks
~/DonnyD

Ister should use pytest instead of writing is own test harness.

To do so will require a bit of refactoring (e.g. have a proper package instead of lumping everything into three files, so this involves a bit of work:-).

I would be willing to help with that, if you are willing to move into that direction.

Swupd's output has been updated since the current progress display with added. Adapt the progress to better display what is happening in swupd.

Hi

I already install clear linux OS on my board. And i install some packages and libraries on it. I want to close this image and install it on another board. How to disable the network and only install it from local disk?

On some systems a /dev/mmcblk0rpmb exists. fdisk -l fails on this device, crashing ister and aborting the install.

the rpmb device is some sort of secure partition not for OS use, and should just be ignored, not crashed on