sumo cannot boot from emmc about meta-intel-edison

[htot]

@alext-mkrs I found Andy's e-mail 9-Sep-2016 and it seems a little different then I remember:

Looks like you have no driver enumerated for your rootfs.
Basically either you need to use special initramfs / initrd where you
have all necessary modules and you use pivot_root to switch rootfs, or
you compile in all those modules. Since one bug(?) in the regulator
framework when you built-in sdhci-pci it will prevent to WiFi be
enumerated, you have to workaround it [1] like

echo 0000:00:01.3 > /sys/bus/pci/drivers/sdhci-pci/unbind
echo 0000:00:01.3 > /sys/bus/pci/drivers/sdhci-pci/bind

https://communities.intel.com/message/419376#419376

So, building sdhc drivers as modules config was from Andy and I wanted to track his kernel branch with as little changes as possible.

[htot]

I can confirm all of the above. There appear to be multiple problems that we did not have with Rocko:

1. With Sumo Yocto changed to incorporating bzImage into edison-image-edison.hddimg instead of vmlinuz. This is not a big issue as we only need to change the file name in meta-intel-edison-bsp/recipes-bsp/u-boot/files/edison.env.
2. It looks like we need to put the baud rate in the console (and earlyprintk) option to see any error messages with linux 4.16. Without this fix, when there are no errors, linux will still boot but very quietly.
3. With bzImage the kernel crashes as mmcblk0p8 does not appear. Adding rootwait to the kernel arg it waits forever. A usb disk does appear if present, sdhc not. As the mmc driver is built as module we need the initramfs to boot from emmc same as for sdhc. However Yocto builds bzImage-edison.bin into edison-image-edison.hddimg instead of bzImage-initramfs-edison.bin. @alext-mkrs do you know how to change which kernel image to build into the hddimage?

I could test the above because my mmcblk0p9 still contains bootable kernels so I can still boot from USB. mmcblk0p7 is a 9MB fs in a 33MB partition, too small to hold the initramfs kernel (12MB). I created a new fat32 partition in p7, copied bzImage-initramfs-edison.bin there and renamed to bzImage. Now it boots normally.

[lybtongji]

I used a temporary solution just now.
After built the image.
I ran the command bitbake -c cleanall edison-image to cleanup the hddimg files.
Then added bzImage-initramfs-edison.bin to hddimg folder manually.

mkdir -p ~/os/out/linux64/build/tmp/work/edison-poky-linux/edison-image/1.0-r0/edison-image-1.0/hddimg
cp -v ~/os/out/linux64/build/tmp/work/edison-poky-linux/linux-yocto/4.16.0-r0/deploy-linux-yocto/bzImage-initramfs-edison.bin ~/os/out/linux64/build/tmp/work/edison-poky-linux/edison-image/1.0-r0/edison-image-1.0/hddimg/vmlinuz

Rebuilt the image:

bitbake edison-image

Got the hddimg file:

~/os/out/linux64/build/tmp/deploy/images/edison/edison-image-edison.hddimg

Then replaced the old hddimg file by the new one and flashed it to the edison.
Now, I can boot into the system.

[htot]

Good one.

I hope @alext-mkrs will know how to fix this permanently. In the past he found VM_DEFAULT_KERNEL = "bzImage-initramfs-edison.bin" which I think did the trick.

Edit: Unfortunately that was removed by this commit: openembedded/openembedded-core@5493bb5

[alext-mkrs]

Not OTMH, but I'll take a look, there should be something.

[htot]

I have the bad feeling they want us to use WIC, but have no clue how that works.

[alext-mkrs]

Well, not sure that's the thing as wic is AFAIU is used for something a bit different like image partitioning - but anyway, generally an easy solution here is just to build to module into the kernel, which we can't due to specific Edison peculiarity. And the image-live/live-vm-common classes simply don't have that functionality, because they support a plain kernel and an initrd setups, and probably no one wanted an initramfs one bad enough.

I think I have a solution, it's a bit hackish, but well within Yocto mechanisms, so unless someone from the Yocto team responds to your commit comment with a better approach, this would work ok. I'll share it in a couple of minutes, preparing a branch to test.

[alext-mkrs]

BTW, @htot, do you remember/know why don't we currently build the mmc drive into the kernel? I sort of assumed that's because of that i2c-6 problem, but re-reading the commit message for that one now, I don't think that's actually related. Built-in module would solve that problem. Initramfs was introduced only for ACPI table loading AFAIU, so I don't see why we couldn't just have the most essential drivers built-in. That will help to avoid the fix I'm about to share.

[alext-mkrs]

@lybtongji, please give it a try with the above two patches. One ensures proper initramfs kernel is added to the hddimg and another one corrects the U-Boot env file. The second one you can actually do manually using setenv, this is just for fix completeness.

[lybtongji]

@alext-mkrs Not working...
The boot message still shows:

[    3.386858] VFS: Cannot open root device "mmcblk0p8" or unknown-block(0,0): error -6
[    3.394831] Please append a correct "root=" boot option; here are the available partitions:
[    3.403430] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)

bzImage was replaced by bzImage-initramfs-edison.bin:

recover@ilab:~/os$  ll ~/os/out/linux64/build/tmp/work/edison-poky-linux/linux-yocto/4.16.0-r0/deploy-linux-yocto/
total 18384
drwxrwxr-x  2 recover recover     4096 8月  13 00:50 ./
drwxrwxr-x 15 recover recover     4096 8月  13 01:08 ../
lrwxrwxrwx  2 recover recover       44 8月  13 00:50 bzImage -> bzImage--4.16.0-r0-edison-20180812160734.bin
-rw-r--r--  2 recover recover  7445984 8月  13 00:50 bzImage--4.16.0-r0-edison-20180812160734.bin
lrwxrwxrwx  2 recover recover       44 8月  13 00:50 bzImage-edison.bin -> bzImage--4.16.0-r0-edison-20180812160734.bin
-rw-r--r--  2 recover recover 10791680 8月  13 00:50 bzImage-initramfs-4.16.0-r0-edison-20180812160734.bin
lrwxrwxrwx  2 recover recover       53 8月  13 00:50 bzImage-initramfs-edison.bin -> bzImage-initramfs-4.16.0-r0-edison-20180812160734.bin
-rw-rw-r--  2 recover recover   575511 8月  13 00:50 modules--4.16.0-r0-edison-20180812160734.tgz
lrwxrwxrwx  2 recover recover       44 8月  13 00:50 modules-edison.tgz -> modules--4.16.0-r0-edison-20180812160734.tgz
recover@ilab:~/os$  ll ~/os/out/linux64/build/tmp/work/edison-poky-linux/edison-image/1.0-r0/edison-image-1.0/hddimg/
total 10856
drwxr-xr-x 2 recover recover     4096 8月  13 01:08 ./
drwxr-xr-x 3 recover recover     4096 8月  13 01:08 ../
-rw-r--r-- 1 recover recover 10791680 8月  13 01:08 bzImage
-r--r--r-- 1 recover recover    68599 8月  13 01:08 ldlinux.sys
-r--r--r-- 1 recover recover   186500 8月  13 01:08 libcom32.c32
-r--r--r-- 1 recover recover    24148 8月  13 01:08 libutil.c32
-rw-r--r-- 1 recover recover      592 8月  13 01:08 syslinux.cfg
-rw-r--r-- 1 recover recover    27104 8月  13 01:08 vesamenu.c32

But still not in hddimg file:

recover@ilab:~/os$  sudo mount ~/os/out/linux64/build/tmp/deploy/images/edison/edison-image-edison.hddimg /mnt/
[sudo] password for recover:
recover@ilab:~/os$  ll /mnt/
total 7718
drwxr-xr-x  2 root root   16384 1月   1  1970 ./
drwxr-xr-x 27 root root    4096 8月   8 06:47 ../
-rwxr-xr-x  1 root root 7445984 8月  13 01:08 bzImage*
-r-xr-xr-x  1 root root  122308 8月  13 01:08 ldlinux.c32*
-r-xr-xr-x  1 root root   69632 8月  13 01:08 ldlinux.sys*
-rwxr-xr-x  1 root root  186500 8月  13 01:08 libcom32.c32*
-rwxr-xr-x  1 root root   24148 8月  13 01:08 libutil.c32*
-rwxr-xr-x  1 root root     592 8月  13 01:08 syslinux.cfg*
-rwxr-xr-x  1 root root   27104 8月  13 01:08 vesamenu.c32*

[htot]

No, initramfs is not for acpi-tables.

We needed to build sdhc driver as module due to some power or regulator related issue since fc0f361. I forgot exactly how, maybe it's similar to how we fix i2c-6. Should be in the very long thread in the edison forum. Like 27-okt-2016: "keeping in mind regulator for eMMC and SD card detection" from @andy-shev. Maybe I understood wrong at the time, or situation changed and it's not needed anymore. But it was a big step for me to understand initramfs and get yocto to build that, so it must have seemed really necessary.

And since we then we have the initramfs, why not put the acpi-tables in there? Although loading the cpio in memory from u-boot works fine too.

[alext-mkrs]

@htot, understood, thanks, will look deeper into that then when I have time.

@lybtongji, thanks for testing. Looks like the approach I took is doing the file substitution too late, at the end of the day. And the variable they now use in live-vm-common looks too generic to mess up with directly. I'll see what else could be done here.

[alext-mkrs]

@htot, thanks, that now makes more sense to me. I wonder though if this has changed since then, I remember there were some kernel patches mentioning SDHC/MMC power or something along these lines...

@lybtongji, here comes the second try - please see the same branch (I've force-pushed the updated two commits). As they have no hooks whatsoever in that new live-vm-common implementation, the only [more or less meaningful] way I see right now is the per-recipe variable override. That means the kernel file name will need to change as well, to the one that's generated. I've checked it in a very limited way (just by building and not from scratch), but at least it now builds the hddimg with initramfs kernel.

[lybtongji]

@alext-mkrs How can I find the commit of yours?
I used the command git cherry-pick -n <SHA> to apply your commits.
But git log alext-mkrs/master --not origin/master shows nothing.
I can not find the commits in https://github.com/alext-mkrs/meta-intel-edison/commits/master either.

---

UPDATE:
These two commits ( 82ebb5d, 2b10370 ) work fine.

[htot]

@lybtongji wrong branch. You need @alext-mkrs fix-hddimg-gen branch.

@alext-mkrs I just built and flashed your fix-hddimg-gen rebased onto top of sumo64-acpi (after make clean, make setup. I used linux version of Flash Tool Lite to flash. It is working just fine, you are a wizard.

2 things could use improvement:

1. bootargs_console could use 115200n8. As is, it all boots but very quiet (nothing) until root: prompt.
2. bootargs_debug turn off acpi, even when we are building acpi U-Boot and kernel.

I propose you test this with no-acpi/32b see if it solves your boot problems.

[lybtongji]

@htot Thanks. I find it now.

[alext-mkrs]

@htot, thanks for checking this out. I haven't modified the rest of the standard U-Boot env pieces for now, just for @lybtongji to check the kernel itself. If that works I think we'll need a separate issue to do an overhaul of those. I'm currently using the "dual-booting" scheme on mine (4.16 kernel on the SD card while the official image is still on eMMC), to have a reference, so I haven't bumped into this just yet.

[htot]

@alext-mkrs I'm not complaining :-) It's more of a note to self that we add 2 more commits to these before pulling.

[htot]

@lybtongji I created PR #42 (from here https://github.com/htot/meta-intel-edison/tree/sumo32 ) that is equal to current master + fix-hddimg-gen + fixes the baud rate. If you have already tested, can you let us know the results?
I tested the sumo64-acpi version myself and that is working fine.

[lybtongji]

@htot It's fine. I have no problem with booting now. 😄

[htot]

Thanks