Real Directory instead of OverlayFS about minimal

[ivandavidov]

Could you please be more specific and give me an example on what you actually want to achieve? It would be helpful if you could specify what your expectations are, i.e. give example of what files/directories you want to install/copy, where you'd like to install/copy these files, what behavior you expect after the copy/install process, etc.

[AwlsomeAlex]

@ivandavidov Sorry, this is what I want to happen (Or theorize at least) The computer for common distributions boots up with a vmlinuz (kernel.xz) then boots an initrd.img (rootfs.xz) then mounts the file system found on the Hard Drive (sda1, etc.) as actual directories (/home, /etc) however with the initramfs mounting the overlay (OverlayFS) it isn't mounting an actual directory, but a stored filesystem. What I would want to do on the installer is create an initramfs on an ext2 partition (sdb1, etc.) IN Minimal Linux Live, then extract the OverlayFS into that ext2 File System, and set the Initramfs to boot into that physical file system and mount everything in its specific places. So basically I want the installed Minimal Linux Live to boot into a filesystem that can be read without linux actually running, as tools exist in Windows to view ext2 file system, and if you ever wanted to recover data on that file system it was real and not an overlay directory that had to be specifically mounted. I hope that makes a little more sense, if now this is what I want to do:

Boot Order:

• Boot (vmlinuz [kernel.xz]) [Calls InitramFS]
• InitramFS (initrd.img [rootfs.xz]) [Mounts physical Directories and Initialize them {A /etc/fstab will have to be created.}]
• Actual Directories (/home, /etc, etc.) [Call on login and log user into (/root or /home/xxxx)

EDIT: Also, which Linux Distro did you model the Initialization of MLL from? I'm thinking about doing something similar to Tiny Core Linux for AwlsomeLinux, but I don't think it's as simple as creating an (/etc/fstab) and modifying the init.

[ivandavidov]

Minimal Linux Live is not based on any other live Linux OS. I got most of the inspiration from Slax but the actual implementation in MLL differs quite a lot. The MLL boot process is much simpler than the one in Slax.

One interesting feature of MLL is that the overlay structure can be placed on any supported partition. It doesn't have to be on the same device where the kernel/initramfs is located. In this way you can use one HDD partition for the kernel/initramfs and another HDD partition for the overlay structure. This will give you mixed boot structure where you use 2 HDD partitions.

If you want complete boot from another FS structure on one HDD partition only without using overlayfs, then first of all you need to understand the difference between switch_root and pivot_root. I guess you can use Google for that purpose.

Basically, all you need to do is this:

• Use fdisk or any other similar tool to create proper partition on your HDD.
• Format the partition with Ext2.
• Create the directory/file structure that you want to use. You'll have to copy the initramfs structure on your FS and then you'll have to copy the overlay structure on top of your FS as well.
• You need to use the pivot_root command to switch from initramfs to your FS and continue the boot process.

Of course, it's easier said than done. MLL only goes half the way (switch_root to overlayfs). The problem with the pivot_root call is that it can't be used from initramfs directly. You need to switch_root to another temporary FS and only after that you can call pivot_root and use your new FS structure. You don't need to go through overlayfs as an interim step but you definitely need to switch_root to something first.

I suggest you start with manual experiments and forget about the installer for a while. You need to be doing your experiments from rescue shell (sh with PID 1) or otherwise you won't be able to call the /sbin/init process in the end. Once you figure out what steps you need to do, only then you should start thinking about the steps in the installer.

Here are some useful links for starters:

http://www.slax.org/bg/blog/18438-initramfs-pivot-root-solution.html

https://landley.net/writing/rootfs-programming.html

https://linux.die.net/man/8/pivot_root

And of course the "Escaping initramfs" section here:

http://www.slax.org/en/documentation.php#internals

Yep, you need to understand all this stuff or otherwise you'll be shooting in the dark with your experiments. Building the OS is actually much simpler than booting the same OS. :)

Good luck!

[AwlsomeAlex]

@ivandavidov Wow this looks confusing, this is probably gonna never happen due to me running into this battle blind but I will experiment. If not I'll probably end up forking Tiny Core or something like that. Thanks for the help!

EDIT: I do see the confusion that I am facing, for I am looking at Tiny Core for a guide and you are looking at Slax, but I find Slax WAY to confusing to comprehend, and applaud you for simplifying a nightmare, but will probably end up sticking to a Tiny Core-like approach, as I am VERY new at this, and this type of field not containing guides, will reproduce something that exists.

[yhaenggi]

@AwlsomeAlex when you include filesystems and drivers into the kernel, you can use a minimal directory on a HDD too. The boot script will parse all disks/partitions found for a minimal folder. I had to enable firewire on my fork for specific devices, take a look there.

[ivandavidov]

Yes, MLL will traverse through all devices/partitions that the kernel can recognize. I based my approach on the work done by @ladiko for the network improvements in his fork.

@AwlsomeAlex - I recommend you take a look at ladiko's USB installer. It's a good starting point because it is quite simple and you can reuse the same approach to install MLL on HDD:

https://github.com/ladiko/minimal/blob/master/src/create_boot_stick.sh

[AwlsomeAlex]

I understand now, the overlay folder will be extracted to the Hard Drive and activated when system boots, and due to it being a folder, it will be accessible even if another Linux Operating System is used to explore it, which is exactly what Im looking for! Does that sound right? @ivandavidov @K1773R

[ivandavidov]

@AwlsomeAlex - yes, your understanding is correct. I have already implemented a script which generates virtual HDD (as "img" file) with sample overlay structure. You can attach this HDD image file in QEMU or VirtualBox which also supports the "img" format.

https://github.com/ivandavidov/minimal/blob/master/src/generate_hdd.sh

https://github.com/ivandavidov/minimal/blob/master/src/qemu32.sh

https://github.com/ivandavidov/minimal/blob/master/src/qemu64.sh

And just to be on the safe side - remove the "minimal" folder from the ISO image in order to ensure that the "minimal" folder from the HDD will be detected during the boot process.

[AwlsomeAlex]

@ivandavidov So basically all I have to do is instead of Overlay being placed into the rootfs.xz during Installation, it has to be placed in the Hard Drive in its own "overlay" directory? So basically:

/ (ISO Image)

• - isolinux (SysLinux Componients)
• - kernel.xz (Linux Kernel Archive)
• - rootfs.xz (InitramFS Archive)
• -- rootfs.xz/overlay (OverlayFS)

Install:

• Create 2 Directories on sdx (sdx1 -- boot/sdx2 -- root)
• Move Kernel and RootFS (Without Overlay) --> /sdx1/boot
• Move Overlay Directory --> /sdx2/
• Configure SysLinux & Install to /boot

/ (ext2 File System - HDD)

• - /boot {sda1 -- 32MB}
• -- /boot/isolinux (Syslinux Componients)
• -- /boot/kernel.xz (Linux Kernel Archive)
• -- /boot/rootfs.xz (InitramFS Archive)
• - /overlay (Overlay Directory) {sda2 -- xMB}
• -- /overlay/[/etc, /usr, /lib](Files in OverlayFS)

If I do something like this, will I have to create an /etc/fstab or will Busybox Init automatically identify the OverlayFS?

[ivandavidov]

@AwlsomeAlex - how in the name of Tangra (the ancient Bulgarian all-mighty God) were you able to generate both nano and ncurses as additional overlay bundles (and then test these bundles) if you don't know where the overlay stuff is located on the ISO image?

No, your understanding from your last post is not correct!!!

Take a look here:

http://www.slideshare.net/IvanDavidov4/minimal-linux-live/17

The presentation is in Bulgarian language but all we need is this particular slide. The whole overlay structure resides in the /minimal folder. I believe I have documented it quite well in the scripts. Also, sine you have worked on the overlay structure, I thought you were already familiar with it. My bad...

This is the usual ISO image structure:

ISO image
|
+-- kernel.xz (the Linux kernel)
|
+-- rootfs.xz (the initial RAM filesystem, based on BusyBox)
|
+-- /src (the folder with all MLL scripts that were used to build this particular ISO image)
|
+-- /minimal (just a folder, part of the overlay structure)
     |
     +-- /rootfs (this is where all overlay files/folders are placed)
     |
     +-- /work (this folder is empty and it is used by the OverlayFS driver)

And this is the other supported overlay format in case you are running MLL from VFAT device, e.g. USB flash:

Some random root device directory here
|
+-- kernel.xz (the Linux kernel)
|
+-- rootfs.xz (the initial RAM filesystem, based on BusyBox)
|
+-- /src (the folder with all MLL scripts that were used to build this particular ISO image)
|
+-- minimal.img (sparse image file which emulates HDD - formatted with Ext2)
    |
    +-- /rootfs (this is where all overlay files/folders are placed)
    |
    +-- /work (this folder is empty and it is used by the OverlayFS driver)

As you see, in both cases the overlay structure is not part of the initramfs structure. It is completely separate structure which can reside on the same device where MLL boot files are, or it could reside on another device. In both cases MLL will detect the overlay structure as long as the device is formatted with filesystem which is recognized by the kernel.

Q: If I do something like this, will I have to create an /etc/fstab or will Busybox Init automatically identify the OverlayFS?
A: If you do something like this (and assuming you manage to boot) you will find out that nothing happens automatically or out of the blue, just like that. You need to handle the overlay structure during the /init process (just as I do in MLL) or alternatively create /etc/fstab and put whatever content you want there.

[AwlsomeAlex]

Okay @ivandavidov , I understand now. I havent work with Overlay for a bit now, due to me rewritting the Core script, then the Overlay, and it was misunderstood on the top of my head. Thanks for clearing that out. So basically leaving the init alone will automatically detecting and mount the Overlay, but as long as it's a directory, it can be read outside of MLL.

[AwlsomeAlex]

@ivandavidov I see what happened, I confused the boot process of Tiny Core/Slax with the initialization of MLL, in other words, I just was looking past the obvious and not looking back at the source code. Sorry about that. :/