I checked out origin/master and built according to your instructions.
I am not sure if this is expected behavior, hence the issue.
(1) I get an empty screen that prints out garbage bitmaps when I type.
(2) The command line gives me the following.

scorbie@home:~/Projects/lilith$ make run RELEASE=1
CR src/main.cr
LD64 build/main.o build/boot.o => build/kernel64
objcopy --output-target=binary build/kernel64 build/kernel64.bin
AS32 src/asm/bootstrap.s
i686-elf-as src/asm/bootstrap.s -o build/bootstrap.o
LD build/bootstrap.o => build/kernel
qemu-system-x86_64 -kernel build/kernel -monitor telnet:127.0.0.1:7777,server,nowait -m 512M -serial stdio -no-shutdown -no-reboot -vga std -device intel-hda,debug=9 -device hda-duplex,cad=0,debug=9
qemu-system-x86_64: warning: TCG doesn't support requested feature: CPUID.01H:ECX.vmx [bit 5]
hda-duplex: hda_audio_init: cad 0
hda-duplex: dac: format: 2 x PCM-S16 @ 48000 Hz
hda-duplex: adc: format: 2 x PCM-S16 @ 48000 Hz
hda-duplex: hda_audio_reset
hda-duplex: hda_audio_reset
intel-hda: intel_hda_update_irq: level 0 [intx]
no handler for 14
no handler for 15
no handler for 15
unknown device type!
no handler for 1
no handler for 12
no main detected.

scorbie@home:~/Projects/lilith$ make run RELEASE=1 NO_CPUEX=1
CR src/main.cr
LD64 build/main.o build/boot.o => build/kernel64
objcopy --output-target=binary build/kernel64 build/kernel64.bin
AS32 src/asm/bootstrap.s
i686-elf-as src/asm/bootstrap.s -o build/bootstrap.o
LD build/bootstrap.o => build/kernel
qemu-system-x86_64 -kernel build/kernel -monitor telnet:127.0.0.1:7777,server,nowait -m 512M -serial stdio -no-shutdown -no-reboot -vga std -device intel-hda,debug=9 -device hda-duplex,cad=0,debug=9
qemu-system-x86_64: warning: TCG doesn't support requested feature: CPUID.01H:ECX.vmx [bit 5]
hda-duplex: hda_audio_init: cad 0
hda-duplex: dac: format: 2 x PCM-S16 @ 48000 Hz
hda-duplex: adc: format: 2 x PCM-S16 @ 48000 Hz
hda-duplex: hda_audio_reset
hda-duplex: hda_audio_reset
intel-hda: intel_hda_update_irq: level 0 [intx]
no handler for 14
no handler for 15
no handler for 15
unknown device type!
no handler for 1
no handler for 12
no main detected.

While I'm pretty certain not using submodules was a design decision,
I'd like to hear about your opinion on having build dependencies such as crystal, binutils, etc... as submodules.
BTW I would need some time if I get to implement this idea.
Edit: Congrats you have 900 stars!
I'll star your repo again cause I really like it

The current incremental garbage collector/memory allocator is great and all but leaves a lot to be desired. I'm thinking of rewriting a new one.

GC

• Make it work
• Don't scan the stack/data segments in order to get the root objects
  • Time to hack the compiler again
  • LLVM provides some intrinsic functions for getting GC stack variables, might wanna use that!
• Scale well with big number of nodes
  • The newly rewritten GC does 1 cycle per allocation which doesn't scale
• Use something other than linked lists
  • The current GC stores nodes in 3 singly-linked lists for each color. This isn't great when we want to realloc or change a node's color individually.
  • Might wanna have an allocation bitmap and mark bitmap for each memory pool
• Needs integration with the scheduler so that it can be run when processor is halted

Ideas

• Store marked items in a bitmap, each bitmap associated with a pool, and each bit associated with a slot in that pool
  • Also store mark bits within another bitmap instead of a Gc header
• Store grayed objects in something else
  • a linked-list-based stack? (the current gc does this)
  • a flat fixed-size stack? (maybe we could limit the amount of gray objects being processed per cycle)

Memory allocator

• Make it work
• Allow page size allocations (>1024 bytes and < 4096 bytes - header)
• Use the allocator for userspace malloc

No pun intented and taken from Reddit

https://www.reddit.com/r/opensource/comments/dkf9h7/fully_working_posixlike_x8664_os_in_crystal/f4dxcbp/

When called from an interrupt, the memory range between the stack start and the %rsp register doesn't cover all of the local variables used by the function being interrupted.

I think this can be solved by:

• Passing the equivalent of -fno-omit-frame-pointer to LLVM (or at least doing something that forces LLVM to store local variables in an address higher than %rbp but lower than %rsp)
• Having LLVM generate stack maps and scanning the map
• Having LLVM generate precise stack information (basically last one but requires a lot of compiler patches)

Some interrupts occur within the kernel stack, mid-syscall, this might corrupt some of the local variables on critical sections of the syscall handler. Might be better to have a separate interrupt stack.

Here are some drivers I want to write:

Storage

• ATAPI support

File systems

• FAT32 driver
• ISO9660 driver

Audio

• PC speaker
• Sound Blaster 16
• Intel HDA

Networking

• NE2000

The kernel currently doesn't boot correctly when compiled in debug mode, might be because stack size is too small and LLVM doesn't inline functions as often as in release mode.

I might wanna add a flag which sets the kernel stack size to 32kb (and maybe the intrq stack size to 16kb), and maybe sprinkle in some @[AlwaysInline] annotations.

Tried to build this amazing project but unfortunately just get an error when running "make toolchain/crystal/.build/crystal" and I currently lack the knowledge to fix it..

cd toolchain/crystal && make release=1
make[1]: Entering directory '/home/nix/Source/9-OS/lilith/toolchain/crystal'
Using /usr/bin/llvm-config [version=10.0.1]
CRYSTAL_CONFIG_LIBRARY_PATH="" CRYSTAL_CONFIG_BUILD_COMMIT="ad3fae661" ./bin/crystal build --release -o .build/crystal src/compiler/crystal.cr -D without_openssl -D without_zlib
In src/gc/boehm.cr:9:1

9 | {% if flag?(:freebsd) %}
^
Warning: expanding macro

""
There was a problem expanding macro 'macro_140249756127904'

Called macro defined in src/gc/boehm.cr:9:1

9 | {% if flag?(:freebsd) %}

Which expanded to:

1 |

2 | @[Link("gc", static: true)]
Warning: specifying static linking for individual libraries is deprecated

A total of 1 warnings were found.
In src/time.cr:1322:7

1322 | in(Location.local)
^
Error: expecting identifier 'end', not 'in'
make[1]: *** [Makefile:133: .build/crystal] Error 1
make[1]: Leaving directory '/home/nix/Source/9-OS/lilith/toolchain/crystal'
make: *** [Makefile:205: toolchain/crystal/.build/crystal] Error 2
""

Used crystal from Arch Linux.
Thanks for response!

CR src/main.cr
Error: Invalid option: --mcmodel
make: *** [Makefile:47: build/main.o] Error 1

The patched version of crystal appears to have built correctly, details below:

spencerking@localhost ~/D/l/t/c/.build> ./crystal version
Crystal 0.30.0 [cbf651aff] (2019-08-30)

LLVM: 8.0.0
Default target: x86_64-unknown-linux-gnu

Not trying to be rude or anything, just thought it would be safe to do the following?

1. The wallpaper

• The original source of the painting? (Which I am genuinely curious BTW)
• Attribution to "Machikado Mazoku"? You may get a fresh peach heart shower
  I'm not sure how animation copyright works to suggest this.

2. The ISO Image?

• It contains GRUB which is GPL licensed, according to the warning here.
• Disclosure that the ISOs are under the GNU GPL, perhaps with a license?

I've found that a clean install of Fedora 30 is pretty lacking when it comes to building the project. I'm using this issue to document all of the packages I need to install as I work towards a build. Going forward it may make sense to create a nix or guix configuration, or perhaps split the custom components into their own repositories with clear documentation.

Packages:
llvm-devel
pcre-devel
libevent-devel
patch
texinfo
zlib-devel

The Process' process data is sometimes corrupted before process creation (when the PROCESS_CREATE_DRV syscall gets called). This might be because the kernel threads' stacks aren't scanned whenever it transfers an object from the global heap over.

Hello,

I would like to know how if this project is abandoned, or just paused for the moment... I loved this first time I saw it, and would love to see this working again with latest Crystal version, etc!

I sincerely hope you are ok,
aunetx

After overwriting files a couple times, the file system reads out garbage. Not sure if this is because the the cluster chain reading algorithm is implemented wrong or something to do with following it.

Either way I should probably abandon FAT16, it's only a temporary solution before a real file system is implemented, I'm thinking of doing MINIX FS.

Installing mruby package currently fails with the following message:

CC    mrbgems/mruby-pack/src/pack.c -> build/lilith/mrbgems/mruby-pack/src/pack.o
/home/scorbie/Projects/lilith/pkgs/.build/mruby/mrbgems/mruby-pack/src/pack.c: In function 'unpack_l':
/home/scorbie/Projects/lilith/pkgs/.build/mruby/mrbgems/mruby-pack/src/pack.c:248:62: error: expected ')' before 'PRIu32'
       snprintf(msg, sizeof(msg), "cannot unpack to Fixnum: %" PRIu32, ul);
                                                              ^~~~~~~
                                                              )
/home/scorbie/Projects/lilith/pkgs/.build/mruby/mrbgems/mruby-pack/src/pack.c:248:60: warning: spurious trailing '%' in format [-Wformat=]
       snprintf(msg, sizeof(msg), "cannot unpack to Fixnum: %" PRIu32, ul);
                                                            ^
/home/scorbie/Projects/lilith/pkgs/.build/mruby/mrbgems/mruby-pack/src/pack.c: In function 'unpack_q':
/home/scorbie/Projects/lilith/pkgs/.build/mruby/mrbgems/mruby-pack/src/pack.c:310:62: error: expected ')' before 'PRId64'
       snprintf(msg, sizeof(msg), "cannot unpack to Fixnum: %" PRId64, sll);
                                                              ^~~~~~~
                                                              )
/home/scorbie/Projects/lilith/pkgs/.build/mruby/mrbgems/mruby-pack/src/pack.c:310:60: warning: spurious trailing '%' in format [-Wformat=]
       snprintf(msg, sizeof(msg), "cannot unpack to Fixnum: %" PRId64, sll);
                                                            ^
/home/scorbie/Projects/lilith/pkgs/.build/mruby/mrbgems/mruby-pack/src/pack.c:316:62: error: expected ')' before 'PRIu64'
       snprintf(msg, sizeof(msg), "cannot unpack to Fixnum: %" PRIu64, ull);
                                                              ^~~~~~~
                                                              )
/home/scorbie/Projects/lilith/pkgs/.build/mruby/mrbgems/mruby-pack/src/pack.c:316:60: warning: spurious trailing '%' in format [-Wformat=]
       snprintf(msg, sizeof(msg), "cannot unpack to Fixnum: %" PRIu64, ull);
                                                            ^
/home/scorbie/Projects/lilith/pkgs/.build/mruby/mrbgems/mruby-pack/src/pack.c: In function 'unpack_l':
/home/scorbie/Projects/lilith/pkgs/.build/mruby/mrbgems/mruby-pack/src/pack.c:248:62: error: expected ')' before 'PRIu32'
       snprintf(msg, sizeof(msg), "cannot unpack to Fixnum: %" PRIu32, ul);
                                                              ^~~~~~~
                                                              )
/home/scorbie/Projects/lilith/pkgs/.build/mruby/mrbgems/mruby-pack/src/pack.c:248:60: warning: spurious trailing '%' in format [-Wformat=]
       snprintf(msg, sizeof(msg), "cannot unpack to Fixnum: %" PRIu32, ul);
                                                            ^
/home/scorbie/Projects/lilith/pkgs/.build/mruby/mrbgems/mruby-pack/src/pack.c: In function 'unpack_q':
/home/scorbie/Projects/lilith/pkgs/.build/mruby/mrbgems/mruby-pack/src/pack.c:310:62: error: expected ')' before 'PRId64'
       snprintf(msg, sizeof(msg), "cannot unpack to Fixnum: %" PRId64, sll);
                                                              ^~~~~~~
                                                              )
/home/scorbie/Projects/lilith/pkgs/.build/mruby/mrbgems/mruby-pack/src/pack.c:310:60: warning: spurious trailing '%' in format [-Wformat=]
       snprintf(msg, sizeof(msg), "cannot unpack to Fixnum: %" PRId64, sll);
                                                            ^
/home/scorbie/Projects/lilith/pkgs/.build/mruby/mrbgems/mruby-pack/src/pack.c:316:62: error: expected ')' before 'PRIu64'
       snprintf(msg, sizeof(msg), "cannot unpack to Fixnum: %" PRIu64, ull);
                                                              ^~~~~~~
                                                              )
/home/scorbie/Projects/lilith/pkgs/.build/mruby/mrbgems/mruby-pack/src/pack.c:316:60: warning: spurious trailing '%' in format [-Wformat=]
       snprintf(msg, sizeof(msg), "cannot unpack to Fixnum: %" PRIu64, ull);
                                                            ^
rake aborted!
... some other message dump ...

Which probably means that there's no inttypes.h in i386-elf-lilith's libc ? (PRIu64, PRId64, PRIu32, PRId32 is an inttypes.h macro )

Whenever we close a window (I tried with cfm), 4kb of memory isn't freed up causing a leak.

Time is displayed wrongly.

It's currently 13:27, and on the VM I have 12:27... I understand that, as it's probably taking GMT time.

But I don't understand what's up with the date... It says 00/02/2020, which is definitely wrong, as it's currently 29/01/2020.

Time captured 3min after the issue creation

Other than that, this is amazing work!

i tryed build...
[leonid@leonid-pc lilith]$ make toolchain/crystal/.build/crystal
cd toolchain/crystal && make release=1
make[1]: вход в каталог «/home/leonid/Рабочий стол/lilith/toolchain/crystal»
Using /usr/bin/llvm-config [version=9.0.0]
g++ -c -o src/llvm/ext/llvm_ext.o src/llvm/ext/llvm_ext.cc -I/usr/include -std=c++11 -fno-exceptions -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS
cc -fPIC -c -o src/ext/sigfault.o src/ext/sigfault.c
ar -rcs src/ext/libcrystal.a src/ext/sigfault.o
CRYSTAL_CONFIG_LIBRARY_PATH="" CRYSTAL_CONFIG_BUILD_COMMIT="9e28451fb" ./bin/crystal build --release -o .build/crystal src/compiler/crystal.cr -D without_openssl -D without_zlib
cc: ошибка: стол/lilith/toolchain/crystal/src/ext/libcrystal.a: no such file or directory
Error: execution of command failed with code: 1: cc "${@}" -o '/home/leonid/.cache/crystal/home-leonid-Рабочий стол-lilith-toolchain-crystal-src-ecr-process.cr/macro_run' -rdynamic -lpcre -lm -lgc -lpthread /home/leonid/Рабочий стол/lilith/toolchain/crystal/src/ext/libcrystal.a -levent -lrt -ldl -L/usr/lib -L/usr/local/lib
make[1]: *** [Makefile:133: .build/crystal] error 1
make[1]: выход из каталога «/home/leonid/Рабочий стол/lilith/toolchain/crystal»
make: *** [Makefile:204: toolchain/crystal/.build/crystal] error 2

I managed to build the thing! Yay!
But I had to patch little tweaks and edits that seemed to belong in origin/master.
Here are the two things I remember, although there might be something more I might have forgot.

1. userspace/toolchain/Makefile:50-51 : ../libc -> ../libraries/libc
   I assume @ffwff refactored the project.
   Relevant git diff:

diff --git a/userspace/toolchain/Makefile b/userspace/toolchain/Makefile
index 964ee0b..f9c2e6d 100644
--- a/userspace/toolchain/Makefile
+++ b/userspace/toolchain/Makefile
@@ -47,8 +47,8 @@ build_gcc: gcc
        make install
 
 # libc
-build_libc: ../libc
-       cd ../libc && \
+build_libc: ../libraries/libc
+       cd ../libraries/libc && \
        make -j$(NPROC) RELEASE=$(RELEASE) && \
        make install \
                LIBDIR=$(shell pwd)/tools/lib/gcc/i386-elf-lilith/8.3.0/ \

2. pkgs/generate-packages.sh pkgs/missio userspace/libraries/libcrystal/compile
   Shebang line is #!/bin/sh while the script uses bash conditionals (i.e. [[ ]])
   Not sure to make it a sh script or a bash script so raised an issue rather than a hasty pull request. If you pick one of the two, I'm happy to make a pull request.

i try compile os

$ make toolchain/crystal/.build/crystal
cd toolchain/crystal && make release=1
make[1]: вход в каталог «/home/leonid/lilith/toolchain/crystal»
Makefile:69: *** Could not locate llvm-config, make sure it is installed and in your PATH, or set LLVM_CONFIG

Hello,

When trying to build the userspace (with cd userspace/toolchain and then make), binutils are correctly patched and built but gcc fails with an error.

The commit id 4ac50a4913e does not seems to be correct, and git checkout 4ac50a4913e (userspace/toolchain/Makefile line 21) fails with the error:
error: le spécificateur de chemin '4ac50a4913e' ne correspond à aucun fichier connu de git (in french, literally the specified commit does not exist in the current git directory)

With that error, the compiler cannot be patched and built. Any idea of where could this come from ? Or what is the correct commit id (or git branch) to use so the patch works ?

Sorry if that's a newbie question, and excuse me for the English grammar

Release: 0.0.5
Qemu options: qemu-system-x86_64 -hda disk.img -vga std -m 512M -serial stdio

I cd'd into /share/papes and ran pape pape.png and the OS froze and my machine's fans spun up rapidly.

Here is qemu's output in my terminal:

rbp=2d0 
rdi=2
rsi=0
r15=100000080
r14=ffffffff
r13=100002130
r12=2d000000500
r11=246
r10=0
r9=0
r8=17
rdx=4414fc
rcx=0
rbx=1
rax=17
int_no=6
errcode=0
rip=4375a0
cs=2b
rflags=202
userrsp=7ffffffd88
ss=23
process: pape
6 0