winash / kgrad

A simple Neural network framework written in Kotlin. This is largely a learning exercise for me, but I hope it can be useful to others as well, who would like a more in-depth understanding of how neural networks work.

Most of the code in the project is inspired from Torch, Micro-grad, TinyGrad and many other sources. It also serves as a companion to https://d2l.ai/ (Dive into Deep Learning) which is a great resource for learning about deep learning. Start at the Tensor class and work your way up from there.

In the code there are many references to the book eg: Here is the MLP implementation from the book in Kotlin https://d2l.ai/chapter_builders-guide/model-construction.html

 class MLP : Module() {
            private val weight = Tensor.rand(20, 20)
            private val linear = LazyLinear(20)

            override fun forward(input: Tensor): Tensor {
                var forward = linear.forward(input)
                forward = forward.mmul(weight)
                forward = forward.add(Tensor.ones(2, 20))
                var fw = forward.relu()
                fw = linear.forward(fw)
                while (fw.abs().sum().data.getDouble(0) > 1) {
                    fw = fw.div(Tensor.fillNum(2.0, *fw.data.shape()))
                }
                return fw.sum()
            }
        }

        val model = MLP()
        val tensor = model.forward(Tensor.rand(2, 20))
        println(tensor.data.getDouble(0))