auxten / go-sqldb

A pure golang SQL database for database theory research

Go SQL DB

中文

"Go SQL DB" is a relational database that supports SQL queries for research purposes. The main goal is to show the basic principles and key design of a relational database to database enthusiasts. Therefore, to easily understand, a lot of tricks but not very rigorous design was adopted, and the amount of code was controlled at about 2000 lines (including 400 lines of unit tests).

Features

1. Pure Golang implementation, does not rely on any third-party packages. Goconvey was only introduced in unit tests
2. Unit test coverage ≈ 73.5%

Storage Engine

1. Special Thanks to Let's Build a Simple Database
2. Data retrieval structure based on B+Tree
3. Disk persistence engine based on 4KB paging
4. Close to POD (Plain Old Data) speed serialization & deserialization

SQL Parser

1. Tokenizer is implemented based on text/scanner
2. Support simple SELECT, INSERT syntax
   1. SELECT supports WHERE of numeric type
   2. Support LIMIT, but not support ORDER BY temporarily
3. If you want to know how the SQL Parser that can be used in the production environment is implemented, please refer to the SQL Parser that I stripped from CockroachDB and supports the SQL-2011 standard

Execution Planner

1. Select Implementation based on Volcano Model
2. HTTP-based query and insert interface

Known Issues

1. No DDL is implemented for the time being, only a fixed Schema

   struct Row {
       Id uint32
       Sex byte
       Age uint8
       Username [32]byte
       Email [128]byte
       Phone [64]byte
   }

2. For limited support for SQL syntax, see Test Cases
3. Tokenizer is based on a clever implementation of the Golang language itself, there will be problems with the support of special characters in some strings, which can be solved by quoting strings with "

How to run

1. Run

   go run . test.db

2. INSERT

   Execute INSERT INTO table (id, username, email) VALUES (10, auxten, "auxtenwpc_gmailcom")

   BY accessing: http://localhost:8080/exec?q=INSERT%20INTO%20table%20(id,%20username,%20email)%20VALUES%20(10,%20auxten,%20%22auxtenwpc_gmailcom%22)

3. SELECT

   Query SELECT * FROM table WHERE id > 3 LIMIT 10

   BY accessing: http://localhost:8080/query?q=SELECT%20*%20FROM%20table%20WHERE%20id%20%3E%203%20LIMIT%2010

Thanks

1. SQL-2011 SQL Parser
2. Marshal/Unmarshal Code generation
3. Document-oriented, embedded SQL database: genji
4. CockroachDB
5. Let's Build a Simple Database

go-sqldb's Issues

Support for DDL

Hey @auxten!
First of all congratulations on launching this project!! I'm actually excited to crate the first Issue!!
I was thinking if I could help out in implementing the DDL. I've myself implemented a simple DB in Python and although it did not support DDL, it was possible to create custom tables(all columns are strings) and the start. You can check it out here NaiveDB.
I'm a newbie to the world of Golang, and therefore would like some guidance from you on how i would go about helping you out.

error running examples (firefox only?)

when trying example i get error.

this is due to firefox being a json browser and the examples are returning plain/text but saying they are returning json.

writer.Header().Set("Content-Type", "application/json")
q := request.URL.Query()
query := q.Get("q")
if query != "" {
	var (
		ast *parser.SelectTree
	)

	p := &parser.Parser{}
	if p.GetSQLType(query) != parser.SELECT {
		_, _ = fmt.Fprintf(writer, "not a SELECT statement")
		return
	}

for all the error printing lines, this will fix;

example;
http.Error(writer, "not a SELECT statement", http.StatusBadRequest)
from
_, _ = fmt.Fprintf(writer, "not a SELECT statement")

but the data written will also have to be converted to json.

Question of execution plan

感谢你开源的项目啊，对于学习数据库很有帮助。
请教一个问题：我看测试用例 SELECT id, username, FROM table WHERE id > 5 AND id < 7 LIMIT 3 查询的时候，过程是，先遍历所有的row，再在内存中过滤，最后返回满足要求的row。 我理解的是应该会先根据where条件在B+树种过滤好再返回吧，而不是在内存中，而且也只会返回 id, username 其他的字段不会返回（这个字段过滤是在内存中吗？还是说在存储层就已经过滤了？）。还是我哪里理解错了？谢谢。

Translation of the question:

Thank you for your open source project, it is very helpful for learning database theory.
Ask a question: I see the test case SELECT id, username, FROM table WHERE id> 5 AND id <7 LIMIT 3 When querying, the process is to first traverse all rows, then filter in memory, and finally return the row that meets the requirements. . What I understand is that it should be filtered in the B-Tree based on where conditions before returning, not in memory, and only id will be returned, and other fields of username will not be returned (is this field filtered in memory? Or Say it has been filtered at the storage layer?). Or am I getting it wrong? Thank you.