bitlap / bitlap

:rocket: A table schema-less OLAP Analytics Engine for Big Data.

Welcome to bitlap 👋

🏠 Homepage

✨ Demo

username:root, no password

Author

👤 IceMimosa

• Github: @IceMimosa

👤 jxnu-liguobin

• Github: @jxnu-liguobin

Code Contributors

This project exists thanks to all the people who contribute. [Contribute].

🤝 Contributing

Contributions, issues and feature requests are welcome!
Feel free to check issues page. You can also take a look at the contributing guide.

Show your support

Give a ⭐️ if this project helped you!

📝 License

Copyright © 2023 bitlap.
This project is Apache License 2.0 licensed.

---

This README was generated with ❤️ by readme-md-generator

bitlap's Issues

Add CSV load sql command

• Load SQL command

load data 'xxx.csv' into table <table_name>

Support sql `use database`

测试覆盖提升

Task list:

• bitlap-core
• bitlap-common
• bitlap-server
• bitlap-testkit
• bitlap-client 忽略
• bitlap-network
• bitlap-cli 忽略

Implement getConnection() method of `BitlapDataSource`

pool ?

Improve serialize/deserialize col_val

SQL supports some simple cases.

CLI binary install

unit-test for core code

The issue should be closed only when the unit-test of the main code of all modules is completed.
So the issue is a long-term track record.

Use enumeratum instead of Scala enum

implementation of `BitlapConf` & `BitlapConfKey`

Reference to spark's ConfigBuilder：https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/internal/config/Tests.scala

BitlapConfKey is designed as follows, and the priority is the same as below：

• with name and group to get value from bitlap.setting configuration
• with sys to get value from system properties, default is bitlap.${name}
• with env to get value from system environment, default is BITLAP_${upper_trans_dot(name)}
• default value
• value data type
• value validator
• conf key version
• ......

Add `stop` or `exit` command for bitlap CLI

It is similar to https://github.com/bitlap/bitlap/issues/331

Query failed `count (distinct pv) as uv`

描述错误
简明扼要地描述该错误是什么。

重现

select os, sum (vv) as vv, sum (pv) as pv, count (distinct pv) as uv
from bitlap_test_table
where _time >= 0
group by os

删掉count (distinct pv) as uv，正常。

预期的行为
简明扼要地描述你期望发生的情况。

屏幕截图

版本（请填写以下信息）：

• 操作系统 [例如：iOS]
• 浏览器 [例如：Chrome, safari]
• 版本 [例如：22]

额外的背景
在此添加关于问题的任何其他背景。

支持Spark SQL集成

Add documents

Write standard documents for public methods or class to produce kotlin documents that can be deployed online.

Add operators for query metric rows without dimensions.

implementation of EventBus system(pub-sub)

• consider kotlin coroutines
• design simple bitlap event system

Split bitmap codes from bitlap-common

Add bitlap-raft

We will copy zio-raft into bitlap, while bitlap is ready to release, move it out.

use sofa-core

NOTE: We should fix raft code, there is no need to run multiple raft servers in one service.

支持取消查询计划

Refactor CLI in the future , use `zio` technology stack

1. cli => zio-cli
2. http => zio-http
3. ftp => zio-ftp
4. s3 => zio-s3
5. stream => zio-stream

remove java librarys

Implementation of `BitlapCli`

Implementation of BitlapCli, it hasn't started yet.

• Based on sqlLine project
• Working with BitlapDriver #36

Add NOTICE

try https://github.com/Jasig/maven-notice-plugin

[JDBC] Next optimization

1. check connection and session before using
2. implemnet type mapping： bytes -> long,bytes -> boolean, bytes -> short
3. check session lock while invoke with session
4. use connection pool

Support to query with multiple dimensions

Spark3 integration

Fix codefactor issue

https://www.codefactor.io/repository/github/bitlap/bitlap/issues

实现 `getTables` 和 `getDatabases`方法

getColumns ?

Synchronize RoaringBitmap(RBM) to the latest commit

current commit is: fc3754f

Support multiple-hosts for JDBC

zio dependency upgrade to 2.x

Generate fake data

Column type is not equal to data type.

Support PreparedStatement

Add scalafix

But maven plugin for scalafix seems to be bad.

client and jdbc

1. client
2. jdbc

Client wrapper protocol, JDBC uses client to execute SQL

Integrate `bitlap-client`, provide a webpage to query data

Add new module named: zim-bitlap or zim-data

Introducing codecov

Provide test coverage report during pr

Add a unified code formatting tool for the JVM lang

spotless maven

Support load data overwrite

• Currently load data 'xx' into table xxx; is supported.

• The code below needs support.

load data 'xx' overwrite table xxx

• PS: carbondata may not support

Split `Processable` macro from `scala-macro-tools`

1. Create a new module named bitlap-macro
2. Move scala-macro-tools to bitlap organization

Add operators for query metric rows with one dimension.

Integrate with Kodein-DI framework

Add assembly.xml to zip package

CBM support BigByteArray

Reference：

• how to store large data 2gb in java

Add some arguments which parsed from URI

• query timeout
• session timeout
• retry times
• initFile
• and so on

Load data by ftp

Support load data by FTP.

Refactor `server`, `network`, use `zio-grpc` and `zio-raft` to replace `sofa.jraft`

https://github.com/zio/zio
https://github.com/ariskk/zio-raft
https://github.com/scalapb/zio-grpc

NOTE: Provisionally, the client uses synchronous calls.

• network
• server
• client
• raft

查询时支持指定自定义的数据库Schema

Steps:

• 1.open session: initialize session, set database into session context (current database)
• 2.execute query: get database from session context, all operations should be handled with current database
• 3.use database;, will update session context too, all operations also should be handled with current database

Implement the remaining types of the JDBC API

Implement `BitlapDriver`

Refer to the Java library sqlite-jdbc to implement BitlapDriver.