rishavbhowmik / xdb Goto Github PK
View Code? Open in Web Editor NEWExperimental storage engine for building databases.
Experimental storage engine for building databases.
rust-clippy-analyze
CodeQL Action v1 will be deprecated on December 7th, 2022. Please upgrade to v2. For more information, see https://github.blog/changelog/2022-04-27-code-scanning-deprecation-of-codeql-action-v1/
Check here
xdb/.github/workflows/rust.yml
Line 93 in d3fc621
Line 456 in 34510a6
read_block(
&mut self,
block_index: BlockIndex,
data_start_index: Option<BlockLength>,
data_end_index: Option<BlockLength>
) {
// ...
}
Make log traversal more efficient. No need to read full segment payload while traversing.
Errors in the code base are not standardised so far.
All Errors should be collected in a single file for each sub-package
Error {
code: 500, // HTTP like error code
message: format!(
"{}/{} - {}",
MODULE_NAME,
"<error_brief>",
"<error_description>"
)
}
5c9792ae0edfe8233987fea622c94084747883e4
):We are using Cargo workspace to manage multiple packages(library or binary) in this repo.
[workspace]
members = [
"util", # Dir: /util, Description: Reusable functions and structs, such as custom error struct
"storage", # Dir: /storage, Description: Module to manage storage file with blocks of fixed size. ie, the primitive storage engine
"logchain", # Dir: /logchain, Description: Abstract function for Storage struct, to store data as a chain of blocks.
"index", # Dir: /index, Description: Inbuild HashMapIndex, BtreeIndex, etc... logic buit for logchain, along with kv store(Manage CRUD for keyvalue store).
]
block
: A segment of fixed size on storage file.
block_index
: The serial index of a block on storage file.
block_data
: The data of a block on storage file.
block_header
: The header of a block on storage file.
block_len
: The length of a block on storage file.
log
/ logchain
: A chain of blocks. (This one makes me cry)
kv
: module to manage Key-value store on block storage.
tupple
: A tuple in Key-value store.
block
is a segment of fixed size on storage file (we have tell this docs). But a block can be any entity of any size.logchain
This is a crazy meaningless name.index
module, we have kv
module, which is a has primitive function to manage crud for KV store. So, kv
module deserves to be a separate library, and should not be restructed to inbuilt index data structues present in index
module.Happens
(which denotes to expected errors which can be part of the logic). This brings in requirement to perform error handling, which may not suit dependency injection as good practice. We have used this extensively in index
module, so we must reconsider this.page
: A segment of fixed size on storage file. (Rename from block
)
page_index
: The serial index of a page on storage file.
page_payload
: The data intended to be stored in a page.
page_header
: Initial bytes of a page to store little-endian representation for size of page_payload in bytes.
page_padding
: Unused space of a page.
page_body
: All the bytes inside a page. page_header + page_payload + page_padding
pagestore
: Module to manage storage file with pages of fixed size. ie, the primitive storage engine. (Rename from storage
)
pagelist
: A chain of pages. (Rename from logchain
)
Follow convention
More about pages
About firebirdDB
[workspace]
members = [
"util", # {Error, Logger, Cursor}
"storage_engine/page_store", # {PagePayload, PageStore, PageStoreHeader, PageIndex}
"storage_engine/page_list", # {create, append, delete}
"storage_engine/kv_serializer", # {serialize_kv, deserialize_kv, serialize_kv_list, deserialize_kv_list}
"storage_engine/index_traits", # {IndexOpration, UnqiueIndexOpration, IndexSerializer, IndexClone}
# More in future
"kv_db_lite",
"kv_db",
"document_db_lite",
"document_db",
"graph_db",
]
/// Returns data: Vec<u8>
pub fn read_log(
storage: &mut Storage,
start_segment_block_index: BlockIndex
) -> Result<Vec<u8>, Error> {
// traverse log
// return log data as Vec<u8>
}
logchain/tests/logchain_storage_flows.rs
Tracking issue for:
store and test set of indexes in Storage as logs.
Line 275 in d3fc621
fn make_chunks(data, chunk_len) -> (blocks_required, chunks);
Originally posted by @rishavbhowmik in #5 (comment)
Line 598 in 14cd92d
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.