jasperav / catalytic

🚀 Catalytic is an ORM for ScyllaDb and Cassandra which provides zero-cost abstractions for querying and generating Rust structs based on the database. Scylla and Cassandra are both NoSQL databases which can scale horizontally and hold large amounts of data.

Catalytic: ScyllaDB and Cassandra Object-Relation Mapper

This library contains several crates with the following features:

• Automatic map tables to Rust structs: catalytic_table_to_struct. See Usage. See the generated dir for the generated code.
• Compile time checked queries: catalytic_macro (catalytic_macro::query). See Usage.
• Plugin derive macro to generate business logic: catalytic_macro (catalytic_macro::mirror)
• Automatic JSON mapping. When there is a column with type text, you can implement the Transformer trait and map it to a type that implements serde::Serialize and serde::Deserialize
• All queries are executed as a prepared statement
• Support for Materialized Views (and mapping between the base table if the columns are the same)

Not all types are supported yet due to scylladb/scylla-rust-driver#104

Query types

Depending on a query, a certain query type can be derived. These can be found here. These are the commonly used query types:

• SelectMultiple: Can be used as an iterator to iterate over the rows
• SelectUnique: Selects an optional unique row by full primary key
• SelectUniqueExpect: Same as SelectUnique, but fails if the row doesn't exist
• SelectUniqueExpect with Count as entity type: has the special select_count method for queries like "select count(*) from ..."

There are also structs for CRUD operations.

Usage

Automatic map tables to Rust

You can see an example on how to generate Rust structs in the example dir.

• In the build.rs file the Transformer trait is implemented and used for json mapping
• In the generated dir you can see the generated structs

How to implement it yourself (step-by-step guide how to mimic the example crate):

• Add a build-dependency: catalytic_table_to_struct = "0.1"
• Create a build.rs file
• Optionally implement the Tranformer trait (or use DefaultTransformer)
• Call catalytic_table_to_struct::generate
• Build the project

An example of the structure of the generated code for table 'child' is:

• Child: a struct with owned values. Can be converted to ChildRef, PrimaryKey and PrimaryKeyRef
• ChildRef: a struct where the values are all borrowed. This struct can perform insertions. Can be converted to Child, PrimaryKey and PrimaryKeyRef
• PrimaryKey: a struct which holds the owned values of the primary key fields of Child. Can be converted to PrimaryKeyRef.
• PrimaryKeyRef: a struct where the values are all borrowed. This struct can perform updates, deletions and selects. Can be converted to PrimaryKey
• UpdatableColumn: an enum which holds the enum variants with owned values of columns that can be updated. Can be converted to UpdatableColumnRef.
• UpdatableColumnRef: an enum where the values are all borrowed. This is used by PrimaryKeyRef for updates. Can be converted to UpdatableColumn.

Macros

Crate catalytic_macro holds several macros that can be used to ensure compile/type checked queries. Examples on how this can be used can be found in lib.rs, method qmd.

How to implement it yourself:

• Add dependency: catalytic_macro = "0.1"
• The query macro returns the correct query type for the given query
• The query_base_table macro transforms a select query to the materialized view table, to a select query of the base table
• mirror and primary_key can be used for other derive macros

Used in production

https://www.beezleapp.com/ - Beezle Social allows users to connect which are nearby, discover local events now

catalytic's Issues

Handling tables with column `type`

Howdy!

With catalytic_table_to_struct, is it possible to rename column names? I'm messing around with a very large database with various type columns.

How can I rename them to, say, r#type?

Upload to crates

This can be done if the following issues are closed:

• scylladb/scylla-rust-driver#261
• scylladb/scylla-rust-driver#297
• scylladb/scylla-rust-driver#308
• scylladb/scylla-rust-driver#309
• scylladb/scylla-rust-driver#332

When above issues are closed:

1. Check that https://github.com/Jasperav/scylla-rust-driver is the same as the original driver
2. Remove https://github.com/Jasperav/scylla-rust-driver
3. Update to the crates scylla rust driver in this crate
4. Push to crates
5. Add a reference to this crate in the scylla rust driver repo

Re-export scylla driver

Because else a mismatch can easily occur in Scylla versions

CI has Trybuild: Overwrite env variable

Because the output is different than the local compiler although they are using the same version. Check with rust 1.56 if this is still different

Can't compile Catalytic 0.1.20 version with scylla_macros 0.2.2

Hello, thank you for this great crate.

My issue is suddenly, I can't compile this crate, I get this error:

failed to resolve: could not find LegacySerializedValues in _macro_internal
--> ../.cargo/registry/src/index.crates.io-6f17d22bba15001f/catalytic-0.1.20/src/table_metadata.rs:34:17
|
34 | #[derive(Debug, scylla::ValueList, scylla::FromRow)]
| ^^^^^^^^^^^^^^^^^
| |
| could not find LegacySerializedValues in _macro_internal
| help: a struct with a similar name exists: SerializedValues
|
= note: this error originates in the derive macro scylla::ValueList (in Nightly builds, run with -Z macro-backtrace for more info)

Problem might be related to scylla_macros crate. I could compile until yesterday. I compared Cargo.lock files and only difference was scylla_macros crate version is now 0.2.2, which was 0.2.1 until yesterday. 0.2.2 version came yesterday.

RUSTSEC-2021-0141: dotenv is Unmaintained

Hi,

It looks dotenv crate is no longer maintained which cargo audit complaints about.
Hopefully there is an up to date fork dotenvy.

Is it possible to replace it with the maintained one?

People can't get answer to their questions(dotenv-rs/dotenv#74, dotenv-rs/dotenv#87) about future of this crate.

And, thanks for your valuable effort.

Fix SerializedValues::with_capacity

When scylladb/scylla-rust-driver#385 is fixed

`catalytic_table_to_struct` can't make some materialized views

At work, we have a table api_keys_by_id, with a materialized view api_keys_by_orgId. However, when catalytic_table_to_struct creates the MV, it doesn't do much. Here's the generated code, shortened a little:

// Generated file
pub const SELECT_ALL_QUERY: &str = "select  from api_keys_by_orgId";
pub const SELECT_ALL_COUNT_QUERY: &str = "select count(*) from api_keys_by_orgId";
pub const SELECT_UNIQUE_QUERY: &str = "select  from api_keys_by_orgId where ";
#[derive(
    scylla :: FromRow, scylla :: ValueList, catalytic_macro :: Mirror, Debug, Clone, PartialEq,
)]
pub struct ApiKeysByOrgId {}
impl ApiKeysByOrgId {
    pub fn primary_key(&self) -> PrimaryKeyRef {
        PrimaryKeyRef {}
    }
    pub fn primary_key_owned(&self) -> PrimaryKey {
        self.primary_key().into_owned()
    }
    pub fn into_primary_key_owned(self) -> PrimaryKey {
        PrimaryKey {}
    }
}

This is the get_columns function:

select
    column_name,
    kind,
    position,
    type as data_type
from system_schema.columns
where keyspace_name = 'spore'
and table_name = 'api_keys_by_orgId';

and it's response:

User defined types (UDT) Types are supported?

SCYLLA-UDT
I'm creating the udt types on build and specify the struct

UDT = Team, Teams and League
~build.rs


src/types_helper/betwise.rs

Do I need to use the text type on creation of table for columns that use UDT?

Docs Improvements

Hey! I'm Daniel, Developer Advocate at ScyllaDB and I truly liked your project! It's my first time trying Rust and the way that Catalytic works is fascinating!

I'm wondering if there's any way of me contributing to docs improvements on this project. I mean, it's a great tool but there's not so much information on how to use it and for me from a few hours ago it would be real important :p

In any case I'm here to offer some help to struct some cool documentation for this project! What do you think?

Cheers!

Macro-based code generation instead of using `build.rs`

Hi,

First, thanks for your amazing crate. Although I really enjoy it, there's some notable difficulities while using it, mostly with the build.rs codegen.

My proposal for this problem is creating a macros that take similar data like the build.rs (query, transformer,...) and user would just need to derive that one trait. This idea is somewhat similar to sea-orm's DeriveEntityModel.

primary_key_owned should call primary_key().to_owned()

That way it can take &self instead of self

Is the project gonna keep being maintained?

I am using ScyllaDB with Leptos and I would love to use Catalytic. There doesn't seem to be much activity here. Is it a good idea to start using the project?

Implement TIMESTAMP/TIMEOUT for inserts

The ORM does not support TIMESTAMP and TIMEOUTs for inserts yet: https://docs.scylladb.com/getting-started/dml/#insert-statement.

This can be implemented by following these steps:

1. Add the new method name in query_ident.rs. The query ident mod can be used by other users to interact with always up to date methods, so they never have to type the method names themselves and run into run time errors
2. Add the new insert method somewhere in write_struct.rs, after the insert method (see line 177).
3. Add a test in lib.rs. I want each new change to be tested.

To see the changes of the generated code easily:

1. Change a line somewhere in build.rs. (there is no cargo:rerun-if-changed). This should trigger re-generation of the structs. The structs are in version control, so it should be easy to see what has been changed.
2. Run the test in lib.rs.