daniel-boll / scylla-javascript-driver

🚀 JavaScript driver for ScyllaDB, harnessing Rust's power through napi-rs for top performance. Pre-release stage. 🧪🔧

🚀 JavaScript driver for ScyllaDB. Pre-release stage. 🧪🔧

⚠️ Disclaimer ⚠️

This repository and the associated npm package are currently in a 🐣 pre-release state and are being used for testing 🧪 purposes. They are subject to change without notice 📝. Eventually, they will be moved to the ExpressoTS organization under the scylladb-driver package name. Users are encouraged to use this driver with caution ❗ and not in production environments until the official release under the ExpressoTS organization.

🚀 Getting Started 🚀

These instructions will get you a copy of the project up and running 🏃 on your local machine for development and testing purposes.

📋 Prerequisites 📋

• Docker: We use Docker 🐳 to run the Scylla database easily without the need for a complex local setup.
• Node.js: Make sure you have Node.js installed on your system to run JavaScript code.

🌟 Quickstart 🌟

1. Start ScyllaDB in Docker:

   Run a ScyllaDB instance using the following Docker command:

   docker run --name scylladb -d --rm -it -p 9042:9042 scylladb/scylla --smp 2

   This command pulls the Scylla image if it's not already present on your system, and starts a new 🌟 container with the Scylla database.

2. Use the JavaScript Driver:

   Here's a simple script to connect to the database and execute a query:

   import { Cluster } from "@lambda-group/scylladb";
   
   const cluster = new Cluster({
     nodes: ["127.0.0.1:9042"],
   });
   
   const session = await cluster.connect("system_schema");
   
   const result = await session
     .execute("SELECT * FROM scylla_tables limit ?", [1])
     .catch((err) => console.error(err));
   
   console.log(result);

   This script connects to the ScyllaDB instance running on your machine, performs a query, and logs the result.

📥 Installing 📥

To install this package, use the following command:

npm install @lambda-group/scylladb

🙏 Acknowledgments 🙏

• Thanks to the developers of ScyllaDB for creating such a high-performance database.
• Thanks to the Rust community for providing the robust scylla crate.
• Thanks to the napi-rs project for enabling efficient Rust and Node.js integrations.

scylla-javascript-driver's Issues

Query feature: Use Keyspace (Not Fully Documented Issue)

Todo

I still need to document this issue

https://rust-driver.docs.scylladb.com/stable/queries/usekeyspace.html

Integrate TLS Support in ScyllaDB Driver Connection

Overview:
This issue focuses on the integration of TLS (Transport Layer Security) into the ScyllaDB driver, enhancing the security of data transmitted between the client and ScyllaDB nodes. This feature is based on the Scylla Rust Driver's TLS documentation and is a follow-up to issue #2, "Enhance connect Method to Support Authentication Parameters," since it involves modifications to the connection parameters.

Detailed Behavior:
The TLS functionality should allow users to establish secure connections by specifying an ssl object in the connection parameters. This object will contain:

1. The file path to the trusted certificate authority (CA) certificate (in PEM format).
2. The SSL verify mode, dictated by an enum with options NONE (no verification) or PEER (verify the peer's certificate).

The proposed API change could look like this:

const session = await cluster.connect({
  keyspace: "keyspace",
  ssl: {
    caFilepath: "/path/to/ca.cert.pem",
    verifyMode: VerifyMode.PEER, // VerifyMode.NONE or VerifyMode.PEER
  },
});

Dependencies:

• This task is dependent on issue #2 since both require changes to the connect method parameters. The implementation should ensure compatibility with the authentication parameters introduced in issue #2.
• The task requires modifications to the project's CI configuration to include the OpenSSL dependency in the GitHub Actions' containers.

Implementation Considerations:

• The driver should be capable of loading the CA file from the provided path and setting the verification mode accordingly.
• Proper error handling should be implemented for scenarios like invalid paths, unreadable CA files, or unsupported verification modes.
• Changes to the GitHub Actions pipeline must be made to ensure that the OpenSSL dependency is available during CI builds.

Environment:

• This update will affect all environments and is crucial for users who transmit sensitive data or operate under strict security compliance requirements.

Related Issues or Discussions:

• Related to issue #2: "Enhance connect Method to Support Authentication Parameters."

Labels:

• enhancement
• security
• CI/CD

Assignees and Mentions:

• To be assigned to contributors with experience in Node.js, Rust, and CI/CD configurations, or those who worked on issue #2. Relevant contributors can be '@' mentioned or assigned.

Impact:

• Providing TLS support is critical for securing data in transit, especially for applications that handle sensitive information or operate in regulated industries.

Progress/Updates:

• This issue serves as the initial task creation. Progress will be tracked through comments, including discussions, decisions, and implementation phases.

Query feature: PreparedStatement (Not Fully Documented Issue)

TODO

I still need to document this issue

https://rust-driver.docs.scylladb.com/stable/queries/prepared.html

Enhance `connect` Method to Support Authentication Parameters

Overview:
This task aims to extend the connect method's functionality within the ScyllaDB driver to support authentication, following the Scylla Rust Driver documentation. The enhancement is crucial for applications requiring secure access to ScyllaDB clusters through authentication credentials.

Detailed Behavior:
The proposed enhancement will modify the connect function to accept multiple types, allowing for an optional authentication object. The authentication object will contain username and password fields for ScyllaDB authentication. This change aims to offer flexibility for establishing authenticated sessions without altering the method's current behavior when authentication details are not provided.

The method should support the following invocations:

1. Connecting with just the keyspace (current behavior):

   const session = await cluster.connect("keyspace");

2. Connecting with keyspace and an authentication object (new behavior):

   const session = await cluster.connect("keyspace", {
     auth: {
       username: "username",
       password: "password",
     },
   });

3. Connecting with an options object that can contain keyspace and authentication details (new behavior):

   const session = await cluster.connect({
     keyspace: "keyspace",
     auth: {
       username: "username",
       password: "password",
     },
   });

Implementation Considerations:

• The function should perform a check to determine the parameter's type(s) passed to the connect method and behave accordingly.
• If introducing multiple types to the connect function proves problematic, the function should be transitioned to use the latter approach (option 3 above), and the initial approach (option 1) should be deprecated with clear communication to the users about this change.
• Proper error handling should be implemented for scenarios where authentication fails or inputs are invalid.

Assignees and Mentions:

• Depending on who has relevant experience or interest, they can be '@' mentioned or assigned to this task.

Impact:

• This task is crucial for users who require secure connections to their ScyllaDB clusters, ensuring that their databases are protected against unauthorized access.

Progress/Updates:

• This issue marks the initiation of this task. Updates on the progress, including discussions, decisions, and implementation phases, will be documented in this thread.

Query feature: Paged Query (Not Fully Documented Issue)

Todo

I still need to document this issue

https://rust-driver.docs.scylladb.com/stable/queries/paged.html

Resolve OpenSSL Configuration Issue on Windows for ScyllaDB Node.js Driver

While working on the ScyllaDB Node.js driver written in Rust, I encountered a build issue on Windows platform related to OpenSSL configuration during the build process. The problem arose when executing cargo build --release. Below is the relevant information about the platform and error log:

Platform Information:

napi:build Platform: {
  "platform": "win32",
  "arch": "x64",
  "abi": "msvc",
  "platformArchABI": "win32-x64-msvc",
  "raw": "x86_64-pc-windows-msvc"
}
napi:build Dylib name: scylladb_driver

Error Log:

error: failed to run custom build command for `openssl-sys v0.9.93`

Caused by:
  process didn't exit successfully: `D:\a\scylla-javascript-driver\scylla-javascript-driver\target\release\build\openssl-sys-3852be1711edf4c9\build-script-main` (exit code: 101)
  --- stdout
  cargo:rerun-if-env-changed=X86_64_PC_WINDOWS_MSVC_OPENSSL_NO_VENDOR
  X86_64_PC_WINDOWS_MSVC_OPENSSL_NO_VENDOR unset
  cargo:rerun-if-env-changed=OPENSSL_NO_VENDOR
  OPENSSL_NO_VENDOR unset
  running "perl" "./Configure" "--prefix=D:\\a\\scylla-javascript-driver\\scylla-javascript-driver\\target\\release\\build\\openssl-sys-debfca1904808b02\\out\\openssl-build\\install" "--openssldir=SYS$MANAGER:[OPENSSL]" "no-dso" "no-shared" "no-ssl3" "no-tests" "no-comp" "no-zlib" "no-zlib-dynamic" "--libdir=lib" "no-md2" "no-rc5" "no-weak-ssl-ciphers" "no-camellia" "no-idea" "no-seed" "no-capieng" "no-asm" "VC-WIN64A"

  --- stderr
  Can't locate Locale/Maketext/Simple.pm in @INC (you may need to install the Locale::Maketext::Simple module) (@INC contains: /d/a/scylla-javascript-driver/scylla-javascript-driver/target/release/build/openssl-sys-debfca1904808b02/out/openssl-build/build/src/util/perl /usr/lib/perl5/site_perl /usr/share/perl5/site_perl /usr/lib/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib/perl5/core_perl /usr/share/perl5/core_perl /d/a/scylla-javascript-driver/scylla-javascript-driver/target/release/build/openssl-sys-debfca1904808b02/out/openssl-build/build/src/external/perl/Text-Template-1.56/lib) at /usr/share/perl5/core_perl/Params/Check.pm line 6.
  BEGIN failed--compilation aborted at /usr/share/perl5/core_perl/Params/Check.pm line 6.
  Compilation failed in require at /usr/share/perl5/core_perl/IPC/Cmd.pm line 59.
  BEGIN failed--compilation aborted at /usr/share/perl5/core_perl/IPC/Cmd.pm line 59.
  Compilation failed in require at /d/a/scylla-javascript-driver/scylla-javascript-driver/target/release/build/openssl-sys-debfca1904808b02/out/openssl-build/build/src/util/perl/OpenSSL/config.pm line 19.
  BEGIN failed--compilation aborted at /d/a/scylla-javascript-driver/scylla-javascript-driver/target/release/build/openssl-sys-debfca1904808b02/out/openssl-build/build/src/util/perl/OpenSSL/config.pm line 19.
  Compilation failed in require at ./Configure line 23.
  BEGIN failed--compilation aborted at ./Configure line 23.
  thread 'main' panicked at C:\Users\runneradmin\.cargo\registry\src\index.crates.io-6f17d22bba15001f\openssl-src-300.1.5+3.1.3\src\lib.rs:577:9:

  Error configuring OpenSSL build:
      Command: "perl" "./Configure" "--prefix=D:\\a\\scylla-javascript-driver\\scylla-javascript-driver\\target\\release\\build\\openssl-sys-debfca1904808b02\\out\\openssl-build\\install" "--openssldir=SYS$MANAGER:[OPENSSL]" "no-dso" "no-shared" "no-ssl3" "no-tests" "no-comp" "no-zlib" "no-zlib-dynamic" "--libdir=lib" "no-md2" "no-rc5" "no-weak-ssl-ciphers" "no-camellia" "no-idea" "no-seed" "no-capieng" "no-asm" "VC-WIN64A"
      Exit status: exit code: 2

This error seems to originate from a Perl script being unable to find certain Perl modules while trying to configure OpenSSL. The error prevents the successful compilation and build of the ScyllaDB Node.js driver on Windows platform. It's crucial to resolve this issue to ensure Windows support for the driver.

This issue is open for further investigation and contributions to resolve the problem and re-establish Windows support for the ScyllaDB Node.js driver.

Implement Basic `Query` Class Functionality for Simple Queries

Overview:
This issue outlines the need for the implementation of the Query class within our ScyllaDB driver, focusing exclusively on enabling simple queries as per the Scylla Rust Driver documentation. This initial implementation will serve as the foundation for executing basic CQL commands, with more complex query capabilities to be added in subsequent enhancements.

Detailed Behavior:
The primary objective is to provide users with the ability to perform simple CQL queries against their ScyllaDB instances. The Query class should encapsulate the logic required to construct and execute these queries. Once this class is implemented, the session object's query method should be able to accept a CQL query string, send it to the ScyllaDB cluster, and handle the response appropriately.

Here's a conceptual example of how the query method might be used with the Query class:

import { Cluster, Query } from "@lambda-group/scylladb";

const cluster = new Cluster({ nodes: ["127.0.0.1:9042"] });
const session = await cluster.connect("keyspace");

const query = new Query("SELECT * FROM my_table WHERE id = ?", [1]);
const result = await session.query(query);

Scope Limitation:
It's important to note that this issue strictly pertains to the simplest use case of the Query class. Features like prepared statements, batch queries, and paginated queries are out of scope for this specific task and will be addressed in future issues.

Implementation Considerations:

• The Query class should be designed with extensibility in mind, as additional functionality will be added in the future.
• Proper error handling should be implemented to address scenarios like invalid CQL syntax, issues with the connection to the database, and server-side errors returned by ScyllaDB.
• The result set returned by the query method should be in a format that's easy to use and understand from a developer's perspective.

Related Issues or Discussions:

• This is a standalone issue, but it lays the groundwork for future enhancements to the Query class.

Assignees and Mentions:

• Contributors with experience in Node.js and familiarity with ScyllaDB or CQL should be considered for assignment. They can be '@' mentioned or assigned directly.

Impact:

• This basic implementation of the Query class is a critical step towards a fully functional ScyllaDB driver. It will enable developers to execute CQL commands from their applications using the driver.

Progress/Updates:

• This issue marks the beginning of this feature. Updates on development progress, including discussions, decisions made, code reviews, and phases of implementation, will be documented in this thread.

Contributors should also ensure that this feature is accompanied by appropriate unit tests to verify correct functionality and by updated documentation that explains how to perform simple queries using the new Query class.

Implement `Metrics Retrieval` Functionality in ScyllaDB Driver

Overview:
This issue proposes the implementation of a metrics retrieval feature within the ScyllaDB driver, closely following the guidelines and behavior found in the Scylla Rust Driver's metrics documentation. This feature is essential for monitoring the performance of database operations, providing users with valuable insights into various metrics like query counts, error rates, and latency statistics.

Detailed Behavior:
The feature will introduce a Metrics class in JavaScript, which will provide methods to retrieve various operational metrics from a ScyllaDB session. Unlike the Rust implementation that uses direct attribute access, the JavaScript Metrics class will include getter methods for these attributes, ensuring consistency with common JS practices.

Here's a conceptual representation of what the JavaScript API might look like:

const metrics = session.getMetrics();

console.log(`Queries requested: ${metrics.getQueriesNum()}`);
console.log(`Iter queries requested: ${metrics.getQueriesIterNum()}`);
console.log(`Errors occurred: ${metrics.getErrorsNum()}`);
console.log(`Iter errors occurred: ${metrics.getErrorsIterNum()}`);
console.log(`Average latency: ${metrics.getLatencyAvgMs()}`);
console.log(`99.9 latency percentile: ${metrics.getLatencyPercentileMs(99.9)}`);

Each getter function in the Metrics class will correspond to a specific metric, allowing users to retrieve and log/monitor these statistics within their applications.

Implementation Considerations:

• The Metrics class should be designed with scalability in mind, allowing for the easy addition of new metrics getters if the ScyllaDB Rust driver's metrics module is updated in the future.
• Proper error handling should be integrated, particularly for methods like getLatencyAvgMs and getLatencyPercentileMs, which may involve complex calculations and potential error states.
• The feature's implementation should consider asynchronous behavior, as metrics retrieval may require communication with the ScyllaDB cluster, potentially necessitating the use of Promises or async/await syntax.

Assignees and Mentions:

• Ideal assignees would be contributors with experience in both Rust and JavaScript, especially those familiar with database performance metrics. They can be '@' mentioned or assigned directly.

Impact:

• The integration of a metrics retrieval feature is crucial for users needing to monitor the health and performance of their ScyllaDB operations, directly influencing maintenance and optimization decisions.

Progress/Updates:

• This issue signifies the beginning of this feature's journey. Updates, including development progress, discussions, decisions, and implementation stages, will be shared in this thread.

Contributors are encouraged to include comprehensive testing to ensure the accuracy and reliability of the metrics retrieved, as well as to provide clear documentation detailing the available metrics and demonstrating how to retrieve and interpret them.

Implement Crate Feature to Disable SSL on Windows for ScyllaDB Node.js Driver

The driver is currently facing a build issue on Windows due to problems encountered with OpenSSL configuration. To work around this issue and re-establish Windows support, this task proposes the introduction of a crate feature that will disable SSL on Windows platforms. This will simplify the build process and ensure functionality, albeit without SSL support for the time being.

Here's a breakdown of the task:

1. Feature Flag Creation:
   • Create a new feature flag, e.g., disable-ssl, in the Cargo.toml file.
   • Ensure this feature is conditional for Windows platforms only.

[features]
disable-ssl = ["cfg(windows)"]

2. Conditional Compilation and Dependency Exclusion:
   • Use Rust's conditional compilation attributes to disable the SSL-related code and dependencies when the disable-ssl feature is enabled.
   • Adjust the dependency declarations in Cargo.toml to exclude SSL dependencies on Windows.

#[cfg(not(feature = "disable-ssl"))]
extern crate openssl;

[dependencies]
scylla = { version = "0.10.1", features = ["ssl"], optional = true }
openssl = { version = "0.10", features = ["vendored"], optional = true }

[target.'cfg(windows)'.dependencies]
scylla = { version = "0.10.1", default-features = false } # There might be more things necessary

3. Documentation:

   • Update the project documentation to explain the new feature flag and its implications.
   • Include clear instructions for Windows users on how to use this feature to build the project, while making them aware of the temporary lack of SSL support.

4. Testing:

   • Ensure that the project builds successfully on Windows with the disable-ssl feature enabled.
   • Test the functionality of the driver on Windows to ensure it operates as expected without SSL.

5. Future Work:

   • Mention that this is a temporary workaround, and that a more robust solution to fix the OpenSSL issue on Windows and re-enable SSL support is in the pipeline.

This issue serves as a call for contributions to implement this feature, and invites discussion on any alternative solutions or potential implications. By addressing the OpenSSL configuration problem in a phased manner, we can promptly restore Windows compatibility for the ScyllaDB Node.js driver while working towards a complete solution.

Query feature: BatchStatement (Not Fully Documented Issue)

Todo

I still need to document this issue

https://rust-driver.docs.scylladb.com/stable/queries/batch.html

Fetch schema (Not Fully Documented Issue)

Todo

I still need to document this issue

https://rust-driver.docs.scylladb.com/stable/schema/schema.html

Query feature: Timeouts (Not Fully Documented Issue)

Todo

I still need to document this issue

https://rust-driver.docs.scylladb.com/stable/queries/timeouts.html

Query feature: Lightweight Transactions (Not Fully Documented Issue)

Todo

I still need to document this issue

https://rust-driver.docs.scylladb.com/stable/queries/lwt.html

Implement Optional Compression Methods in Cluster Constructor

Overview:
This issue proposes the enhancement of adding optional compression methods to the Cluster constructor, aligning with the Scylla Rust Driver's capabilities as documented here. The feature should allow users to specify their preferred compression (LZ4, Snappy, or None) when establishing a connection to a Scylla cluster.

Detailed Behavior:
The new feature will expand the Cluster constructor's functionality by introducing a compression parameter. This parameter will accept an enum, Compression, that includes the options NONE (default), LZ4, and SNAPPY. The specified compression algorithm will be used in the communication between the driver and Scylla nodes, potentially reducing network traffic and improving performance, especially when large volumes of data are being transferred.

Proposed Implementation:
The implementation will leverage napi-rs to bridge the Rust Scylla driver's capabilities with Node.js. Here's a preliminary look at how the API might be structured:

import { Cluster, Compression } from "@lambda-group/scylladb";

const cluster = new Cluster({
  nodes: ["127.0.0.1:9042"],
  compression: Compression.LZ4, // options: Compression.NONE (default), Compression.LZ4, Compression.SNAPPY
});

Assignees and Mentions:

• TBD depending on who is interested or has the relevant expertise. '@' mention or assign the issue to them.

Progress/Updates:

• This is the initial issue creation. Updates will follow as discussions proceed, decisions are made, and implementation begins.

Query feature: Schema Agreement (Not Fully Documented Issue)

Todo

I still need to document this issue

https://rust-driver.docs.scylladb.com/stable/queries/schema-agreement.html