octonica / clickhouseclient

ClickHouse .NET Core driver

ClickHouse .NET Core driver

This is an implementation of .NET Core driver for ClickHouse in a form of ADO.NET DbProvider API. This driver supports all ADO.NET features (with some exclusions like transaction support).

Features

• supports binary protocol
• compression (send and recieve)
• timezones
• most clickhouse column types are supported (aggregating ones are under development)
• full support for .net async ADO.NET API
• no unsafe code
• tested- used in production
• c# named tuple and record support
• Dapper support (example in #19)
• Linq To DB support

Usage

Install from NuGet:

dotnet add package Octonica.ClickHouseClient

ConnectionString syntax: Host=<host>;Port=<port>;Database=<db>;Password=<pass>, e.g. "Host=127.0.0.1;Password=P@ssw0rd; Database=db additionally, if you want to build a connection string via code you can use ClickHouseConnectionStringBuilder.

Entry point for API is ADO .NET DbConnection Class: Octonica.ClickHouse.ClickHouseConnection.

Extended API

In order to provide non-ADO.NET complaint data manipulation functionality, proprietary ClickHouseColumnWriter API exists. Entry point for API is ClickHouseConnection.CreateColumnWriter() method.

Simple SELECT async verison

var sb = new ClickHouseConnectionStringBuilder();
sb.Host = "127.0.0.1";
using var conn = new ClickHouseConnection(sb);
await conn.OpenAsync();
var currentUser = await conn.CreateCommand("select currentUser()").ExecuteScalarAsync();

Insert data with parameters

var sb = new ClickHouseConnectionStringBuilder();
sb.Host = "127.0.0.1";
using var conn = new ClickHouseConnection(sb);
conn.Open();
using var cmd = conn.CreateCommand("INSERT INTO table_you_just_created SELECT {id}, {dt}");
cmd.Parameters.AddWithValue("id", Guid.NewGuid());
cmd.Parameters.AddWithValue("dt", DateTime.Now, System.Data.DbType.DateTime);
var _ = cmd.ExecuteNonQuery();

For more information see Parameters.

Bulk insert

using var conn = new ClickHouseConnection("Host=127.0.0.1");
conn.Open();
using var cmd = conn.CreateCommand("CREATE TABLE IF NOT EXISTS table_with_two_fields(id Int32, name String) engine Memory");
await cmd.ExecuteNonQueryAsync();

//generate values
List<int> ids = Enumerable.Range(1, 10_000).ToList();
List<string> names = ids.Select(i => $"Name #{i}").ToList();

//insert data
await using (var writer = await conn.CreateColumnWriterAsync("insert into table_with_two_fields(id, name) values", default))
{
	await writer.WriteTableAsync(new object[] { ids, names }, ids.Count, default);
}

Build requirements

In order to build the driver you need to have .NET SDK 5.0 or higher.

clickhouseclient's Issues

The time zone with the identifier 'UTC' was not found

Hi and thanks for the library!

The library fails for me on Open when it tries to get server time zone, which is UTC in my configuration.
The exception is:

System.TimeZoneNotFoundException: The time zone with the identifier 'UTC' was not found.
   at Octonica.ClickHouseClient.Utils.TimeZoneHelper.GetTimeZoneInfo(String timeZone)
   at Octonica.ClickHouseClient.Types.DateTimeTypeInfo.Configure(ClickHouseServerInfo serverInfo)

I checked the failing lines:

if (!TimeZoneInfo.TryConvertIanaIdToWindowsId(timeZone, out windowsId))
   throw new TimeZoneNotFoundException("The time zone with the identifier '" + timeZone + "' was not found.");

and indeed TryConvertIanaIdToWindowsId doesn't find UTC on my Windows 10 system (running on .Net 6).

The

timeZoneInfo = TimeZoneInfo.FindSystemTimeZoneById(windowsId)

works though, so I guess instead of throwing an exception we could fallback to FindSystemTimeZoneById and fail if it can't find timezone too?

Unexpected NullReferenceException during connection closing.

Hi there!

I have an error:

System.NullReferenceException
at Octonica.ClickHouseClient.ClickHouseTcpClient.Dispose()
at Octonica.ClickHouseClient.ClickHouseConnection.Dispose(Boolean disposing)
at System.ComponentModel.Component.Finalize()

It throws from time to time and I can't understand why but I think null check in Dispose() would be nice anyway.

Something like that:

public void Dispose()
{
    _semaphore.Dispose();
    _reader.Dispose();
    _writer.Dispose();

    if(_client.Client != null)
        _client.Client.Close(0);
    
    _client.Dispose();
}

Add support for UInt128, Int128, UInt256, Int256

Any value of these types can be converted to System.Numerics.BigInteger.

Throws exception "The value is null" when tries to get an empty string

I have not-null column named "http_method" with string type. Some values in that column can be empty string.
When I try to get value of that column on some iteraton of read loop, it throws the "The value is null" exception.

Actual behaviour: Throws exception

Expected behaviour: Returns string.Empty.

Connection issue

Hello. I'm trying to open connect via simple operation connection.Open() and always receive this error:

System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
 ---> Octonica.ClickHouseClient.Exceptions.ClickHouseException: Internal error. Not supported message code (00000048) received from the server.
   at Octonica.ClickHouseClient.ClickHouseBinaryProtocolReader.ReadMessage(Boolean async, CancellationToken cancellationToken)
   at Octonica.ClickHouseClient.ClickHouseConnection.Open(Boolean async, CancellationToken cancellationToken)
   at Octonica.ClickHouseClient.Utils.TaskHelper.WaitNonAsyncTask(ValueTask task)
   at Octonica.ClickHouseClient.ClickHouseConnection.Open()

There are no problems with clickhouse. Another library is working fine with same connection parameters.

ClickHouseDataReader.GetSchemaTable method not implemented

bulk insert support?

Hi @SergeyMirvoda!
ClickHouseClient support bulk insert?

ClickHouse.Ado bulk instert 1000000 records use 1.8 second! it is so fast!

if ClickHouseClient can support it too,it will be very nice.

and thank you for your dapper par @ support in 1.1.9! good job.

Dapper par error

using Dapper;
using Octonica.ClickHouseClient;

using var conn = new ClickHouseConnection(".....");
conn.Open();
var sql = "insert into people values(@id,@name)";
conn.Execute(sql, new { id = 1, name = "lina" });
var data = conn.Query("select * from people");

Octonica.ClickHouseClient.Exceptions.ClickHouseServerException:“DB::Exception: Cannot parse expression of type Int32 here: @id,@name)”

Add support for Decimal256

Decimal256(S) is an alias for Decimal(76, S). Currently all values of type Decimal(P, S) are converted to decimal and there is no intention to change it.

But Decimal256 and Decimal128 can store values which are bigger than the max value of decimal. It's not clear which .NET type could be used for big decimal values. Maybe it should be some custom type.

Длительное ожидание во время получения данных из БД

@SergeyMirvoda
Добрый день.
Я сейчас пробую различные варианты работы и наткнулся на такую особенность.
Выполняю запрос
SELECT id from replacing_table where has({p}, uniq_id), где в качестве {p} передаю массив из 100000 элементов.
Первые 40000 возвращаются очень быстро. Затем идёт какое-то ожидание (несколько минут) в строке await reader.ReadAsync(). Затем опять быстро получаются следующие 40000 записей и опять ожидание. Подскажите, с чем это связано и как можно решить эту проблему?

Cannot find implementation of GetUInt* methods in reader.

This document declarates about implemented support of unsigned types with certain get methods such as GetUInt32() etc. In fact, there are no implementations of this methods.

use datetime in bulk load under dotnet 6.0

When working under dotnet 6.0 with date fields, a timezone error appears. When the same code is running under dotnet 5.0, such an error does not occur.

TCP over HTTP advantages

Hi. I'm sorry for asking the question here.
What are the advantages of TCP over the HTTP ClickHouse interface? (performance, reliability, security, maintainability, support of different formats like parquet and arrow, streaming support)
As I understand, It is much harder to support Native format and interface because of lack of specification. Most likely, Apache Parquet+ HTTP does not lose in speed to the native interface.
Why have you decided to implement the library on the native interface? Thank you in advance.

Very slow "await reader.ReadAsync()"

Hello. It takes me a very long time to read the data. At the same time, the code in the loop practically does not affect the execution speed.

await using ClickHouseDataReader reader = await cmd.ExecuteReaderAsync(); while (await reader.ReadAsync()) { ... get data }

For example this python code. it works much faster. 3-4 times faster. The request text is the same.

result = client.execute(query)

Why not use ConfigureAwait(false) ?

ConfigureAwait(false) can improve performance and avoid deadlocks like microsoft's article said.
article link

Send large table to server by parts

ClickHouseColumnWriter sends an entire table to the server in one message. It works slow for a large tables.
Splitting one big message into several smaller messages will increase performance.

Error when use in

Code

using var cnn = new ClickHouseConnection(settings.ToString());

await cnn.OpenAsync().ConfigureAwait(false);

var cmd = cnn.CreateCommand("SELECT * FROM cktest WHERE id IN ({ids})");
cmd.Parameters.AddWithValue("ids", new long[] { 1, 2 });

var res = await cmd.ExecuteReaderAsync().ConfigureAwait(false);

Error

Octonica.ClickHouseClient.Exceptions.ClickHouseServerException : DB::Exception: Table test._b518ecd58bd34345b613ae87297fa501 doesn't exist

[Compatibility] add `ClickHouseConnection(string connectionString)` constructor

While connection constructor with signature .ctor(string connectionName) is not a part of ADO.NET contract, it is a common thing for provider to have it (I actually don't know any mature provider without such constructor).

This is quite important for integration with libraries (e.g. ORM) that use dynamic provider instantiation using reflection as they could expect such constructor.

Add support for Date32

From the documentation:

[Date32 value is] stored in four bytes as the number of days since 1925-01-01. Allows storing values till 2283-11-11.

Bool type not supported in data reader

ClickHouseException: The type "Bool" is not supported.
         at Octonica.ClickHouseClient.ClickHouseCommand.ExecuteDbDataReader(CommandBehavior behavior, Boolean ignoreProfileEvents, Boolean async, CancellationToken cancellationToken)
         at Octonica.ClickHouseClient.Utils.TaskHelper.WaitNonAsyncTask[T](ValueTask`1 task)
         at Octonica.ClickHouseClient.ClickHouseCommand.ExecuteDbDataReader(CommandBehavior behavior)

Array UUID not parsed by client ExecuteNonQueryAsync by ParameterizedArray(UUID)

Hello sorry for my bad language.

Example:

[Test]
public async Task ShouldInsertParameterizedArray()
{
    await connection.ExecuteStatementAsync("TRUNCATE TABLE IF EXISTS test.uuid_array");
    await connection.ExecuteStatementAsync(
        "CREATE TABLE IF NOT EXISTS test.uuid_array (arr Array(UUID)) ENGINE TinyLog");

    var command = connection.CreateCommand();
    command.AddParameter("values", new[] {Guid.NewGuid(), Guid.NewGuid(),});
    command.CommandText = "INSERT INTO test.uuid_array VALUES ({values:Array(UUID)})";
    await command.ExecuteNonQueryAsync();

    var count = await connection.ExecuteScalarAsync("SELECT COUNT(*) FROM test.uuid_array");
    Assert.AreEqual(1, count);
}

Exception:

ClickHouse.Client.ClickHouseServerException (0x0000001B): Code: 27. DB::ParsingException: Cannot parse input: expected '\'' before: 'toUUID(\'8a22df62-aaae-4aff-b552-1a28168cd9a2\'),toUUID(\'e1f8dd19-ca37-47b8-8db1-ae66f3d6a246\')]':  at row 0: While executing ValuesBlockInputFormat. (CANNOT_PARSE_INPUT_ASSERTION_FAILED) (version 22.1.2.2 (official build))

   at ClickHouse.Client.ADO.ClickHouseConnection.HandleError(HttpResponseMessage response, String query) in C:\Projects\ClickHouse.Client\ClickHouse.Client\ADO\ClickHouseConnection.cs:line 252
   at ClickHouse.Client.ADO.ClickHouseConnection.PostSqlQueryAsync(String sqlQuery, CancellationToken token, ClickHouseParameterCollection parameters) in C:\Projects\ClickHouse.Client\ClickHouse.Client\ADO\ClickHouseConnection.cs:line 242
   at ClickHouse.Client.ADO.ClickHouseCommand.ExecuteNonQueryAsync(CancellationToken cancellationToken) in C:\Projects\ClickHouse.Client\ClickHouse.Client\ADO\ClickHouseCommand.cs:line 64
   at ClickHouse.Client.Tests.ParameterizedInsertTests.ShouldInsertParameterizedArray()

I don't know may be this problem on clickhouse http interface because if you try do this:

It's work !!!

And you can delete toUUID('uuid')

And this worked

---

In my current solution, i just replace Guid[] to String[]:

if (parameter.Value is Guid[] guidsValue)
    parameter.Value = guidsValue.Select(e => e.ToString()).ToArray();

// and
 command.CommandText = "INSERT INTO test.uuid_array VALUES ({values:Array(String)})"

It resolve problem, but it's bad solution.

Please change logic on Guid[] in:

• HttpParameterFormatter
• InlineParameterFormatter

for format result will be equals:

[toUUID('4953ecd0-d212-4828-8cc5-11dd386f78ca')] => ['4953ecd0-d212-4828-8cc5-11dd386f78ca']

SSL certificate

need support ssl certificate root. Connection yandex cloud requires ssl connection certificate

INSERT INTO VALUES error

Hi, I got error when I run the code like

using var cnn = new ClickHouseConnection(settings.ToString());

await cnn.OpenAsync().ConfigureAwait(false);

using var cmd = cnn.CreateCommand("INSERT INTO cktest VALUES ({id:Int64}, {name:String})");

cmd.Parameters.AddWithValue("id", 1);
cmd.Parameters.AddWithValue("name", "name");

var res = await cmd.ExecuteNonQueryAsync().ConfigureAwait(false);

Error:

Octonica.ClickHouseClient.Exceptions.ClickHouseServerException : DB::Exception: Element of set in IN, VALUES or LIMIT or aggregate function parameter is not a constant expression (result column not found): CAST(__subquery_11347167823684683694_3925735668227058289, 'Int64'): While executing ValuesBlockInputFormat

I found a issues that said "Unfortunately, ClickHouse server doesn't support parameters in VALUES (...)."(#19 (comment))
but I could get right result when i used clickhouse client cli, like

clickhouse-client --user=default --password --param_id=1 --param_name="name" -q "INSERT INTO default.cktest VALUES ({id:Int64}, {name:String})"

Are you sure the clickhouse not support parameters in VALUES?

GetOrdinal should throw IndexOutOfRangeException

According to IDataRecord API GetOrdinal should throw an exception, as for now, it returns -1.

Such an exception is not very helpful.
Also in the exception message, it is nice to have a column name.

using var writer = await conn.CreateColumnWriterAsync("INSERT into t2 VALUES", token);
var columns = new object?[writer.FieldCount];
var index = columns[writer.GetOrdinal("the_middle_of_nowhere")]; //should throw

Replace query parameters with their values

Currently parameters' values are passed along with the query as a table with the only one row. In the query each parameter is replaced with the SELECT subquery (see implementation details). However, there are some placed in the query where such subqueries are not allowed (#42, #46, #48).

Another way to pass parameters is implemented in the ClickHouse command-line client. This client substitutes parameters with their values. Similar behavior could be implemented in the ClickHouseClient as well.

In order to keep a backward compatibility the current behavior should remain the default behavior of ClickHouseClient. Which means that that the new behavior must be explicitly requested by the client code.

The strategy for passing parameters to the query can be defined at three levels:

1. Parameter. Affects only this parameter;
2. Command. Affects all command's parameters;
3. Connection. Affects all parameters in all commands executed with this connection.

Illegal type of column for parameter

Hi!

I have Clickhouse table with some fields. One of them is

I'm trying to perform select statement:

var sb = new ClickHouseConnectionStringBuilder();
sb.Host = "127.0.0.1";
using var conn = new ClickHouseConnection(sb);
conn.Open();
using var cmd = conn.CreateCommand("SELECT * FROM MyTable WHERE {UserId}");
cmd.Parameters.AddWithValue("UserId", 1);
cmd.ExecuteReaderAsync()
....

Application sends proper parameter type

But after all that client throws exception:

DB::Exception: Illegal type Int32 of column for constant filter. Must be UInt8 or Nullable(UInt8).

Table-valued parameters

It's possible to pass a set of values as an array. But some operations with arrays are much slower than operation with tables.

Here is an example of slow running query (from #22):

using var cmd = conn.CreateCommand("select toInt32(number) from numbers(100000) where has({p}, toInt32(number))");
var array = Enumerable.Range(0, 100_000).ToArray();
cmd.Parameters.AddWithValue("p", array);

I assume that this query could work much faster with a table passed as a parameter's value.

select toInt32(number) from numbers(100000) where number in temp_table.p

In this example temp_table is the name of the table and p is the name of the column of this table.

Exception on progress message

When executing a long-term selection
Clickhouse version 20.9.3.45

Unhandled exception. System.FormatException: One of the identified items was in an invalid format.
   at Octonica.ClickHouseClient.ClickHouseBinaryProtocolReader.Read7BitInt32(Boolean async, CancellationToken cancellationToken)
   at Octonica.ClickHouseClient.Protocol.ServerProgressMessage.Read(ClickHouseBinaryProtocolReader reader, Boolean async, CancellationToken cancellationToken)
   at Octonica.ClickHouseClient.ClickHouseBinaryProtocolReader.ReadMessage(Boolean throwOnUnknownMessage, Boolean async, CancellationToken cancellationToken)
   at Octonica.ClickHouseClient.ClickHouseTcpClient.Session.WithCancellationToken[T](CancellationToken token, Func`2 execute)
   at Octonica.ClickHouseClient.ClickHouseTcpClient.Session.ReadMessage(Boolean async, CancellationToken cancellationToken)
   at Octonica.ClickHouseClient.ClickHouseDataReader.Read(Boolean nextResult, Boolean async, CancellationToken cancellationToken)
   at Octonica.ClickHouseClient.ClickHouseDataReader.Read(Boolean nextResult, Boolean async, CancellationToken cancellationToken)
...

Question about the performance in concurrency operation case

Hi, I want to use clickhouse in a distributed system and run clickhouse server in sharding mode. But I didn't find the code about connection pool, so I'm confused about the performance in high concurrency operation case. Whether the sufficient performance just need one connection ?
I'm not good at English, thanks for your reading.

Map(key, value) type support

hello,
Is Map(key, value) type supported?
It looks like a very important feature. Please provider help.

Command behavior "SequentialAccess, CloseConnection" not supported with Dapper Query

Hi!

killwort/ClickHouse-Net#53
I have the same issue with ClickHouseClient.

Thank you!

Remove hardcoded default database name

The default database is defined in the server's configuration file. It may be not default.

Производительность при вставке

Добрый день.
Выполняю bulk insert. Таблица из 700 колонок. Кол-во записей в одной вставке 50000. Время вставки ~ 50 секунд.
При этом, clickhouse-клиент, который работает поверх http то же самое делает ~ 15 сек.
Подскажите пожалуйста, существуют ли какие-либо настройки, позволяющие увеличить производительность insert`ов?

Метод, который вызывается для insert`ов:

public async Task BulkInsert(string tableName, string[] columnNames, object[][] rows)
        {
            var sw = new Stopwatch();
            sw.Start();
            
            var list = new List<object[]>();
            for (int i = 0; i < columnNames.Length; i++)
            {
                list.Add(rows.Select(x => x[i]).ToArray());
            }
            
            Console.WriteLine($"provider - {sw.Elapsed}");
            
            await using var connection = await CreateConnection();
            await using var writer = await connection.CreateColumnWriterAsync(
                $"INSERT INTO {tableName}({string.Join(",", columnNames)}) values", CancellationToken.None);
            await writer.WriteTableAsync(list, rows.Length, CancellationToken.None);
            
            Console.WriteLine($"provider - {sw.Elapsed}");
        }

Support "where in" queries

I've got a query:

select * from data where p1 in ({p1s}) and p2 in ({p2s}) and p3 in ({p3s})

And this code:

cmd.Parameters.AddWithValue("p1s", string.Join(",", p1s.Distinct()), DbType.String);

Don't work. Is there a way to run this query?

Error from Dapper with DateTime column in clickhouse

I'm providing an example implementation to show the error

create table test ( aDate DateTime)
Engine = MergeTree()
Order by aDate;

insert into test (aDate) values ('2022-01-01 12:13:34');

public class TestError
{
public DateTime ADate { get; set; }

public static IEnumerable GetData()
{
var sql = "select aDate as ADate from test";
using var connection = new ClickHouseConnection($"Host={host};Database={db};");
connection.Conn.Query(sql);
}
}

This results in the following error:

System.Data.DataException: Error parsing column 0 (ADate=01/01/2022 12:13:34 +00:00 - Object)
---> System.InvalidCastException: Object must implement IConvertible.
at System.Convert.ChangeType(Object value, Type conversionType, IFormatProvider provider)
at Deserializeadc23d33-683a-47b3-93b7-b6c5eebe8df8(IDataReader )
--- End of inner exception stack trace ---
at Dapper.SqlMapper.ThrowDataException(Exception ex, Int32 index, IDataReader reader, Object value) in /_/Dapper/SqlMapper.cs:line 3706
at Deserializeadc23d33-683a-47b3-93b7-b6c5eebe8df8(IDataReader )
at Dapper.SqlMapper.QueryImpl[T](IDbConnection cnn, CommandDefinition command, Type effectiveType)+MoveNext()
at System.Collections.Generic.List1..ctor(IEnumerable1 collection)
at System.Linq.Enumerable.ToList[TSource](IEnumerable1 source) at Dapper.SqlMapper.Query[T](IDbConnection cnn, String sql, Object param, IDbTransaction transaction, Boolean buffered, Nullable1 commandTimeout, Nullable`1 commandType)

This query works fine with a different clickhouse ADO provider using Dapper.

I'm using:
Octonica.ClickHouseClient 2.2.8
Dapper 2.0.123
.Net 6.0

Query returns properly with: ClickHouse.Client 4.2.2

Push new version nuget, please

Exception TODO: Implement support for overflow

CH version 20.8.3.18 on Ubuntu 18.04

LowCardinality(T) Type Support

Simply speaking LowCardinality feature is the engine hint to treat type column of T as a dictionary.

From the driver perspective LowCardinality(T) is simply T e.g. LowCardinality(String) is String.

As for now, driver throws NotSupported Exception

enum type support

Looks like enum type is a very important feature.

As for me by default enums should be treated like c# string type when insert and select.
Also, the driver should provide some helper class to register c# enums to allow usage of suck enum values in parameter's value like in npgsql.

Cannot perform simple select with integers

Attempt 1:

using var reader = connection.ExecuteReader($"SELECT number FROM numbers(1000000)");
while (reader.Read())
    reader.GetInt32(0);

Result: Unable to cast object of type 'System.UInt64' to type 'System.Int32'

Attempt 2:

using var reader = connection.ExecuteReader($"SELECT number FROM numbers(1000000)");
while (reader.Read())
    reader.GetInt64(0);

Result: Unable to cast object of type 'System.UInt64' to type 'System.Int64'.

While ADO.NET spec does not requre implicit conversion (e.g. UInt64 => Int32), other clients do handle such situation

.NET 6 DateOnly and TimeZoneInfo improvements

Improvements of the API for working with dates, times, and time zones were announced in the article Date, Time, and Time Zone Enhancements in .NET 6.

There are at least two changes that could be made for the .NET 6 version of ClickHouseClient:

1. Map the ClickHouse type Date to the .NET type DateOnly instead of DateTime. It's going to be a breaking change;
2. Use new TimeZoneInfo methods to convert a time zone from Windows to IANA and vice versa. This change is not going to affect the ClickHouseClient's public API.

Вставка Guid-значения

Добрый день.
Вставляю запись с Guid = 74D47928-2423-4FE2-AD45-82E296BF6058
Получаю данные из таблицы, а в ней Guid = 2879D474-2324-E24F-AD45-82E296BF6058
Последние две части совпадают, а в первых трёх числа смещены.
Подскажите, в чём проблема?

Mutations Error

Hi, i got error when i use mutations operation delete like

using var cnn = new ClickHouseConnection(settings.ToString());

await cnn.OpenAsync().ConfigureAwait(false);

using var cmd = cnn.CreateCommand("INSERT INTO cktest SELECT @id, @name, 1");

cmd.Parameters.AddWithValue("id", 1);
cmd.Parameters.AddWithValue("name", "ok");

var text = cmd.CommandText;

var res = await cmd.ExecuteNonQueryAsync().ConfigureAwait(false);

using var cmd2 = cnn.CreateCommand("ALTER TABLE cktest delete WHERE id=@id");

cmd2.Parameters.AddWithValue("id", 1);

res = await cmd2.ExecuteNonQueryAsync().ConfigureAwait(false);

Error

Data still exist after i run detele code and I found mutations logs use "SELECT database,table, command, create_time, is_done FROM system.mutations LIMIT 10" , I got result like

database	table	command	create_time	is_done
default	cktest	DELETE WHERE id = (SELECT _d18374be9ae14385a55461be204f5a71.id FROM _d18374be9ae14385a55461be204f5a71)	2022-03-20 17:17:45	0

the operation blocked and it may be because the error command

I got right result when i run "ALTER TABLE cktest DELETE WHERE id=1" in cli

The mutations logs like

database	table	command	create_time	is_done
default	cktest	DELETE WHERE id = 1	2022-03-20 17:18:25	1

Clickhouse Server Version: v22.2.3.5

Support arrays in parameter's value

The type Array(T) is supported, but there is no way to pass array as the parameter.

Bulk insert Array(LowCardinality(String)) not working

I'm running into an issue when I try to bulk insert data that contains a Array(LowCardinality(String)) column. I have the following code

await connection.OpenAsync();

await using var cmd = connection.CreateCommand("CREATE TABLE IF NOT EXISTS table_with_cardinality(id Int32, data Array(LowCardinality(String))) engine Memory");
await cmd.ExecuteNonQueryAsync();

var columns = new Dictionary<string, object?>()
{
    ["id"] = Enumerable.Range(1, 10).ToList(),
    ["data"] = Enumerable.Range(1, 10).Select(o => new [] {"test1", "test2"}),
};

await using var writer = await connection.CreateColumnWriterAsync("insert into table_with_cardinality(id,data) values", default);
await writer.WriteTableAsync(columns, 10, default);

The code executes without exceptions, when I look into the table no data is added. If I run the exact same code, and change the Array(LowCardinality(String)) to Array(String) I see data in the table. Am I doing something wrong or this is something that is not working properly?

There is no supertype for types String, UUID because some of them are String/FixedString and some of them are not

DB::Exception: There is no supertype for types String, UUID because some of them are String/FixedString and some of them are not
Octonica.ClickHouseClient.Exceptions.ClickHouseServerException: DB::Exception: There is no supertype for types String, UUID because some of them are String/FixedString and some of them are not

support WITH TOTALS

We should support WITH TOTALS modifier for SELECT statement.

Looks like NextResult is an appropriate API for this feature.
FYI same issue in JDBC
ClickHouse/clickhouse-java#161

Here is the sample scenario

 var sb = new ClickHouseConnectionStringBuilder();
sb.Host = "click";
sb.User = "default";
var conn = new ClickHouseConnection(sb);
conn.Open();
var cmd = conn.CreateCommand("select * from system.one group by dummy");
var r = cmd.ExecuteNonQuery();
Assert.True(r == 1);

//with totals modifier ignored by the driver
cmd = conn.CreateCommand("select * from system.one group by dummy with totals");
r = (byte)cmd.ExecuteNonQuery();
using var reader = cmd.ExecuteReader();
var rows = 0;
while (reader.Read())
{
  rows++;
}
Assert.True(rows == 1);//one resulting row 
reader.NextResult();//throws System.NotImplementedException for now
while (reader.Read())
{
  rows++;//one totals row
}
Assert.True(rows == 2);

Передача массива в качестве параметра

@victor-sushko, @SergeyMirvoda
Добрый день.
Подскажите пожалуйста, имеется ли возможность передать массив в качестве параметра?

Пробую такой вариант:
await using var cmd = connection.CreateCommand("select * from numbers(100) where number in {p}");
var array = new[] {4, 8, 15, 16, 23, 42};
cmd.Parameters.AddWithValue("p", array);
var reader = await cmd.ExecuteReaderAsync();

Получаю ошибку:
Unhandled exception. Octonica.ClickHouseClient.Exceptions.ClickHouseServerException: DB::Exception: CAST AS Array can only be perforamed between same-dimensional Array or String types: while executing 'FUNCTION in(number : 0, _subquery26 :: 1) -> in(number, _su bquery26) UInt8 : 2'

Unable to cast object of type 'System.DateTime' to type 'System.DateTimeOffset'.

Unable to cast object of type 'System.DateTime' to type 'System.DateTimeOffset'.

Supports IReadOnlyList<object> implementation for ColumnWriter

Hello and thank you for the awesome client. Are there plans to implement IReadOnlyList<object> support for CreateColumnWriter inside type info? For example:

ClickHouseClient/src/Octonica.ClickHouseClient/Types/UuidTypeInfo.cs

Lines 38 to 41 in de7d8d3

	public override IClickHouseColumnWriter CreateColumnWriter<T>(string columnName, IReadOnlyList<T> rows, ClickHouseColumnSettings? columnSettings)
	{
	if (!(rows is IReadOnlyList<Guid> guidRows))
	throw new ClickHouseException(ClickHouseErrorCodes.TypeNotSupported, $"The type \"{typeof(T)}\" can't be converted to the ClickHouse type \"{ComplexTypeName}\".");

It may be useful when data is built at runtime and usually has an object[][] type. So the bulk insert method may looks like this:

public async Task InsertManyAsync(string query, object[][] rowColumns)
{
   await using (var writer = await _connection.CreateColumnWriterAsync(query))
   {
      // here is no knowledge what is each collection about inside "rowColumns"
      await writer.WriteTableAsync(rowColumns, rowColumns.Length, default);
   }
}

This flow throws an exception now: The type "System.Object" can't be converted to the ClickHouse type "UUID".