ClickHouse on low memory machine about clickhouse

[den-crane]

how many memory will be enough for our data size?

Maybe 8G maybe 128G. You should test.

[den-crane]

The max_server_memory_usage_to_ram_ratio is set to 0.4.

Why ?

8GB * 0.4 = 3.2 GB -- you allowed Clickhouse to use only 3.2GB. Mark cache size by default is 5GB.

Given our current setup and data size, what would be a good configuration for a low memory machine? If an upgrade is needed, how much memory would be required to handle our current data size?

It makes no sense since you don't allow clickhouse to use 60% of RAM.

[stevelukis]

@den-crane Because we want to leave enough memory for our web server.

If we were to move our ClickHouse server to a separate machine and restore max_server_memory_usage_to_ram_ratio to default , how many memory will be enough for our data size?

[alexey-milovidov]

Lower the size of the mark cache in the config.xml. It can be set as low as 500 MB, but it cannot be set to zero.

Lower the number of query processing threads down to 1.
Lower the max_block_size to 8192. Values as low as 1024 can still be practical.

Lower max_download_threads to 1.
Set input_format_parallel_parsing and output_format_parallel_formatting to 0.

To flush the memory that is cached by the memory allocator, you can run the

SYSTEM JEMALLOC PURGE

command.

It is not recommended to use S3 or Kafka integrations on low-memory machines because they require a lot of memory for buffers.