Missing MV output in server-side DESCRIBE about scylladb

[piodul]

Ahh, OK, I see. There are CREATE INDEX ON test.test (col) and CREATE INDEX ON test.test (val). IIUC, secondary indexes are internally implemented as materialized views (@nyh please correct me if I'm wrong), and client-side describe currently prints the MVs in addition to the secondary indexes (test.test_col_idx_index and test.test_val_idx_index respectively correspond to the aforementioned indexes).

I think this is a correct change in behavior. We only want to emit a single CREATE INDEX statement as it already takes care of creating the underlying materialized view. The CREATE MATERIALIZED VIEW is unneeded and would only cause problems if one tries to restore schema from DESCRIBE SCHEMA output.

In my opinion, it's a non-issue. I'll go ahead and close but @sylwiaszunejko, @nyh, @Jadw1 - feel free to reopen if I'm wrong about the conclusion here.

[piodul]

Before DESCRIBE this queries are executed:

CREATE KEYSPACE test WITH REPLICATION = {'class' : 'NetworkTopologyStrategy', 'replication_factor' : 1};
                CREATE TABLE test.users ( userid text PRIMARY KEY, firstname text, lastname text, age int);
CREATE INDEX myindex ON test.users (age);
CREATE TABLE test.test (id int, col int, val text, PRIMARY KEY(id, col));
CREATE INDEX ON test.test (col);
CREATE INDEX ON test.test (val)

@sylwiaszunejko, are you sure these are the only CREATE statements used in the test? Creating materialized views is done via CREATE MATERIALIZED VIEW statements and I can't see any such statements here.

[sylwiaszunejko]

@piodul Yes, I am sure, I am talking for example about this dtest: https://github.com/scylladb/scylla-dtest/blob/next/cqlsh_tests/cqlsh_tests.py#L617 in this line https://github.com/scylladb/scylla-dtest/blob/next/cqlsh_tests/cqlsh_tests.py#L639 expected output has MV outputs: https://github.com/scylladb/scylla-dtest/blob/next/cqlsh_tests/cqlsh_tests.py#L826.

I also tested it locally by building scylla master, starting one node and doing these commands through cqlsh, there was no CREATE MATERIALIZED VIEW statements, but in DESCRIBE there were MV outputs.

[Jadw1]

Before DESCRIBE this queries are executed:

CREATE KEYSPACE test WITH REPLICATION = {'class' : 'NetworkTopologyStrategy', 'replication_factor' : 1};
                CREATE TABLE test.users ( userid text PRIMARY KEY, firstname text, lastname text, age int);
CREATE INDEX myindex ON test.users (age);
CREATE TABLE test.test (id int, col int, val text, PRIMARY KEY(id, col));
CREATE INDEX ON test.test (col);
CREATE INDEX ON test.test (val)

@sylwiaszunejko, are you sure these are the only CREATE statements used in the test? Creating materialized views is done via CREATE MATERIALIZED VIEW statements and I can't see any such statements here.

+1

I've quickly tried it locally (my cqlsh is using server-side desc):

cqlsh> create keyspace ks1 with REPLICATION = {'class' : 'SimpleStrategy', 'replication_factor' : 1};
cqlsh> use ks1;
cqlsh:ks1> create table t (a int, b int, c int, primary key(a,b));
cqlsh:ks1> create materialized view mv as select a, b from t where b is not null primary key (a, b);
cqlsh:ks1> desc keyspace ks1;

CREATE KEYSPACE ks1 WITH replication = {'class': 'org.apache.cassandra.locator.SimpleStrategy', 'replication_factor': '1'} AND durable_writes = true;

CREATE TABLE ks1.t (
    a int,
    b int,
    c int,
    PRIMARY KEY (a, b)
) WITH CLUSTERING ORDER BY (b ASC)
    AND bloom_filter_fp_chance = 0.01
    AND caching = {'keys': 'ALL', 'rows_per_partition': 'ALL'}
    AND comment = ''
    AND compaction = {'class': 'SizeTieredCompactionStrategy'}
    AND compression = {'sstable_compression': 'org.apache.cassandra.io.compress.LZ4Compressor'}
    AND crc_check_chance = 1
    AND default_time_to_live = 0
    AND gc_grace_seconds = 864000
    AND max_index_interval = 2048
    AND memtable_flush_period_in_ms = 0
    AND min_index_interval = 128
    AND speculative_retry = '99.0PERCENTILE'
    AND paxos_grace_seconds = 864000
    AND tombstone_gc = {'mode': 'timeout', 'propagation_delay_in_seconds': '3600'};


CREATE MATERIALIZED VIEW ks1.mv AS
    SELECT a, b
    FROM ks1.t
    WHERE b IS NOT null
    PRIMARY KEY (a, b)
    WITH CLUSTERING ORDER BY (b ASC)
    AND bloom_filter_fp_chance = 0.01
    AND caching = {'keys': 'ALL', 'rows_per_partition': 'ALL'}
    AND comment = ''
    AND compaction = {'class': 'SizeTieredCompactionStrategy'}
    AND compression = {'sstable_compression': 'org.apache.cassandra.io.compress.LZ4Compressor'}
    AND crc_check_chance = 1
    AND default_time_to_live = 0
    AND gc_grace_seconds = 864000
    AND max_index_interval = 2048
    AND memtable_flush_period_in_ms = 0
    AND min_index_interval = 128
    AND speculative_retry = '99.0PERCENTILE'
    AND paxos_grace_seconds = 864000
    AND tombstone_gc = {'mode': 'timeout', 'propagation_delay_in_seconds': '3600'};

[sylwiaszunejko]

@Jadw1
I am talking about the situation where there is no create materialized view statement done before running describe. And in your test you used one, so I am not sure how it is relevant,