ClickHouse: fail to qualify "WITH <expression> AS <identifier>" about sqlglot

[georgesittas]

I see, thanks for clarifying, we'll take a look soon.

[herry13]

@georgesittas I understood the reason.

Is there a chance for you to review and merge a PR that resolves a case where the "" is a column expression? I think we can implement your suggestion to replace all instances after "WITH". If the answer is yes, then I'll find some time to implement it.

What do you think?

[herry13]

This is related to #3175. The previous bug report is about parsing (line maybe_parse), not qualifying.

[georgesittas]

I wonder, does ClickHouse simply replace all instances of mydate in the query that follows the WITH definition? Perhaps this expansion could happen before hitting the qualifier.

The reason this fails is because it hasn't been a priority for us to take this special CTE AST case into account, since it's very ClickHouse specific. My guess is that we shouldn't pollute the optimizer code around CTEs to take this into account, but should instead preprocess the AST first to canonicalize it into something we can deal with.

We'd need to understand the semantics of the <expression> AS <identifier> syntax in CH a bit more fix this.

[herry13]

does ClickHouse simply replace all instances of mydate in the query that follows the WITH definition?

I believe this is correct if <expression> is a column expression.

If it is a select ... then the possible routes are:

1. if the select's result is 1 row and 1 column (scalar value) (e.g. with (select col from db.table limit 1) as result select result;), then it behaves the same as the column expression where all instances are replaced;
2. if the select's result is 1 row and multiple columns, then all column values are merged as a tuple, then all instances are replaced with this tuple (e.g. with (select * from db.table limit 1) as result select result;);
3. if the select's result is multiple rows (e.g. with (select * from db.table limit 2) as result select result;), regardless the number of columns, then it raises an error: Code: 125. DB::Exception: Scalar subquery returned more than one row.

The above use cases are based on my observation running the example queries directly on clickhouse.

[georgesittas]

Hey @herry13, I think I'll close this one after all, at least for now. It doesn't seem trivial and since it's not a very high priority item for us, I'd like to avoid introducing a new transformation in the optimizer just to deal with this special case.

In general, it's very tricky to fully support ClickHouse, especially in the optimizer, as they have various non-standard SQL extensions not supported by other dialects.

We'll be happy to accept PRs. If you're interested to work on this, please reach out in Slack to discuss the implementation details.