Comments (11)
#10392 is the upgrade to sqlparser -- I think it is pretty close but @tisonkun hit an issue during upgrade.
We may need a 0.46.1 for resolving the regressions:
- sqlparser-rs/sqlparser-rs#1278
- And what's still unclear how to fix, #10392 (comment)
I've locally confirmed that array.slt
is last failure for cargo test --test sqllogictests
.
from arrow-datafusion.
Provide something I surveyed.
I think we can follow how Calcite handles the quoted issue. The SqlDialect
of Calcite has a check rule identifierNeedsQuote
.
It can be overridden according to the specific data source, such as BigQuery:
They implement some rules, such as regex patterns or reserved word lists. I think a dialect-specific rule is a nice choice.
from arrow-datafusion.
Indeed, there is already a function on the sqlparser::dialect trait that takes this into account:
The dialect specific implementations just need to be expanded on. For now they just always return a conservative quote character.
from arrow-datafusion.
The dialect specific implementations just need to be expanded on. For now they just always return a conservative quote character.
I think it's same as the dialect in datafusion::unparser::dialect
However, what we need is a checker to check if the identifier needs to be quoted.
I think I can make a PR for
DefaultDialect
first.from arrow-datafusion.
As the mentioned in dialect.rs
I think we need to use the Dialect in sqlparser-rs instead and extract identifier_needs_quote
in #10573 to sqlparser-rs. Just like sqlparser-rs/sqlparser-rs#1170
from arrow-datafusion.
Yes, these are basically the same object. The one in DataFusion was put there temporarily until the trait extension in the sqlparser repo is landed and pushed to crates.io. This may have happened in the meantime.
from arrow-datafusion.
#10392 is the upgrade to sqlparser -- I think it is pretty close but @tisonkun hit an issue during upgrade.
from arrow-datafusion.
I'm not sure but I think we can merge #10573 first because it also fix many unpasring tests. Then, I'll create PR for sqlparser to add the check rule in dialect.
from arrow-datafusion.
FWI #10573 is merged!
from arrow-datafusion.
Do we split off a ticket reduce the nr of brackets emitted?
from arrow-datafusion.
Do we split off a ticket reduce the nr of brackets emitted?
Excellent call -- I filed #10633
from arrow-datafusion.
Related Issues (20)
- Circular relationship when determining state fields for AggregateUDF HOT 5
- Support join filter in NestedLoopJoin in fizz join test cases HOT 1
- `Int64` as default type for `make_array` function empty or null case
- `array_slice` panicked when called with empty args HOT 2
- `cli_quick_test` failing on windows (stack overflow) after sqlparser `0.47.0` upgrade
- Implement `ScalarValue::IntervalMonthDayNano` -> String Support
- Implement `ScalarValue::TimestampNanosecond` -> String Support
- Implement `ScalarValue::TimestampMillisecond` -> String Support HOT 1
- Support convert LogicalPlan::EmptyRelation to SQL String HOT 1
- Improve overflow errors HOT 1
- Efficiently and correctly Extract Page Index statistics into `ArrayRef`s HOT 7
- Add ability to receive an iterator over the inputs of a LogicalPlan instead of a Vec. HOT 6
- Support `array_any_value`
- Projects require unique expressions names error in substrait producer/consumer HOT 7
- Substrait consumer doesn't respect final output column names HOT 1
- `extract` doesn't accept quoted field names HOT 2
- Convert `stddev` to udaf HOT 4
- x NOT IN y works but NOT (x IN y) doesn't
- Convert `approx_distinct` to UDAF HOT 1
- Convert `approx_median` to UDAF HOT 7
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from arrow-datafusion.