xwb1989 / sqlparser Goto Github PK
View Code? Open in Web Editor NEWSQL Parser implemented in Go
License: Apache License 2.0
SQL Parser implemented in Go
License: Apache License 2.0
Does this parser detect sql injection? How?
Some users may mistakenly interpret the position as the line number.
when usingsqlparser.ParseStrictDDL
I receive an error
syntax error at position 290 near 'foreign'
when parsing valid mySQL:
CREATE TABLE `comments` (
`id` INT UNSIGNED NOT NULL AUTO_INCREMENT,
`content` TEXT DEFAULT NULL,
`created` DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
`last_edited` TIMESTAMP,
`user_id` VARCHAR(18) NOT NULL,
`post_id` INT UNSIGNED NOT NULL,
PRIMARY KEY (`id`),
FOREIGN KEY (`user_id`) REFERENCES users(id) ON UPDATE CASCADE ON DELETE CASCADE,
FOREIGN KEY (`post_id`) REFERENCES posts(id) ON UPDATE CASCADE ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
I expected that foreign key would be parseable?
I use sqlparser parse sql, and the code is below:
sql := "rename table a to b;"
tree, err := sqlparser.Parse(sql)
if err != nil {
fmt.Println("sql parser failed!")
return
}
out := sqlparser.String(tree)
fmt.Println(out)
the output is : "rename table a b",
the sql can't run success in mysql, forget add the "to"?
SELECT @variable := id
FROM randomtable;
I'm currently using sqlparse and I'm currently facing the issue that := doesn't seem supported when assigning values within a select.
is there plans to support this in future releases?
Thank you!
guys, i'm newbe to parser. how can i use sqlparser in my go project:)
Hello,
I want to translate SQL like 'select * from tb where col1=1 and col2='aaa'' to 'select * from tb where col1=??? and col2='???''
Is there any good ways?
the sql like this
CREATE TABLE test (
id INTEGER PRIMARY KEY AUTOINCREMENT,
created_at DATETIME,
updated_at DATETIME,
deleted_at DATETIME
)
Sometimes I just want to parse sql by some usual keyword like where, and, or... Is there any way to bypass some keyword like use, desc...?
I am sure this repo is not maintained continuously for almost 2 years. Even the README has a mistaken example. I strongly recommend not using this library.
A syntax error is produced in the case where a DELETE statement uses the USING keyword.
Examples:
Query: delete from l using l inner join t on l.id = t.id
Error: syntax error at position 20 near 'using'
Even simple GRANT:
GRANT SELECT ON db2.`xx` TO 'jeffrey'@'localhost'; -- error: syntax error at position 6 near 'grant'
or LOCK:
lock tables T1 read; -- error: syntax error at position 5 near 'lock'
select 1 as 测试 from dual
report
panic: syntax error at position 15 near � [recovered]
select 1 from dual where utl_encode.base64_decode(rawtohex(1))
report
panic: syntax error at position 52 [recovered]
Queries beginning with 'SELECT STRAIGHT_JOIN' fail with the message 'syntax error at position 21 near straight_join'
I create a table with
UpdateDatetime
TIMESTAMP NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP COMMENT '更新时间',
it shows
syntax error at position 471 near 'current_timestamp'
HI.
When I use sqlparser to parse sql with "?", and then call function stmt.Format, the "?" in the sql will be replaced with ":v1". Is this a bug?
Here is an example.
sql := "select * from tbl where id = ?"
stmt, _ := sqlparser.Parse(sql)
buf := sqlparser.NewTrackedBuffer(nil)
stmt.Format(buf)
newSql := buf.String()
fmt.Println(newSql) // the newSql is "select * from tbl where id = :v1"
Thank you in advance for your answer.
hi,
I propose to write an enchancement to this package that allows me to write tools such as a better formatter and validator to work with the ast nodes. I think a simple visitor pattern implementation will do.
func (node Select) Accept(v Visitor) { v.VisitSelect(node) }
If you argee with the feature and implementation then I will create a PR for it.
Is there any way that I can compare semantically to SQL senteces using sqlparser?
For example, this two sentences should be the same...
1.) SELECT * FROM tbl WHERE a = 2 AND b = 4
2.) SELECT * FROM tbl WHERE 4 = b AND a = 2
Thanks in advance!
Is it possible to obtain the position of each node in the AST? I believe it's not currently being tracked. Any sense of what would be involved to add support for that? I'd be happy to implement if someone could point me in the right direction.
Ideally, it would look something like this:
{
Start: { Offset: 23, Line: 5, Column: 6 },
End: { Offset: 25, Line: 5, Column: 8 }
}
1,不支持索引名称为 xxx-2018xxxx 这个形式,也就是说不支持索引名称中有”-"
2,不支持多个索引一起查询
it do not support primary key like:
create table test
(
channel
varchar(255) not null,
id
varchar(255) not null,
primary key c_id
(channel, id)
)
it only support like:
create table test
(
channel
varchar(255) not null,
id
varchar(255) not null,
primary key (channel, id)
)
Thank you for providing a tremendously helpful library for managing SQL code! Could you help identify exactly which SQL dialects are supported?
For example, is sqlparser limited to MySQL? Or does it support PostgreSQL? What about MSSQL? SQLite?
And what of the different ISO SQL standards? Is sqlparser up to date with any of the particular ISO SQL standard editions?
Thanks again for providing the open source community with sqlparser, a valuable system.
Hi,
great library. :) Just one concern:
the following snipped doesn't create valid SQL:
r := strings.NewReader(`CREATE TABLE answer (
id bigint(20) NOT NULL AUTO_INCREMENT,
element_id varchar(255) NOT NULL,
element_value varchar(255) NOT NULL,
version int(11) DEFAULT NULL,
participant bigint(20) NOT NULL,
PRIMARY KEY (id),
KEY FKABCA3FBEBE857B66 (participant),
CONSTRAINT FKABCA3FBEBE857B66 FOREIGN KEY (participant) REFERENCES participant (id)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;")`)
tokens := sqlparser.NewTokenizer(r)
for {
stmt, err := sqlparser.ParseNext(tokens)
if err == io.EOF {
break
}
fmt.Println(sqlparser.String(stmt))
if err != nil {
fmt.Println(err)
}
}
Output:
create table answer
syntax error at position 391 near ')'
What could I do here?
CREATE TABLE t1 (
ts TIMESTAMP(6) DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6)
);
ignoring error parsing DDL 'CREATE TABLE t1 (ts TIMESTAMP(6) DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6));': syntax error at position 60
Discovered while looking at mysqldump
's default output, which contains a couple things that currently cause syntax errors.
A mysqldump
file consists of SQL statements that create tables and insert rows into them. By default it places LOCK TABLES
and UNLOCK TABLES
statements around its inserts, but currently these are not handled by the parser.
select * from S3Object where cast(age AS int) > 12
This query is flagged as an error, even though it is a valid cast in SQL.
Parse sql "insert into trading.test(i, time) values(1, NOW())" will cause syntax error.
sql := "SELECT * FROM table WHERE a = 'abc'"
stmt, err := sqlparser.Parse(sql)
if err != nil {
panic(err)
}
I got this error
http: panic serving [::1]:50432: syntax error at position 20 near 'table'
goroutine 19 [running]:
net/http.(*conn).serve.func1(0xc0000a2960)
/usr/local/go/src/net/http/server.go:1769 +0x139
panic(0x7624e0, 0xc000087020)
/usr/local/go/src/runtime/panic.go:522 +0x1b5
github.com/flume-cloud-services/database/controllers.CreateQuery(0x8565a0, 0xc000136000, 0xc000112400)
/home/rerwan/go/src/github.com/flume-cloud-services/database/controllers/query.go:25 +0x27b
net/http.HandlerFunc.ServeHTTP(0x7ece30, 0x8565a0, 0xc000136000, 0xc000112400)
/usr/local/go/src/net/http/server.go:1995 +0x44
github.com/flume-cloud-services/database/middleware.AuthMiddleware.func1(0x8565a0, 0xc000136000, 0xc000112400)
/home/rerwan/go/src/github.com/flume-cloud-services/database/middleware/auth.go:60 +0x351
net/http.HandlerFunc.ServeHTTP(0xc000090440, 0x8565a0, 0xc000136000, 0xc000112400)
/usr/local/go/src/net/http/server.go:1995 +0x44
net/http.(*ServeMux).ServeHTTP(0xadf1a0, 0x8565a0, 0xc000136000, 0xc000112400)
/usr/local/go/src/net/http/server.go:2375 +0x1d6
net/http.serverHandler.ServeHTTP(0xc0000932b0, 0x8565a0, 0xc000136000, 0xc000112400)
/usr/local/go/src/net/http/server.go:2774 +0xa8
net/http.(*conn).serve(0xc0000a2960, 0x856ee0, 0xc00009e440)
/usr/local/go/src/net/http/server.go:1878 +0x851
created by net/http.(*Server).Serve
/usr/local/go/src/net/http/server.go:2884 +0x2f4
I am using the latest version of sqlparser
And Golang 1.12
select cast(INSIDE_APP_NO as INTEGER) from refund;
syntax error at position 37 near 'integer'.
Minimal example:
package main
import (
"fmt"
"github.com/xwb1989/sqlparser"
)
func main() {
sql := "SELECT DATE_SUB(NOW(), INTERVAL 1 MONTH)"
tree, err := sqlparser.Parse(sql)
if err != nil {
fmt.Println("Error: ", err)
panic("oh no")
}
fmt.Println(tree)
}
$ go run main.go
Error: syntax error at position 34 near 1
panic: oh no
[trace info]
Looks like it's failing to handle the MONTH
after INTERVAL
.
Dose this package support sqlite3
The current examples are not very helping. Please create more examples
sql as follow can run in mysql:
CREATE TABLE `table10_int_autoinc` (
`col_bigint` bigint,
`col_time` time,
`col_double` double,
`col_bigint_key` bigint,
`col_char_20` char(20),
`col_datetime` datetime,
`col_varchar_20_key` varchar(20),
`col_varchar_20` varchar(20),
`col_date` date, `col_float` float,
`col_float_key` float,
`col_char_20_key` char(20),
`col_time_key` time,
pk integer auto_increment,
key (`col_bigint_key`),
key (`col_date_key`),
primary key (pk))
but it can not parse in this parser, because it do not support anonymous index definition like 'key (columns)'
Hi, are there any examples of working with WHERE conditions/data?
Does the parser return the position (start + end) of nodes in the original text?
Hi there!
Thank you so much for taking the time to extract this into its own package, it's awesome! I was wondering if you could provide any guidance if there were any functions exposed that takes a parsed query and can recompile it back into an SQL statement?
Thank you so much for your time!
"select 'name 1' from test where names = 'a1' and names ='a3' LIMIT 10"
Column Names with spaces in them are not handled correctly by the parser, rather than being recognized as columns they are parsed as SQL vals and classified as ints. I tried surrounding 'name 1' with double quotes as well and this also did not work. If you could fix this that would be great, as it currently is not compatible with the behavior described for case sensitivity and spaces defined by the Amazon S3 Select API.
Thanks!
curious how much work it'd be to pull in the latest changes? seems like the vitess's parser is still under active development.
Discovered while looking at mysqldump
's default output, which contains a couple things that currently cause syntax errors.
A mysqldump
file consists of SQL statements that create tables and insert rows into them. However at the beginning of the file several comments are added -- some human readable, others with optional settings in them -- but what is important is that some of them are separated by semicolons, resulting in, after the comments are removed, effectively empty statements. For example:
/* some comment*/;
/* another comment */;
CREATE TABLE ...
Note: Lex
usually discards comments until it finds a token to return but in the above case that is nil since it is an empty statement.
Currently the parser does not handle empty statements -- obviously when parsing individual statements this could be relatively easily handled by the caller, but when tokenizing and parsing a stream with ParseNext
, this case is harder to handle in the caller.
sqlparser is fantastic for analyzing MySQL snippets! Could we extend this library to handle other common variants?
stmt, err := sqlparser.Parse(`select * from information_schema.processlist`)
if err != nil {
log.Println("解析错误:", err.Error())
return
}
log.Println(stmt)
result:
2019/03/29 15:31:38 解析错误: syntax error at position 45 near 'processlist'
Functions in SELECT operate correctly, but not in WHERE. I get a syntax error.
This is a fantastically complete and fast parser, but I have one problem case to report. It doesn't seem to understand queries where the "using" shorthand is used in a join.
Reproduce with:
Parsing: select * from t1 join t2 using(id)
yields error: syntax error at position 31 near 'using'
Hi,
I got another one:
r := strings.NewReader(`SET NAMES utf8;`)
tokens := sqlparser.NewTokenizer(r)
for {
stmt, err := sqlparser.ParseNext(tokens)
if err == io.EOF {
break
}
fmt.Println(sqlparser.String(stmt))
if err != nil {
fmt.Println(err)
}
}
This results only in this output:
set
The rest is missing?
I got an error for a query which contain OFFSET. It should be a valid query.
syntax error at position 42 near OFFSET for query - SELECT * FROM AccessToken LIMIT 10 OFFSET 13
I am using sqlparse to get table names from queries. My queries are complex and contain joins as well.
Fetching table name from an insert statement is easy but with select i am getting hard time
Here is my code:
stmt, err := sqlparser.Parse(sql)
if err != nil {
fmt.Println("Error: " + err.Error())
}
switch stmt := stmt.(type) {
case *sqlparser.Select:
var sel = stmt
var tbl = sel.From[0]
switch tblst := tbl.(type) {
case *sqlparser.JoinTableExpr:
var s = tblst
var ss = s.RightExpr
switch expr := ss.(type) {
case *sqlparser.AliasedTableExpr:
var cExpr = expr.Expr
fmt.Println(reflect.TypeOf(cExpr))
fmt.Printf("%#v\n", cExpr)
}
}
case *sqlparser.Insert:
var ins = stmt
fmt.Println(ins.Table.Name)
}
I am stuck after that because it returns a table indent and the fields are not exported
Sorry, i am new to golang so not sure if its the right way to do this
Can you please guide me to extract all tables from a complex select query with joins?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.