vertical-blank / sql-formatter

SQL formatter written with only Java Standard Library, without dependencies.

sql-formatter

Java port of great SQL formatter https://github.com/zeroturnaround/sql-formatter.

Written with only Java Standard Library, without dependencies.

Demo

Demo is running on Google Cloud Function, with native-compiled shared library by GraalVM.

This does not support:

• Stored procedures.
• Changing of the delimiter type to something else than ;.

Usage

Maven

<dependency>
  <groupId>com.github.vertical-blank</groupId>
  <artifactId>sql-formatter</artifactId>
  <version>2.0.5</version>
</dependency>

Gradle

implementation 'com.github.vertical-blank:sql-formatter:2.0.5'

Examples

You can easily use com.github.vertical_blank.sqlformatter.SqlFormatter :

SqlFormatter.format("SELECT * FROM table1")

This will output:

SELECT
  *
FROM
  table1

You can also pass FormatConfig object built by builder:

SqlFormatter.format('SELECT * FROM tbl',
  FormatConfig.builder()
    .indent("    ") // Defaults to two spaces
    .uppercase(true) // Defaults to false (not safe to use when SQL dialect has case-sensitive identifiers)
    .linesBetweenQueries(2) // Defaults to 1
    .maxColumnLength(100) // Defaults to 50
    .params(Arrays.asList("a", "b", "c")) // Map or List. See Placeholders replacement.
    .build()
);

Dialect

You can pass dialect com.github.vertical_blank.sqlformatter.languages.Dialect or String to SqlFormatter.of :

SqlFormatter
    .of(Dialect.N1ql)  // Recommended
     //.of("n1ql")      // String can be passed
    .format("SELECT *");

SQL formatter supports the following dialects:

• sql - Standard SQL
• mariadb - MariaDB
• mysql - MySQL
• postgresql - PostgreSQL
• db2 - IBM DB2
• plsql - Oracle PL/SQL
• n1ql - Couchbase N1QL
• redshift - Amazon Redshift
• spark - Spark
• tsql - SQL Server Transact-SQL

Extend formatters

Formatters can be extended as below :

SqlFormatter
    .of(Dialect.MySql)
    .extend(cfg -> cfg.plusOperators("=>"))
    .format("SELECT * FROM table WHERE A => 4")

Then it results in:

SELECT
  *
FROM
  table
WHERE
  A => 4

Placeholders replacement

You can pass List or Map to format :

// Named placeholders
Map<String, String> namedParams = new HashMap<>();
namedParams.put("foo", "'bar'");
SqlFormatter.of(Dialect.TSql).format("SELECT * FROM tbl WHERE foo = @foo", namedParams);

// Indexed placeholders
SqlFormatter.format("SELECT * FROM tbl WHERE foo = ?", Arrays.asList("'bar'"));

Both result in:

SELECT
  *
FROM
  tbl
WHERE
  foo = 'bar'

Build

Building this library requires JDK 11 because of ktfmt.

sql-formatter's Issues

java.util.Date query parameters not put in correctly

Date query parameters do not come out quoted, and don't use the standard ISO format accepted by most DB platforms.

Formatting issue after LIKE Predicate Escape Character '\'

This only seems to happen when using '' as the escape character. I've tired all the supported languages using the of(...) with the same results. I've provide a simple sample code and console output.

String a = "SELECT * FROM psprcsrqst WHERE prcsname LIKE 'A\_%' ESCAPE '\' AND prcstype = :1";
System.out.println(SqlFormatter.format(a));

Output:
SELECT
*
FROM
psprcsrqst
WHERE
prcsname LIKE 'A_%' ESCAPE '' AND prcstype = :1

Using some other character, in this case '!'
String a = "SELECT * FROM psprcsrqst WHERE prcsname LIKE 'A!_%' ESCAPE '!' AND prcstype = :1";
System.out.println(SqlFormatter.format(a));

Output:
SELECT
*
FROM
psprcsrqst
WHERE
prcsname LIKE 'A!_%' ESCAPE '!'
AND prcstype = :1

Contains spaces.

A space is created as shown in the example below.

@Test
    void test() {
        String sql = "SELECT * FROM TEST WHERE ABC != '4'";
        String format = SqlFormatter.format(sql);
        System.out.println("format: " + format);
    }

// "!=" ==> "! ="
// output : 
format: SELECT
  *
FROM
  TEST
WHERE
  ABC ! = '4'

Thank you very much.

Scala Native support

https://github.com/takayahilton/sql-formatter was recently updated to support Scala Native
This means that vertical-blank's sql-formatter could easily support Scala Native compilation by depending on https://github.com/takayahilton/sql-formatter.

Other tools using vertical-blank's sql-formatter (such as quill) could thus also be ported to SN without having to change the sql-formatter namespace.
And this would add this dependency only for the SN build.

is format() thread safe？

Add @@SESSION as reserved word for MySQL

@@SESSION is a reserved word in MySQL. https://dev.mysql.com/doc/refman/8.0/en/session-state-tracking.html
The formatter is currently adding a space after the first @ as it's considering it as a special word char.

Maybe a same fix as #42 would work? although it's not an operator in MySQL

:= format problem

My script starts as
DECLARE v_number NUMBER := 0;

then SqlFormatter.format(sql). But return like this
DECLARE v_number NUMBER: = 0;
This script throw exception in Oracle. It seperates := operator.

v2.0.0 String '||' issue

select '123' || 'abc' from table1

after

select
    '123' | | 'abc'
from
    table1

---

extra space between the ||

Extra space inserted inside backtick strings

If I format an SQL string like this:

SELECT * FROM `tableName`;

The resulting formatted string is like this:

SELECT * FROM ` tableName `;

The extra spaces break my query. Is there a way to fix this, or an option to disable this particular extra space?

Release 1.0.4

Hi,
Would it be possible to make a release 1.0.4 before the big catch up ?

Regards,

Comments and the "BETWEEN ... AND " clause are not formatted correctly.

If we put comments and all the sql in one line, it would not identify the sql and still in one line.

The between ... and ... clause should in the same line, right now the behavior is that before every and it would have a newline.

And just for suggestion, maybe your guys could consider some caching or other ways to improve the performance, it's almost 100X slower than hibernate core sql formatter when it comes to long queries.

Is there any way to set "Indentation style: Tabular, Right"?

Hi there,
I tested the demo of sql-formatter, https://sql-formatter-org.github.io/sql-formatter/
There is an option "Indentation style: Tabular, Right",
is there any way of veritcal-blank edition to archive this?
Many thanks.

Open to new features?

My team is looking for a SQL formatter that meets our needs. sql-formatter gets pretty close to what we want but I want to add some features that would satisfy our needs:

1. Option to capitalize reserved words
2. certain functions should not have parameters newlined on commas
3. boolean statements shouldn’t be wrapped across lines with the operator at the beginning of subsequent lines unless the line would exceed some set length
4. JOIN should be at the same indentation level as the parent FROM instead of within it (this one is strange to me, the aforementioned make sense)
5. ON should be inside a JOIN

I'd like to make these be toggleable options. If I made the changes, would you accept a PR?

Format Merge

Please enhance the library to format this SQL.

merge into DW_STG_USER.ACCOUNT_DIM target using (
   select
      COMMON_NAME m_commonName,
      ORIGIN m_origin,
      USAGE_TYPE m_usageType,
      CATEGORY m_category
   from
      MY_TABLE
   where
      USAGE_TYPE = :value
) source on source.m_usageType = target.USAGE_TYPE when matched then
update
set
   target.COMMON_NAME = source.m_commonName,
   target.ORIGIN = source.m_origin,
   target.USAGE_TYPE = source.m_usageType,
   target.CATEGORY = source.m_category
where
   (
      (source.m_commonName <> target.COMMON_NAME)
      or(source.m_origin <> target.ORIGIN)
      or(source.m_usageType <> target.USAGE_TYPE)
      or(source.m_category <> target.CATEGORY)
   )
   when not matched then
insert
   (
      target.COMMON_NAME,
      target.ORIGIN,
      target.USAGE_TYPE,
      target.CATEGORY
   )
values
   (
      source.m_commonName,
      source.m_origin,
      source.m_usageType,
      source.m_category
   )

Question: getTableNames from SQL Statement?

I know this is a SQL Formatter, but can it also be used to extract table names from a SQL Statement!

Previously I have worked with:

• SQL Table Name Parser -- https://github.com/mnadeem/sql-table-name-parser
  Which uses "regex" to extract table names -- but it's not really reliable

• JSqlParser -- https://github.com/JSQLParser/JSqlParser
  Which is a full-blown parser, so if the parser hasn't implemeted various stuff (for example: old-style-outer-joins in T-SQL, it will fail)

So I thought... with a little bit of changes... it might be easy to implement: Set<String> SqlFormatter.getTableNames(sql) in this project!
Would it be hard to enhance this package to also extract table names (at a "best effort", 100% correctness is not important, just as long as it's better than the SQL Table Name Parser)

Your library is awesome! Thanks for writing it!

Thats it.

Escape character + escaped double quote makes formatted SQL invalid

In Postgres such a query is valid:

select A as "A\""B" from "T1";

The single result column will be named A\"B.
The backslash escape isn't used as escape here, the first inner " will escape the following ".
To demonstrate the logic a bit more a few examples:

• A\B => A\B
• A""B => A"B

Now the formatter chokes on this.

select A as "A\""B" from "T1";

will be formatted as

select A as "A\"" B " from " T1 ";

Which is not even executable anymore.
Is there any configuration that could alleviate this problem ?

Extra whitespace inserted in string concatenation operator

In Dialect.PostgreSql, when using the string concatenation operator (||), extra whitespace is inserted.

    val res = SqlFormatter
      .of(Dialect.PostgreSql)
      .format("select aa || bb from zzz")

    println(res)
    // select
    //   aa | | bb
    //       ^
    // from
    //   zzz

If you run dialectConfig.plusOperators("||"), no extra whitespace will be inserted.

    val res = SqlFormatter
      .of(Dialect.PostgreSql)
      .extend { dialectConfig -> dialectConfig.plusOperators("||") }
      .format("select aa || bb from zzz")

    println(res)
    // select
    //   aa || bb
    // from
    //   zzz

Similar whitespace will be inserted in Dialect.N1ql and Dialect.TSql.

I can't judge whether the response is appropriate, but after applying the following patch, plusOperators("||") is no longer needed.

diff --git a/src/main/java/com/github/vertical_blank/sqlformatter/languages/PostgreSqlFormatter.java b/src/main/java/com/github/vertical_blank/sqlformatter/languages/PostgreSqlFormatter.java
index e8c4f0a..15e76ae 100644
--- a/src/main/java/com/github/vertical_blank/sqlformatter/languages/PostgreSqlFormatter.java
+++ b/src/main/java/com/github/vertical_blank/sqlformatter/languages/PostgreSqlFormatter.java
@@ -531,7 +531,7 @@ public class PostgreSqlFormatter extends AbstractFormatter {
         .operators(
             Arrays.asList(
                 "!=", "<<", ">>", "||/", "|/", "::", "->>", "->", "~~*", "~~", "!~~*", "!~~", "~*",
-                "!~*", "!~", "!!", "@@", "@@@"))
+                "!~*", "!~", "!!", "@@", "@@@", "||"))
         .build();
   }

jdbc named parameter support

hi,
in jdbc,the named parameter is :param
but i find parameter in lib is @param
any idea about replcase @ to : ?
thanks~

Issue when column name contains escaped enclosing character

The query

SqlFormatter.of(Dialect.PlSql)
		.extend(cfg -> cfg.plusOperators("->"))
		.format("SELECT `My field with \\` back () <<<<<< tick and a verryyyyyyyy long name` FROM MY_TABLE");

produces

SELECT
  `My field with \` back < < < < < < tick
  and a verryyyyyyyy long name ` FROM MY_TABLE

Which is no longer a valid query as the name has been split over multiple lines and whitespaces have been included.
I think that the first ` is detected as the end of the name even though it is escaped with a \.

Is there any way to specify such an escape pattern in column names ?

Formatter inserts space in between "//" (this is vertica's integer division operator)

Hi, I'm trying to use sqlFormatter with vertica which uses "//" for integer division. It doesn't look like there's an exact dialect of of the box for vertica. Wondering if there's a way to disable adding the extra space?

Is there any javadoc for this library?

I'm specially interested in the following questions:

1. anyway to convert keywords to upper case?

@@ operator splitting in Postgres

The @@ operator is used in PostgreSQL for text search
https://www.postgresql.org/docs/13/functions-textsearch.html
However, the formatter obtained by SqlFormatter.of(Dialect.PostgreSql) splits those two @ symbols with space, making a query using that operator invalid.

Named placeholders are broken due to whitespace

Name placeholders like WHERE name = :name are broken and do not replace at all.

Example:

SELECT supplier_name, city FROM suppliers
WHERE supplier_id = :name

this formats as:

SELECT
  supplier_name,
  city
FROM
  suppliers
WHERE
  supplier_id =: name

Notice the =: and the space between the : and name - =: name

You can see this right on the demo page:

https://www.vertical-blank.com/sql-formatter/

Oracle quotation operation fails

Using StandardSql or PlSql dialect, sql-formatter incorrectly interprets the Oracle quotation (q'...') operator.
For input string Select q'{test message}' from dual
gives wrong result
Select q '{test message}' from dual

need additional space in ALTER TABLE ADD

We have:
ALTER TABLE public.Orders ADD CONSTRAINT "fk_Orders_ref_Customers" FOREIGN KEY (customer_id) REFERENCES public.Customers (id)

After standart SqlFormatter.format(sqlScript, FormatConfig.builder().build()):
ALTER TABLE
public.Orders
ADD
CONSTRAINT "fk_Orders_ref_Customers" FOREIGN KEY (customer_id) REFERENCES public.Customers (id)

Between public.Orders and ADD elements only row delimiter, without any spaces.
And after replace all \n or \r we get invalid script:
ALTER TABLE public.OrdersADD CONSTRAINT

Splitting table name with $ on PostgreSQL

Table names like "MY_SCHEMA.C$_MY_TABLE" will be formatted into three parts"MY_SCHEMA.C $ _MY_TABLE".