GithubHelp home page GithubHelp logo

erik / squabble Goto Github PK

View Code? Open in Web Editor NEW
64.0 3.0 7.0 188 KB

An extensible linter for SQL queries and migrations.

Home Page: https://squabble.readthedocs.org

License: GNU General Public License v3.0

Python 100.00%
postgres sql linter static-analysis

squabble's Introduction

squabble

build status Documentation Status PyPI version

Catch unsafe SQL migrations.

$ squabble sql/migration.sql
sql/migration.sql:4:46 ERROR: column "uh_oh" has a disallowed constraint [1004]
ALTER TABLE big_table ADD COLUMN uh_oh integer DEFAULT 0;
                                               ^
# Use --explain to get more information on a lint violation
$ squabble --explain 1004
ConstraintNotAllowed
     When adding a column to an existing table, certain constraints can have
     unintentional side effects, like locking the table or introducing
     performance issues.
     ...

Squabble can also be integrated with your editor to catch errors in SQL files.

$ echo 'SELECT * FROM WHERE x = y;' | squabble --reporter=plain
stdin:1:15 CRITICAL: syntax error at or near "WHERE"

Currently, most of the rules have been focused on Postgres and its quirks. However, squabble can parse any ANSI SQL and new rules that are specific to other databases are appreciated!

Installation

$ pip3 install squabble
$ squabble --help

Note

Squabble is only supported on Python 3.5+

If you’d like to install from source:

$ git clone https://github.com/erik/squabble.git && cd squabble
$ python3 -m venv ve && source ve/bin/activate
$ python setup.py install
$ squabble --help

Configuration

To see a list of rules, try

$ squabble --list-rules

Then, to show more verbose information about a rule (such as rationale and configuration options)

$ squabble --show-rule AddColumnDisallowConstraints

Once a configuration file is in place, it can be passed explicitly on the command line, or automatically looked up.

$ squabble -c path/to/config ...

If not explicitly given on the command line, squabble will look for a file named .squabblerc in the following places (in order):

  • ./.squabblerc
  • (git_repo_root)/.squabblerc
  • ~/.squabblerc

Per-File Configuration

Configuration can also be applied at the file level by using SQL line comments in the form -- squabble-enable:RuleName or -- squabble-disable:RuleName.

For example, to disable RuleA and enable RuleB just for one file, this could be done:

-- squabble-disable:RuleA
-- squabble-enable:RuleB config=value array=1,2,3
SELECT email FROM users WHERE ...;

To prevent squabble from running on a file, use -- squabble-disable. Note that this will also disable syntax checking. Note that this flag will take precedence over any other configuration set either on the command line or in the rest of the file.

Example Configuration

{
  "reporter": "color",

  "plugins": [
    "/some/directory/with/custom/rules"
  ],

  "rules": {
    "AddColumnsDisallowConstraints": {
      "disallowed": ["DEFAULT", "FOREIGN", "NOT NULL"]
    }
  }
}

Prior Art

squabble is of course not the first tool in this space. If it doesn't fit your needs, consider one of these tools:

  • sqlcheck - regular expression based (rather than parsing), focuses more on SELECT statements than migrations.
  • sqlint - checks that the syntax of a file is valid. Uses the same parsing library as squabble.
  • sqlfluff - focused more on style and formatting, seems to still be a work in progress.

Acknowledgments

This project would not be possible without:

  • libpg_query - Postgres query parser
  • pglast - Python bindings to libpg_query
  • Postgres - …obviously

The logo image used in the documentation is created by Gianni - Dolce Merda from the Noun Project.

squabble's People

Contributors

erik avatar avilaton avatar philiptrauner avatar

Stargazers

Ilan Keshet avatar  avatar Max Belov avatar Prasannan N avatar Alexander avatar Artur Sak avatar wangxiaolei avatar Tin C. avatar Neal Fultz avatar Sakti Dwi Cahyono avatar Andrey Torsunov avatar  avatar g.vyacheslav avatar Steve Coffman avatar Rohit Trivedi avatar Laurence Dougal Myers avatar John M. Owen avatar Harry Dobrev avatar  avatar Jesse Carter avatar ranjanprj avatar Kristian Dupont avatar Jonathan Dobson avatar Byron Gibson avatar Yasser Fadl avatar Nico Müller avatar George avatar Lautaro Bringas avatar  avatar Alex Pentland avatar Paulo Haddad avatar Jeff Carpenter avatar  avatar Alex Thompson avatar Marc Guirand avatar Juan Rosero avatar Oleksandr Moskalenko avatar Marcin Popławski avatar Saad Shahd avatar Itamar Turner-Trauring avatar Osvaldo Santana Neto avatar Danilo Shiga avatar Gabriel Laskar avatar Michele Lacchia avatar Zhao Xiaohong avatar Kahlil Hodgson avatar Adrienne Franke avatar Luiz Menezes avatar Sergey Mezentsev avatar Henry Goodwin avatar Rafael Francischini avatar George Kussumoto avatar Zoltán Szeredi avatar k2uhl avatar dot avatar Yasha avatar Michael Born avatar  avatar Daniel Flook avatar  avatar Bruno Paulino avatar Ahmad Moussawi avatar  avatar Lewis Zhang avatar

Watchers

 avatar James Cloos avatar  avatar

squabble's Issues

Investigate adding LSP support

This is likely a pretty fundamental change to the architecture, but it would definitely be nice to add language server protocol support.

Doesn't seem like there's an existing implementation for SQL generally (there's one for T-SQL).

Failure to load rules on python3.5

Hi, thanks for building squabble first of all.
I had run it in the past and now I came back to give it another go. I found an issue running in 3.5 which is not affecting python3.8. The simplest form of test I can provide is

from squabble import cli
cli.dispatch_args({"--verbose": True, "--list-presets": False, "--config": None, "--preset": None, "--reporter": None, "--list-rules": True})

returns empty for 3.5, while it prints rules for 3.8. I was unable to make further progress due to time constraints but I thought I could leave a note here just in case.
If I find out what is happening I'll be back for sure.

Add xUnit compatible reporter

Would be useful to have a reporter which dumps results in the standard (ish) XML format used by JUnit etc. so that this can integrate more easily with existing tooling.

It's obviously much more geared toward test runs, so we'll need to see if lint output makes any kind of sense here.

https://xunit.net/docs/format-xml-v2

Issue with latest version of pglast

I was attempting to use the latest version of pglast with squabble, as the specified version of pglast was released back in 2019.

Keep getting these particular errors (see last line):

$ squabble --list-rules
Traceback (most recent call last):
  File "/usr/bin/squabble", line 33, in <module>
    sys.exit(load_entry_point('squabble==1.4.0', 'console_scripts', 'squabble')())
  File "/usr/lib/python3.9/site-packages/squabble/__main__.py", line 12, in main
    status = cli.main()
  File "/usr/lib/python3.9/site-packages/squabble/cli.py", line 45, in main
    return dispatch_args(args)
  File "/usr/lib/python3.9/site-packages/squabble/cli.py", line 75, in dispatch_args
    rule.load_rules(plugin_paths=base_config.plugins)
  File "/usr/lib/python3.9/site-packages/squabble/rule.py", line 59, in load_rules
    _load_builtin_rules()
  File "/usr/lib/python3.9/site-packages/squabble/rule.py", line 51, in _load_builtin_rules
    importlib.import_module('squabble.rules.' + mod_name)
  File "/usr/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/usr/lib/python3.9/site-packages/squabble/rules/disallow_float_types.py", line 28, in <module>
    class DisallowFloatTypes(BaseRule):
  File "/usr/lib/python3.9/site-packages/squabble/rules/disallow_float_types.py", line 48, in DisallowFloatTypes
    _INEXACT_TYPES = set(
  File "/usr/lib/python3.9/site-packages/squabble/rules/disallow_float_types.py", line 49, in <genexpr>
    _parse_column_type(ty)
  File "/usr/lib/python3.9/site-packages/squabble/rules/disallow_float_types.py", line 25, in _parse_column_type
    return format_type_name(type_name)
  File "/usr/lib/python3.9/site-packages/squabble/util.py", line 39, in format_type_name
    return '.'.join([p.string_value for p in type_name.names])
  File "/usr/lib/python3.9/site-packages/squabble/util.py", line 39, in <listcomp>
    return '.'.join([p.string_value for p in type_name.names])
  File "/usr/lib/python3.9/site-packages/pglast/node.py", line 190, in __getattr__
    value = getattr(self.ast_node, attr)
AttributeError: 'String' object has no attribute 'string_value'

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.