GithubHelp home page GithubHelp logo

innogames / igcommit Goto Github PK

View Code? Open in Web Editor NEW
236.0 30.0 28.0 211 KB

Git pre-receive hook to check commits and code style

License: MIT License

Python 100.00%
syntax-checker git git-hook csslint htmlhint python golint puppet-lint flake8 pep8

igcommit's Introduction

Git Pre-Receive Hook to Validate Commits

It is exhausting to edit files again and again to have a consistent style. This project provides a Git pre-receive hook to validate pushed commits on the Git server side. The hook avoids all issues by rejecting any commit not matching the rules to get in to the repository in the first place.

The pre-receive hook runs some checks on commits on its own, and searches for programming language specific syntax checkers on the PATH of the server to check changed files with them. The process is pretty fast, because only the added and modified files on the pushed commits are passed to the syntax checkers, also in parallel. It wouldn't slow you down unless your commits are touching hundreds of files.

Installation

It is possible to install the tool with `pip`:

pip install igcommit

Link the script to hooks/pre-receive on you Git repositories on your Git server:

ln -s igcommit-receive /home/git/repositories/myproject.git/hooks/pre-receive

Features

  • Validate received commits one by one not just the last one
  • Only validate added or modified files on the commits
  • Report all problems before failing
  • Check for duplicate commit summaries
  • Check for misleading merge commits
  • Validate committer and author timestamps
  • Validate committer and author names and email addresses
  • Check commit message best practices (80 lines, first line summary...)
  • Check commit summary formatting
  • Validate commit tags against a list [BUGFIX], [FEATURE], [WIP]...
  • Check for changed file paths
  • Accept commits tagged as [HOTFIX], [MESS], [TEMP], or [WIP] with issues
  • Check executable bits and shebangs
  • Check for allowed branch names
  • Check symlinks
  • Check CSS files with csslint
  • Check Go files with golint
  • Check HTML files with htmlhint
  • Check Puppet files with puppet parser validate and puppet-lint
  • Check Python files with flake8 or pycodestyle and pyflakes
  • Check Ruby files with rubocop
  • Check shell scripts with shellcheck
  • Check JavaScript files with eslint, jshint, jscs, or standard
  • Check CoffeeScript files with coffeelint
  • Check PHP files with phpcs
  • Run the external check commands in parallel
  • Validate JSON, XML, YAML file formats

Here is an example problem output:

=== CheckDuplicateCommitSummaries on CommitList ===
ERROR: summary "Add nagios check for early expiration of licenses" duplicated 2 times

=== CheckCommitSummary on 31d0f6b ===
WARNING: summary longer than 72 characters

=== CheckCommitSummary on 6bded65 ===
WARNING: past tense used on summary

=== CheckCommand "flake8" on src/check_multiple.py at 6bded65 ===
INFO: line 10 col 5: E225 missing whitespace around operator
INFO: line 17 col 80: E501 line too long (122 > 79 characters)
INFO: line 17 col 85: E203 whitespace before ','

=== CheckCommitMessage on 6fdbc00 ===
WARNING: line 7 is longer than 80
WARNING: line 9 is longer than 80

Configuration

The script itself is currently configuration free. Though, some of the syntax checkers called by the script uses or requires configurations. Those configuration files has to be on the top level of the Git repository.

Syntax Checker Configuration File
coffeelint coffeelint.json, or package.json optional
csslint .csslintrc optional
eslint eslint.(jsyml|json), or package.json required
flake8 .flake8, setup.cfg, or tox.ini optional
htmlhint .htmlhintrc optional
jscs .jscsrc, .jscs.json, or package.json required
jshint .jshintrc, or package.json optional
phpcs phpcs.xml, or phpcs.xml.dist optional
puppet-lint .puppet-lint.rc optional
pycodestyle setup.cfg, or tox.ini optional
rubocop .rubocop.yml optional

Pros and Cons of Pre-receive Hook

Continuous Integration Server

A continuous integration server can run such checks with the many other things it is doing. Moving this job from it has many benefits:

  • Synchronous feedback
  • More efficient
  • Disallow any commit violating the rules
Pre-commit Hook

Even though, pre-receive hook gives later feedback than pre-commit hook, it has many advantages over it:

  • No client side configuration
  • Plugins has to be installed only once to the Git server
  • Everybody gets the same checks
  • Enforcement, nobody can skip the checks
  • Commit checking (pre-commit hook only gets what is changed in the commit)
IDE Integration

The same advantages compared to pre-commit hooks applies to IDE integration. Though, IDE integration gives much sooner and nicer feedback, so it is still a good idea, even with the pre-receive hook.

Dependencies

The script has no dependencies on Python 3.4 or above. The script executes the validation commands using the shell. The necessary ones for checked repositories need to be installed separately. See the complete list of commands on the config.py. The commands which are not available on the PATH is not going to be used.

Testing

I found it useful to check what the script would have complained if it had been active on different Git repositories. You can run a command like this to test this inside a Git repository against last 50 commits:

git log --reverse --oneline HEAD~50..HEAD |
    sed 's:\([^ ]*\) .*:\1 \1 refs/heads/master:' |
    python ../igcommit/igcommit-receive

Changes

Version 2.0
  • Fix line numbers on syntax errors for executables being 1 off
  • Recognize and validate symlinks
  • Validate committer and author timestamps
  • Validate contributor names and email addresses
  • Reduce commit message line length limits
  • Complain about file extensions on executables
Version 2.1
  • Add [TEMP] to recognized commit tags
  • Fix getting the changes of the initial commit (Zheng Wei)
  • Fix various file descriptor leaks
  • Check commit summaries more strictly
  • Check shebangs of non-executable files too
  • Don't check on empty file contents
  • Improve unicode support on Python 2
  • Fix checking symlink targets
Version 2.2
  • Fix eslint configuration (Jerevia)
  • Accepts commits with [TEMP] with issues
  • Stop skipping empty files
  • Make sure not to get unknown file contents
  • Move file extensions to config
  • Increase timestamp comparison tolerance for 1 more minute
  • Handle spaces on shebangs
Version 2.3
  • Handle check command failing immediately
  • Support pushed tags
  • Fix failing on file check with bogus return code
  • Include list of commit tags on warning
  • Fix recognising commit tags [REVIEW] and [SECURITY]
Version 2.4
  • Fix recognising removed configuration files
  • Support coffeelint
Version 2.5
  • Fix unicode issue on Python 2 for XML, YAML, and JSON (jcoetsie)
Version 3.0
  • Drop Python 2 support
  • Fix handling filenames with spaces (Friz-zy)
Version 3.1
  • Stop complaining about the same commit for Git tags
  • Fix checking contributor names and email addresses
  • Stop complaining about file extensions we don't know about
  • Filter out checking format of files under templates/ directories
  • Improve code quality and style
Version 3.2
  • Reduce severity of length for merge commit summary to warning
Version 3.3
  • Add optional branch name check via regular expressions

License

The script is released under the MIT License. The MIT License is registered with and approved by the Open Source Initiative1.


  1. https://opensource.org/licenses/MIT

igcommit's People

Contributors

felixoid avatar friz-zy avatar hasegeli avatar kofrezo avatar seqizz avatar thefredfox avatar woellchen avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

igcommit's Issues

Commit#get_changed_files returns empty list for initial commit of a repos

I edited config.py to keep only phpcs check in it. Then I tested it with serval files of non-standard style in a initial commit. I found that the push is done without any remote output.

Here is the command executed, which output nothing:

git diff-tree -r --no-commit-id --break-rewrites --no-renames --diff-filter=AM 5620d9d89b45f2901adf13e0ab04e8dd07c39192

On submodule problem

I used "git submodule add - B URL Src / pages / SPD" and submitted ". Git modules" file and "Src / pages / SPD" file at the same time, so the following error occurred:

subprocess.CalledProcessError : Command'['/opt/gitlab/embedded/libexec/git-core/git', 'cat-file', '-s', '*********************************']' returned non-zero exit status 128.

IMPROVEMENT: checking ulimit before processing a commit

OSError occurred while pushing a commit of huge number(more than 2200) of files to remote.

remote: === CheckCommand "phpcs" on code/Doctrine/Common/Cache/WinCacheCache.php at 9f6ba964 ===
remote: INFO:
remote: INFO: FILE: code/Doctrine/Common/Cache/WinCacheCache.php
remote: INFO: ----------------------------------------------------------------------
remote: INFO: FOUND 13 ERRORS AND 7 WARNINGS AFFECTING 18 LINES
remote: INFO: ----------------------------------------------------------------------
remote: INFO: 2 | ERROR | You must use "/**" style comments for a file comment
remote: INFO: 25 | WARNING | @link tag is not allowed in class comment
remote: INFO: 26 | WARNING | @SInCE tag is not allowed in class comment
remote: INFO: 27 | WARNING | @author tag is not allowed in class comment
remote: INFO: 28 | WARNING | @author tag is not allowed in class comment
remote: INFO: 29 | WARNING | @author tag is not allowed in class comment
remote: INFO: 30 | WARNING | @author tag is not allowed in class comment
remote: INFO: 31 | WARNING | @author tag is not allowed in class comment
remote: INFO: 35 | ERROR | Doc comment for parameter "$id" missing
remote: INFO: 37 | ERROR | Missing @return tag in function comment
remote: INFO: 43 | ERROR | Doc comment for parameter "$id" missing
remote: INFO: 45 | ERROR | Missing @return tag in function comment
remote: INFO: 51 | ERROR | Doc comment for parameter "$id" missing
remote: INFO: 51 | ERROR | Doc comment for parameter "$data" missing
remote: INFO: 51 | ERROR | Doc comment for parameter "$lifeTime" missing
remote: INFO: 53 | ERROR | Missing @return tag in function comment
remote: INFO: 59 | ERROR | Doc comment for parameter "$id" missinTraceback (most recent call last):
remote: File "/root/githook/igcommit/igcommit/prereceive.py", line 97, in main
remote: state = Runner().run()
remote: File "/root/githook/igcommit/igcommit/prereceive.py", line 28, in run
remote: for check in iter_buffer(self.expand_checks(checks), 16):
remote: File "/root/githook/igcommit/igcommit/utils.py", line 50, in iter_buffer
remote: for elem in iterable:
remote: File "/root/githook/igcommit/igcommit/prereceive.py", line 41, in expand_checks
remote: for check in self.expand_checks_to_input(next_checks, line):
remote: File "/root/githook/igcommit/igcommit/prereceive.py", line 57, in expand_checks_to_input
remote: for check in self.expand_checks_to_commit_list(checks, commit_list):
remote: File "/root/githook/igcommit/igcommit/prereceive.py", line 67, in expand_checks_to_commit_list
remote: for check in self.expand_checks_to_commit(next_checks, commit):
remote: File "/root/githook/igcommit/igcommit/prereceive.py", line 78, in expand_checks_to_commit
remote: for check in self.expand_checks_to_file(next_checks, changed_file):
remote: File "/root/githook/igcommit/igcommit/prereceive.py", line 90, in expand_checks_to_file
remote: for check in prepare_checks(checks, changed_file):
remote: File "/root/githook/igcommit/igcommit/base_check.py", line 94, in prepare_checks
remote: prepared_check = check.prepare(obj)
remote: File "/root/githook/igcommit/igcommit/file_checks.py", line 191, in prepare
remote: new._prepare_proc()
remote: File "/root/githook/igcommit/igcommit/file_checks.py", line 226, in _prepare_proc
remote: [self.get_exe_path()] + self._prepared_args()
remote: File "/root/githook/igcommit/igcommit/git.py", line 314, in pass_content
remote: target_proc = Popen(proc_args, stdin=stdin, stdout=PIPE, stderr=STDOUT)
remote: File "/usr/lib64/python2.7/subprocess.py", line 711, in init
remote: errread, errwrite)
remote: File "/usr/lib64/python2.7/subprocess.py", line 1216, in _execute_child
remote: errpipe_read, errpipe_write = self.pipe_cloexec()
remote: File "/usr/lib64/python2.7/subprocess.py", line 1168, in pipe_cloexec
remote: r, w = os.pipe()
remote: OSError: [Errno 24] Too many open files
remote: g
remote: INFO: 61 | ERROR | Missing @return tag in function comment
remote: INFO: 69 | ERROR | Missing @return tag in function comment
remote: INFO: 77 | ERROR | Missing @return tag in function comment
remote: INFO: ----------------------------------------------------------------------
remote: INFO:
remote: INFO: Time: 32ms; Memory: 4Mb
remote: INFO:

How can I install this?

Hello, Guru
I want to try your works.
However I cannot solve its dependency -- intenum? how can I install intenum in Ubuntu-14.04?
Thanks in advance.

Call from bash

Hello!

I've been looking for a pre-receive hook for a long time, and this repo looks awesome!

Just a question regarding the use: I'm implementing a git pre-receive hook in bash, and I'd like to call igcommit from my bash script. Is this possible?

The use of bash is enforced by my git server.

Thanks!

UnicodeEncodeError: 'ascii' codec can't encode character u'\u20ac' in position 425: ordinal not in range(128)

Hi,

It seems there is a problem with encoding unicode characters, in this case the European Euro Sign €

remote: === CheckCommitSummary on 1d03c552 ===
remote: WARNING: commit title not capitalized
remote:
remote:
remote: An error occurred, but the commits are accepted.
remote: Traceback (most recent call last):
remote:   File "/usr/local/lib/python2.7/dist-packages/igcommit/prereceive.py", line 116, in main
remote:     state = Runner().run()
remote:   File "/usr/local/lib/python2.7/dist-packages/igcommit/prereceive.py", line 34, in run
remote:     check.print_problems()
remote:   File "/usr/local/lib/python2.7/dist-packages/igcommit/base_check.py", line 75, in print_problems
remote:     for severity, problem in self.evaluate_problems():
remote:   File "/usr/local/lib/python2.7/dist-packages/igcommit/base_check.py", line 85, in evaluate_problems
remote:     for severity, problem in self.get_problems():
remote:   File "/usr/local/lib/python2.7/dist-packages/igcommit/file_checks.py", line 327, in get_problems
remote:     self.load_func(self.committed_file.get_content().decode('utf-8'))
remote:   File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1311, in XML
remote:     parser.feed(text)
remote:   File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1651, in feed
remote:     self._parser.Parse(data, 0)
remote: UnicodeEncodeError: 'ascii' codec can't encode character u'\u20ac' in position 425: ordinal not in range(128)

Any idea?

I don't see how this can fail. json.loads is available in Python 2.7.16 and Python 3.7.3 on my machines:

I don't see how this can fail. json.loads is available in Python 2.7.16 and Python 3.7.3 on my machines:

Python 2.7.16 (default, Oct 10 2019, 22:02:15)
[GCC 8.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from json import loads
>>> 
Python 3.7.3 (default, Apr  3 2019, 05:39:12) 
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>d> from json import loads
>>> 

Originally posted by @kofrezo in #20 (comment)

Issue on committing empty php file

$ touch 2.php
$ git add 2.php
$ git commit -m 'test'
$ git push origin master
Counting objects: 3, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (2/2), done.
Writing objects: 100% (3/3), 321 bytes | 0 bytes/s, done.
Total 3 (delta 0), reused 0 (delta 0)
remote: === CheckCommand "phpcs" on 2.php at f5aafab3 ===
remote: ERROR: You must supply at least one file or directory to process.
remote: INFO:
remote: INFO: Usage: phpcs [-nwlsaepqvi] [-d key[=value]] [--colors] [--no-colors] [--stdin-path=]
remote: INFO: [--report=] [--report-file=] [--report-=] ...
remote: INFO: [--report-width=] [--generator=] [--tab-width=]
remote: INFO: [--severity=] [--error-severity=] [--warning-severity=]
remote: INFO: [--runtime-set key value] [--config-set key value] [--config-delete key] [--config-show]
remote: INFO: [--standard=] [--sniffs=] [--exclude=] [--encoding=]
remote: INFO: [--extensions=] [--ignore=] [--bootstrap=]
remote: INFO: [--file-list=] ...
remote: INFO: Set runtime value (see --config-set)

Is this by design?

Give samples of the configuration files to be used

I cannot see any of the configuration files to be mentioned in your code base

Kindly provide the configuration files for pycodestyle because currently there is ambiguity as to what configurations on must put into the setup.cfg or tox.ini file

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.