seddonym / import-linter

Import Linter allows you to define and enforce rules for the internal and external imports within your Python project.

Import Linter

Import Linter allows you to define and enforce rules for the imports within and between Python packages.

• Free software: BSD license
• Documentation: https://import-linter.readthedocs.io.

Overview

Import Linter is a command line tool to check that you are following a self-imposed architecture within your Python project. It does this by analysing the imports between all the modules in one or more Python packages, and compares this against a set of rules that you provide in a configuration file.

The configuration file contains one or more 'contracts'. Each contract has a specific type, which determines the sort of rules it will apply. For example, the forbidden contract type allows you to check that certain modules or packages are not imported by parts of your project.

Import Linter is particularly useful if you are working on a complex codebase within a team, when you want to enforce a particular architectural style. In this case you can add Import Linter to your deployment pipeline, so that any code that does not follow the architecture will fail tests.

If there isn't a built in contract type that fits your desired architecture, you can define a custom one.

Quick start

Install Import Linter:

pip install import-linter

Decide on the dependency flows you wish to check. In this example, we have decided to make sure that myproject.foo has dependencies on neither myproject.bar nor myproject.baz, so we will use the forbidden contract type.

Create an .importlinter file in the root of your project to define your contract(s). In this case:

[importlinter]
root_package = myproject

[importlinter:contract:1]
name=Foo doesn't import bar or baz
type=forbidden
source_modules=
    myproject.foo
forbidden_modules=
    myproject.bar
    myproject.baz

Now, from your project root, run:

lint-imports

If your code violates the contract, you will see an error message something like this:

=============
Import Linter
=============

---------
Contracts
---------

Analyzed 23 files, 44 dependencies.
-----------------------------------

Foo doesn't import bar or baz BROKEN

Contracts: 1 broken.


----------------
Broken contracts
----------------

Foo doesn't import bar or baz
-----------------------------

myproject.foo is not allowed to import myproject.bar:

-   myproject.foo.blue -> myproject.utils.red (l.16)
    myproject.utils.red -> myproject.utils.green (l.1)
    myproject.utils.green -> myproject.bar.yellow (l.3)

import-linter's Issues

tests and CI for `import-linter`

I am opening this issue to talk about CI and testing approach in this great package. We can start from discussing following points:

1. Why travis is not fully integrated with project? Usually CI systems use github's checks to give info about build result etc which is really handy - example of github checks with travis
2. What do you say for migrating from travis to github actions and workflows?
3. Is tox really needed? Maybe travis build matrix or github workflows strategy matrix will be enough?
4. What do you think about services like codecov? It's free for open source projects and can help us to get 100% code coverage and then keep it as this level

Forbid self imports

There's a very useful feature in pylint called self-import.
What it does?

It disallows to write code as this one:

# ex.py
import ex  # self import
from ex import *  # self import

print('init!')

I guess it might be a good idea to forbid this kind of imports by import-linter.

I have tried to implement it inside wemake-python-styleguide, but I am not sure that it is the best place to do it.
Original: wemake-services/wemake-python-styleguide#699

What do you think?

Forbidden contracts should support self forbidding

A potential use case for a forbidden contract is along these lines:

[importlinter]
root_package = tests

[importlinter:contract:tests-restrictions1]
name = Explicit import restrictions for tests
type = forbidden
source_modules =
  tests
forbidden_modules =
  tests

Perhaps this contract should prevent any module within tests importing any other module within tests. Currently, it will fail with the error 'Modules have shared descendants.' This is because it's trying to see if any direct imports exist between the two packages, and Grimp thinks that doesn't make sense.

Documentation

Domain tests for number of files and dependencies analysed

It should be possible to tell, across all contracts, how many files and dependencies were analysed.

Target files not only packages.

Hi, thank you for the useful project!

Is it possible to setup import linter to validate my test suite, not my packages?

The source code layout looks like this:

src/
  _priavte/
    __init__.py
  public/
    __init__.py
tests/
  test_one.py
  test_two.py

I want to test modules only touches public package and not private.

Is it possible to archive with import linter?

Best regards,
Artem.

Ability to group modules together

Currently the various contracts require one module per line.
It would be nice to be able to have multiple per line and to have that be semantically different from when they are on separate lines.
Specifically I'm thinking that where there are two on one line it would say nothing about the relationship between the two just that they have the same relationship to the others.
Formally where there are two on one line it would be equivalent to evaluating the rule twice, one time with each of them in that space.
With regard to one of my current projects this would let me write this:

[importlinter:contract:4]
name = Config can't import Runner et al
type = forbidden
source_modules =
    kedro.config
forbidden_modules =
    kedro.runner
    kedro.io
    kedro.pipeline

[importlinter:contract:5]
name = Runner et al can't import Config
type = forbidden
source_modules =
    kedro.runner
    kedro.io
    kedro.pipeline
forbidden_modules =
    kedro.config

as

[importlinter:contract:4]
name = Config and Runner et al are independent
type = independence
modules =
    kedro.config
    kedro.runner kedro.io kedro.pipeline

Domain tests for ignored paths - layers contract

It should be possible to ignore paths in a layers contract.

Domain test for details of broken contracts

The offending import chains, together with details of each direct import, should be available: e.g.

mypackage.data.foo imports mypackage.domain.bar:

mypackage.data.foo ->
mypackage.utils ->
mypackage.domain.bar

Command line argument: config filename

Contracts on source code and tests

Hi, @seddonym! Thanks for this package once again 🙂

I try to achieve the following:

1. Source code cannot import anything from tests/
2. Anything inside tests/ directory cannot import from tests/ as well

By doing this I try to encourage people to use pytest with its fixtures and plugins approach.

Currently I have this configuration:

[importlinter]
root_package = wemake_python_styleguide
include_external_packages = True

[importlinter:contract:tests-restrictions]
name = Explicit import restrictions for tests
type = forbidden

source_modules =
  wemake_python_styleguide
  tests

forbidden_modules =
  tests

But, it does not work as I want it to work:

» lint-imports
Module 'tests' does not exist.

What can this be achieved? Thanks again.
Original issue: wemake-services/wemake-python-styleguide#720

Containers for forbidden and independence

Containers seem to really be a prefix system. Repeat all of these rules prefixing them with each of the container names. That seems like a generally useful bit of functionality that would be handy on most if not all contract types. Do you agree?

Custom contract types don't inherit fields

As per bug reported #44.

If you just subclass a contract, you get the following error when running the linter:

'ListField' object is not iterable

The workaround is to redeclare the fields:

class CustomLayersContract(LayersContract):
    containers = LayersContract.containers
    layers = LayersContract.layers
    ignore_imports = LayersContract.ignore_imports

Improve handling of single values passed to multiple value field

It's usually ok to pass just one value to a multivalue field. Instead of erroring, it should just make a one-element list.

Debug mode

Would be useful to be able to pass --debug and then it would print the stack trace of errors.

Improve factoring of contracts

There's a lot of shared code and it's not easy to understand, particularly in relation to descendants - nested loops and the like.

One possible route might be to improve as_subpackages support within Grimp.

General review of whole code base

Review whole code base for opportunities for simplication, especially around types.

Also maybe the separation of domain and application isn't useful.

Errors aren't outputted to users

This PR indicates that errors are not always output to users' consoles: #55

I haven't been able to reproduce this myself yet.

Command line arguments: verbosity

Support for multiple packages within a single directory

hey,
thanks for the package, this is really what I need for my project!

There's one thing I don't get though. What if I want to enfoce import rules at the root of my project, not at one of its modules?

For example, I have a django project with a core module with some utilities and an apps module with several django apps. Is there a way to prohibit anything from the apps module to import the core module? What should be placed in the root_package setting then?

Complexity contract

Hi, @seddonym!

I have an idea about the new contract type: package complexity. I am not sure about the technical part of this issue, but here's the motivation.

[importlinter:contract:grouped-modules]
name = We ensure that all modules are properly grouped
type = complexity

source_modules =
  wemake_python_styleguide

max_complexity = 5

What do we do here? When a package has more than 5 modules, we raise a violation. Why do we need it? Because this allows users to better group their modules together. And not just leave them in the main package with like 15 slightly-related files.

I guess, that it is a good fit for this project. And what do you think?

Improve functional tests

Forbid transitive imports

Hi!

I am thing about if this is on scope of this project.
I really dislike transitive imports in my projects.

Like:

# module.py
from a import b

And then:

from module import b  # ???

What do you think about it?

Test for parsing file into contract

Probably yaml?

Fix broken build on master

Discover .importlinter file too

Get travis-ci pull request integration working again

It looks like this has now been turned off on travis-ci.org, need to migrate to travis-ci.com.

Forbidden Contracts: Only Forbid Direct Imports

I was wondering if these was a way to configure the Forbidden Contract to only forbid direct imports. Perhaps with a boolean flag. This would be quite useful instead of having to add all the indirect imports to ignore imports directly. In this case, I want to allow indirect imports to a module, but only allow indirect imports. I want to forbid direct imports.

Duplicate ignored_imports lead to confusing error

If you include an ignored import twice in a contract, you get the following error (as it tries to remove it the second time):

The edge mypackage.foo-mypackage.bar not in graph.

Support for including external packages

Exclude modules from the graph, according to wildcard expressions

Allow ignored regex expressions in layers contract.

As a example y don't want my *.tests.* modules to be processed.

Better reporting of syntax errors in code

If the code under analysis has a syntax error, the output of lint-imports is a bit unclear.

For example:

test.py

from someimport other # missing space between some and import

...
And when I run lint-imports that's what I see:

» lint-imports
invalid syntax (, line 9)

Thanks @sobolevn for reporting this.

Make parsing of multiple lines from INI files more robust

As part of this, ignore paths aren't parsed correctly.

Forbidden imports contract

Ignore imports parsed from config

Improve reporting when an import is not in the graph

Usually says: either source testpackage.high.blue.one or target pytest is not in G.

Ignore paths for independence contract

Feature request: pyproject.toml intergration

Would be cool if import-linter would intergrate with pyproject.toml (PEP-518).

[tool.importlinter]
root_package = "myproject"

[[tool.importlinter.contract]]
name = "Foo doesn't import bar or baz"
type = "forbidden"
source_modules = [ "myproject.foo" ]
forbidden_modules = [
  "myproject.bar",
  "myproject.baz"
]

Improve error reporting when unrecognised contract type

Currently it just prints the unrecognised contract type name.

Domain tests for ignored paths - independence contract

List all the missing ignore-imports at once

Similar issue on Windows as for `layer-linter`

Currently getting the same error as layer-linter got initially on Windows.

Missing layer in container 'pymedphys.coll': module pymedphys.coll._level1 does not exist

Could there be an issue with Windows path separator as before?

Add error message when __init__.py is missing in a root package

The tool is awesome and works very well, is just that it fails silently on this condition which gives a bad experience

Validation system for contracts

E.g. in _strings_to_direct_imports

Optional layers

i.e. supporting the same behaviour as in layer linter.

Support for OnlyAllowContract

Thank you for your great linter. This is exactly what I was looking for.

It would be great if it would support an OnlyAllow Contract, which should only allow the defined imports and no others. This would especially be handy with include_external_packages=False. This way you could simple code your architecture and don't have to define the inverse of your architecture every time.

Thanks in advance!

switch to poetry

poetry is becoming new standard when it comes to dependency management of python packages and it could help this great project in following ways:

• one source of truth for all dependencies (also dev) - currently dev dependencies are defined in tox.ini and main in setup.py
• easy way to run black or isort locally - when I was working on PR I had some troubles with applying black on the code - I have installed the freshest version from PyPI, but it formats some things diffrently than version used in the tox - e check that is listed is tox.ini. When this issue will be rejected, at least some more info should be added to CONTRIBUTING.rst
• easy way to add new dev dependencies like for instance isort or other that this project can benefit from
• easy integration with CI
• easy integration with dependabot and similar solutions, so always the most recent dependencies can be used (at least dev ones)
• all features listed on poetry web page like easier publishing

Support wildcard references to modules in contracts

I have this directory structure: https://github.com/wemake-services/wemake-python-styleguide/tree/ae821ddb0b9c234870953ed5a17cd3858a52ee3c/wemake_python_styleguide/visitors

Every module in any module should be respect type = independent contract.
But, I have like dozens of modules, and new ones are created like once a week.

What do I want? I want to enforce independent contract on all modules in a package and sub-packages. And mark several exceptions: base and decorators in my case. Something like this:

[importlinter:contract:independence]
name = Independence contract (all shall be free!)
type = independence

modules =
  wemake_python_styleguide.visitors.**.*
  # Or at least something like this will do:
  wemake_python_styleguide.visitors.ast.*
  wemake_python_styleguide.visitors.tokenize.*
  wemake_python_styleguide.visitors.filenames.*
ignore_imports =
  # It is ok to be related on these modules:
  wemake_python_styleguide.visitors.base
  wemake_python_styleguide.visitors.decorators

Is it possible?
Original issue: wemake-services/wemake-python-styleguide#720

Add topics for this repository

I'm probably going to forget about this entirely by the time I'll need to implement this in my project.
It'd be easier to find this project if it had topics such as linter, python etc.

Contract type plugin system

At the moment the config file requires the classes to be named.

Better to have some built in contract types, and then you declare / configure extra contract types by specifying class.