seddonym / layer_linter Goto Github PK

• Layer Linter version: 0.12.0
• Python version: 3.7
• Operating System: Windows 10

Description

I appear to be having issues with layer-linter on Windows.

What I did

My layers.yml file looks like the following:

https://gitlab.com/pymedphys/pymedphys/blob/cd8c143ab72121ae8fa53d4797d4be9eff178292/layers.yml

Of note is the following:

    layers:
        - (_level3)
        - (_level2)
        - _level1

It appears that although _level1 does exist within the modules I am getting errors like the following:

Missing layer in container 'pymedphys.coll': module pymedphys.coll._level1 does not exist.

Caveat

Weirdly the error appears to occur on Windows but not Linux

See the lack of error within the following pipeline:

https://gitlab.com/pymedphys/pymedphys/-/jobs/152621905

I'll see if I can set up an AppVeyor account and see if it is replicated on there.

Layer linter works on Python 3.7, but it isn't in the tox run yet due to travis not supporting it.

Currently pyup is only checking requirements_dev.txt. The main requirements are actually in setup.py.

It would be good if layer-lint --version printed the version.

• Layer Linter version: 0.11.0
• Python version: 3.6.7
• Operating System: macOS

What happened

My Python project has this (simplified) structure:

mypackage/
    api/__init__.py
        api.py
    data/__init__.py
         api.py
    config/__init__.py
           config.py

Note that the contents of mypackage/data/__init__.py looks like:

from .api import some_function
__all__ = ['some_function'] 

Since I would like to impose the hierarchy mypackage.api > mypackage.data > mypackage.config, I wrote a layers.yml:

Simple Contract:
  containers:
    - mypackage
  layers:
    - api
    - data
    - config

What went wrong

When I run layer-lint mypackage, layer-lint complains that:

1. mypackage.data imports mypackage.api:

What I expected

I would not expect the contract to be broken: the contract requires that mypackage.data does not import the package mypackage.api. Here, mypackage.data imports it's own private module mypackage.data.api which should be legal under the contract.

Also, I noticed the documentation says that:

Remember, a Python module can either be a .py file or a directory with an __init__.py file inside.

In my experience, a .py file is referred to as a module, while a directory with an __init__.py is referred to as a package.

It is not very obvious if someone includes a layer that is invalid or nonexistent. This is a particular problem when getting started (e.g. misunderstanding the format of layers.yml), but it could also happen due to a typo.

However, it may not be correct to error if a layer cannot be found, as it should be possible (at least in the case of multiple package contracts) to leave out certain layers for certain packages.

Layer Linter currently doesn't import modules that contain the word 'migrations', in order to avoid importing Django migrations.

It would be better if this could be configurable on a project-by-project basis. Also excluding 'node_modules' might be a sensible default, as Python files are sometimes included there.

"A closed layered architecture means that any given layer (or tier) can only use the services of the next immediate layer, whereas in an open layer architecture a given layer can make use of any of the layers (or tiers) below it. Closed layer architecture promotes independence between layers by reducing dependencies."
https://stackoverflow.com/a/904280/1005499

The contract might look something like this:

My contract:
    containers:
        - mypackage.apps.foo
        - mypackage.apps.bar
    layers:
        - api
        - [closed] views
        - [closed] forms
        - models

This would prevent anything from anywhere else in the codebase from reaching down below the closed layers to layers underneath. This would effectively make api and views in each app a public interface, while forms and modules are private to the app.

This commit fixed a bug with quiet mode, but we should have a test to go with it too:

f1bf820

• Layer Linter version: 0.10.3
• Python version: 3.6
• Operating System: Ubuntu

Description

#74 does the opposite of what I want - I wanted to have layers defined for multiple containers, which will not necessarily all contain all of the layers. I have downgraded to 0.10.1 for now, as it seems to function how I would like. Adding syntax for optional layers, or a configuration option, would be helpful.

This shouldn't be a success:

============
Layer Linter
============

---------
Contracts
---------

Analyzed 0 files, 0 dependencies.
---------------------------------

Architecture KEPT

Contracts: 1 kept, 0 broken.

• Layer Linter version: 0.12.0
• Python version: 3.6
• Operating System: Ubuntu 18.04

Description

Hi! :) We've just begun implementing this library within our project:

https://gitlab.com/pymedphys/pymedphys/blob/varian-logfiles/layers.yml

It's amazing! :) Thank you very much. Also, it plays quite nice with pytest:

https://gitlab.com/pymedphys/pymedphys/blob/varian-logfiles/tests/test_project_architecture/test_layering.py

I do have one request. In your rocky river architecture (http://seddonym.me/2018/09/16/rocky-river-pattern/) you produced a diagram. In that particular diagram the river itself was also layers/levelled. You essentially had 5 layers in that rocky river. I have circled the layers below:

What I was hoping was to be able to do is enforce a structure where those modules are on the same level as each other, unable to depend on each other, but all able to depend on those lower than them.

A way I thought that this might look in the yaml file is:

Top Level Modules:
    containers:
        - pymedphys
    layers:
        - logfile
        - trf
        - msq
        -
            - mudensity
            - pddprofiles
            - gamma
            - coll
        - 
            - electronfactors
            - film
            - sinogram
            - dcm
            - utilities
        -
            - deliverydata
            - geometry
            - plt
            - filehash
            - xarray
        - libutils

Essentially any module within a nested list are all considered on the same level as each other. Any of those cannot import from each other, but can import from those lower than them.

What do you think?

I love the idea of this tool, and I've started to add it to my projects at home and at work!

I was adding some color/additional-requests to #91 , and I realized what I was asking for was actually different enough that I should open a different issue.

Given a package structure like:

package/
  adapters/
    front_end.py
    back_end.py
  core/
    data.py
    logic.py

...in plain-ish english, what I'd like to have a contract for is a hexagonal-ish architecture:

• anything in the project can import from the standard lib
• anything in the project can import from core.data
• any adapter can import from third party sources
• adapters.front_end may import from core.logic
• nothing else is allowed (adapters.front_end may not import adapters.back_end)

I've mocked out a few of possible representations of this... I like the last one the best:

Nested Patterned:
    package:
        *:
            - standard library
            - core.data
        adapters:
            *:
                - third party
            front_end:
                - ..core.logic

Flat:
    package.adapters.front_end:
        - standard library
        - third party
        - package.core.logic
        - package.core.data
    package.core.logic:
        - standard library
        - package.core.data
    package.core.data:
        - standard library
    package.adapters.back_end:
        - standard library
        - third party
        - package.core.data


Flat Patterned:
    package.*:
        - standard library
        - package.core.data
    package.adapters.*:
        - third party
    package.adapters.front_end:
        - package.core.logic

This... is possibly an unreasonably large departure from the current contract format, so I understand if you'd rather not go down this route. If not, though - do you have a preferred/recommended/planned way to implement the same contract?

Regardless, thanks for your work on this project - it's a big step up as-is!

It would be great if the linter could have a quiet mode, where it doesn't output anything on success.

Currently when we do a release, the test run actually fails:

https://travis-ci.org/seddonym/layer_linter/builds/450960486

This is because the conditional deploy in the .travis.yml requires Python 3.6, which multiple items in the Travis matrix use. After the first one succeeds, the rest fail.

This commit fixed a bug that prevented path whitelisting from working, but we should have a test that goes with it:

0c9b64d

It might be useful to have the ability to add layer linter to the build process without a codebase fully adhering to the contract. Perhaps some exclusions could be added to the layers.yml.

The intended use case would be to allow layer linter to prevent further breaking of the contract, despite their being some imperfections (that could be treated as technical debt).

Eg, I'd like the ability to say that modules in utils may not import from from any path that matches apps.*.models.

• Layer Linter version: >=0.12.0,<0.13.0
• Python version: 3.6.6
• Operating System: macos

Description

I am trying to use click@7 with layer-linter.
And the version resolution fails.

What I Did

I am using poetry to manage my dependencies.
To reproduce my issue:

1. Install the latest version of poetry
2. Create a new project with poetry init
3. Run poetry add click and specify a version 7
4. Run poetry add --dev layer-lint

You will see something like this:

» poetry add click
Using version ^7.0 for click

Updating dependencies
Resolving dependencies... (0.1s)

[SolverProblemError]
Because no versions of layer-linter match >0.12.0,<0.13.0
 and layer-linter (0.12.0) depends on click (>=6.7,<7.0), layer-linter (>=0.12.0,<
0.13.0) requires click (>=6.7,<7.0).
So, because wemake-python-styleguide depends on both click (^7.0) and layer-linter
 (^0.12.0), version solving failed.

I think it's because there's RST directives that PyPI can't understand - fixing now 🤞

Currently import errors break the command but without it being obvious what's going on. Instead any issues should fail loudly when running the command. The files with the import errors can then either be fixed, or be excluded (see #19).

My contract:
    containers:
        - mypackage.apps.*
    layers:
        - views
        - models

In this case, any apps within mypackage.apps would be linted to check that their models don't import from views.

The workaround at the moment is just to list all the containers explicitly.

• Layer Linter version: 0.4.0 (but layer-lint --version does not work)
• Python version: 2.7.12
• Operating System: Ubuntu 16.04

Description

Running the program

What I Did

$ layer-lint test
Could not find package 'test' in your path.
$ echo $PYTHONPATH 
/home/flap/Dev/jlmcc/apps
$ pwd
/home/flap/Dev/jlmcc/apps
$ ls test
__init__.py  [...]
$ more layers.html 
rental:
    packages:
        - test
        - test/rental
    layers:
        - core
        - domain

I expect to have more infos in the doc to make this first __import__ works

I am using a sort-of-hexagonal-architecture.

So, my use-case is: I want to forbid imports of django (in my case, can be flask, etc) inside my domain level.

Currently it seems like an impossible task.
But, it seems rather practical for people who care about their architecture.

Proposed syntax:

Global architecture:
  containers:
    - common
  layers:
    - domain
    - @django

Or even new field can be applied:

Global architecture:
  containers:
    - common
  layers:
    - domain
    ...
  constraints:
    - @django

The bot created this issue to inform you that pyup.io has been set up on this repo.
Once you have closed it, the bot will open pull requests for updates as soon as they are available.

Pyup is more for pegged versions, I'm not sure this is appropriate for layer_linter.

If there is a syntax error in a container module, the linter will crash. Better to handle the SyntaxError gracefully.

• Layer Linter version: latest
• Python version: 3.6
• Operating System: macOS

Description

Layer linter takes a while to run on a moderately sized codebase. This is understandable as there's a lot to do, but I noticed that it only maxes out a single logical core. Multi-threaded execution could provide a significant speed boost in the same way that it does for tools like flake8.

What I Did

Ran layer-lint with a single contract covering 400 packages with 14 layers. Time on my MBP is 67 seconds.

Hi, thanks for the awesome tool.

I am thinking about adding it to the https://github.com/wemake-services/wemake-django-template
But, the only feature that it totally required for me is setup.cfg support.
Since it it the standard way to configure python tools.

I guess it is suitable for the configuration you have to write something like:

[tool:layer-lniter]
layers = 
   first
   second

How does it sound to you?