gauge-sh / tach Goto Github PK

When trying to update to v0.5.0, and running as pre-commit hook, I'm getting an installation error on my CI:

$ pre-commit run --all-files
...
[INFO] Initializing environment for https://github.com/Never-Over/tach.
...
[INFO] Installing environment for https://github.com/Never-Over/tach.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
An unexpected error has occurred: CalledProcessError: command: ('/home/circleci/.cache/pre-commit/repobo16dqb4/py_env-python3.12/bin/python', '-mpip', 'install', '.')
return code: 1
stdout:
    Processing /home/circleci/.cache/pre-commit/repobo16dqb4
      Installing build dependencies: started
      Installing build dependencies: finished with status 'done'
      Getting requirements to build wheel: started
      Getting requirements to build wheel: finished with status 'done'
      Preparing metadata (pyproject.toml): started
      Preparing metadata (pyproject.toml): finished with status 'error'
stderr:
      error: subprocess-exited-with-error
      
      × Preparing metadata (pyproject.toml) did not run successfully.
      │ exit code: 1
      ╰─> [6 lines of output]
          
          Cargo, the Rust package manager, is not installed or is not on PATH.
          This package requires Rust and Cargo to compile extensions. Install it through
          the system's package manager or via https://rustup.rs/
          
          Checking for Rust toolchain....
          [end of output]
      
      note: This error originates from a subprocess, and is likely not a problem with pip.
    error: metadata-generation-failed
    
    × Encountered error while generating package metadata.
    ╰─> See above for output.
    
    note: This is an issue with the package mentioned above, not pip.
    hint: See above for details.
Check the log at /home/circleci/.cache/pre-commit/pre-commit.log

Exited with code exit status 3

The error seems to be because Rust isn't installed. I assumed that was because you didn't provide pre-built wheels but I just checked on PyPI and I can see there are lots of them, so I'm a bit confused as to why it failed...

We're using a Docker image cimg/python:3.12 which is the default Python image from Circle CI. I can try to dive deeper if you want more details about the flavour of linux it uses...

The way the dependencies of Tach are defined is very strict (only using ==)

dependencies = [
    "pyyaml==6.0.1",
    "pydantic==2.7.1",
    "rich==13.7.1",
    "prompt-toolkit==3.0.43",
    "stdlib-list==0.10.0; python_version < '3.10'",
    "eval-type-backport==0.2.0; python_version < '3.10'"
]

This means that folks that want to install tach have to have these exact dependencies or it won't install.
As tach will most of the time not be installed in its own venv, this is problematic.

Could you be open to a PR that would relax the dependencies?

The implicit root boundary appears in the modules list in tach.yml with the path: <root>

Right now, when this module exists, it is always at the beginning of the modules list (sorted lexicographically). This is not ideal, because as a reader (especially a new user), the meaning of <root> is hard to parse before first seeing the rest of the modules. This is because a reasonable definition of the root boundary is the 'leftover' code - any code which is not covered by another module.

Given this, it is easier to read the explicit modules first, and then see the 'leftovers' at the end of the file. Tach should explicitly sort this module last when writing the project config to yml.

Right now, if tach test determines that all tests can be skipped, the exit code from pytest is 5. This might mean that we should adjust the way the plugin operates (there may be a more native way to indicate tests should be skipped), adjust pytest's configuration, or maybe we need to diverge from pytest's behavior here and manually return zero in this case.

Example:

.venv/bin/tach test
============ Cached results found!  ============
==================================================== test session starts =====================================================
platform linux -- Python 3.12.3, pytest-8.2.2, pluggy-1.5.0
rootdir: /home/evan/Programming/tach
configfile: pyproject.toml
plugins: mock-3.14.0
collected 85 items                                                                                                           
[Tach] Skipped 10 test files (85 tests) since they were unaffected by current changes.
[Tach] > Skipped '/home/evan/Programming/tach/python/tests/test_affected_modules.py'
[Tach] > Skipped '/home/evan/Programming/tach/python/tests/test_parsing.py'
[Tach] > Skipped '/home/evan/Programming/tach/python/tests/test_check.py'
[Tach] > Skipped '/home/evan/Programming/tach/python/tests/test_file_tree.py'
[Tach] > Skipped '/home/evan/Programming/tach/python/tests/test_module_tree.py'
[Tach] > Skipped '/home/evan/Programming/tach/python/tests/test_cli.py'
[Tach] > Skipped '/home/evan/Programming/tach/python/tests/test_changed_files.py'
[Tach] > Skipped '/home/evan/Programming/tach/python/tests/test_report.py'
[Tach] > Skipped '/home/evan/Programming/tach/python/tests/test_imports.py'
[Tach] > Skipped '/home/evan/Programming/tach/python/tests/test_cache.py'

=================================================== no tests ran in 0.28s ====================================================
============ END Cached results  ============
make: *** [Makefile:41: test] Error 5

I am working on a Python project that has the exact same problem described here. We also have a homemade tool similar to (and less mature than) Tach.

One thing our tool supports and Tach does not is "module visibility".
When developers add a module, they could express "This module can be imported only in module X and Y (or any module in package Z)" with our custom tool.
This feature is inspired by "target visibility rule in Bazel"

I wonder if it is possible for Tack to support this feature.

Let me start by saying that I love this! It was super easy to get setup and start seeing value.

I was wondering if y'all have considered adding a "strict" mode that causes tach check to fail if it sees dependencies in the tach.yml file that aren't actually present. While this might not be part of tach's original vision, it would allow the tach.yml file to also represent the current DAG of dependencies in a project. Not sure how often this would happen in practice, but it may be useful to help trim the transient dependencies that existed before adding tach to a CI pipeline.

Thanks!

Right now, the default behavior of sync is additive, meaning that it will not remove any modules or configured dependencies. Instead it will add dependencies to existing modules to fix any boundary errors detected.

When using the --prune option, sync ignores the currently configured dependencies, and instead sets the dependencies for each module to the minimal set matching the actual imports found in the codebase. This behavior is more consistent with the idea of "syncing" your dependencies, and should be the default behavior when calling tach sync with no CLI arguments.

The current additive behavior should be enabled by passing a renamed flag: --add
That is, tach sync will prune dependencies to the minimal set, while tach sync --add will add any missing dependencies.

NOTE: This issue interacts with #145 , and may require coordination if both are worked on simultaneously

Right now, public only operates at the level is which it is defined. The expected behavior would be that it encompasses all members and sub-members of the module that are enclosed in the same boundary.

Ref #51

This is a dependency for allowing centralized config in pyproject.toml (#49)

We should change the type of 'constraints' in tach.yml from a dictionary with tags each as keys into a list of dictionaries, with each dictionary specifying its 'name' as a string. This avoids any issues with special characters like dots and colons when using TOML.

Current:

constraints:
  scope:filesystem:
    depends_on:
    - scope:utils
  scope:parsing:
    depends_on:
    - scope:core
    - scope:filesystem
  scope:root:
    depends_on:
    - scope:utils
    - scope:core
    - scope:filesystem
    - scope:parsing
exclude:
- tests/
- docs/
exclude_hidden_paths: true

Proposed:

constraints:
  - name: scope:filesystem
    depends_on:
    - scope:utils
  - name: scope:parsing
    depends_on:
    - scope:core
    - scope:filesystem
  - name: scope:root
    depends_on:
    - scope:utils
    - scope:core
    - scope:filesystem
    - scope:parsing
exclude:
- tests/
- docs/
exclude_hidden_paths: true

While profiling Tach during #82 , I noticed that parse_project_config was taking ~100ms, even though it should be parsing a relatively tiny file into a relatively simple data structure. Instead of trying to improve this in Python, it makes more sense to just extract this to the Rust extension and use something like serde.

Either in the same PR or a separate one, it would also make sense to move the dump_project_config_to_yml into the Rust extension as well.

Ideally all of this can be done in a way that is minimally coupled to YML, since we may want to migrate to TOML in the future.

I was trying Gauge on https://github.com/posthog/posthog, and ran gauge show --web, but the page it leads to is empty. https://show.gauge.sh/?uid=6eb4ed80-76cc-43fc-941e-f0238848a221

One of my main complaints about the JavaScript ecosystem is the root of the project is cluttered with hundreds of individual tool-specific configuration files, python has mostly been able to avoid this by centralizing most configuration to the pyproject.toml file.

I would like to be able to configure tach entirely from the pyproject.toml file.

Potential implementation details: It is possible (and maybe even common?) for someone to use both pyproject.toml and the existing configuration files. There should be a single, correct, determinstic way to load and merge configs from all these files. In most projects that do this the pyproject.toml file is loaded first, and the tach-specific config files overlay on top of them if they are present.

Potential issues: Native ability to read TOML files only recently came to the python standard library, for older versions you will have to bring in another 3rd-party dependency.

Example from another tool: https://import-linter.readthedocs.io/en/stable/toml.html#toml-support

Right now there's the ability to exclude specific submodules, like tests:

exclude:
- tests
- docs

It would additionally be helpful to exclude all imports under an if TYPE_CHECKING block as not being counted against breaking module boundaries. importer-linter is an example of a tool that supports this.

from module_a import ...

if TYPE_CHECKING:
   from module_b import ... # Boundary for module_b is not violated

As mentioned in #101, it's confusing that exclude_paths uses regex instead of file globbing.

Conceptually, this behavior should match that of other linters like ruff. Using that example, it makes sense to support globset:
https://docs.astral.sh/ruff/settings/#exclude
https://docs.rs/globset/latest/globset/#syntax

This will require updating the docs to explain the new behavior.

Does this project support python 3.8? From the pyproject.toml it appears that it does, but the codebase uses python-3.9-style type annotations which subscript the builtin dict type.

$ tach -h
Traceback (most recent call last):
  File "/home/brianm/repos/medigator/venv/bin/tach", line 5, in <module>
    from tach.cli import main
  File "/home/brianm/repos/medigator/venv/lib/python3.8/site-packages/tach/cli.py", line 8, in <module>
    from tach.check import check, BoundaryError
  File "/home/brianm/repos/medigator/venv/lib/python3.8/site-packages/tach/check.py", line 5, in <module>
    from tach import filesystem as fs
  File "/home/brianm/repos/medigator/venv/lib/python3.8/site-packages/tach/filesystem/__init__.py", line 1, in <module>
    from tach.filesystem.service import (
  File "/home/brianm/repos/medigator/venv/lib/python3.8/site-packages/tach/filesystem/service.py", line 46, in <module>
    def _get_file_cache() -> dict[str, FileInfo]:
TypeError: 'type' object is not subscriptable

$ python -V
Python 3.8.19

$ pip show tach
Name: tach
Version: 0.2.2
...

Can you add the ability to configure the root directory tach check pre commit runs from?

Looking to support a monolith structure like so:

app/
    /frontend
    /backend

Hi! This tool looks very promising to my use case but I've stumbled upon what I believe is a bug.

I have a directory layout which looks similar to this one:

my_project/
├── package1
│   └── tests
└── package2
    └── tests

expect I have many more sub-packages. Let's say package1 cannot import anything from package2, but I want to allow all imports in every tests package. I provided the following values to the exclude setting in my tach.yml:

exclude:
- my_project/*/tests/
- tests

But tach is still reporting illegal imports from package2 in package1.tests.

To see this behavior, you can add:

- path: notarealmodule
  depends_on: []
...

into your tach.yml, and then run tach sync.

Right now, the module notarealmodule will remain in the tach.yml file.
This PR should instead remove all modules listed which cannot be resolved from the configured source root.

While planning work on Issue #49 , I realized that we should restrict and validate package 'tags'. This makes TOML support much more straightforward, and will probably be useful in general if we end up using 'tags' as input to other commands or functionality.

We should use a regex like the following:
^[a-zA-Z0-9]([a-zA-Z0-9_-]*[a-zA-Z0-9])?$
which would allow only alphanumeric characters, underscores, and dashes, while also always starting and ending with an alphanumeric character.

Describe the issue

There's a broken image in the Getting Started doc.

I use a Python monorepo with many scripts, libraries and many microservices (and their number grows over time), where all Python modules are directly importable from Git repository root without any need of packaging/editable installation:

bin/
    __init__.py
    script1.py
    script2.py
    ...
lib/
    __init__.py
    library1/
        __init__.py
    library2/
        __init__.py
        module1.py
        module2.py
    ...
svc/
    __init__.py
    service1/
        __init__.py
    service2/
        __init__.py
        module1.py
        module2.py
    ...

Rules:

• Libraries can be imported by libraries, scripts and services
• Services cannot be imported by other services
• Scripts cannot be imported
• Only public symbols can be imported when crossing a boundary of a library/service

How can I configure tach to setup the import rule enforcement without the need for a reconfiguration when a service/library is added?

I think I'd need a support for glob patterns in path and depends_on + support for import from siblings and parents:

modules:
- path: lib/**
  depends_on:
  - lib/**
  strict: true
  siblings: true
  parents: true
- path: svc/**
  depends_on:
  - lib/**
  strict: true
  siblings: true
  parents: true

Glob patterns would avoid the need for manual specification of individual modules/packages and their dependencies.
Siblings would allow imports from the same filesystem folder.
Parents would allow imports from modules located in parent filesystem folders.

Would it make sense to add support for such scenario? Or is already there a way how to achieve it?

I'm in the process of untangling the dependencies of a project, and trying to figure out why a certain module, say utils, depend on another module, say blog. The goal is to remove blog from the depends_on: list of utils.

The most helpful way I have found is to comment out blog from the depends_on: list of utils and then run tach check to get a list of modules that causes that dependency. This is a bit cumbersome though.

Would it be possible to add something like a --filter option to tach report to only show dependencies of a certain module (i.e. tach report utils --filter=blog) to achieve the same the same goal? Right now tach report lists every single dependency, including dependencies within the module itself, which is just too much output to be useful.

When the source_root in a project has been adjusted, the absolute path used in the create_clickable_link function is not correct.

[The issue is that the path should be ...Programming/tach/python/tach/__main__.py because the source_root is 'python']

The display path should stay the same, but the absolute path needs to be modified since it is relative to the source root.

This change is a bit subtle, since ProjectConfig.source_root is itself relative to the project root.

Here is my pre-commit config file

default_language_version:
  python: python3.12
repos:
  # Check dependencies between apps
  - repo: https://github.com/Never-Over/tach
    rev: "v0.2.0"
    hooks:
      - id: tach

When I try to commit, I get:

tach.....................................................................Failed
- hook id: tach
- exit code: 2
usage: tach [-h] {pkg,check,install,sync,clean} ...
tach: error: unrecognized arguments: --root

Seems like it was added in #62

I don't see any mention of the --root option in the help of tach check:

 ➜ tach check --help       
usage: tach check [-h] [--strict] [-e file_or_path,...]

Check existing boundaries against your dependencies and package interfaces

options:
  -h, --help            show this help message and exit
  --strict              Raise errors if any dependency constraints are unused.
  -e file_or_path,..., --exclude file_or_path,...
                        Comma separated path list to exclude. tests/, ci/, etc.

What am I missing?

Some projects would like to ensure that their project's dependencies remain unidirectional, or in other words, that there are no cycles in their dependency graph.

A simple way to begin supporting this is to add a setting like forbid_circular_dependencies which can be set in tach.yml (ProjectConfig within the Tach codebase). When this setting is true, tach check should fail with a specific error message when a circular dependency is detected.

This should likely be done as a "pre-flight check" which would validate the modules list, and could be delegated to the ModuleTree internally. It does not need to apply any new logic to actual detected imports.

I'd like to use this project in my Python 3.7 project. Is it something you consider supporting? What is preventing it

I don't mind to do some work to make it happen if it can be accepted into the project.

Right now, the tach.yml in our repo is formatted with 2 space indents at each level of nesting, and there is a language server comment directive at the top of the file.

After running any variant of tach sync however, the language server directive is eliminated, and the formatting reverts to be minimally indented.

It would be much nicer if tach sync could preserve formatting, but at a minimum it should preserve the language server directive.

Why did you choose not to use all instead Boundary objects and decorators?

I am using tach on my windows OS, and whenever I use to run any tach command it gives me this error

Process Process-1:
Traceback (most recent call last):
  File "C:\Python312\Lib\multiprocessing\process.py", line 314, in _bootstrap
    self.run()
  File "C:\Python312\Lib\multiprocessing\process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "C:\Users\dhana\AppData\Roaming\Python\Python312\site-packages\tach\logging\logger.py", line 48, in handle_log_entry
    if os.fork() != 0:
       ^^^^^^^
AttributeError: module 'os' has no attribute 'fork'
Validation error: Module 'C:\Users\dhana\coding\Personal Projects\DeCATv\lucas\tach\python' is not contained within source root: 'C:\Users\dhana\coding\Personal Projects\DeCATv\lucas\tach\python'

I think this is because in the tach.logging.logger::handle_log_entry, the try catch block is does not catch the AttributeError( Indicates not unix-like OS).
I have resolved this for my own use by re-building the tach module by appending a catch for the AttributeError, this removed the error message and works fine. One other way would be to detect the OS using platform module and only then call the os.fork() function but not feasible for this stage. Also, should I raise a PR on this?

It would be great to have the ability to generate a dependency graph based on the tach.yml file in graphviz format.

See Graphviz example). Tach could simply just output a graphviz file to stdout.

Optional features

• Shows the difference between the config and the actual source code. (togglable via commandline flags)
  • Violations are marked
  • Imports that are in tach.yml but do not exist in the code base are marked

Hello! 👋 Thank you for making and distributing tach! ❤️

Does tach support namespace packages? 🤔 I did not see explicit support in the docs. And I've been running into issues trying to adapt tach to my situation. I'd like to use tach in a project that makes use of multiple namespace packages.

The individual packages are all PEP-420 compliant. There is a specific set of inter-dependencies that must be enforced between the namespaced packages. This DAG is currently encoded in each's own install_requires. However, I'd like to use tach to further enforce and check that these inter-namespace package dependencies are (a) all what we intend and (b) identify and flag any module boundary violations. It'd be great if tach would also inspect the install_requires and verify that the dependency graph is correct too. Here, "module" really means "distinct namespace package." Though I think the concepts are well aligned :)

Is this something that is in the near-future for tach?

It would be great if there was a VSCode extension to help with live import linting, similar to what Pylance or the Ruff LSP can do. This way boundaries being broken can be be caught earlier as opposed to later. I wonder if it might be possible to point the CLI tool against the file whenever a module is saved, some plugins do this without taking any advantage of an LSP, for example https://github.com/InSyncWithFoo/pyright-for-pycharm (not VSCode, but the same idea)

What's this issue?

When using conda, running make deps fail. (It must be failing otherwise as well, didn't give it a try).

How to reproduce it?

Here's some logs. Run make deps in a conda env.

(base) ➜  tach git:(main) ✗ make deps       
python -m pip install --upgrade uv
Requirement already satisfied: uv in /Users/girish/miniconda3/lib/python3.8/site-packages (0.2.25)

[notice] A new release of pip is available: 23.1.2 -> 24.1.2
[notice] To update, run: pip install --upgrade pip
Using Python 3.8.11 interpreter at: /Users/girish/miniconda3/bin/python3
Creating virtualenv at: .venv
Activate with: source .venv/bin/activate
Virtual environment created at .venv
source .venv/bin/activate && \
	uv pip install -r dev-requirements.txt
Resolved 48 packages in 398ms
Installed 48 packages in 57ms
 + annotated-types==0.7.0
 + backports-tarfile==1.2.0
 + build==1.2.1
 + certifi==2024.7.4
 + charset-normalizer==3.3.2
 + coverage==7.6.0
 + docutils==0.20.1
...
...
 + typing-extensions==4.12.2
 + urllib3==2.2.2
 + wcwidth==0.2.13
 + zipp==3.19.2
bash: line 1: maturin: command not found
make: *** [deps] Error 127

I have the following use case:

I want to slowly introduce tach and thus slowly remove depends_on from some modules. If I add a comment in the tach.yml it gets overwritten by sync so desirable behaviour for me would be:

Either:

• Do not delete comments
• Have a deprecation annotation
  • tach could support checking violations with all deprecated dependencies removed and without them being removed)
  • automatically remove deprecation annotations on sync if the dependency is not present anymore

when i start tach sync or tach check for example it gives me this back:

Failed to build regex patterns for excluded paths

I have selected some folder instead of a python file. Maybe you can give some advice within a new error text as that above

In a monorepo, top-level packages are defined like this:

.
├── package_1_root
│   ├── package_1  # python source lives here
│   ... # possibly additional data directories
│   └── tests
├── package_2_root
│   ├── package_2 
│   └── tests
...
└── package_n_root
    ├── package_n 
    └── tests

The source is split into subdirectories, but the whole repository consists of a single python package. This layout can be achieved with e.g. poetry:

[tool.poetry]
name = "monorepo"
packages = [
    { include = "package_1", from = "package_1_root" },
    { include = "package_2", from = "package_2_root" },
#   ...
    { include = "package_n", from = "package_n_root" },
]

Then these packages can be used like this from python code:

>>> import package_1.bar
>>> package_1.bar
<module 'package_1.bar' from '/.../package_1_root/package_1/bar/__init__.py'>

I need to set source_root to e.g. package_1, because otherwise tach thinks that the package_<n>_root is part of the module path.

I'd like to enforce dependencies in and between all of these python_<n> packages, so it would be great to be able to tell tach that individual packages can reside under different prefixes.

Right now, if you send a keyboard interrupt (Ctrl + c) during Tach's execution, you'll get an ugly stack trace.

Instead, we should catch a KeyboardInterrupt at a high level and exit gracefully (with a nonzero exit code).

Rather than relying on devs to remember to run the CLI before pushing (or having a CI step on PRs etc)

Is there any JSON schema published for tach.yml? It would be useful to have one and assign it in IDEs for auto completion and validation.