ag91 / code-compass Goto Github PK

When I try a command like c/show-complexity-over-commits. I get the following error:

Sorry, didn't recognize that command!
Type graph --help to view common commands.

Any help is appreciated.

The experimental slack features error out, eg calling M-x c/slack-main-contributor produce:

Symbol’s value as variable is void: slack-current-team

Use fboundp to see if user has slack installed, if not then show message about it as intended

checkdoc is still used for projects, and is used by maintainers to ready up project for release

Will help with #20

First, thank you for an interesting and inspiring project, and for presenting such an effective introduction at emacsconf.

My company's product, a web app, is built on a ten year-old codebase that was written by a much larger team than we have now; it is important for us to not trip over the large amount of code we didn't write ourselves and understand where old code stands in the way of new projects. Code compass looks like a very nice way to measure and communicate these things. However, I'm finding I need to make certain workarounds as I experiment with code-compass in my company's monorepo, and I think I could expand some of these into general improvements. It's worth noting that I am at the beginning of an incremental investigation of the different functions code-compass provides; once I am satisfied with my ability to generate a hotspots visualization, I will move onto other types of report, which may uncover new issues to be solved for my use case.

I find that generating a visualization like (c/show-hotspots path-to-monorepo "3m") visualizes so many files that it causes javascript performance issues in the generated page and makes it hard to drill down on any particular sub-project. I have observed two distinct causes:

1. the sheer amount of potentially-relevant source code in a ten-year-old monorepo containing many sub-projects is just too big. There is a large monolithic rails app, several backend services, and multiple frontend codebases spanning two major frontend frameworks in both javascript and typescript.
2. the javascript codebases each have their own node_modules directory of 3rd party library code, and these are not filtered out by default

The second is simpler, so I am starting there: since the locally-cached library code is kept out of version control by .gitignore, I believe the only necessary step is providing an --exclude-dir argument to the cloc executable. For my personal workaround, I simply added the argument to the command string directly, for convenience, but for general consumption, extracting a customizable list like the following seems reasonable:

(defcustom c/exclude-directories
  '("node_modules" "vendor" "tmp")
  "A list of directory patterns to exclude from reports. Contents are passed to the cloc executable via its --exclude-dir argument."
  :group 'code-compass)

(defun c/produce-cloc-report (repository)
  "Create cloc report for REPOSITORY."
  (message "Producing cloc report...")
  (shell-command
   (format "(cd %s; cloc ./ --by-file --csv --quiet --skip-uniqueness --exclude-dir=%s) > cloc.csv" repository (string-join c/exclude-directories ",")))
  repository)

If the basic approach for filtering out low-value files I outlined above looks reasonable to you, I can open a pull request in the coming days; I filled in my current best guess for sane defaults, though I'm sure it could be improved.

The first issue, where I would like to filter reports to one or more specific project directories within a monorepo, does not have as obvious a solution, either in terms of implementation or user interface. It might be possible to hack filtered reports together by abusing the --exclude-dir argument—it may even be possible to build a clean API on top of such a hack—but that seems suboptimal for several reasons (for example, the git operations would still operate over the entire repo, and I'm still in the process of figuring out how this may affect the generated reports). I will continue to think on this problem as I continue my experimentation and learn the workings of code-compass better, and would welcome any suggestions or targeted question you may be able to offer.

To reproduce:

1. M-x c/show-coupling-graph-sync
2. M-x c/show-coupling-graph-sync

The second time, it fails:

c/copy-file: Opening output file: Permission denied, /tmp/code-compass-<REDACTED>/coupling_csv_as_edge_bundling.py

This is because most of the script files are read-only (linux permissions).

Probably these should have read-write permissions in the Git repo?

The code assumes that:

• /tmp is used when using java -jar xxx.jar value for c/code-maat-command
• /data is used when using the Docker value for c/code-maat-command

An example of such an assumption:
https://github.com/ag91/code-compass/blob/main/code-compass.el#L336

I use neither a Java command nor Docker (I provide directly a code-maat executable via Nix).
In most places the default used is /tmp when not Docker, but in this line of code (and maybe others?) it uses /data when not Java.

It means that my set-up breaks because it sometimes uses /tmp and sometimes /data.

How about the path where dummy directories are created becomes configurable?

• it makes less assumptions about the running system (e.g. what if one does not have /tmp? Or not write-permission?)
• even for those using Docker, it becomes possible to fine-tune the location

In a similar way, I also suggest (but we can move that in another issue if you will) making the global path where code-compass downloads stuff (e.g. dependencies/d3.v3.min2js) as customizable please 🙏
A use case I have is that I provide code-compass Git repository as a Nix package, i.e. it ends up in a read-only directory, which then crashes when code-compass tries to write to it.

c/slack-main-contributor gives the impression that its related to slack but when you read the docstring you realize its not related:

"Enable the listing of contributors for a file in the *Messages* buffer."

perhaps c/display-file-contributors or something else?

Hi,
Thank you for this great selection of tools! It really helps me to understand some new repositories.

The one thing I would like to suggest is to change the colors for the hotspot graph to a gradient going from blue to green to red.
For me at least this is visually clearer where the hotspots are.

Hi,

Have you considered publishing code-compass on MELPA or GNU ELPA? That would make it simpler to use 🙏

Some defcustoms are not passed to async code. E.g.:

(setq c/code-maat-command "foobar")
(c/show-hotspots)

will incorrectly use the default value of c/code-maat-command rather than "foobar" e.g. in c/run-code-maat.

I managed to make it work with something like that:

@@ -263,6 +263,7 @@
    `(lambda ()
       (setq load-path ',load-path)
       (load-file ,(symbol-file command))
+      (setq c/code-maat-command ,c/code-maat-command)
       (let ((browse-url-browser-function 'browse-url-generic)
             (browse-url-generic-program ,c/preferred-browser))
         (funcall ',command ,repository ,date)))

But I don't know if this is the idiomatic way to go.

Or how is one supposed to customize those defcustoms otherwise?

The function formate is not found in any of the listed dependencies and it looks closely similar to format, so I think it is a typo.

https://github.com/ag91/code-compass/blob/main/code-compass.el#L1170

I have scanned through the code and it looks like this function is in fact missing. Also, there is this file called dot-code-maat.el which is also not available in the package directory

I'm a bit of an emacs newbie, and I'm completely unsure how to use this. I'm using Doom Emacs. Here is the relevant part of my packages.el

;; code-compass https://github.com/ag91/code-compass
(package! async)
(package! dash)
(package! f)
(package! s)
(package! simple-httpd)

(package! code-compass)

I don't see any instructions on how to use this package in the readme. When doing M-x compass nothing shows up, but I'm not sure if that's how to use code-compass

Hi there! I'm in the process of trying out code-compass, and I'm finding that I see the following error in the browser:

The page itself is blank. To get to that result I opened one of my repositories, and ran M-x c/show-hotspots. No errors are reported in emacs.

After pressing pause on trying to make code-compass work (thank you by the way on fixing the variables not being passed to the async process!), I tried again today on my repo.
But c/show-hotspots failed (as in, the displayed zoomable.html does not work with a javascript error) while I had no error in Emacs whatsoever 😭

I just spent some time investigating why and while I found the problem (I will open another issue about that), I think that code-compass should make errors more visible to the user, and fail fast to simplify investigation.

Example:

• when the code-maat command for revisions fails (shell return code 1), the process continues rather than aborting immediately. The generated revisions.csv contains garbage content:

Invalid argument:  git2: Failed to parse the given file - is it a valid logfile?
This is Code Maat, a program used to collect statistics from a VCS.
Version: 1.0-SNAPSHOT

Usage: program-name -l log-file [options]

Options:
  -l, --log LOG                                         Log file with input data
  -c, --version-control VCS                             Input vcs module type: supports svn, git, git2, hg, p4, or tfs
[...]

• the process continues like everything is ok then spectacularly 😄 crashes:

Produce json...
Traceback (most recent call last):
  File "/tmp/code-compass-<REDACTED>/csv_as_enclosure_json.py", line 232, in <module>
    run(args)
  File "/tmp/code-compass-<REDACTED>/csv_as_enclosure_json.py", line 195, in run
    raw_weights = parse_csv(
  File "/tmp/code-compass-<REDACTED>/csv_as_enclosure_json.py", line 90, in parse_csv
    return [parse_action(row) for row in r]
  File "/tmp/code-compass-<REDACTED>/csv_as_enclosure_json.py", line 90, in <listcomp>
    return [parse_action(row) for row in r]
  File "/tmp/code-compass-<REDACTED>/csv_as_enclosure_json.py", line 114, in parse_element_weight
    weight = float(csv_row[weight_column])  # Assert not zero?
ValueError: could not convert string to float: ' a program used to collect statistics from a VCS.'

Below the full logs:

Producing git report...
(Shell command succeeded with no output)
Producing code-maat revisions report for <REDACTED>...
(Shell command failed with code 1 and no output)
Producing cloc report...
(Shell command succeeded with no output)
Produce json...
Traceback (most recent call last):
  File "/tmp/code-compass-<REDACTED>/csv_as_enclosure_json.py", line 232, in <module>
    run(args)
  File "/tmp/code-compass-<REDACTED>/csv_as_enclosure_json.py", line 195, in run
    raw_weights = parse_csv(
  File "/tmp/code-compass-<REDACTED>/csv_as_enclosure_json.py", line 90, in parse_csv
    return [parse_action(row) for row in r]
  File "/tmp/code-compass-<REDACTED>/csv_as_enclosure_json.py", line 90, in <listcomp>
    return [parse_action(row) for row in r]
  File "/tmp/code-compass-<REDACTED>/csv_as_enclosure_json.py", line 114, in parse_element_weight
    weight = float(csv_row[weight_column])  # Assert not zero?
ValueError: could not convert string to float: ' a program used to collect statistics from a VCS.'

⚠️ notice in particular (Shell command failed with code 1 and no output)

Hey, thanks for the package. I see great potential for this! \o/

One thing I noticed is that everything is prefixed with c/ which is not common among package development.
prefix would ideally be code-compass- (bit verbose but that is the norm).

Are you open to updating?
I can make PR and still keep backwards compatibility (can be done with defalias)

Found a bug where the name of the jar is missmatched

Just realized I didn't think something through.

Right now, caching works by overwritting code-compass-coupling-project-map with whatever is in the cache.

This isn't ideal, it would be better preferred if it extended the current hashmap instead.

Reasoning:

1. Working on Project X
2. Run command to get coupled files, so code-compass-coupling-project-map is now populated.
3. Go to Project Y
4. Run command to get coupled files, (we see cache exists for this project so we overwrite code-compass-coupling-project-map, hence loosing all previous coupling info for Project X)

I've noticed that when calling code-compass-coupling-files, the result may sometimes spit out file(s) that no longer exists.

There should be some type of validator against the files it is suggesting.

After installation, running M-x code-compass-show-hotspots gives me the following

Debugger entered--Lisp error: (file-missing "Opening input file" "No such file or directory" "/Users/msanghvi/.emacs.d/elpa/code-compass-20230411.1032/scripts/csv_as_enclosure_json.py")
  signal(file-missing ("Opening input file" "No such file or directory" "/Users/msanghvi/.emacs.d/elpa/code-compass-20230411.1032/scripts/csv_as_enclosure_json.py"))
  async-handle-result((lambda (result) (when (not nil) (code-compass--run-server-and-navigate "/Users/msanghvi/Repos/sdlc/ice-gradle-pkg/" (or nil code-compass-default-port)))) (async-signal (file-missing "Opening input file" "No such file or directory" "/Users/msanghvi/.emacs.d/elpa/code-compass-20230411.1032/scripts/csv_as_enclosure_json.py")) #<buffer *emacs*>)
  async-when-done(#<process emacs> "finished\n")

I've followed the instructions in the README file:

;;;; init.el
(use-package async)
(use-package dash)
(use-package f)
(use-package s)
(use-package simple-httpd)

;; code-compass
;; for Code-Maat (code as crime scene, etc.)
(use-package code-compass)

Do the script needs to be packaged ? I don't see them in Melpa directory:

(Sun Apr 23 17:37:18)  ~/.emacs.d/elpa/code-compass-20230411.1032
OSX 12.6.5 x86_64 msanghvi % ls
code-compass-autoloads.el	code-compass-pkg.el		code-compass.el			code-compass.elc		dependencies/

(Sun Apr 23 17:37:19)  ~/.emacs.d/elpa/code-compass-20230411.1032
OSX 12.6.5 x86_64 msanghvi %  ls dependencies/
code-maat-1.0.1-standalone.jar

Let me know if there is any more information needed or if you need me to test anything.

Would be nice to save a cached version of the coupling results.

Thoughts:
when calculating the coupling results it would be nice to take a snapshot of the current head commit and branch.
Reasoning:

• if head commit changes, then cached results might not be valid
• Same reasoning for current branch
• if we match on both of these, then retrieve results, otherwise re-calculate
• We could save on directory {$code_compass_home}/.coupling_files/commit_hash ? ¯_(ツ)_/¯

Not sure if this makes sense or is too ambitious.

We can add a section in magit buffer to show the files coupled with currently edited files it git

When calling code-compass-find-coupled-files, the results will show the full path of the suggested files (see below)

Would be nice to trim out project root from path since we know in what repo we are working on.