Poly check can't understand pillow dependency about python-polylith

[DavidVujic]

Hi @BeRT2me,

There's a new version of the tool out! v1.9.0 introduces two new options to the poly check and poly libs commands:

--strict
This option will narrow down the comparison between imports and third-party libraries. This should make false positives more rare. This option will allow libraries with - and the corresponding imports, using _, but not pandas and geopandas.

Usage:
poetry poly check --strict

--alias
This option will help the check and libs commands, when the difference between a library name and the import is too big. I chose to add it as a command option, to make it project-specific (and not for all projects in the workspace).

Usage:
poetry poly check --alias opencv-python=cv2

This will treat cv2 as a third-party library when running the analysis.

I hope this will help and make the commands more accurate! There will likely be more cases where the commands will fail on identifying libraries, but I think these options will cover many use cases.

The --strict and --alias options can be used in combination, to both rule out false positives and identifying dependencies.

[DavidVujic]

Thank you for reporting @BeRT2me!

This is probably because the poly check command couldn't see enough similarities between PIL and pillow and I will have a look at it. I have probably missed out the string comparison between different caps or something like that 😄

[DavidVujic]

@BeRT2me I believe I have fixed this issue and just released v1.8.2 of the tool. Please let me know if this has solved your issue, and thank you again for reporting it here! ⭐

[BeRT2me]

@DavidVujic This is great, thank you!

Another one I've discovered are OpenCV packages; which can be installed as any one of:

• opencv-python
• opencv-contrib-python
• opencv-python-headless
• opencv-contrib-python-headless

And all have the import cv2.

Also, the mpl_toolkits import than comes with matplotlib.

[DavidVujic]

Yes, these imports are probably too different from the dependency names, and the builtin difflib that is used in the source code won't recognize them properly.

I think I will add a possibility to ignore, or to make hints, to the poly check and poly libs commands.

[BeRT2me]

Yeah, there are an unfortunate number of libraries that don't align their name and imports.

Although using difflib is a super interesting approach, I do worry about false positives. Maybe just giving the ability to map library names to a list of known imports wouldn't be clearer - and eliminate those false positives.

Something like this in the workspace.toml file:

[tool.polylith.known-libraries]
opencv-python = ["cv2"]
matplotlib = ["matplotlib", "mpl_toolkits"]
pillow = ["PIL"]
scikit-image = ["skimage"]

[DavidVujic]

Good ideas! ⭐ I plan to work on this during next week.

[BeRT2me]

Speaking of false positives, pandas being installed falsely allowed geopandas to be overlooked.