notanameserver / incipyt Goto Github PK

At the moment we produce the following pyproject.toml for the keys license, authors and maintainers:

[[project.authors]]
name = "Julien"
email = "[email protected]"

[[project.maintainers]]
name = "Julien"
email = "[email protected]"

[project.license]
file = "LICENSE"

It would be better if we used inline-maps for those keys:

license = {file = "LICENSE"}
authors = [
  {name = "Julien", email = "[email protected]"},
]
maintainers = [
  {name = "Julien", email = "[email protected]"}
]

Just discovered the tool today from the python mailing list. We should asses if we support it in the future.

https://ofek.dev/hatch/latest/

During the creation of a new repo using incipyt, we are prompted which "Python version" we want. I was confused as I thought for a second it was asking me for the python to use instead of minimal python version the repo will be compatible.

I propose we change the "Python version [3.7]" to something else, e.g. "Minimal supported python version [3.7]"

Some project_urls render nicely on pypi.org, see

I found the actual code responsible for that in warehouse's source code, we might make sure we generate urls name that match

https://github.com/pypi/warehouse/blob/141aaa7fe499f7d9a4ee68aa5e2a944d4ee61ee3/warehouse/templates/packaging/detail.html#L20-L62

Regarding https://github.com/NotANameServer/incipyt/wiki/Build-systems many metadata can be given for build tools, the question is then : what incipyt does about them ?

I see three different policy:

• [1] Ask to the user (directly through command line options or a prompt input);
• [2] Ignore it;
• [3] Ask to the user but he can let it blank;
• [4] Have a default value and ask the user to confirm or change it;
• [5] Set a predefined value;

Note that we also need variable name for them in our code, let's also discuss them here, I note them {VARIABLE} here.

Let's review each metadata:

• name [mandatory], imo only [1] is possible here, {NAME} or {PROJECT_NAME} ?;
• version [mandatory], [1], [4] or [5], {VERSION} ?
• authors.email, [1], [2] or [3], {AUTHOR_EMAIL} ?
• authors.name, [1], [2] or [3], {AUTHOR_NAME} ?
• classifiers, [2] or [5] ? If [5], which ones ?
• dependencies, imo we cannot really do anything for it, so [2],
• description, [3] {DESCRIPTION} ?
• maintainers.email, [2], [3] or [4], {MAINTAINER_EMAIL} ? For policy [4], the default value could be {AUTHOR_EMAIL}.
• maintainers.name, [2], [3] or [4], {MAINTAINER_NAME} ? For policy [4], the default value could be {AUTHOR_NAME}.
• optional-dependencies.dev, imo this one should be populated by any tools that need it, e.g. pytest would add itself here, in #9 I setup a hook system for that.
• readme, [5] and fill README.md with {DESCRIPTION} ?
• requires-python, [2], [4] or [5] ?
• urls.*, imo [2] is the best here, it's too early to ask the user for them.

For setuptools, two more things are needed:

• options.package, it should correspond to the source folder, do we deal with the case multiple packages in one project or not ? If not, name, options.package and source folder are identical up to a normalization policy. I think we can start by restricting ourselves with this "simple" case.
• options.package_data imo the simplest is to decide a name for data folder, and get stick with it : {SRC_FOLDER}/data or {SRC_FOLDER}/assets ? But [3] or [4] is also possible.

Then after all files are created, what incipyt should do:

• Install the package in developer mode in the virtual environment ?
• Build the project ?

In case you give the URL to a repository that is not hosted on github (e.g. git web https://git.drlazor.be/?p=example;a=summary) then the issue/documentation URLs inside of pyproject.toml are wrong:

[project.urls]
Documentation = "https://git.drlazor.be/?p=example;a=summary/wiki"
Issue = "https://git.drlazor.be/?p=example;a=summary/issues"
Repository = "https://git.drlazor.be/?p=example;a=summary"

We should make a map of all online git hosting services with their issue/wiki URL and not fill the issue/doc inside of pyproject.toml when the service is unknown

In PR #15 incipyt should be called from an empty/non-existent folder, however it could be a too much restrictive requirement.

Actually this requirement ensures incipyt do not destroy anything, but maybe we can setup a more finer policy.

Let's consider three cases:

• Git: Just skip git init if the folder is already under supervision ? Also skip git add ?
• package folder: Just skip creation of the folder ? What about __init__.py if it doesn't exist ? Create or skip ?
• setup.cfg: What to do ?

A quite easy and general way to do that is setup policy at tool level; three (condition, policy) -- one for each step, pre/add_to/post, policy being skip/fail -- if the condition is met, then the policy is apply. Is any of them is "fail", incipyt stops. Eventually with a superset policy being "do not overwrite any file".

A related question is, what we do if incipyt failed while some files have been written ? Delete or keep them in place ?

IMO default level should be WARNING. The verbosity could be increased via cumulative -v options and decreased via cumulative -s options. Duty for another PR, IMO INFO is fine for now.

Originally posted by @Julien00859 in #15 (comment)

Which tools should we support firstly
@LeMinaw started a list here
If anyone wants to add/change something to the list, it's here we discuss about it.

I think we should optout for unittest instead of pytest, because unittest is in the standard library.

As Mestery suggested:

Also, we could restrict certain workflows to certain path (e.g., run the linter only on .py change, etc.).
See https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#onpushpull_requestpaths.

I started this wiki page for discussing project structure.

As you might see it's still incomplete and some questions have to be discussed :) feel free to give feedback.

We can add another section to discuss opt-ins tools structure if needed.

The current README contains a minimal tutorial explaining how to install incipyt in development but we still lack a proper in-depth tutorial on how to contribute.

See also #28

Some users (including myself) prefer to use pyenv-virtualenv over venv.

One of the advantages is that the actual folder where the venv is located is outside of the source dir, this makes git clean -dxfs not remove your venv.

We need to define witch build system we want to use for incipyt itself and also the recommanded tool for virtual environement for incipyt developers (as those tools don't imply anything to commit, actually you can use whatever you want).

Regarding https://github.com/NotANameServer/pip-init/wiki/Build-systems I think flit is a good candidate:

• It's a tool dedicated to build package, and only that (compared to pdm and poetry);
• It supports PEP 517 and PEP 621;
• It's configured by a single file pyproject.toml (compared to setuptools);

About virtual environment, actually the only one that supports multiple virtual environment (regardless major/minor version of the package) is the builtin venv, imo it's the best candidate for that.

Disregarding the build system, we need to choose metadata for incipyt:

[project]
name = "incipyt"
version = "0.0.1"
description = Someone to write a nice short description here and README.md ?
readme = "README.md"
authors = [
  {name = "TO-DO", email = "TO-DO"}
]
maintainers = [
  {name = "TO-DO", email = "TO-DO"}
]
requires-python = ">=3.6"
classifiers = [
  "Development Status :: 1 - Planning",
  "License :: OSI Approved :: MIT License",
  "Programming Language :: Python",
  "Programming Language :: Python :: 3 :: Only",
  "Programming Language :: Python :: 3",
  "Programming Language :: Python :: 3.6",
  "Programming Language :: Python :: 3.7",
  "Programming Language :: Python :: 3.8",
  "Programming Language :: Python :: 3.9",
  "Programming Language :: Python :: 3.10",
  "Intended Audience :: Developers",
  "Topic :: Software Development",
  "Topic :: Software Development :: Code Generators"
]

[project.urls]
homepage = "https://github.com/NotANameServer/pip-init"
documentation = "https://github.com/NotANameServer/pip-init" #Temporary
repository = "https://github.com/NotANameServer/pip-init"

Two questions:

• Are all those trove classifiers (see https://pypi.org/classifiers/) relevant ? Others to add ?
• What names/email do we use ?

At the moment the keys are alphabetically sorted, I suggest we instead sort the keys according to their definition in the Declaring project metadata document.

pyproject_keys = [
    "name",
    "version",
    "description",
    "readme",
    "requires-python",
    "license",
    "authors",
    "maintainers",
    "keywords",
    "classifiers",
    "urls",
    "scripts",
    "gui-scripts",
    "entry-points",
    "dependencies",
    "optional-dependencies",
    "dynamic",
]

https://discord.com/channels/323076998576603137/830536421067718695/980203486455136288

As @fblanchetNaN was thinking about it, hereby follow a few notes about how we should test incipyt.

Unit tests:

• test_dumpers: Mock Hierarchy if needed and test dumpers on simple, generic cases
• actions/test_*.py: Mock hierarchy and environment, ensure each action runs needed commands and performs required operations on the hierarchy
• Any other unit test we may need for internals (utils etc)

Integration tests:

• test_cli.py -> Test CLI invoke/interaction and ensures the correct Hierarchy is generated from prompts
• test_hierarchy.py -> Test folder and files generation from hierarchy

For CLI tests, we can mock input as following if needed:

import builtins


def concat():
    return input("a") + input("b")


def sequence(*args):
    iterator = iter(args)
    return lambda *_, **__: next(iterator)


def test_concat(monkeypatch):
    monkeypatch.setattr(builtins, "input", sequence("123", "456"))
    assert concat() == "123456"

For testing folders and file hierarchy, one may use pytest tmp_path fixture

INFO:incipyt.__main__:Running pre-script for Venv.
INFO:incipyt.commands:/usr/bin/python3 -m venv --upgrade-deps simplebin/.env
INFO:incipyt.commands:
ERROR:incipyt.commands:usage: venv [-h] [--system-site-packages] [--symlinks | --copies] [--clear]
            [--upgrade] [--without-pip] [--prompt PROMPT]
            ENV_DIR [ENV_DIR ...]
venv: error: unrecognized arguments: --upgrade-deps

Traceback (most recent call last):
  File "/home/julien/.local/bin/incipyt", line 8, in <module>
    sys.exit(main())
  File "/home/julien/.local/lib/python3.8/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/home/julien/.local/lib/python3.8/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/home/julien/.local/lib/python3.8/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/julien/.local/lib/python3.8/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/home/julien/.local/lib/python3.8/site-packages/incipyt/__main__.py", line 52, in main
    tool.pre(folder)
  File "/home/julien/.local/lib/python3.8/site-packages/incipyt/tools/venv.py", line 20, in pre
    commands.venv(["--upgrade-deps", os.fspath(env_path)])
  File "/home/julien/.local/lib/python3.8/site-packages/incipyt/commands.py", line 104, in venv
    return python_m(["venv"] + args, **kwargs)
  File "/home/julien/.local/lib/python3.8/site-packages/incipyt/commands.py", line 56, in python_m
    return run([project.environ["PYTHON_CMD"], "-m"] + args, **kwargs)
  File "/home/julien/.local/lib/python3.8/site-packages/incipyt/commands.py", line 30, in run
    raise subprocess.CalledProcessError(
subprocess.CalledProcessError: Command '['/usr/bin/python3', '-m', 'venv', '--upgrade-deps', 'simplebin/.env']' returned non-zero exit status 2.

I'm using Linux Mint 20.3 (una) with the python3-venv provided by the system that is https://packages.ubuntu.com/focal/python3-venv

Disclaimer: Do not work on it before PR #15 is closed.

Up to now Setuptools deals with two tasks:

• Properly writes build-system section;
• Manage all metadata of the project;

However, considering PEP 621 the second should become quite independent regarding the first one. So it can be wise to split those in few separated tools and let the CLI be responsible to select them according to the used build system. Then it becomes internal tools as those aren't directly linked to a specific software.

Such tools could be:

• PEP621: Use PEP 621 to store all metadata;
• SetupCfg: Use setup.cfg specified by setuptool;'
• PoetryMetadata: Use tool.poetry in pyproject.toml as Poetry not yet support PEP 621l;

Moreover some elements here can be super-set by other specifications and thus required additional internal enforcing them:

• SemVer: Use semver specification for version scheme, this tool would consists only in calling a Version Hook register by a metadata tool with an appropriate sanitizer;
• PEP440: Use PEP 440 specification for version scheme;
• License: TO-DO

In setuptools, setup.cfg is where you add dependencies. Incipyt setups a project without any dependency thus the [options] requires= option is not set. I suggest we set it empty so it is easy for users to add a dependency.