release-engineering / pushsource

A library to load push items from a variety of sources

pushsource

A Python library for collecting content from various sources, used by release-engineering publishing tools.

• Source
• Documentation
• PyPI

Installation

Install the pushsource package from PyPI.

pip install pushsource

Usage Example

from pushsource import Source

# Get a source of content; sources and their parameters can be
# specified by URL. This source will use a couple of RPMs from
# Fedora koji as the content source.
with Source.get('koji:https://koji.fedoraproject.org/kojihub?rpm=python3-3.7.5-2.fc31.x86_64.rpm,python3-3.7.5-2.fc31.src.rpm') as source:
  # Iterate over the content and do something with it:
  for push_item in source:
    publish(push_item)

Development

Patches may be contributed via pull requests to https://github.com/release-engineering/pushsource.

All changes must pass the automated test suite, along with various static checks.

The Black code style is enforced. Enabling autoformatting via a pre-commit hook is recommended:

pip install -r requirements-dev.txt
pre-commit install

License

This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

pushsource's Issues

Default test does not pass with tox.

Summary

Default test does not pass with tox

Issue Type

• Bug Report

Steps to reproduce

• clone master
• tox # let tox run default test from envlist

Actual results

sh$ tox

//snip

Collecting virtualenv-api==2.1.18
  Using cached virtualenv_api-2.1.18-py2.py3-none-any.whl (9.5 kB)
Collecting wrapt==1.15.0
  Using cached wrapt-1.15.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (81 kB)
Collecting zipp==3.15.0
  Using cached zipp-3.15.0-py3-none-any.whl (6.8 kB)
Collecting pytz>=2015.7
ERROR: In --require-hashes mode, all requirements must have their versions pinned with ==. These do not:
    pytz>=2015.7 from https://files.pythonhosted.org/packages/2e/09/fbd3c46dce130958ee8e0090f910f1fe39e502cc5ba0aadca1e8a2b932e5/pytz-2022.7.1-py2.py3-none-any.whl#sha256=78f4f37d8198e0627c5f1143240bb0206b8691d8d7ac6d78fee88b78733f8c4a (from babel==2.12.1->-r test-requirements.txt (line 27))
WARNING: You are using pip version 21.3.1; however, version 23.0.1 is available.
You should consider upgrading via the '/home/user/projects/pushsource/.tox/py38/bin/python -m pip install --upgrade pip' command.

=============================================== log end ================================================
ERROR: could not install deps [-rtest-requirements.txt]; v = InvocationError('/home/user/projects/pushsource/.tox/py38/bin/python -m pip install -rtest-requirements.txt', 1)
_______________________________________________ summary ________________________________________________
ERROR:   py38: could not install deps [-rtest-requirements.txt]; v = InvocationError('/home/user/projects/pushsource/.tox/py38/bin/python -m pip install -rtest-requirements.txt', 1)

Expected results

Test passed without any failure with git master

Additional Information

There are more ways how to solve it.
A) run pip-compile with older version of python

• pushsource/tox.ini

  Line 36 in e0f1dfd

  basepython = python3.9

BTW it does not work with 3.6

pip-compile run-test: commands[1] | pip-compile -U --generate-hashes requirements.in test-requirements.in -o test-requirements.txt
Could not find a version that matches importlib-resources<5.3,>=5.4 (from -r test-requirements.in (line 22))
Tried: 0.1.0, 0.1.0, 0.2, 0.2, 0.3, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.8, 1.0, 1.0.1, 1.0.1, 1.0.2, 1.0.2, 1.1.0, 1.1.0, 1.2.0, 1.2.0, 1.3.0, 1.3.0, 1.3.1, 1.3.1, 1.4.0, 1.4.0, 1.5.0, 1.5.0, 2.0.0, 2.0.0, 2.0.1, 2.0.1, 3.0.0, 3.0.0, 3.1.0, 3.1.0, 3.1.1, 3.1.1, 3.2.0, 3.2.0, 3.2.1, 3.2.1, 3.3.0, 3.3.0, 3.3.1, 3.3.1, 4.0.0, 4.0.0, 4.1.0, 4.1.0, 4.1.1, 4.1.1, 5.0.0, 5.0.0, 5.0.2, 5.0.2, 5.0.3, 5.0.3, 5.0.4, 5.0.4, 5.0.5, 5.0.5, 5.0.6, 5.0.6, 5.0.7, 5.0.7, 5.1.0, 5.1.0, 5.1.1, 5.1.1, 5.1.2, 5.1.2, 5.1.3, 5.1.3, 5.1.4, 5.1.4, 5.2.0, 5.2.0, 5.2.1, 5.2.1, 5.2.2, 5.2.2, 5.2.3, 5.2.3, 5.3.0, 5.3.0, 5.4.0, 5.4.0
There are incompatible versions in the resolved dependencies:
  importlib-resources (from -r test-requirements.in (line 22))
  importlib-resources>=5.4 (from virtualenv==20.17.1->pre-commit==2.17.0->-r test-requirements.in (line 10))
  importlib-resources<5.3 (from pre-commit==2.17.0->-r test-requirements.in (line 10))
ERROR: InvocationError for command /home/user/projects/pushsource/.tox/pip-compile/bin/pip-compile -U --generate-hashes requirements.in test-requirements.in -o test-requirements.txt (exited with code 2)
_______________________________________________ summary ________________________________________________
ERROR:   pip-compile: commands failed

B) change default python version in envlist in tox.ini (it works with 3.9)

• pushsource/tox.ini

  Line 2 in e0f1dfd

  envlist = py38,static,docs

And ideally upgrade requirements for python in setup.py

pushsource/setup.py

Line 41 in e0f1dfd

python_requires=">=3.6",

C) do not used pinned versions for test (but that would complicate "reproducible tests" locally and in github actions)

• pushsource/tox.ini

  Line 6 in e0f1dfd

  -rtest-requirements.txt

And maybe even some other