oca / oca-ci Goto Github PK

This repository has no specified licence, making it difficult to re-use. Is it possible to add a licence?

manifestoo does everything we need.

Done in 9da97ae

Module

oca-ci/py3.8-odoo15.0 package

Describe the bug

Tests fail on GitHub for both Odoo tests and OCB tests while they pass on Odoo.sh and a local dev instance.

To Reproduce

Affected versions: Unsure (currently latest)

Steps to reproduce the behavior:

1. Set up a local dev instance and include purchase_delivery_split_date addon from https://github.com/dur-pro/purchase-workflow branch 15.0.
2. Appropriately set up virtual environment and install all required dependencies.
3. Run test suite with odoo-bin -c <config file> -d <new db name> --test-enable -i purchase_delivery_split_date --stop-after-init. All tests should pass. Note that running these tests with the version currently on OCA's repo fails 9 tests on dev machine but passes all tests in the GitHub action.
4. See the errors generated from OCA-CI actions here: OCA/purchase-workflow#2203

Expected behavior

Test results should be consistent between Odoo's test suite and the tests run by OCA-CI for the same major version.

Additional context

Tests run locally on MacOS with Python 3.8.18, both using only Odoo community edition and full Enterprise edition sources (same result). Tests run on Odoo.sh all passing.

A docker-compose.yml that provides a postgres and the appropriate environment variables to the odoo-ci container should help running the tests with docker-compose run.

Firstly, thanks for this amazing tools.
I would like to know how to add Odoo enterprise modules among available modules for testing in Github Action?
I have a private repo and the modules depend on the odoo enterprise modules. So, when I added the modules that depend on enterprise modules, CI test is failed? I found these #23 (comment). I don't know how to approach.

Module

The issue first occurred in coopiteasy/cie-custom#109, which contains custom modules.

Describe the bug

Given:

• Module spp_pos_mustard in https://github.com/coopiteasy/cie-custom/tree/12.0 depends on pos_shift_partner_can_shop.
• Module pos_shift_partner_can_shop exists in our repository https://github.com/carmenbianca/circular-rebel-example.
• Another module other_module in our repository depends on spp_pos_mustard. (edit: spp_pos_mustard is added to test-requirements.txt)
• Module pos_shift_partner_can_shop is a rebel module as per https://github.com/OCA/oca-addons-repo-template.

Then:

• The test for pos_shift_partner_can_shop will pass, correctly installing itself and spp_pos_mustard.
• The test for other_module will fail, failing to install spp_pos_mustard because the dependency pos_shift_partner_can_shop is not met.

See https://github.com/carmenbianca/circular-rebel-example for example GitHub CI runs.

To Reproduce

Affected versions: Any

Described above with example repository.

Expected behavior
I expect pos_shift_partner_can_shop to be pip-installed always so that all modules in test-requirements.txt can be installed without error.

Additional context
N/A

Hi.

First, thanks a lot for this tools !

I tried to change the CI of my company for my custom repositories, using oca-ci with github workflow actions.

• for the first one : (https://github.com/grap/grap-odoo-incubator) : all is great. (I had to implement that fix OCA/oca-addons-repo-template#102, thanks @sbidoul).
  note : the grap-odoo-incubator repo only depends on odoo and oca modules.

• for the second one (https://github.com/grap/grap-odoo-business) : I have an error during the installation of the dependencies.

https://github.com/legalsylvain/grap-odoo-business/runs/4869488630?check_suite_focus=true#step:4:205

The CI can't install a module (produt_print_category) that is present in the first repo (grap-odoo-incubator, not odoo nor oca). it tries via pip. how can I change the call of oca_install_addons to have the possiblity to install modules present in non OCA repositories ? The repo is correctly referenced in the oca_dependencies.txt.

thanks in advance, and sorry if my question is not relevant.

Hello @sbidoul

I believe the daily workflow to build the images is not running anymore:

As a consequence oca tests are running with a slightly outdated codebase, which is blocking in some cases.

Could you help to re-enable that workflow?

Thanks!

PS: I didn't find a way to prevent it from being disabled again after 60d of inactivity on the repo :/

Is your feature request related to a problem?

I've develop a werkzeurg middle-ware to redirect http to https in the following PR OCA/server-tools#2558

Before running test (therefor lauching odoo server) unit test require to add module in the server_wide_modules list

Describe the solution you'd like

I'm wondering if there is a proper approach to consider already implemented or to implement that would avoid breaking maintenance task on github action files ?

Or probably this something in copier template https://github.com/OCA/oca-addons-repo-template instead of this project ?

Describe alternatives you've considered

As working alternative I've update the odoo.cfg file running a shell command https://github.com/OCA/server-tools/pull/2558/files#diff-faff1af3d8ff408964a57b2e475f69a6b7c7b71c9978cccc8f471798caac2c88R76

Module

not a modules repository

Describe the bug

The tests hang on

❯ docker run -v $PWD/tests:/mnt/tests ghcr.io/oca/oca-ci/py3.8-odoo14.0:latest /mnt/tests/runtests.sh -v
Unable to find image 'ghcr.io/oca/oca-ci/py3.8-odoo14.0:latest' locally
latest: Pulling from oca/oca-ci/py3.8-odoo14.0
eaead16dc43b: Pull complete
69623f897469: Pull complete
cb04cdf8294b: Pull complete
1cdf5e92710b: Pull complete
8bcb80089e13: Pull complete
0127f8309be7: Pull complete
9eaa49ee8f9e: Pull complete
e9d854d08099: Pull complete
d4d3858bb4a9: Pull complete
c88165554b6f: Pull complete
2074cac1f696: Pull complete
9b40702c51ca: Pull complete
23554f0dbe2d: Pull complete
be60c6f99f89: Pull complete
ee4d20d20c14: Pull complete
1e8d4dcb3909: Pull complete
341ccbaa2c55: Pull complete
74775fe9453f: Pull complete
b769083296ea: Pull complete
c486ab10f179: Pull complete
1a3d2699c705: Pull complete
cc98a71c2060: Pull complete
702d4c016a8e: Pull complete
3078e2daea60: Pull complete
Digest: sha256:430787fcf2f71049a02ebad84733678f6bf5d0dd33670053a3e9060aeeb7d331
Status: Downloaded newer image for ghcr.io/oca/oca-ci/py3.8-odoo14.0:latest
created virtual environment CPython3.8.10.final.0-64 in 91ms
creator CPython3Posix(dest=/tmp/testvenv, clear=False, no_vcs_ignore=False, global=False)
seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/root/.local/share/virtualenv)
added seed packages: pip==22.2.2, setuptools==65.3.0, wheel==0.37.1
activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator
Looking in indexes: https://wheelhouse.odoo-community.org/oca-simple-and-pypi
Collecting pytest
Downloading pytest-7.2.0-py3-none-any.whl (316 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 316.8/316.8 kB 2.1 MB/s eta 0:00:00
Collecting pluggy<2.0,>=0.12
Downloading pluggy-1.0.0-py2.py3-none-any.whl (13 kB)
Collecting attrs>=19.2.0
Downloading attrs-22.1.0-py2.py3-none-any.whl (58 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 58.8/58.8 kB 593.7 kB/s eta 0:00:00
Collecting packaging
Downloading packaging-22.0-py3-none-any.whl (42 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 42.6/42.6 kB 80.6 kB/s eta 0:00:00
Collecting iniconfig
Downloading iniconfig-1.1.1-py2.py3-none-any.whl (5.0 kB)
Collecting tomli>=1.0.0
Downloading tomli-2.0.1-py3-none-any.whl (12 kB)
Collecting exceptiongroup>=1.0.0rc8
Downloading exceptiongroup-1.0.4-py3-none-any.whl (14 kB)
Installing collected packages: iniconfig, tomli, pluggy, packaging, exceptiongroup, attrs, pytest
Successfully installed attrs-22.1.0 exceptiongroup-1.0.4 iniconfig-1.1.1 packaging-22.0 pluggy-1.0.0 pytest-7.2.0 tomli-2.0.1
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.2.0, pluggy-1.0.0 -- /tmp/testvenv/bin/python
cachedir: .pytest_cache
rootdir: /mnt/tests
collecting ... collected 17 items

mnt/tests/test_addons_path.py::test_addons_path PASSED [ 5%]
mnt/tests/test_list_external_dependencies.py::test_list_external_dependencies PASSED [ 11%]
mnt/tests/test_list_external_dependencies.py::test_list_external_dependencies_transitive PASSED [ 17%]
mnt/tests/test_no_addons.py::test_no_addons

To Reproduce

Affected versions:
master
Steps to reproduce the behavior:

1. Clone the repository and move in there
2. Execute

   docker run -v $PWD/tests:/mnt/tests ghcr.io/oca/oca-ci/py3.8-odoo14.0:latest /mnt/tests/runtests.sh -v

Expected behavior
The tests should run

Additional context
Looking at the processes, the tests are waiting for postgres:

maybe postgres should be configured somehow before running the tests?

Say you have a repo with addons A and B.

Say you have a PR with changes to B that depend on a changes in an unmerged PR to A . In that PR, test-requirements.txt has a git reference to A.

In that case, it is unclear if the PR to A will have precendence over A found in the addons path (it is deterministic but fragile and not guaranteed to not change in the future).

When using editable installation mode (#62) there will likely be a conflict between the PR to A and -e A that pip will not be able to resolve.

Hi @sbidoul. Do you have any suggestions or examples on how to add Odoo enterprise modules among available modules for testing in Github Action?

Originale request here: #23 (comment)

Add odoo.cfg

• containing [options]
• in /etc/odoo.cfg (?)
• with ODOO_RC and OPENERP_SERVER env vars pointing to it

When addons in the repo being tested depend on addons from other repos which themselve depend on other addons in the repo being tested, these dependent addons are installed from PyPI instead of from the repo under test.

OCA/wms#815 (comment)

Searchpanel multi with style Hierarchy like default searchpanel (many2one)

In this line:
https://github.com/sbidoul/oca-ci/blob/fdbcf377efb00f94102ee5ed373ad1feab91f439/Dockerfile#L108

There is a git clone. Just wondering why not curl directly a tarball
like https://api.github.com/repos/odoo/odoo/tarball/14.0

Currently oca-ci does not have an Odoo log parser and relies on the Odoo exit code... and would really like to avoid adding one.

This seems to work well, but there are apparently some exceptions. This issue is to track them.

• When a test modules raise exceptions at import time, all tests of the addon are skipped. This is tracked upstream in odoo/odoo#80198

Hello @sbidoul

This is a follow-up to #57

Disclaimer: I might be missing the bigger picture here 😅

Context: OCA/bank-payment#1236

Adding a non-merged dependency from same repository in test-requirements.txt leads to 2 issues:

• (1) tests: can't build wheels because of "dubious ownership in repository"
• (2) runboat: can't build wheels because directory is not a git repo

Both cases raise the exception NoScmFound in whool, here

Workarounds:

• (1): OCA/oca-addons-repo-template#246
• (2): sbidoul/runboat#104

Relevant logs:

(1)

  Building wheel for odoo-addon-account_payment_order (pyproject.toml): started
  Building wheel for odoo-addon-account_payment_order (pyproject.toml): finished with status 'error'
  error: subprocess-exited-with-error
  
  × Building wheel for odoo-addon-account_payment_order (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [31 lines of output]
      fatal: detected dubious ownership in repository at '/__w/bank-payment/bank-payment'
      To add an exception for this directory, call:
      
          git config --global --add safe.directory /__w/bank-payment/bank-payment
      Traceback (most recent call last):
        File "/tmp/pip-build-env-16qd0cbq/overlay/lib/python3.10/site-packages/whool/buildapi.py", line 41, in _scm_ls_files
          subprocess.check_output(
        File "/usr/lib/python3.10/subprocess.py", line 421, in check_output
          return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
        File "/usr/lib/python3.10/subprocess.py", line 526, in run
          raise CalledProcessError(retcode, process.args,
      subprocess.CalledProcessError: Command '['git', 'ls-files']' returned non-zero exit status 128.

(2)

  Building wheel for odoo-addon-account_payment_order (pyproject.toml): started
  Building wheel for odoo-addon-account_payment_order (pyproject.toml): finished with status 'error'
  error: subprocess-exited-with-error
  
  × Building wheel for odoo-addon-account_payment_order (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [29 lines of output]
      fatal: not a git repository (or any parent up to mount point /mnt)
      Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
      Traceback (most recent call last):
        File "/tmp/pip-build-env-7452fbi7/overlay/lib/python3.10/site-packages/whool/buildapi.py", line 41, in _scm_ls_files
          subprocess.check_output(
        File "/usr/lib/python3.10/subprocess.py", line 421, in check_output
          return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
        File "/usr/lib/python3.10/subprocess.py", line 526, in run
          raise CalledProcessError(retcode, process.args,
      subprocess.CalledProcessError: Command '['git', 'ls-files']' returned non-zero exit status 128.

In sbidoul/runboat#48 we noticed that doing an apt-get install could fail due to package information being outdated.

We do an apt-get update in the Dockerfile. Even though we rebuild the images nightly, due to layer caching, there can be a long delay (up to 2 weeks IIRC) between the apt-get update and apt-get install.

The question I have is the following. Is the problem we had in sbidoul/runboat#48 a one-off incident or could it become prevalent ? In which case we'll need to do something about the installation of addons deb dependencies here and here.

When we look at http://archive.ubuntu.com/ubuntu/pool/main/r/rsync/ we can see that a lot of versions are there, which tend to imply that published version remain available "forever". http://archive.ubuntu.com/ubuntu/pool/main/r/rsync/rsync_3.1.3-8ubuntu0.1_amd64.deb is however absent.

So I suspect it was published then removed and we just played bad luck as we cached an apt index that still included it.

I'm not sure about this explanation and if we should do something about it. If there are people around with deb packaging expertise who could shed light on this it would be very welcome. @gurneyalex maybe ?

Is your feature request related to a problem?

Python libraries, such as PyCurl depend on apt/deb dependencies, unfortunately, oca_install_addons gets into a chicken and egg issue where oca_list_external_dependencies won't run until the Python deps are installed, but the Python deps cannot be installed until libcurl4-(openssl|nss|gnutls)-dev apt package is installed.

i.e.

DEBIAN_FRONTEND=noninteractive apt-get install -qq --no-install-recommends $(oca_list_external_dependencies deb)
not found in addons path: component,component_event,connector,queue_job
Aborted!

Describe the solution you'd like

Either:

• There separate step beforehand that installs the deb dependencies first
• Or manifestoo requires a new sub-command to not throw errors?

Describe alternatives you've considered

N/A

Additional context

This seems to be a common issue where the workaround has been to modify the Dockerfile and add external dependencies, such as xmlsec1?

I am happy to do the leg work on this issue, but I'm not clear if it's an OCA-CI or a manifestoo issue, or both? Before I do any work I'd like some guidance or input in the better way to tackle this :)

For example: OCA/manufacture#702

That module is available in the dependent repo

Is your feature request related to a problem?
Yes: some code might fail in Windows so it would be nice to test in Windows too.

Describe the solution you'd like
At least one CI should run the tests on Windows.

Describe alternatives you've considered
Adapt the code or files to work both in Ubuntu and Windows.

Additional context
For example, if a file name contains : (colon) the repository cannot be cloned in Windows (that is what happened in OCA/l10n-italy#3805).

Is your feature request related to a problem?
Yes, some modules require pip packages that require deb packages in order to be installed.
An example is in https://github.com/OCA/server-tools/blob/10.0/base_external_dbsource_odbc: it depends on pyodbc (https://pypi.org/project/pyodbc) that needs unixodbc-dev, as explained in its documentation https://github.com/mkleehammer/pyodbc/wiki/Install#installing-on-linux

Describe the solution you'd like
Configure apt packages on a per-repo basis that will be installed before installing Python libraries.

Describe alternatives you've considered
A step can be added in the github actions that installs the apt packages before installing pip packages, but as far I as understand, that is discouraged.
Currently there is nothing similar for runboat

Additional context
Similar to #33.
I ran into this while trying to add https://github.com/OCA/oca-addons-repo-template to https://github.com/OCA/server-tools/tree/10.0 in OCA/server-tools#2486.
There is some discussion already in https://discord.com/channels/737652535149592587/761225067798462465/1067129843130581083

As reported by @pedrobaeza in OCA/sale-workflow#3232 (comment), it seems that Odoo 17 now requires the pg_trgm extension to be installed.

This requirement was introduced recently in this commit odoo/odoo@85b2ede

@pivi-odoo as author of that commit, maybe you can say if it was intended to add pg_trgm as a hard dependency in 17.0?

If yes, we can likely do it in oca_init_test_database.

Images are not being refreshed.

@sbidoul Last action executed is from 12 days ago