pbelskiy / aiojenkins

Python async client for Jenkins

Async python client for Jenkins

Asynchronous python package of Jenkins API endpoints based on aiohttp.

---

Also pay attention to brand new package with same API set but with sync and async interfaces:

https://github.com/pbelskiy/ujenkins

Installation

pip3 install aiojenkins

Usage

Start new build:

import asyncio
import aiojenkins

jenkins = aiojenkins.Jenkins('http://your_server/jenkins', 'user', 'password')

async def example():
    await jenkins.builds.start('job_name', dict(parameter='test'))

loop = asyncio.get_event_loop()
try:
    loop.run_until_complete(example())
finally:
    loop.run_until_complete(jenkins.close())
    loop.close()

Please look at tests directory for more examples.

Documentation

Read the Docs

Testing

Currently tests aren't using any mocking. I am testing locally with dockerized LTS Jenkins ver. 2.222.3

Prerequisites: docker, tox

docker run -d --name jenkins --restart always -p 8080:8080 jenkins/jenkins:lts
docker exec jenkins cat /var/jenkins_home/secrets/initialAdminPassword
chromium http://localhost:8080  # create admin:admin
tox

Contributing

Feel free to PR

aiojenkins's Issues

TODO for v.1.0

• remove function stubs from readme, try to use readthedocs.io hook
• black code formatter
• use conftest for Jenkins instance and fixtures for tmpdir (pytest)
• add stubs (mypy hook for stubs?)
• use jenkins.utils through public interface only
• internal retrier when network problem \ server rebooting
• reuse aiohttp client session -> #5
• builds.start -> builds.build? %) -> jobs.build() ?
• remove code duplication in requests part
• use POST for build_job
• nodes CRUD
• fix flake8 notes
• add crumb live updater
• add wait is_jenkins_ready function & use it in tests instead of dumb sleep
• token API support
• add isort to CI
• use submodules for nodes, jobs etc
• CI: mypy
• problem with default parameters build (use buildWithParameters) if parameters exist
• return build queue id
• remove leading slash (GET //job/test_build_list_1595076834.379972/api/json?tree=allBuilds%5Bnumber,url%5D HTTP/1.1 )
• get rid off code duplication in tests (job create \ delete)

---

Fully engineered entities:

• nodes
• jobs
• builds

add black and pylint

• add quotes consistency check (black)
• pylint

add new retry + timeout

https://github.com/pbelskiy/quickbuild/blob/master/quickbuild/adapters/aio.py

coveralls fails on PRs

coveralls fails on PRs

Accessing jobs in subfolders

job api only handles jobs in the Jenkins server root folder, it does not seem to be possible to list all jobs or access job information (among others) for jobs in subfolders

Add code coverage to CI

PyPI release on tag

Add actions trigger

aiohttp problem

Traceback (most recent call last):                                                                                                                                                                                                                                                          
  File "/test.py", line 3, in <module>                                                                                                                                                                                                     
    from aiohttp import web                                                                                                                                                                                                                                                                
  File "/home/user/.local/lib/python3.6/site-packages/aiohttp/__init__.py", line 6, in <module>                                                                                                                                                                                           
    from .client import BaseConnector as BaseConnector                                                                                                                                                                                                                                      
  File "/home/user/.local/lib/python3.6/site-packages/aiohttp/client.py", line 30, in <module>                                                                                                                                                                                            
    from yarl import URL                                                                                                                                                                                                                                                                    
  File "/home/user/.local/lib/python3.6/site-packages/yarl/__init__.py", line 1, in <module>                                                                                                                                                                                              
    from ._url import URL, cache_clear, cache_configure, cache_info                                                                                                                                                                                                                         
  File "/home/user/.local/lib/python3.6/site-packages/yarl/_url.py", line 56, in <module>                                                                                                                                                                                                 
    @rewrite_module                                                                                                                                                                                                                                                                         
  File "/home/user/.local/lib/python3.6/site-packages/yarl/_url.py", line 132, in URL                                                                                                                                                                                                     
    _QUERY_PART_QUOTER = _Quoter(safe="?/:@", qs=True, requote=False)                                                                                                                                                                                                   
  File "yarl/_quoting.pyx", line 192, in yarl._quoting._Quoter.__init__                                                                                                                                                                                                                     
TypeError: __init__() got an unexpected keyword argument 'requote'                      

instance destruction problem

GitHub Actions:

Exception ignored in: <bound method Jenkins.__del__ of <aiojenkins.Jenkins object at 0x7ff9997f4320>>
Traceback (most recent call last):
  File "/home/runner/work/aiojenkins/aiojenkins/aiojenkins/__init__.py", line 86, in __del__
    asyncio.get_event_loop().run_until_complete(
  File "/opt/hostedtoolcache/Python/3.5.10/x64/lib/python3.5/asyncio/events.py", line 678, in get_event_loop
    return get_event_loop_policy().get_event_loop()
  File "/opt/hostedtoolcache/Python/3.5.10/x64/lib/python3.5/asyncio/events.py", line 584, in get_event_loop
    % threading.current_thread().name)
RuntimeError: There is no current event loop in thread 'Dummy-39'.
Unclosed client session
client_session: <aiohttp.client.ClientSession object at 0x7ff9881f4630>
Unclosed connector
connections: ['[(<aiohttp.client_proto.ResponseHandler object at 0x7ff9862ce590>, 183.844163169)]']
connector: <aiohttp.connector.TCPConnector object at 0x7ff9881f4ba8>

In aiohttp server termination:

Stopping test.service...
Exception ignored in: <bound method Jenkins.__del__ of <aiojenkins.Jenkins object at 0x7f1aa31d70f0>>
Traceback (most recent call last):
  File "/home/user/.local/lib/python3.6/site-packages/aiojenkins/__init__.py", line 90, in __del__
  File "/usr/lib/python3.6/asyncio/base_events.py", line 460, in run_until_complete
  File "/usr/lib/python3.6/asyncio/base_events.py", line 377, in _check_closed
RuntimeError: Event loop is closed
sys:1: RuntimeWarning: coroutine 'RetryClientSession.close' was never awaited
Unclosed client session
client_session: <aiohttp.client.ClientSession object at 0x7f1aa31a9fd0>
source_traceback: Object created at (most recent call last):
  File "/home/user/test.py", line 51, in <module>
    web.run_app(init())
  File "/home/user/.local/lib/python3.6/site-packages/aiohttp/web.py", line 433, in run_app
    reuse_port=reuse_port))
  File "/usr/lib/python3.6/asyncio/base_events.py", line 471, in run_until_complete
    self.run_forever()
  File "/usr/lib/python3.6/asyncio/base_events.py", line 438, in run_forever
    self._run_once()
  File "/usr/lib/python3.6/asyncio/base_events.py", line 1443, in _run_once
    handle._run()
  File "/usr/lib/python3.6/asyncio/events.py", line 145, in _run
    self._callback(*self._args)
  File "/usr/lib/python3.6/asyncio/coroutines.py", line 126, in send
    return self.gen.send(value)
  File "/home/user/devices.py", line 603, in test_function
    devices = await get_devices()
  File "/usr/lib/python3.6/asyncio/coroutines.py", line 110, in __next__
    return self.gen.send(None)
  File "/home/user/devices.py", line 129, in get_devices
    jenkins_info = await jenkins.get_status()
  File "/usr/lib/python3.6/asyncio/coroutines.py", line 110, in __next__
    return self.gen.send(None)
  File "/home/user/.local/lib/python3.6/site-packages/aiojenkins/__init__.py", line 190, in get_status
    response = await self._request('GET', '/api/json')
  File "/usr/lib/python3.6/asyncio/coroutines.py", line 110, in __next__
    return self.gen.send(None)
  File "/home/user/.local/lib/python3.6/site-packages/aiojenkins/__init__.py", line 185, in _request
    self.crumb = await self._get_crumb()
  File "/usr/lib/python3.6/asyncio/coroutines.py", line 110, in __next__
    return self.gen.send(None)
  File "/home/user/.local/lib/python3.6/site-packages/aiojenkins/__init__.py", line 162, in _get_crumb
    response = await self._http_request('GET', '/crumbIssuer/api/json')
  File "/usr/lib/python3.6/asyncio/coroutines.py", line 110, in __next__
    return self.gen.send(None)
  File "/home/user/.local/lib/python3.6/site-packages/aiojenkins/__init__.py", line 125, in _http_request
    session = await self._get_session()
  File "/usr/lib/python3.6/asyncio/coroutines.py", line 110, in __next__
    return self.gen.send(None)
  File "/home/user/.local/lib/python3.6/site-packages/aiojenkins/__init__.py", line 109, in _get_session
    self._session = Client(cookies=self.cookies)
  File "/home/user/.local/lib/python3.6/site-packages/aiojenkins/__init__.py", line 29, in __init__
    self.client = ClientSession(*args, **kwargs)
Unclosed connector
connections: ['[(<aiohttp.client_proto.ResponseHandler object at 0x7f1aa28084c0>, 16398300.401200768)]']
connector: <aiohttp.connector.TCPConnector object at 0x7f1aa2851048>
source_traceback: Object created at (most recent call last):
  File "/home/user/test.py", line 51, in <module>
    web.run_app(init(loop=loop), port=1991, access_log=None)
  File "/home/user/.local/lib/python3.6/site-packages/aiohttp/web.py", line 433, in run_app
    reuse_port=reuse_port))
  File "/usr/lib/python3.6/asyncio/base_events.py", line 471, in run_until_complete
    self.run_forever()
  File "/usr/lib/python3.6/asyncio/base_events.py", line 438, in run_forever
    self._run_once()
  File "/usr/lib/python3.6/asyncio/base_events.py", line 1443, in _run_once
    handle._run()
  File "/usr/lib/python3.6/asyncio/events.py", line 145, in _run
    self._callback(*self._args)
  File "/usr/lib/python3.6/asyncio/coroutines.py", line 126, in send
    return self.gen.send(value)
  File "/home/user/devices.py", line 603, in testing_function
    devices = await get_devices()
  File "/usr/lib/python3.6/asyncio/coroutines.py", line 110, in __next__
    return self.gen.send(None)
  File "/home/user/devices.py", line 129, in get_devices
    info = await jenkins.get_status()
  File "/usr/lib/python3.6/asyncio/coroutines.py", line 110, in __next__
    return self.gen.send(None)
  File "/home/user/.local/lib/python3.6/site-packages/aiojenkins/__init__.py", line 190, in get_status
    response = await self._request('GET', '/api/json')
  File "/usr/lib/python3.6/asyncio/coroutines.py", line 110, in __next__
    return self.gen.send(None)
  File "/home/user/.local/lib/python3.6/site-packages/aiojenkins/__init__.py", line 185, in _request
    self.crumb = await self._get_crumb()
  File "/usr/lib/python3.6/asyncio/coroutines.py", line 110, in __next__
    return self.gen.send(None)
  File "/home/user/.local/lib/python3.6/site-packages/aiojenkins/__init__.py", line 162, in _get_crumb
    response = await self._http_request('GET', '/crumbIssuer/api/json')
  File "/usr/lib/python3.6/asyncio/coroutines.py", line 110, in __next__
    return self.gen.send(None)
  File "/home/user/.local/lib/python3.6/site-packages/aiojenkins/__init__.py", line 125, in _http_request
    session = await self._get_session()
  File "/usr/lib/python3.6/asyncio/coroutines.py", line 110, in __next__
    return self.gen.send(None)
  File "/home/user/.local/lib/python3.6/site-packages/aiojenkins/__init__.py", line 109, in _get_session
    self._session = Client(cookies=self.cookies)
  File "/home/user/.local/lib/python3.6/site-packages/aiojenkins/__init__.py", line 29, in __init__
    self.client = ClientSession(*args, **kwargs)
  File "/home/user/.local/lib/python3.6/site-packages/aiohttp/client.py", line 213, in __init__
    connector = TCPConnector(loop=loop)
  File "/home/user/.local/lib/python3.6/site-packages/aiohttp/connector.py", line 731, in __init__
    loop=loop)
Stopped test.service.

• add implicit close method

Add crumb updater

add Python3.5 support?

• remove f-string usage
• add badge of supported versions