frenck / python-twentemilieu

Asynchronous Python client for Twente Milieu.

Python: Twente Milieu

Asynchronous Python client for the Twente Milieu API.

About

This package allows you to request waste pickup days from Twente Milieu programmatically. It is mainly created to allow third-party programs to use or respond to this data.

An excellent example of this might be Home Assistant, which allows you to write automations, e.g., play a Google Home announcement in the morning when it is trash pickup day.

Installation

pip install twentemilieu

Usage

import asyncio

from twentemilieu import TwenteMilieu, WasteType


async def main() -> None:
    """Show example on stats from Twente Milieu."""
    async with TwenteMilieu(post_code="1234AB", house_number=1) as twente:
        unique_id = await twente.unique_id()
        print("Unique Address ID:", unique_id)
        pickups = await twente.update()
        print("Next pickup for Non-recyclable:", pickup.get(WasteType.NON_RECYCLABLE))


if __name__ == "__main__":
    asyncio.run(main())

Changelog & Releases

This repository keeps a change log using GitHub's releases functionality. The format of the log is based on Keep a Changelog.

Releases are based on Semantic Versioning, and use the format of MAJOR.MINOR.PATCH. In a nutshell, the version will be incremented based on the following:

• MAJOR: Incompatible or major changes.
• MINOR: Backwards-compatible new features and enhancements.
• PATCH: Backwards-compatible bugfixes and package updates.

Contributing

This is an active open-source project. We are always open to people who want to use the code or contribute to it.

We've set up a separate document for our contribution guidelines.

Thank you for being involved! 😍

Setting up development environment

The easiest way to start, is by opening a CodeSpace here on GitHub, or by using the Dev Container feature of Visual Studio Code.

This Python project is fully managed using the Poetry dependency manager. But also relies on the use of NodeJS for certain checks during development.

You need at least:

• Python 3.11+
• Poetry
• NodeJS 20+ (including NPM)

To install all packages, including all development requirements:

npm install
poetry install

As this repository uses the pre-commit framework, all changes are linted and tested with each commit. You can run all checks and tests manually, using the following command:

poetry run pre-commit run --all-files

To run just the Python tests:

poetry run pytest

Authors & contributors

The original setup of this repository is by Franck Nijhof.

For a full list of all authors and contributors, check the contributor's page.

License

MIT License

Copyright (c) 2019-2024 Franck Nijhof

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

python-twentemilieu's Issues

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Awaiting Schedule

These updates are awaiting their schedule. Click on a checkbox to get an update now.

• ⬆️ Lock file maintenance

Ignored or Blocked

These are blocked by an existing closed PR and will not be recreated unless you click a checkbox below.

• ⬆️ Update dependency pytest to v8

Detected dependencies

devcontainer

.devcontainer/devcontainer.json

• ghcr.io/devcontainers-contrib/features/poetry 2
• ghcr.io/devcontainers/features/github-cli 1
• ghcr.io/devcontainers/features/node 1
• ghcr.io/devcontainers/features/python 1

github-actions

.github/workflows/codeql.yaml

• actions/checkout v4.1.7
• github/codeql-action v3.25.10
• github/codeql-action v3.25.10

.github/workflows/labels.yaml

• actions/checkout v4.1.7
• micnncim/action-label-syncer v1.3.0

.github/workflows/linting.yaml

• actions/checkout v4.1.7
• actions/setup-python v5.1.0
• actions/checkout v4.1.7
• actions/setup-python v5.1.0
• actions/checkout v4.1.7
• actions/setup-python v5.1.0
• actions/checkout v4.1.7
• actions/setup-python v5.1.0
• actions/checkout v4.1.7
• actions/setup-python v5.1.0
• actions/checkout v4.1.7
• actions/setup-python v5.1.0
• actions/setup-node v4.0.2

.github/workflows/lock.yaml

• dessant/lock-threads v5.0.1

.github/workflows/pr-labels.yaml

• jesusvasquez333/verify-pr-label-action v1.4.0

.github/workflows/release-drafter.yaml

• release-drafter/release-drafter v6.0.0

.github/workflows/release.yaml

• actions/checkout v4.1.7
• actions/setup-python v5.1.0
• pypa/gh-action-pypi-publish v1.9.0
• sigstore/gh-action-sigstore-python v2.1.1

.github/workflows/stale.yaml

• actions/stale v9.0.0

.github/workflows/tests.yaml

• actions/checkout v4.1.7
• actions/setup-python v5.1.0
• actions/upload-artifact v4.3.3
• actions/checkout v4.1.7
• actions/download-artifact v4.1.7
• actions/setup-python v5.1.0
• codecov/codecov-action v4.5.0
• SonarSource/sonarcloud-github-action v2.3.0

.github/workflows/typing.yaml

• actions/checkout v4.1.7
• actions/setup-python v5.1.0

npm

package.json

• prettier 3.3.2

nvm

.nvmrc

• node 20.14.0

pep621

pyproject.toml

• poetry-core >=1.0.0

poetry

pyproject.toml

• aiohttp >=3.0.0
• python ^3.11
• yarl >=1.6.0
• aresponses 3.0.0
• codespell 2.3.0
• covdefaults 2.3.0
• coverage 7.5.3
• mypy 1.10.0
• pre-commit 3.7.1
• pre-commit-hooks 4.6.0
• pylint 3.2.3
• pytest 7.4.4
• pytest-asyncio 0.23.7
• pytest-cov 5.0.0
• ruff 0.4.9
• safety 3.2.3
• yamllint 1.35.1

---

• Check this box to trigger a request for Renovate to run again on this repository

Output timestamp instead of a formatted string

Currently the output is a formatted which you can't localise without template sensor in Home Assistant. By leaving this a timestamp we can add device_class timestamp in Home Assistant.

I can make the adjustment, just curious if this is perhaps a formatted string by purpose.

Please add Gemeente Hellendoorn afval

Hello,

I live in de gemeente Hellendoorn and our afvalkalender uses exactly the same API calls, parameters and URLS's as Twentemillieu. Only the Company Code differs. For Hellendoorn it is:

24434f5b-7244-412b-9306-3a2bd1e22bc1

Test zipcode is 7441AA with housenummer 121.

Can you please add this to your code? I can also fork your repository and change it myself, but it would be nice if it was Integrated. And I then might also be Able to use it with Home Assistant. Maybe you have Some ideas to create a switch or parameter between the two of them.

The community of Hellendoorn has a population of about 35.000 people.

Gr. Remco

ImportError: cannot import name 'TwenteMilieu'

Problem/Motivation

I tried to get this working on RPI3 and RPI4, both running Openhabian OS, which is based on Linux bullseye. Unfortunately I don't get it working.

Expected behavior

By using the example.py I expected to get some dates on the screen. Python v3.9.2 is installed.

Actual behavior

By running the example.py, following error is shown:

Traceback (most recent call last):
File "/opt/twentemilieu/example.py", line 6, in
from twentemilieu import TwenteMilieu, WasteType
File "/opt/twentemilieu/twentemilieu.py", line 3, in
from twentemilieu import TwenteMilieu, WasteType
ImportError: cannot import name 'TwenteMilieu' from partially initialized module 'twentemilieu' (most likely due to a circular import) (/opt/twentemilieu/twentemilieu.py)

Did I forget something important during installation?

Multiple identical _pickupTypeText's should be merged

I got to this library because of the integration with HomeAssistant. Good stuff!

However, the Paper pickup is unknown:

After debugging the API, I found the cause. For my address, the following snippet of JSON is returned for the "_pickupTypeText" : "PAPER":

      {  
         "pickupDates":[  
            "2019-08-29T00:00:00"
         ],
         "pickupType":0,
         "_pickupType":0,
         "_pickupTypeText":"PAPER",
         "description":"EN PAP DO42"
      },
      {  
         "pickupDates":[  

         ],
         "pickupType":0,
         "_pickupType":0,
         "_pickupTypeText":"PAPER",
         "description":"EN PAP DO42_"
      }

The lines of code in the parser, override the second (empty) pickupDates from the first:

python-twentemilieu/twentemilieu/twentemilieu.py

Lines 135 to 142 in 77bd609

	for pickup in response["dataList"]:
	waste_type = API_TO_WASTE_TYPE.get(pickup["_pickupTypeText"])
	pickup_date = None
	if pickup["pickupDates"]:
	pickup_date = datetime.strptime(
	min(pickup["pickupDates"]), "%Y-%m-%dT%H:%M:%S"
	)
	self._pickup.update({waste_type: pickup_date}) # type: ignore

I'm not sure if this happens to other _pickupTypeText's as well, but I think the identical _pickupTypeText values should be merged instead of overridden.

Initial Update

The bot created this issue to inform you that pyup.io has been set up on this repo.
Once you have closed it, the bot will open pull requests for updates as soon as they are available.

Twente Milieu stopped supporting x-www-form-urlencoded

Since yesterday, this integration started returning an address doesn't exist error. Removing and re-adding the integration didn't help.

Changing /usr/local/lib/python3.7/site-packages/twentemilieu/twentemilieu.py to do http instead of https requests, and taking a look with tcpdump/wireshark, it sends and receives the following. And indeed the expected dataList (https://github.com/frenck/python-twentemilieu/blob/master/twentemilieu/twentemilieu.py#L114) is empty

POST /api/FetchAdress HTTP/1.1
Host: wasteapi.2go-mobile.com
User-Agent: PythonTwenteMilieu/0.2.0
Accept: application/json
Accept-Encoding: gzip, deflate
Content-Length: 95
Content-Type: application/x-www-form-urlencoded

companyCode=8d97bb56-5afd-4cbc-a651-b4f7314264b4&postCode=1234AB&houseNumber=1&houseLetter=None

HTTP/1.1 200 OK
Cache-Control: no-cache
Pragma: no-cache
Content-Type: application/json; charset=utf-8
Expires: -1
Server: Microsoft-IIS/10.0
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
Date: Thu, 26 Mar 2020 20:14:53 GMT
Content-Length: 312

{"userMessage":{"type":null,"messageCode":0,"description":null},"ExtraOption":{"$type":"System.Dynamic.ExpandoObject, System.Core"},"dataList":null,"intDataList":null,"data":null,"status":false,"messageCode":103,"pdfResponse":null,"token":null,"ID":null,"invalidParameters":null,"total":0,"messageToString":null}

Replaying this with wget, gives the same.

But, using json:

wget http://wasteapi.2go-mobile.com/api/FetchAdress/ --post-data='{"companyCode": "8d97bb56-5afd-4cbc-a651-b4f7314264b4", "postCode": "1234AB", "houseNumber": 1}' -nv -O- --header="Content-Type: application/json"

does return valid data (replaced actual data with ...):

{"userMessage":{"type":null,"messageCode":0,"description":null},"ExtraOption":{"$type":"System.Dynamic.ExpandoObject, System.Core"},"dataList":[{"UniqueId":"...","Street":"...","HouseNumber":"...","HouseLetter":" ","HouseNumberIndication":" ","HouseNumberAddition":" ","ZipCode":"....","City":"...","Community":"...","AddressType":"","Floor":null,"AddressUniqueId":0,"AddressUnique":null}],"intDataList":null,"data":null,"status":true,"messageCode":107,"pdfResponse":null,"token":null,"ID":null,"invalidParameters":null,"total":0,"messageToString":null}

It seems we need to start sending json instead of formdata.

I'd be happy to make a PR doing json requests, when confirmed this is the right approach.

TwenteMileue.unique_id not working with empty house_letter

Hey. The setup of my homeassistant integration of TwenteMilieu was broken, but I could not complete the flow. I got the error "Address not found in Twente Milieu service area.". Since it worked before, I started debugging.

I found out that this snippet works (removed address from example, note the space in house_letter):

from twentemilieu import TwenteMilieu
import asyncio

async def test():
    twentemilieu = TwenteMilieu(
        post_code="postcode",
        house_number="number",
        house_letter=" ",
    )

    unique_id = await twentemilieu.unique_id()

loop = asyncio.get_event_loop()
loop.run_until_complete(test())

But this doesn't:

from twentemilieu import TwenteMilieu
import asyncio

async def test():
    twentemilieu = TwenteMilieu(
        post_code="postcode",
        house_number="number",
        house_letter="",
    )

    unique_id = await twentemilieu.unique_id()

loop = asyncio.get_event_loop()
loop.run_until_complete(test())

Provinding an empty string in the houseLetter field of the POST reqeust to https://wasteapi.2go-mobile.com/api/FetchAdress seems to work fine.

Luckily I got my integration working again by entering a space in the setup flow of homeassistant.