As of late 2022, serious rewrite from BeautifulSoup to Selenium is needed because BeautifulSoup is struggling to download the whole page. This results in just 5-10% images downloaded.
Looking for a modern, nicely looking apartment but tired with all these old flats from the '90s? Let's automate that :)
Scrape apartment offers from OLX․pl, analyse them using artificial intelligence (AI) model to get only good looking flats and (optionally) run IFTTT automation (eg. send email; add a to-do task) when new offers matching search criteria (eg. ≥ 2 rooms, ≤ 2k PLN, ≥ 30m²) are found.
With support for native macOS & Windows 10 notifications.
- Get offers from OLX․pl
- Pagination support
- Artificial intelligence (AI) model to assess if an aparment is modern or not
- IFTTT (eg. send an email or add a to-do task if new offer is found)
- Cross platform (macOS & Windows)
- Native notifications (macOS & Windows)
Coming soon™.
- 0.14.2: Fixed offer counter & Windows 10 notification icon not showing up.
- 0.14.1: Added two additional calculations to know how much time it takes to train the model and how much time it takes to download and classify the images.
- 0.14: Refactored code; moved some more code to functions; added support for more than one URL with offers (eg. useful w/ multiple locations); tried to achieve feature parity between Windows and macOS; improved model accuracy; changed comparison mode so that previously seen offers are not shown again; cleaner output / general look & feel; removed unused code; other small improvements.
- 0.13: Open folders for manual review of the images.
- 0.12: Added
coloramato have colored output; changed URL; added more images to model's dataset; added fix forfeedModel.pycrashing due to duplicates found in target location. - 0.11.3: Forced to look only on 3 subpages.
- 0.11.2: Increased # of epochs to improve model's accuracy; added more images to model's dataset.
- 0.11.1: Changed title in notifications.
- 0.11: Automated runs using Task Scheduler (Windows) & Automator (macOS). Added more images to model's dataset. Disabled IFTTT automation for testing.
- 0.10.2: Cleaned up the code. Added comments.
- 0.10.1: Cleaned up the code.
- 0.10: Added macOS compatibility. Enabled IFTTT automation. Improved model accuracy - newly downloaded images are used to feed the model so it can improve itself over time.
- 0.9: Pagination support.
- 0.8: Enabled Windows 10 & macOS notifications when new offers are found.
- 0.7: Downloading images to unique folders. Added data comparison logic to show which offers are new vs previous script run.
- 0.6: Adding only "modern" offers to a text file. Changed local images renaming logic.
- 0.5: Started to build some
if/elselogic for when offer is modern/ancient. - 0.4: Added v1 of AI model to assess which offer shows new ("modern") and which old ("ancient") apartments.
- 0.3: Downloading offers' main images'.
- 0.2: Getting offers' URLs + offers' main images' URLs.
- 0.1: Initial release.
Using SemVer.
- TensorFlow
- BeautifulSoup
- alive-progress
- win10toast
- win10toast-persist
- win10toast-click
- pync
- GD Shortener
- wget
- Pillow
- NumPy
- Matplotlib
- colorama
- Images used to train the model were downloaded from Google Images. Respective licenses apply.
- Flaticon / Freepik
- IFTTT
- Connect a Python Script to IFTTT by Enrico Bergamini
- Use IFTTT web requests to send email alerts by Anthony Hartup
- Image classification
If you found a bug or want to propose a feature, feel free to visit the Issues page.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent