Looking for property can be arduous to say the least. In recent times, there have been many SaaS providers with platforms that allow the ease of search of propety. However, there is still the manual search which might be tiring and boring for the user. The idea of the Telegram bot is to automate this process of manually clicking a button. The bot will schedule a scrape once every day at a particular time, and send the user a list of property from various websites based on particular desired requirements from the user.
property-telebot's Introduction
property-telebot's People
property-telebot's Issues
Build a Basic Webscrapping Script
The idea is that the User has a list of requirements of what they want in a property (eg. XXX sq meters, Location in Clementi ...), and enters it into a property website.
The script would need to scrap that data with the given queries.
What is needed to be found out:
- How does queries work in the Property website
- Does the website have any protection against webscrapping
- How to store the data from scrapping
Draft out Architecture of the Bot
Find out requirements for Property
The user will want to look at a particular set of property metrics such as
- Location
- Square Meters
- Type of Housing (Condo/HDB etc...)
- Cost
Setting up base Telegram Bot
The idea behind the Telegram Bot is that it scrapes the website everyday at a certain hour, with the given requirements of the Property that the user wants.
For the first iteration, it could be that the bot has a schedule to scrap the top listed property in a pre-coded region in Singapore.
Base skeleton of the Telegram bot should be:
- Scheduler
- Authenticating the Telegram Bot
- Posting out data to the user
Run first iteration of the Bot
The first Iteration of the Bot should look like:
- The User authenticates and accesses the Telebot
- The Bot schedules a scrape every 10am (SGT)
- The values are stored inside a database (Excel/SQL)
- The User is able to see the values from their Telegram application
Research on Webscrapp-able Property Websites
So previously when I was attempting to web scrap for an idea that my friends were pursuing, there might be an issue where the websites block the user from calling the URL an excessive amounts of time? Im not sure.
But some of the websites that I have been recommended for property would be:
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. ๐๐๐
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google โค๏ธ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.