GithubHelp home page GithubHelp logo

whogotpwned / amazoncrawler Goto Github PK

View Code? Open in Web Editor NEW
6.0 1.0 0.0 6.86 MB

Simple Crawler For Amazon Stock Information Gathering

Python 71.55% Shell 17.17% PowerShell 6.82% Xonsh 4.46%
python selenium python-selenium-automation amazon

amazoncrawler's Introduction

The Simple Amazon Stock Crawler

Simple Crawler For Amazon Stock Information Gathering.

If you like my project, "Star" in the corresponding project right corner, please. If you want to buy me a cup of coffee, use the Bitcoin QR Code right below.

Buy Me A Coffee

Installation

Adapt Config.xml

The xml file contains all the items to be observed. All items should be enclosed within the tag. Each item on the other hand should be enclosed within an tag, as shown in the following:

<?xml version="1.0" encoding="utf-8"?>
<data>
    <selenium-options>
        <headless-mode>
            True
        </headless-mode>
    </selenium-options>
    <item name="flamingo-ball">
        <path>
            https://www.amazon.de/ballonfritz%C2%AE-Flamingo-Ballon-Riesenballon-Geburtstagsgeschenk/dp/B07D24S144/ref=sr_1_5?__mk_de_DE=%C3%85M%C3%85%C5%BD%C3%95%C3%91
            keywords=flamingo+ballon qid=1561379051 s=gateway sr=8-5
        </path>
    </item>
</data>

If you are going to base your config file on example_config.xml do not forget to rename it to config.xml. The given chromedriver in bin/chromedriver is actually the one for macos. Depending on your operating system you might have to replace it by the correct one, see Chromedriver Chromium.

Create Virtualenvironment and Install Dependencies

virtualenv -p python3 myenv
cd myenv
git clone https://github.com/localhostport80/amazoncrawler
source venv/bin/activate
pip install -r requirements.txt

Usage

There is one main file in this project, namely amazoncrawler.py. Feel free to type "?" or "help" inside the CLI that opens when running it. You will find commands for retrieving the stock information as well as commands for plotting csv files.

python amazoncrawler.py
Starting prompt...
> ?

Documented commands (type help <topic>):
========================================
exit  help  quit  retrieve_stock_of_all_items_in_config  save_plots  show_plots

Telegram-Bot

There is an additional main file called amazoncrawler_bot.py which makes it possible to interact with amazoncrawler.py via Telegram. Use BotFather to generate your unique private API token and add your private chat-id to the config file. The bot only grants admission to those users who's ids are listed in the XML.

python amazoncrawler_bot.py

amazoncrawler's People

Contributors

whogotpwned avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.