insiares / scraper

Quote Scraper

A simple scraper tool crawling the quotes to scrape website

Table of Contents

1. About The Project
   • Built With
2. Getting Started
   • Prerequisites
   • Installation
3. Usage
4. Roadmap
5. Contributing
6. License
7. Contact
8. Acknowledgments

About The Project

A scraping service built on Scrapy, served with Flask, and deployed using Acorn.

(back to top)

Built With

• 
• 
• 
• 
• 

(back to top)

Getting Started

This is an example of how you may give instructions on setting up your project locally. To get a local copy up and running follow these simple example steps.

Prerequisites

-- This is an example of how to list things you need to use the software and how to install them.

• npm

  npm install npm@latest -g

Installation

1. Get a free API Key at https://example.com
2. Clone the repo

   git clone https://github.com/Insiares/scraper.git

3. Install NPM packages

   npm install

4. Enter your API in config.js

   const API_KEY = 'ENTER YOUR API';

(back to top)

Usage

Use this space to show useful examples of how a project can be used. Additional screenshots, code examples and demos work well in this space. You may also link to more resources.

For more examples, please refer to the Documentation

(back to top)

Roadmap

• Create our spider
• mount mongodb
  • Implement an item pipeline to populate our dabatabase from the spider findings
• Build an API on top of your system
  • Present it in a nice html/css
• run unit tests
• implement logging
• Write dockerfile
• deploy on acorn
  • CI with acorn/dockerhub : add credential to GH actions and acorn logic to build and push to dockerio
  • attempt to CD with acorn autoupdate
• security test
• documentation

See the open issues for a full list of proposed features (and known issues).

(back to top)