costajob / stock_prices

A simple collector, visualiser and forecaster program for closing stock prices written in Python.

Table of Contents

• Scope
• Requirements
  • Versions
    • Python 2
  • Chart
  • Footprint
• Design
  • SRP
  • Data
  • Tests
• Usage
  • Installation
  • Start Server
    • Docker
  • API
• Performance

Scope

This is the implementation of the python code kata Data Analysis and Visualisation for financial stock data.

Requirements

Versions

The library is compatible and it has been tested with the following python versions:

• 3.4.8

• 3.6.4

• 3.7.1

Python 2

Support for old python 2 is not guaranteed, since some components makes use of the concurrent.futures introduced on standard library since version 3.2.

Chart

The chart.js library is used in order to plot data into a non-static chart.
The minimized script is loaded by an external CDN, thus requiring a network connection to properly works.

Footprint

To grant resiliency (and courtesy of the Python's broad standard library) the external dependencies footprint is kept to a minimum:

• Flask: a WSGI-compliant HTTP micro-framework, required by objectives

• Gunicorn: a pre-fork WSGI-compliant HTTP server, used to distribute the load among available cores

• Meinheld: a WSGI-compliant Web server, based on green threads and asynchronous network I/O to reduce latency

• BeautifulSoup: a HTML parser library aimed at screen-scraping

Design

SRP

The code design follows the single responsibility principle by using a dedicated class for any specific task. Each class is confined within meaningful modules:

• data: data related objects, such as Fetcher, Parser and Downloader

• entity: plain value objects, such as Stock

• computer: computation logic objects, such as Forecaster and Global

Data

The stock closing data are fetched by remote HTML documents. The fetching is quite inefficient, since the whole document need to be downloaded and scanned in order to collect just last month prices, but the objectives do not advise of any available APIs.
HTML documents are fetched once at server start and cached at ./cache/corn.html (etc.) to avoid further network latency. Just delete the cached HTML documents and restart gunicorn to fetch fresh copies.

Tests

The library is covered, by fast, isolated unit and doc testing (the latter to grant reliable documentation):

python -m unittest discover -s stockp -p '*'
..........
----------------------------------------------------------------------
Ran 10 tests in 0.166s

OK

Usage

Depending on your python version use the appropriate binaries available on your workstation:

Installation

Install the external dependencies via pip:

pip install -r requirements.txt

Start Server

Development

To start the WSGI server on development just execute the main.py file with the python binary:

python main.py

Production

To start the server on production use the gunicorn executable by spawning as many workers as you need and by specifying the HTTP port:

gunicorn -w 4 -k meinheld.gmeinheld.MeinheldWorker -b :8888 main:app

Docker

This application can be built and deployed as a Docker container by relying on the python:3.7.1 official image:

docker build -t stockp .

Once the container has been built, just run it by:

docker run -d -p 8888:8888 stockp

API

The library exposes a single HTTP endpoint at port 8888 (or at the port you bound at server start), which renders a HTML chart representation of last month stock prices and their forecast for Nasdaq, Corn and Gasoline stocks:

Performance

Courtesy of the gunicorn and meinheld it is possible to squeeze decent throughput by stressing the server via the wrk tool:

wrk -t 4 -c 100 -d30s --timeout 2000 http://127.0.0.1:8888
Running 30s test @ http://127.0.0.1:8888
  4 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    11.10ms    3.62ms  51.38ms   75.78%
    Req/Sec     2.27k   268.19     2.85k    68.08%
  271111 requests in 30.01s, 584.07MB read
Requests/sec:   9034.28
Transfer/sec:     19.46MB