ramonpin / mredu

A simple simulator of a system which implements map/reduce paradigm.

MREDU

A simple simulator of a system which implements map/reduce paradigm similarly to how Apache Hadoop does. Its objective is to be used as an educational tool to learn how to code map/reduce algorithms without needing to install complex components.

Requirements

To use it you will need:

• A Python interpreter and standard library. Tested with 2.7 and 3.5 versións of cpython.

• Install required packages from PyPi:

  $> pip install mredu

Examples

There are several examples of use of the simulator in the examples directory.

• example1: Does some calculations from a list of tuples.
• example2: Calculates the histogram of the number of words per line in the file quijote.txt from data folder.
• example3: The ubiquitous word-count example written to run on the simulator and applied to the same quijote.txt file.
• example4: Inverse k,v -> v,k agrupation

Docs

Currently there are no docs... sorry...

mredu's Issues

Add support for combiner simulation

Add support for combiner functions to the simulation pipeline. This will allow to teach how to use this functionality.