asaini / apriori

Python Implementation of Apriori Algorithm for finding Frequent sets and Association Rules

Python Implementation of Apriori Algorithm

Set up

Edit without local environment setup

---

Acknowledgements

The code attempts to implement the following paper:

Agrawal, Rakesh, and Ramakrishnan Srikant. "Fast algorithms for mining association rules." Proc. 20th int. conf. very large data bases, VLDB. Vol. 1215. 1994.

---

Interactive Streamlit App

To view a live interactive app, and play with the input values, please click here. This app was built using Streamlit 😎, the source code for the app can be found here

Running the Streamlit app locally

To run the interactive Streamlit app with dataset

$ pip3 install -r requirements.txt
$ streamlit run streamlit_app.py

---

CLI Usage

To run the program with dataset provided and default values for minSupport = 0.15 and minConfidence = 0.6

python apriori.py -f INTEGRATED-DATASET.csv

To run program with dataset

python apriori.py -f INTEGRATED-DATASET.csv -s 0.17 -c 0.68

Best results are obtained for the following values of support and confidence:

Support : Between 0.1 and 0.2

Confidence : Between 0.5 and 0.7

---

Datasets

INTEGRATED-DATASET.csv

The dataset is a copy of the “Online directory of certified businesses with a detailed profile” file from the Small Business Services (SBS) dataset in the NYC Open Data Sets <http://nycopendata.socrata.com/>_

tesco.csv

Toy dataset of items from shopping cart

---

License

MIT-License

apriori's Issues

Invalid Mode Ru

hello iam already follow the instructions, already installl requement.txt but i got error in invalid mode ru, can u help me solve this? i really need this system ;(

how did u convert dataset into integrated dataset. please explain

error found when run this code

Traceback (most recent call last):
File "", line 178, in
File "", line 77, in runApriori
File "", line 62, in getItemSetTransactionList
TypeError: 'AsyncStream' object is not iterable

what should i do to correct this error

wrong syntax

hi, I run your code but I have received this message

File "apriori.py", line 118
for item, support in sorted(items, key=lambda (item, support): support):
^
SyntaxError: invalid syntax

Where is the error?

Invalid syntax: `key=lambda (item,`

I ran this inside my jupyter notebook and got the following error

Title "<ipython-input-303-3f4dffe6b184>", line 118
    for item, support in sorted(items, key=lambda (item, support): support):
                                                 ^
SyntaxError: invalid syntax

Many issues

No python3 optimization: print value should be print(value)... and you try to use indexing in dictionaries which will give errors.

Add lift (interestness) in rules

Hi,
I want to add lift (lift(X ⇒ Y ) =conf (X ⇒ Y )/supp(Y )) after each rules. In your code, may like lift = getSupport(item)/(getSupport(element)*getSupport(remain)). But an error occured (ValueError: too many values to unpack) when I do so. Can you modified the code and add the lift value in each rules. Many thanks.

Best.

wangxiong

Real datasets needed; lift needed.

I have implemented this lift feature with more datasets. See my repo here:
https://github.com/zoeleesss/Apriori

Redundant frozenset() in the `getItemSetTransactionList()` method?

In the line here: https://github.com/asaini/Apriori/blob/master/apriori.py#L52

There is no need to frozenset() the records, as the records in the data_iterator are already frozensets.

no pruning in Apriori function

This is what we should implement in code with pruning.

1. Let k=1
2. Generate frequent itemsets of length k
3. Repeat until no new frequent itemsets are identified

• Generate length (k+1) candidate itemsets from length k frequent itemsets
• Prune candidate itemsets containing subsets of length k+1 that are infrequent
• Count the support of each candidate by scanning the DB
• Eliminate candidates that are infrequent, leaving only those that are frequent

Huge Dataset issue

I ran your apriori.py with many custom data,,,
but when the lines of data get bigger , my python exec doesn't end not showing any results.

datafile is opened without close function

Inside the function dataFromFile(fname) ,the file is opened. But the file close function is not called, which is not safe. Suggest that call the with open...as... instead of open function.

Very Slow

I gave a database of 22 MB. It took around 600sec to solve it. Is this expected? Are there any other implementation in Python of Apriori by which execution time can be brought under 20sec?
Also, can you give me a rough estimate of how much it will take to solve a database of 10MB, 15MB, 20MB using the best possible Apriori implementation in Python.

Small question

Hello in your code you have a line (19) https://github.com/asaini/Apriori/blob/master/apriori.py#L19
where you use the code

return chain(*[combinations(arr, i + 1) for i, a in enumerate(arr)])

What does the * operator stand for before the array of combinations?

*[combinations(arr, i + 1) for i, a in enumerate(arr)]

i havent seen it before, i suspect it expands the arguments for the chain.
How is this called so i can look it up?

Thank you

Not same generation as paper

In the paper from Agrawal, the generation of candidates is different (formulated in sql query).
It is said that from itemsets (1,2,3), (1,2,4), (1,3,4), (1,3,5), (2,3,4), you have only (1,2,3,4) and (1,3,4,5) generated. With your approach, you have another, which is (1,3,4,5).

def joinSet(itemSet, length):
  """Join a set with itself and returns the n-element itemsets"""
  return set([i.union(j) for i in itemSet for j in itemSet if len(i.union(j)) == length])
  
  
itemset = [frozenset({1, 2, 3}), frozenset({1, 2, 4}), frozenset({1, 3, 4}), frozenset({1, 3, 5}), frozenset({2, 3, 4})]

print(joinSet(itemset, 4))

{frozenset({1, 2, 3, 4}), frozenset({1, 2, 3, 5}), frozenset({1, 3, 4, 5})}

Moreover, I think you switched the name of variables currentLSet and currentCSet (L for Large itemset and C for Candidate itemset) here

python3 support

I've created a new python3 branch to start supporting the code in both 2.7 and 3.4. In the coming days I'll be doing some work to add python3 support

Code does not implement APriori correctly

Hi. I don't think that this code implements the APriori algorithm correctly.

The reason I say this is due to the subset comparison in the "returnItemsWithMinSupport()" function. The code is comparing each item with a given transaction row by relying on a subset comparison rather than doing a comparison that preserves the column identity of the item.

For example, if we have the following transactions with column headers A, B as follows:

A,B

a,b
b,a

The support for the rule {A=a} is 50% since there is one transaction in which A=a. But the way that the code is implemented, we get a support of 100% because A=a and B=a are both (incorrectly) deemed support for either {A=a} or {B=a}.

Enable Sourcegraph

I want to use Sourcegraph for Apriori code search, browsing, and usage examples. Can an admin enable Sourcegraph for this repository? Just go to https://sourcegraph.com/github.com/asaini/Apriori. (It should only take 30 seconds.)

Thank you!

Wrong output

Your program gives incorrect output. I tried example 2 from http://en.wikipedia.org/wiki/Apriori_algorithm. It displays only 2 rules and all the rules it prints show 1.0 confidence, however it is not so by manual inspection.