lasseregin / svm-w-smo

Simple implementation of a Support Vector Machine using the Sequential Minimal Optimization (SMO) algorithm for training.

SVM

Simple implementation of a Support Vector Machine using the Sequential Minimal Optimization (SMO) algorithm for training.

Supported python versions:

• Python 2.7
• Python 3.4

Python package dependencies

• Numpy (http://docs.scipy.org/doc/numpy-1.10.1/user/install.html)

Documentation

Setup model (following parameters are default)

from SVM import SVM
model = SVM(max_iter=10000, kernel_type='linear', C=1.0, epsilon=0.001)

Train model

model.fit(X, y)

Predict new observations

y_hat = model.predict(X_test)

svm-w-smo's Issues

I think I might have found a bug.

Hello! Excuse me.
I think this kernel method quadratic have error.

This is my test code.

import warnings
warnings.filterwarnings('ignore')
import numpy as np
import random

# load yourself model
from SVM import SVM

from sklearn.svm import SVC
from sklearn.metrics import accuracy_score
from sklearn.datasets.samples_generator import make_blobs
from sklearn.model_selection import train_test_split


def test_SVM():
    i = 1
    np.random.seed(12345)
    while True:
        # generate dataset
        X, Y = make_blobs(
            n_samples=np.random.randint(2, 100), 
            n_features=np.random.randint(2, 100),
            centers=2, random_state=i, 
        )
        X, X_test, Y, Y_test = train_test_split(X, Y, test_size=0.3, random_state=i)
        # ignore split error(train/test data only 1 class)
        if 0 not in Y or 1 not in Y:
            continue
        # generate param
        C = random.uniform(0.1, 0.9)
        max_iter = random.uniform(50, 500)
        kernel = "quadratic"
        tol = random.uniform(0.000001, 0.1)
        # fit and predict
        clf = SVM(C=C, max_iter=max_iter, kernel=kernel, tol=tol)
        clf.fit(X, Y)
        pred = clf.predict(X_test)

        print(accuracy_score(Y_test, pred))


if __name__ == "__main__":
    test_SVM()

calc_w function issue

Hey Lasse,

When I tried your SVM, it seems calc_w was not working properly as alpha*y calculated the dot product of these two vectors so that your function doesn't really return the right w. I replace your function with mine below, which works. Let me know what you think, I can do a pull request to correct it.

    def calc_w(self, alpha, y, X):

        new = np.multiply(alpha,y)
        return np.dot(X.T, new)

Yitao

calc_w function

Hey Lasse,

The calc_w function in your SVM function seems wrong, it should return a vector but it ends up with a matrix. I rewrote the calc_w as follow, it worked, let me know what you think.

 def calc_w(self, alpha, y, X):
        X_t = X.transpose()
        X_ta = np.matmul(X_t, X)
        X_tai = np.linalg.inv(X_ta)
        ols = np.matmul(X_tai, X_t)
        ols = np.matmul(ols, y)
        return ols