eakbas / tf-svm

Tensorflow Linear SVM

Tensorflow Linear SVM

A demonstration of how you can use TensorFlow to implement a standard L2-regularized support vector machine (SVM) in primal form.

linear_svm.py optimizes the following SVM cost using gradient descent:

where

The first part of the cost function, i.e. the regularization part, is implemented by the regularization_loss expression, and the second part is implemented by the hinge_loss expression in the code.

Run the code using

python linear_svm.py --train linearly_separable_data.csv --svmC 1 --verbose True --num_epochs 10

On a linearly separable, 2D data, the code gives the following decision boundary:

The code here is inspired by the repository try-tf.

tf-svm's Issues

L2-SVM Equation

We should replace the SVM equation in the README.md. The L2-SVM equation is as follows:

Reference: Tang, Y. (2013). Deep Learning using Linear Support Vector Machines. From arXiv preprint arXiv:1306.0239: https://arxiv.org/abs/1306.0239.

Wrong computation of vector norm

You compute the regularization loss by the following:

regularization_loss = 0.5 * tf.reduce_sum(tf.square(W))

But to compute for the norm of a vector, you must get the square root of the sum of squared elements in the vector, i.e.

tf.sqrt(tf.reduce_sum(W ** 2))

That might throw an error of the data type being int64, so,

tf.sqrt(tf.reduce_sum(W.astype(np.float32) ** 2))