xavier-i / cetus

C++ Library of machine learning under development, includes SVM, linear regression, logistic regression etc.

CETUS: a C++ machine learning library

Table of Contents

1. Dependencies
2. Bug Reporting
3. Reference

Dependencies

Cetus has the following dependencies:

  Armadillo     >= 6.500.0

If you are compiling Armadillo by hand, ensure that LAPACK and BLAS are enabled. Or apt-get install will be an easier way.

Bug Reporting

If you find a bug in Cetus or have any issue, feel free to open an issue on Github

Reference

• Conrad Sanderson and Ryan Curtin. Armadillo: a template-based C++ library for linear algebra. Journal of Open Source Software, Vol. 1, pp. 26, 2016.
• Platt, J., "Sequential Minimal Optimization: A Fast Algorithm for Training Support Vector Machines." (1998)
• Tipping, M. E., & Bishop, C. M. , "Mixtures of Probabilistic Principal Component Analyzers" (1999)

cetus's Issues

Haven't implement gradient descent in linear regression

Run time of computing a matrix's inverse is o(n^3), kinda slow when n is very large. Usually we will consider gradient descent when n is almost 1000,000.

Adding normalize and FitIntercept parameter to Train() method in Linear Regression

If normalize is true, the regressors X will be normalized before regression. This parameter is ignored when FitIntercept is set to False. When the regressors are normalized, note that this makes the hyperparameters learnt more robust and almost independent of the number of samples. The same property is not valid for standardized data. However, if people wish to standardize, please use preprocessing(which has not be implemented yet). StandardScaler before calling fit on an estimator with normalize=false.

If FitIntercept set to false, no intercept will be used in calculations (e.g. data is expected to be already centered).

Cost Function of Logistic Regression

Sometimes generate nan or -nan

Decide Matrix type and way of initialization

Determine leave matrix as a pointer or something else, and which constructor to use