Training in the Primal
Following the recent interest on the solution of the primal optimization problem of SVM-like classifiers, we developed a library to solve the primal problem of Laplacian SVMs (see the related JMLR paper). Training times and complexity are appreaciably reduced with respect to the original dual formulation, thanks to some early stopping conditions of the gradient descent based on the stability of the prediction. See our JMRL paper for all details. In the current implementation, it requires the whole kernel matrix to fit in memory.
Note: our library is somewhat generic, and it embeds also other classifiers in its framework, such as Laplacian RLSC (Regularized Least Square Classifier), RLSC, SVM (trained in the primal). The original dual formulation of SVM and Laplacian SVM is solved by LibSVM (2.89).
If you use this library, you are kindly requested to cite the reference paper, for which we provide the BibTeX code:
@article{melacci2011primallapsvm,
title={{Laplacian Support Vector Machines Trained in the Primal}},
author={Melacci, Stefano and Belkin, Mikhail},
journal={Journal of Machine Learning Research},
volume={12},
month={March},
year={2011},
issn={1532-4435},
pages={1149--1184},
numpages={36},
publisher={JMLR.org}
}
Details
The library is written in Matlab 7.6. An example on how to use it can be found on the "example.m" script, whereas a complete GUI for playing with Laplacian SVMs (and other classifiers) on some toy datasets can be ran from the "demo.m" file (after positioning the in the "gui" folder, otherwise the Matlab demo will start). It is very intuitive to use, so that you should not have any problems with it.
First, run the "setpaths" command, and then use the Matlab help function to see all the details of each script. If an unregularized bias is added to the kernel expansion of primal LapSVM, then the Laplacian is assumed to be not normalized.
In order to use the original dual LibSVM code, the included MEX interface must be complied with "make_libsvm.m" (this requires a C/C++ compiler installed in your machine!). The binary for Windows Vista 64-bit is included.
Downloads
Primal LapSVM (ver. 0.1)
Credits
The primal Laplacian SVM library has been developed by Stefano Melacci.
The code for building the graph Laplacian and computing the Kernel matrix is just a minor rewrite of the code written by Vikas Sindwhani for the paper "V. Sindhwani, P. Niyogi, M. Belkin. Beyond the Point Cloud: from Transductive to Semi-supervised Learning, International Conference on Machine Learning (ICML), 2005".
The general code structure of the primal solver follows the one of primal SVM of Oliver Chapelle. The reference paper for primal SVM is "O. Chapelle, Training a support vector machine in the primal, Neural Computation, MIT Press, 2007".