A Bahadur Type Representation of the Linear Support Vector Machine and its Relative Efficiency

The support vector machine has been used successfully in a variety of applications. Also on the theoretical front, its statistical properties including Bayes risk consistency have been examined rather extensively. Taking another look at the method, we investigate the asymptotic behavior of the linear support vector machine through Bahadur type representation of the coefficients established under appropriate conditions. Their asymptotic normality and statistical variability are derived on the basis of the representation. Furthermore, direct theoretical comparison is made with likelihood based approach to classification such as linear discriminant analysis and logistic regression in terms of the asymptotic relative efficiency, where the efficiency of a classification procedure is defined using the excess risk from the Bayes risk.