Support Vector Machines

The Support Vector Machine can sometimes be cleaner and more powerful than other methods.

In logistic regression, we use:

• if $y=1$, then $h_\theta(x) \approx 1$ and $\Theta^Tx \gg 0$
• if $y=0$, then $h_\theta(x) \approx 0$ and $\Theta^Tx \ll 0$

With a SVM, we modify the first term of the cost function so that when $\theta^Tx = z$ is greater than 1, it outputs 0. When $z$ is less than 1, we use a straight line. We make a similar modification to the second term, so that when $z$ is less than $-1$ it outputs $0$.