6.3.4. Vectorized Logistic Regression Algorithm

Like we did for linear regression we can vectorize the expressions for logistic regression and ‘hide’ the summation over the examples in the learning set in matrix vector multiplications. Without proof we give the vectorized cost function:

\[J(\v\theta) = \frac{1}{m} \left( -\v y \T \log(g(\hv X\v\theta)) - (\v 1 - \v y)\T \log( \v 1 - g(\hv X \v\theta)) \right)\]

where \(g\) is the logistic function and \(\v 1\) is an \(m\times1\) vector of ones. The gradient vector is:

\[\frac{\partial J(\v\theta)}{\partial\v\theta} = \frac{1}{m} \hv X\T (g(\hv X\v\theta) - \v y)\]

This leads to the following algorithm:

\[\text{iterate until done:}\qquad \v\theta := \v\theta - \alpha \frac{1}{m} \hv X\T ( g(\hv X \v\theta) - \v y)\]