=====================
 Logistic Regression
=====================


Logistic regression, despite its name, it a classification
technique. We have seen that the Bayes classifier assigns the class
$\hat y = \classify(\v c)$ to an object characterized with feature
vector $\v x$ based on:

.. math::
   \classify(\v x) = \arg\max_y \P(Y=y\given \v X=\v x)

For the Bayesian classifier the a posteriori probability
:math:`\P(Y=y\given \v X = \v x)` is then expressed in the class
conditional probabilities of the data and the a priori probabilities
of the classes. Effectively the joint distribution of the feature
vector and class is estimated. A Bayesian classifier is therefore a
**generative classifier** (given the joint distribution you could
generate examples).

The logistic regression classifier is an example of a **discriminative
classifier** and it starts with the same a posteriori probability as
given above. But unlike the Bayes classifier it does not calculate
this a posteriori probability from (an estimate of) the joint
distribution but it estimates the a posteriori probability directly
from the training set using a 'simple' parameterized model. As such
the logistic regression classifier concentrates more on the **decision
boundaries** of the different classes and not on an estimator for the
probability.


.. toctree::

   LogRegrModel
   LogRegrMLE
   LogRegrGradDesc
   LogRegrVectorized
   LogRegrExtended
   LogRegrMultiClass