1. Mathematical Tools

• 1.1. Linear Algebra Recap
  • 1.1.1. Vectors
  • 1.1.2. Basis Vectors
  • 1.1.3. Linear Mappings
  • 1.1.4. Matrices
  • 1.1.5. Linear Equations, Determinants and Inverse Matrices
  • 1.1.6. Vector Norm, Inner Products and Orthogonality
  • 1.1.7. Eigenvectors and Eigenvalues
  • 1.1.8. Singular Value Decomposition
• 1.2. Univariate Functions
  • 1.2.1. Definition
  • 1.2.2. Differentiating Univariate Functions
    • 1.2.2.1. Derivatives of Standard Functions
    • 1.2.2.2. Rules for Derivatives
  • 1.2.3. Integrating Univariate Functions
  • 1.2.4. Scaling and Shifting Functions
  • 1.2.5. Extrema of Univariate Functions
  • 1.2.6. Exercises
• 1.3. Multivariate Functions
  • 1.3.1. Plotting a multivariate function
  • 1.3.2. Differentiating a multivariate function
  • 1.3.3. Properties of Differentiation
  • 1.3.4. Integration of Multivariate Functions
  • 1.3.5. Symbolic Math Computations
  • 1.3.6. Exercises
• 1.4. The Chain Rule of Differentiation
  • 1.4.1. A chain of functions
  • 1.4.2. The Chain Rule
  • 1.4.3. The Leibnitz Notation
  • 1.4.4. Multivariate Chain Rule
  • 1.4.5. Exercises
• 1.5. Matrix Calculus
  • 1.5.1. The derivative of a scalar function with respect to a vector
  • 1.5.2. The derivative of a vector with respect to a scalar
  • 1.5.3. The derivative of a vector with respect to a vector
  • 1.5.4. Some important derivatives in the machine learning context
  • 1.5.5. Examples from Machine Learning
    • 1.5.5.1. Linear Regression
    • 1.5.5.2. Logistic Regression
• 1.6. Computational Graphs and the Chain Rule of Differentiation
  • 1.6.1. Computational Graph
  • 1.6.2. Automatic Differentiation
  • 1.6.3. Example
  • 1.6.4. Numerical Example
  • 1.6.5. Vectorial Graphs
  • 1.6.6. PyTorch
• 1.7. Some Bits of Information Theory
  • 1.7.1. Entropy
  • 1.7.2. Bit Encoding
  • 1.7.3. Cross Entropy
  • 1.7.4. References