Machine Learning
Contents I. Introduction
Objectives
Related Fields 1. Statistics
Literature Machine Learning:
Software Programming:
Software Statistics:
Software Lab Class Setup
Software Lab Class Setup
Chapter ML:I I. Introduction
Examples of Learning Tasks (1) Car Shopping Guide
Examples of Learning Tasks (2) Risk Analysis for Credit Approval
Examples of Learning Tasks (2) Risk Analysis for Credit Approval
Examples of Learning Tasks (3) Image Analysis
Examples of Learning Tasks (3) Image Analysis
Specification of Learning Tasks Definition 1 (Machine Learning
Remarks:
Specification of Learning Tasks Learning Paradigms
Specification of Learning Tasks Learning Paradigms
Specification of Learning Tasks (4) Example Chess: Kinds of Experience
Specification of Learning Tasks (4) Example Chess: Kinds of Experience
Specification of Learning Tasks (4) Example Chess: Kinds of Experience
Specification of Learning Tasks (4) Example Chess: Ideal Target Function γ()
Specification of Learning Tasks (4) Example Chess: Ideal Target Function γ()
Specification of Learning Tasks (4) Example Chess: Real World → Model World
Specification of Learning Tasks (4) Example Chess: Real World → Model World
Specification of Learning Tasks (4) Example Chess: Real World → Model World
Remarks:
Remarks (continued):
Specification of Learning Tasks
Specification of Learning Tasks Real World → Model World
Specification of Learning Tasks Real World → Model World
Specification of Learning Tasks Real World → Model World
Specification of Learning Tasks Real World → Model World
Specification of Learning Tasks Real World → Model World
Specification of Learning Tasks Real World → Model World
Remarks:
Remarks (continued) :
Specification of Learning Tasks The LMS Algorithm for Fitting y(x)
Specification of Learning Tasks The LMS Algorithm for Fitting y(x)
Remarks: (?) We consider the feature vector x in its extended form when used as operand in a scalar
Remarks (continued):
Chapter ML:I I. Introduction
Elements of Machine Learning (1) Model Formation: Real World → Model World
Elements of Machine Learning (2) Design Choices in the Machine Learning Stack: LMS
Elements of Machine Learning (2) Design Choices in the Machine Learning Stack: LMS
Elements of Machine Learning (2) Design Choices in the Machine Learning Stack: LMS
Elements of Machine Learning (2) Design Choices in the Machine Learning Stack: LMS
Elements of Machine Learning (2) Design Choices in the Machine Learning Stack: LMS
Related questions:
Elements of Machine Learning (3) Feature Space Structure
Elements of Machine Learning (3) Feature Space Structure
Elements of Machine Learning (3) Feature Space Structure
Remarks:
Elements of Machine Learning (4) Discriminative versus Generative Approach to Classification
Elements of Machine Learning (4) Discriminative versus Generative Approach to Classification
Elements of Machine Learning (4) Discriminative versus Generative Approach to Classification
Remarks:
Elements of Machine Learning (5) Frequentist versus Subjectivist Paradigm to Learning
Elements of Machine Learning (5) Frequentist versus Subjectivist Paradigm to Learning
Elements of Machine Learning (5) Frequentist versus Subjectivist Paradigm to Learning
Remarks:
Elements of Machine Learning (5) Frequentist versus Subjectivist Paradigm to Learning
Elements of Machine Learning (5) Frequentist versus Subjectivist Paradigm to Learning
Elements of Machine Learning (5) Frequentist versus Subjectivist Paradigm to Learning
Elements of Machine Learning (5) Frequentist versus Subjectivist Paradigm to Learning
Remarks:
Chapter ML:I I. Introduction
Notation Overview Data, Sets, and Distributions
Notation Overview Indexing
Notation Overview Functions
Notation Overview Algorithms
Classification Approaches Overview Search in hypothesis space
Chapter ML:II
Concept Learning: Search in Hypothesis Space Simple Classification Problems
Concept Learning: Search in Hypothesis Space Example Learning Task
Remarks:
Concept Learning: Search in Hypothesis Space Simple Classification Problems
Concept Learning: Search in Hypothesis Space Simple Classification Problems
Remarks:
Concept Learning: Search in Hypothesis Space Simple Classification Problems
Remarks:
Concept Learning: Search in Hypothesis Space Simple Classification Problems
Concept Learning: Search in Hypothesis Space Simple Classification Problems
Concept Learning: Search in Hypothesis Space Order of Hypotheses
Concept Learning: Search in Hypothesis Space Order of Hypotheses
Concept Learning: Search in Hypothesis Space Order of Hypotheses
Remarks:
Remarks on entailment:
Remarks on entailment: (continued)
Concept Learning: Search in Hypothesis Space Inductive Learning Hypothesis
Concept Learning: Search in Hypothesis Space Find-S Algorithm
Remarks:
Concept Learning: Search in Hypothesis Space Find-S Algorithm
Concept Learning: Search in Hypothesis Space Find-S Algorithm
Concept Learning: Search in Hypothesis Space Find-S Algorithm
Concept Learning: Search in Hypothesis Space Find-S Algorithm
Concept Learning: Search in Hypothesis Space Discussion of the Find-S Algorithm
Concept Learning: Version Space Definition 5 (Version Space)
Concept Learning: Version Space Definition 5 (Version Space)
Remarks:
Concept Learning: Version Space Definition 6 (Boundary Sets of a Version Space)
Concept Learning: Version Space Theorem 7 (Version Space Representation)
Remarks:
Concept Learning: Version Space Candidate Elimination Algorithm
Concept Learning: Version Space Candidate Elimination Algorithm
Remarks:
Concept Learning: Version Space Candidate Elimination Algorithm (pseudo code)
Concept Learning: Version Space Candidate Elimination Algorithm (pseudo code)
Concept Learning: Version Space Illustration of the Candidate Elimination Algorithm
Concept Learning: Version Space Illustration of the Candidate Elimination Algorithm
Concept Learning: Version Space Illustration of the Candidate Elimination Algorithm
Concept Learning: Version Space Illustration of the Candidate Elimination Algorithm
Concept Learning: Version Space Illustration of the Candidate Elimination Algorithm
Concept Learning: Version Space Discussion of the Candidate Elimination Algorithm
Concept Learning: Version Space Question 1: Selecting Examples from D
Concept Learning: Version Space Question 1: Selecting Examples from D
Concept Learning: Version Space Question 2: Partially Learned Concepts
Concept Learning: Version Space Question 2: Partially Learned Concepts
Concept Learning: Version Space Question 2: Partially Learned Concepts
Concept Learning: Version Space Question 2: Partially Learned Concepts
Concept Learning: Version Space Question 2: Partially Learned Concepts
Concept Learning: Version Space Question 3: Inductive Bias
Concept Learning: Version Space Question 3: Inductive Bias
Concept Learning: Version Space Question 3: Inductive Bias
Concept Learning: Version Space Question 3: Inductive Bias
Concept Learning: Version Space Question 3: Inductive Bias
Concept Learning: Version Space Question 3: Inductive Bias
Chapter ML:II II. Machine Learning Basics
From Regression to Classification Regression versus Classification
From Regression to Classification Regression versus Classification
From Regression to Classification Regression versus Classification
From Regression to Classification The Linear Regression Model
From Regression to Classification The Linear Regression Model
From Regression to Classification The Linear Regression Model
Remarks (residuals):
Remarks (randomness and distributions):
From Regression to Classification One-Dimensional Feature Space
From Regression to Classification One-Dimensional Feature Space
From Regression to Classification One-Dimensional Feature Space
From Regression to Classification One-Dimensional Feature Space
From Regression to Classification One-Dimensional Feature Space
From Regression to Classification One-Dimensional Feature Space
From Regression to Classification One-Dimensional Feature Space
From Regression to Classification One-Dimensional Feature Space
From Regression to Classification Higher-Dimensional Feature Space
From Regression to Classification Higher-Dimensional Feature Space
From Regression to Classification Higher-Dimensional Feature Space
From Regression to Classification Higher-Dimensional Feature Space
From Regression to Classification Higher-Dimensional Feature Space
Remarks:
From Regression to Classification Linear Regression for Classification
From Regression to Classification Linear Regression for Classification
From Regression to Classification Linear Regression for Classification
From Regression to Classification Linear Regression for Classification
From Regression to Classification Linear Regression for Classification
From Regression to Classification Linear Regression for Classification
From Regression to Classification Linear Regression for Classification
Remarks: (?) We consider the feature vector x in its extended form when used as operand in a scalar
From Regression to Classification Linear Regression for Classification
From Regression to Classification Linear Regression for Classification
From Regression to Classification Linear Regression for Classification
From Regression to Classification Linear Regression for Classification
From Regression to Classification Linear Regression for Classification
From Regression to Classification Linear Regression for Classification
From Regression to Classification Linear Regression for Classification
Remarks:
From Regression to Classification Linear Model Function Variants
From Regression to Classification Linear Model Function Variants
From Regression to Classification Linear Model Function Variants
From Regression to Classification Linear Model Function Variants
From Regression to Classification Non-Linear Decision Boundaries
From Regression to Classification Non-Linear Decision Boundaries
From Regression to Classification Non-Linear Decision Boundaries
From Regression to Classification Non-Linear Decision Boundaries
From Regression to Classification Non-Linear Decision Boundaries
From Regression to Classification Methods of Least Squares: Iterative versus Direct Methods
From Regression to Classification Methods of Least Squares: Iterative versus Direct Methods
From Regression to Classification Methods of Least Squares: Iterative versus Direct Methods
Remarks:
From Regression to Classification Properties of the Solution
From Regression to Classification Properties of the Solution
Remarks:
Chapter ML:II
Evaluating Effectiveness Misclassification Rate
Evaluating Effectiveness Misclassification Rate
Remarks:
Remarks: (continued)
Evaluating Effectiveness Misclassification Rate
Evaluating Effectiveness Misclassification Rate
Evaluating Effectiveness Illustration 1: Label Noise
Evaluating Effectiveness Illustration 1: Label Noise
Evaluating Effectiveness Illustration 1: Label Noise
Evaluating Effectiveness Illustration 1: Label Noise
Remarks:
Remarks: (continued)
Evaluating Effectiveness Illustration 2: Bayes [Optimal] Classifier and Bayes Error
Evaluating Effectiveness Illustration 2: Bayes [Optimal] Classifier and Bayes Error
Evaluating Effectiveness Illustration 2: Bayes [Optimal] Classifier and Bayes Error
Evaluating Effectiveness Illustration 2: Bayes [Optimal] Classifier and Bayes Error
Evaluating Effectiveness Illustration 2: Bayes [Optimal] Classifier and Bayes Error
Remarks (Bayes classifier):
Evaluating Effectiveness Illustration 3: Marginal and Conditional Distributions
Evaluating Effectiveness Illustration 3: Marginal and Conditional Distributions
Evaluating Effectiveness Illustration 3: Marginal and Conditional Distributions
Evaluating Effectiveness Illustration 3: Marginal and Conditional Distributions
Evaluating Effectiveness Illustration 3: Marginal and Conditional Distributions
Evaluating Effectiveness Illustration 3: Marginal and Conditional Distributions
Remarks:
Evaluating Effectiveness Illustration 4: Probability Distribution in a Regression Setting
Evaluating Effectiveness Illustration 4: Probability Distribution in a Regression Setting
Evaluating Effectiveness Illustration 4: Probability Distribution in a Regression Setting
Evaluating Effectiveness Illustration 4: Probability Distribution in a Regression Setting
Evaluating Effectiveness Illustration 4: Probability Distribution in a Regression Setting
Evaluating Effectiveness Illustration 4: Probability Distribution in a Regression Setting
Remarks:
Evaluating Effectiveness Estimating Error Bounds
Evaluating Effectiveness Estimating Error Bounds
Evaluating Effectiveness Estimating Error Bounds
Evaluating Effectiveness Estimating Error Bounds
Remarks:
Evaluating Effectiveness Training Error
Remarks:
Evaluating Effectiveness Holdout Error
Evaluating Effectiveness Holdout Error
Evaluating Effectiveness Holdout Error
Evaluating Effectiveness Holdout Error
Remarks:
Evaluating Effectiveness k-Fold Cross-Validation
Evaluating Effectiveness k-Fold Cross-Validation
Remarks:
Evaluating Effectiveness Comparing Model Variants
Remarks:
Evaluating Effectiveness Model Selection: Single Validation Set
Evaluating Effectiveness Model Selection: Single Validation Set
Evaluating Effectiveness Model Selection: Single Validation Set
Evaluating Effectiveness Model Selection: k validation sets
Evaluating Effectiveness Model Selection: k validation sets
Evaluating Effectiveness Model Selection: k validation sets
Remarks:
Evaluating Effectiveness Misclassification Costs
Remarks:
Chapter ML:III III. Linear Models
Logistic Regression Binary Classification Problems
Logistic Regression Binary Classification Problems
Logistic Regression Linear Regression
Logistic Regression Linear Regression
Logistic Regression Linear Regression
Logistic Regression Linear Regression
Logistic Regression Linear Regression
Remarks: (?) Recap.
Logistic Regression Sigmoid (Logistic) Function
Logistic Regression Sigmoid (Logistic) Function
Logistic Regression Sigmoid (Logistic) Function
Logistic Regression Sigmoid (Logistic) Function
Logistic Regression Interpretation of the Logistic Model Function
Logistic Regression Interpretation of the Logistic Model Function
Logistic Regression Interpretation of the Logistic Model Function
Logistic Regression Interpretation of the Logistic Model Function
Logistic Regression Interpretation of the Logistic Model Function
Logistic Regression Interpretation of the Logistic Model Function
Logistic Regression Interpretation of the Logistic Model Function
Remarks (probabilistic view to classification) :
Remarks (probabilistic view to classification) : (continued)
Remarks (derivation of Lσ (w)) :
Remarks (derivation of Lσ (w)) : (continued) = argmax
Logistic Regression Recap. Linear Regression for Classification
Logistic Regression Recap. Linear Regression for Classification
Logistic Regression Logistic Regression for Classification
Logistic Regression Logistic Regression for Classification
Logistic Regression Logistic Regression for Classification
Logistic Regression Logistic Regression for Classification
Logistic Regression Logistic Regression for Classification
Logistic Regression Logistic Regression for Classification
Logistic Regression The BGDσ Algorithm
Logistic Regression The BGDσ Algorithm
Remarks:
Logistic Regression Machine Learning Stack for Logistic Regression
Logistic Regression Machine Learning Stack for Logistic Regression
Logistic Regression Machine Learning Stack for Logistic Regression
Logistic Regression Machine Learning Stack for Logistic Regression
Logistic Regression Machine Learning Stack for Logistic Regression
Logistic Regression Non-Linear Decision Boundaries
Logistic Regression Non-Linear Decision Boundaries
Logistic Regression Non-Linear Decision Boundaries
Logistic Regression Non-Linear Decision Boundaries
Logistic Regression Non-Linear Decision Boundaries
Remarks:
Chapter ML:III III. Linear Models
Loss Computation in Detail Loss Computation in the Machine Learning Stack
Remarks:
Loss Computation in Detail Linear Regression
Loss Computation in Detail Linear Regression
Loss Computation in Detail Linear Regression
Loss Computation in Detail Linear Regression
Loss Computation in Detail Linear Regression
Loss Computation in Detail Linear Regression
Loss Computation in Detail Linear Regression
Remarks:
Loss Computation in Detail Logistic Regression
Loss Computation in Detail Logistic Regression
Loss Computation in Detail Logistic Regression
Loss Computation in Detail Logistic Regression
Remarks:
Remarks (different roles of loss functions) :
Chapter ML:III
Overfitting Definition 9 (Overfitting)
Overfitting Definition 9 (Overfitting)
Overfitting Definition 9 (Overfitting)
Overfitting Definition 9 (Overfitting)
Overfitting Definition 9 (Overfitting)
Overfitting Example: Linear Regression
Overfitting Example: Linear Regression
Overfitting Example: Linear Regression
Overfitting Example: Linear Regression
Overfitting Example: Linear Regression
Overfitting Example: Linear Regression
Overfitting Example: Linear Regression
Overfitting Definition 9 (Overfitting
Overfitting Mitigation Strategies
Overfitting Mitigation Strategies
Overfitting Mitigation Strategies
Chapter ML:III
Regularization Regularization in the Machine Learning Stack
Regularization Bound the Absolute Values of the Weights w
Regularization Bound the Absolute Values of the Weights w
Regularization Bound the Absolute Values of the Weights w
Regularization Bound the Absolute Values of the Weights w
Regularization Bound the Absolute Values of the Weights w
Remarks:
Remarks (continued) :
Regularization The Vector Norm as Regularization Function
Remarks:
Regularization The Vector Norm as Regularization Function
Regularization The Vector Norm as Regularization Function
Regularization The Vector Norm as Regularization Function
Remarks:
Regularization The Vector Norm as Regularization Function
Regularization The Vector Norm as Regularization Function
Regularization The Vector Norm as Regularization Function
Regularization The Vector Norm as Regularization Function
Regularization The Vector Norm as Regularization Function
Regularization The Vector Norm as Regularization Function
Regularization The Vector Norm as Regularization Function
Regularization The Vector Norm as Regularization Function
Regularization The Vector Norm as Regularization Function
Regularization The Vector Norm as Regularization Function
Remarks:
Regularization Regularized Linear Regression
Regularization Regularized Linear Regression
Regularization Regularized Linear Regression
Regularization Regularized Linear Regression
Regularization Regularized Linear Regression
Regularization Regularized Linear Regression
Regularization Regularized Linear Regression
Chapter ML:III
Gradient Descent in Detail Principle
Gradient Descent in Detail Principle
Gradient Descent in Detail Principle
Gradient Descent in Detail Principle
Remarks:
Gradient Descent in Detail
Gradient Descent in Detail (continued)
Gradient Descent in Detail (continued)
Gradient Descent in Detail (continued)
Gradient Descent in Detail (continued)
Gradient Descent in Detail (1) Linear Regression + Squared Loss
Gradient Descent in Detail (1) Linear Regression + Squared Loss
Remarks (derivation of ∇L2 (w)) :
Gradient Descent in Detail The BGD Algorithm
Gradient Descent in Detail The BGD Algorithm
Remarks: (?) Recap.
Remarks: (continued)
Gradient Descent in Detail Global Loss versus Pointwise Loss
Remarks:
Gradient Descent in Detail The IGD Algorithm
Gradient Descent in Detail The IGD Algorithm
Remarks (IGD) :
Remarks (:::::::: recap. different roles of loss functions) :
Gradient Descent in Detail (2) Linear Regression + 0\u002f1 Loss
Gradient Descent in Detail (2) Linear Regression + 0\u002f1 Loss
Gradient Descent in Detail (continued)
Gradient Descent in Detail (continued)
Remarks:
Gradient Descent in Detail
Gradient Descent in Detail (continued)
Gradient Descent in Detail (continued)
Gradient Descent in Detail (continued)
Gradient Descent in Detail (continued)
Gradient Descent in Detail (3) Logistic Regression + Logistic Loss + Regularization
Gradient Descent in Detail (3) Logistic Regression + Logistic Loss + Regularization
Remarks: ~ = (w1 , . . . , wp )T , and the
Remarks (derivation of ∇Lσ (w)) :
Remarks (derivation of ∇Lσ (w)) : (continued) (3)
Chapter ML:IV IV. Neural Networks
Perceptron Learning Biological Inspiration
Perceptron Learning Biological Inspiration
Remarks (facts about the human brain):
Perceptron Learning History
Perceptron Learning The Perceptron of Rosenblatt
Perceptron Learning The Perceptron of Rosenblatt
Perceptron Learning The Perceptron of Rosenblatt
Perceptron Learning The Perceptron of Rosenblatt
Remarks:
Perceptron Learning Binary Classification Problems
Perceptron Learning The PT Algorithm
Perceptron Learning The PT Algorithm
Remarks: (?) Recap.
Perceptron Learning Weight Adaptation: Illustration in Input Space
Perceptron Learning Weight Adaptation: Illustration in Input Space
Remarks: ~ = (w1 , . . . , wp )T , and the
Perceptron Learning Example
Perceptron Learning Example
Perceptron Learning Example: Illustration in Input Space
Perceptron Learning Example: Illustration in Input Space
Perceptron Learning Example: Illustration in Input Space
Perceptron Learning Example: Illustration in Input Space
Perceptron Learning Example: Illustration in Input Space
Perceptron Learning Example: Illustration in Input Space
Perceptron Learning Example: Illustration in Input Space
Perceptron Learning Example: Illustration in Input Space
Perceptron Learning Example: Illustration in Input Space
Perceptron Learning Example: Illustration in Input Space
Perceptron Learning Perceptron Convergence Theorem
Perceptron Learning Perceptron Convergence Theorem
Perceptron Learning Perceptron Convergence Theorem: Proof
Perceptron Learning Perceptron Convergence Theorem: Proof
Perceptron Learning Perceptron Convergence Theorem: Proof
Perceptron Learning Perceptron Convergence Theorem: Proof
Perceptron Learning Perceptron Convergence Theorem: Proof
Perceptron Learning Perceptron Convergence Theorem: Proof
Perceptron Learning Perceptron Convergence Theorem: Proof
Perceptron Learning Perceptron Convergence Theorem: Proof
Perceptron Learning Perceptron Convergence Theorem: Proof
Perceptron Learning Perceptron Convergence Theorem: Proof
Perceptron Learning Perceptron Convergence Theorem: Proof
Perceptron Learning Perceptron Convergence Theorem: Proof
Perceptron Learning Perceptron Convergence Theorem: Proof
Perceptron Learning Perceptron Convergence Theorem: Proof
Perceptron Learning Perceptron Convergence Theorem: Proof
Perceptron Learning Perceptron Convergence Theorem: Discussion
Remarks:
Perceptron Learning PT Algorithm versus Gradient Descent
Perceptron Learning PT Algorithm versus Gradient Descent
Perceptron Learning PT Algorithm versus Gradient Descent
Remarks (PT versus GD) :
Chapter ML:IV
Multilayer Perceptron Basics Definition 1 (Linear Separability)
Multilayer Perceptron Basics Definition 1 (Linear Separability)
Multilayer Perceptron Basics Linear Separability
Multilayer Perceptron Basics Linear Separability
Multilayer Perceptron Basics Overcoming the Linear Separability Restriction
Multilayer Perceptron Basics Overcoming the Linear Separability Restriction
Multilayer Perceptron Basics Overcoming the Linear Separability Restriction
Multilayer Perceptron Basics Overcoming the Linear Separability Restriction
Multilayer Perceptron Basics Overcoming the Linear Separability Restriction
Multilayer Perceptron Basics Overcoming the Linear Separability Restriction
Remarks:
Remarks (history) :
Multilayer Perceptron Basics Overcoming the Non-Differentiability Restriction
Multilayer Perceptron Basics Overcoming the Non-Differentiability Restriction
Multilayer Perceptron Basics Overcoming the Non-Differentiability Restriction
Multilayer Perceptron Basics Overcoming the Non-Differentiability Restriction
Multilayer Perceptron Basics Unrestricted Classification Problems
Multilayer Perceptron Basics Unrestricted Classification Problems: Example
Multilayer Perceptron Basics Unrestricted Classification Problems: Example
Multilayer Perceptron Basics Sigmoid Function
Multilayer Perceptron Basics Sigmoid Function
Remarks (derivation of (σ(z))0 ) :
Remarks (limitation of linear thresholds) :
Chapter ML:IV
Multilayer Perceptron with Two Layers Network Architecture
Multilayer Perceptron with Two Layers Network Architecture
Multilayer Perceptron with Two Layers Network Architecture
Multilayer Perceptron with Two Layers Network Architecture
Multilayer Perceptron with Two Layers (1) Forward Propagation
Remarks:
Multilayer Perceptron with Two Layers (1) Forward Propagation
Multilayer Perceptron with Two Layers (1) Forward Propagation
Multilayer Perceptron with Two Layers (1) Forward Propagation: Batch Mode
Multilayer Perceptron with Two Layers (2) Backpropagation
Multilayer Perceptron with Two Layers (2) Backpropagation
Multilayer Perceptron with Two Layers (continued)
Multilayer Perceptron with Two Layers (continued)
Multilayer Perceptron with Two Layers (continued)
Remarks:
Multilayer Perceptron with Two Layers (2) Backpropagation
Multilayer Perceptron with Two Layers (2) Backpropagation
Multilayer Perceptron with Two Layers (2) Backpropagation
Multilayer Perceptron with Two Layers The IGD Algorithm
Multilayer Perceptron with Two Layers The IGD Algorithm
Multilayer Perceptron with Two Layers The IGD Algorithm
Multilayer Perceptron with Two Layers The IGD Algorithm
Multilayer Perceptron with Two Layers The IGD Algorithm
Remarks:
Chapter ML:IV
Multilayer Perceptron at Arbitrary Depth Network Architecture
Multilayer Perceptron at Arbitrary Depth (1) Forward Propagation
Multilayer Perceptron at Arbitrary Depth (2) Backpropagation
Multilayer Perceptron at Arbitrary Depth (2) Backpropagation (continued)
Multilayer Perceptron at Arbitrary Depth (2) Backpropagation (continued) [two layers]
Multilayer Perceptron at Arbitrary Depth (2) Backpropagation (continued) [two layers]
Multilayer Perceptron at Arbitrary Depth (2) Backpropagation
Multilayer Perceptron at Arbitrary Depth The IGD Algorithm
Multilayer Perceptron at Arbitrary Depth The IGD Algorithm
Multilayer Perceptron at Arbitrary Depth The IGD Algorithm
Multilayer Perceptron at Arbitrary Depth The IGD Algorithm
Multilayer Perceptron at Arbitrary Depth The IGD Algorithm
Remarks (derivation of ∇hs L2 (w)) :
Remarks (derivation of ∇hs L2 (w)) : (continued)
Remarks (derivation of ∇hs L2 (w)) : (continued)
Remarks (derivation of ∇hs L2 (w)) : (continued)
Remarks (derivation of ∇hs L2 (w)) : (continued) h
Remarks (derivation of ∇hs L2 (w)) : (continued) h
Remarks (derivation of ∇o L2 (w) and ∇h L2 (w) for MLP at depth one) :
Chapter ML:IV
Advanced MLPs Output Normalization: Softmax
Advanced MLPs Output Normalization: Softmax
Advanced MLPs Output Normalization: Softmax
Advanced MLPs Output Normalization: Softmax
Advanced MLPs Output Normalization: Softmax
Advanced MLPs Output Normalization: Softmax
Advanced MLPs Output Normalization: Softmax
Remarks:
Advanced MLPs Loss Function: Cross-Entropy
Advanced MLPs Loss Function: Cross-Entropy
Advanced MLPs Loss Function: Cross-Entropy
Advanced MLPs Cross-Entropy in Classification Settings
Advanced MLPs Cross-Entropy in Classification Settings
Advanced MLPs Cross-Entropy in Classification Settings
Remarks:
Advanced MLPs Activation Function: Rectified Linear Unit (ReLU)
Advanced MLPs Regularization: Dropout
Advanced MLPs Learning Rate Adaptation: Momentum
Advanced MLPs Learning Rate Adaptation: Momentum
Remarks:
Chapter ML:IV
Automatic Gradient Computation The IGD Algorithm
Automatic Gradient Computation The IGD Algorithm
Automatic Gradient Computation The IGD Algorithm
Automatic Gradient Computation Reverse-Mode Automatic Differentiation in Computational Graphs
Automatic Gradient Computation Reverse-Mode Automatic Differentiation in Computational Graphs
Remarks:
Automatic Gradient Computation Autodiff Example: Setting
Automatic Gradient Computation Autodiff Example: Computational Graph
Automatic Gradient Computation Example: Forward and Reverse Trace
Automatic Gradient Computation Example: Forward and Reverse Trace
Automatic Gradient Computation Example: Forward and Reverse Trace
Automatic Gradient Computation Example: Forward and Reverse Trace
Automatic Gradient Computation Example: Forward and Reverse Trace
Automatic Gradient Computation Example: Forward and Reverse Trace
Automatic Gradient Computation Example: Forward and Reverse Trace
Remarks: ∂
Automatic Gradient Computation Reverse-mode Autodiff Algorithm for Scalar-valued Functions
Remarks:
Chapter ML:VI VI. Decision Trees
Decision Trees Basics Classification Problems with Nominal Features
Decision Trees Basics Decision Tree for the Concept “EnjoySurfing”
Decision Trees Basics Decision Tree for the Concept “EnjoySurfing”
Decision Trees Basics Definition 1 (Splitting, Induced Splitting)
Decision Trees Basics Definition 1 (Splitting, Induced Splitting)
Decision Trees Basics Definition 1 (Splitting, Induced Splitting)
Decision Trees Basics Definition 1 (Splitting, Induced Splitting)
Decision Trees Basics Definition 1 (Splitting, Induced Splitting)
Decision Trees Basics Definition 1 (Splitting, Induced Splitting)
Remarks:
Decision Trees Basics Definition 2 (Decision Tree)
Decision Trees Basics Definition 2 (Decision Tree)
Decision Trees Basics Definition 2 (Decision Tree)
Remarks:
Decision Trees Basics Notation
Remarks:
Decision Trees Basics Algorithm Template: Construction
Decision Trees Basics Algorithm Template: Classification
Remarks:
Decision Trees Basics When to Use Decision Trees
Decision Trees Basics On the Construction of Decision Trees
Decision Trees Basics Assessment of Decision Trees
Decision Trees Basics Assessment of Decision Trees
Decision Trees Basics Assessment of Decision Trees: Size
Decision Trees Basics Assessment of Decision Trees: Size
Decision Trees Basics Assessment of Decision Trees: Size
Decision Trees Basics Assessment of Decision Trees: Size
Decision Trees Basics Assessment of Decision Trees: Size
Decision Trees Basics Assessment of Decision Trees: Size
Decision Trees Basics Assessment of Decision Trees: Classification Error
Decision Trees Basics Assessment of Decision Trees: Classification Error
Decision Trees Basics Assessment of Decision Trees: Classification Error
Remarks:
Remarks (misclassification costs):
Chapter ML:VI VI. Decision Trees
Impurity Functions Splitting
Impurity Functions Splitting
Impurity Functions Splitting
Impurity Functions Splitting
Impurity Functions Definition 4 (Impurity Function ι)
Impurity Functions Definition 5 (Impurity of an Example Set ι(D))
Impurity Functions Definition 5 (Impurity of an Example Set ι(D))
Remarks:
Impurity Functions Impurity Functions Based on the Misclassification Rate
Impurity Functions Impurity Functions Based on the Misclassification Rate
Impurity Functions Impurity Functions Based on the Misclassification Rate
Impurity Functions Impurity Functions Based on the Misclassification Rate
Impurity Functions Impurity Functions Based on the Misclassification Rate
Impurity Functions Impurity Functions Based on the Misclassification Rate
Impurity Functions Definition 7 (Strict Impurity Function)
Impurity Functions Definition 7 (Strict Impurity Function)
Remarks:
Impurity Functions Impurity Functions Based on Entropy
Remarks:
Impurity Functions Impurity Functions Based on Entropy
Impurity Functions Impurity Functions Based on Entropy
Impurity Functions Impurity Functions Based on Entropy
Remarks [::::::: Bayes :::
Remarks: (continued)
Impurity Functions Impurity Functions Based on Entropy
Impurity Functions Impurity Functions Based on Entropy
Impurity Functions Impurity Functions Based on Entropy
Impurity Functions Impurity Functions Based on Entropy
Impurity Functions Impurity Functions Based on Entropy
Impurity Functions Impurity Functions Based on Entropy
Impurity Functions Impurity Functions Based on Entropy
Impurity Functions Impurity Functions Based on the Gini Index
Impurity Functions Impurity Functions Based on the Gini Index
Impurity Functions Impurity Functions Based on the Gini Index
Impurity Functions Impurity Functions Based on the Gini Index
Chapter ML:VI VI. Decision Trees
Decision Tree Algorithms ID3 Algorithm
Decision Tree Algorithms ID3 Algorithm
Decision Tree Algorithms ID3 Algorithm
Decision Tree Algorithms ID3 Algorithm
Decision Tree Algorithms ID3 Algorithm (pseudo code)
Decision Tree Algorithms ID3 Algorithm (pseudo code)
Decision Tree Algorithms ID3 Algorithm (pseudo code)
Decision Tree Algorithms ID3 Algorithm (pseudo code)
Remarks:
Decision Tree Algorithms ID3 Algorithm: Example
Decision Tree Algorithms ID3 Algorithm: Example
Decision Tree Algorithms ID3 Algorithm: Example
Decision Tree Algorithms ID3 Algorithm: Example
Decision Tree Algorithms ID3 Algorithm: Example
Remarks:
Decision Tree Algorithms ID3 Algorithm: Example
Decision Tree Algorithms ID3 Algorithm: Example
Decision Tree Algorithms ID3 Algorithm: Example
Decision Tree Algorithms ID3 Algorithm: Search Space
Remarks (search space versus hypothesis space):
Decision Tree Algorithms ID3 Algorithm: Inductive Bias
Decision Tree Algorithms ID3 Algorithm: Inductive Bias
Decision Tree Algorithms ID3 Algorithm: Inductive Bias
Remarks (inductive bias):
Decision Tree Algorithms CART Algorithm
Decision Tree Algorithms CART Algorithm
Decision Tree Algorithms CART Algorithm
Decision Tree Algorithms CART Algorithm
Remarks:
Decision Tree Algorithms CART Algorithm
Decision Tree Algorithms CART Algorithm
Decision Tree Algorithms CART Algorithm
Decision Tree Algorithms CART Algorithm
Decision Tree Algorithms CART Algorithm
Chapter ML:VI VI. Decision Trees
Decision Tree Pruning Overfitting
Decision Tree Pruning Overfitting
Decision Tree Pruning Overfitting
Remarks:
Decision Tree Pruning Overfitting
Decision Tree Pruning Overfitting
Remarks:
Decision Tree Pruning Overfitting
Decision Tree Pruning (a) Stopping
Decision Tree Pruning (b) Pruning
Decision Tree Pruning (b) Pruning
Decision Tree Pruning (b) Pruning
Decision Tree Pruning (b) Pruning
Decision Tree Pruning (b) Pruning
Decision Tree Pruning (b) Pruning: Reduced Error Pruning
Decision Tree Pruning (b) Pruning: Reduced Error Pruning
Decision Tree Pruning (b) Pruning: Reduced Error Pruning
Remarks (pruning extensions) :
Chapter ML:VII VII. Bayesian Learning
Approaches to Probability Area Overview
Approaches to Probability Area Overview
Approaches to Probability Definition 1 (Random Experiment, Random Observation)
Approaches to Probability Definition 1 (Random Experiment, Random Observation)
Remarks: ❑
Approaches to Probability Definition 2 (Sample Space, Event Space)
Approaches to Probability Definition 2 (Sample Space, Event Space)
Approaches to Probability Definition 2 (Sample Space, Event Space)
Approaches to Probability Definition 2 (Sample Space, Event Space)
Approaches to Probability Definition 3 (Important Event Types)
Approaches to Probability Definition 3 (Important Event Types)
Remarks: ❑
Approaches to Probability How to Capture the Nature of Probability
Approaches to Probability How to Capture the Nature of Probability
Remarks: ❑
Approaches to Probability How to Capture the Nature of Probability
Remarks: ❑
Approaches to Probability How to Capture the Nature of Probability
Approaches to Probability How to Capture the Nature of Probability
Approaches to Probability How to Capture the Nature of Probability
Approaches to Probability How to Capture the Nature of Probability
Approaches to Probability How to Capture the Nature of Probability
Remarks: ❑
Remarks (frequentist versus subjectivist) : ❑
Approaches to Probability Axiomatic Approach to Probability
Approaches to Probability Axiomatic Approach to Probability
Approaches to Probability Axiomatic Approach to Probability
Approaches to Probability Axiomatic Approach to Probability
Approaches to Probability Axiomatic Approach to Probability
Remarks: ❑
Chapter ML:VII
Conditional Probability Basic Definition
Conditional Probability Basic Definition
Conditional Probability Basic Definition
Conditional Probability Basic Definition
Conditional Probability Basic Definition
Conditional Probability Basic Definition
Remarks (conditional probability) : ❑
Remarks (conditional event algebra) : ❑
Conditional Probability Total Probability
Conditional Probability Total Probability
Conditional Probability Total Probability
Conditional Probability Total Probability
Conditional Probability Total Probability
Conditional Probability Total Probability
Conditional Probability Total Probability
Remarks: ❑
Conditional Probability Independence of Events
Conditional Probability Independence of Events
Conditional Probability Independence of Events
Conditional Probability Independence of Events
Conditional Probability Independence of Events
Conditional Probability Independence of Events
Conditional Probability Independence of Events
Conditional Probability Independence of Events
Conditional Probability Independence of Events
Conditional Probability Independence of Events
Chapter ML:VII
Bayes Classifier Generative Approach to Classification Problems
Bayes Classifier Bayes Theorem
Bayes Classifier Bayes Theorem
Bayes Classifier Bayes Theorem
Bayes Classifier Example: Reasoning About a Disease
Bayes Classifier Example: Reasoning About a Disease
Bayes Classifier Example: Reasoning About a Disease
Bayes Classifier Example: Reasoning About a Disease
Bayes Classifier Example: Reasoning About a Disease
Bayes Classifier Example: Reasoning About a Disease
Bayes Classifier Example: Reasoning About a Disease
Bayes Classifier Example: Reasoning About a Disease
Bayes Classifier Combined Conditional Events: P (Ai | B1, . . . , Bp)
Bayes Classifier Combined Conditional Events: P (Ai | B1, . . . , Bp)
Bayes Classifier Combined Conditional Events: P (Ai | B1, . . . , Bp)
Remarks [:::::::::::: information:::::
Remarks: (continued)
Bayes Classifier Naive Bayes
Bayes Classifier Naive Bayes
Bayes Classifier Naive Bayes
Remarks:
Remarks: (continued)
Bayes Classifier Naive Bayes
Bayes Classifier Naive Bayes
Bayes Classifier Naive Bayes
Remarks:
Bayes Classifier Naive Bayes: Classifier Construction Summary
Bayes Classifier Naive Bayes: Classifier Construction Summary
Remarks:
Bayes Classifier Naive Bayes: Example
Bayes Classifier Naive Bayes: Example
Bayes Classifier Naive Bayes: Example
Bayes Classifier Naive Bayes: Example
Bayes Classifier Naive Bayes: Example
Bayes Classifier Naive Bayes: Example
Remarks:
Chapter ML:VII
Exploitation of Data Data Events
Exploitation of Data Data Events
Exploitation of Data Data Events
Exploitation of Data Data Events
Remarks:
Remarks: (continued)
Exploitation of Data Typical Learning Settings
Exploitation of Data Typical Learning Settings
Exploitation of Data Typical Learning Settings
Exploitation of Data Typical Learning Settings
Exploitation of Data Typical Learning Settings
Exploitation of Data Typical Learning Settings
Exploitation of Data Typical Learning Settings
Remarks (predictor-response vs. outcome-only setting) : (1),. . . , (4)
Remarks (discriminative vs. generative approach) : (1), (2), (3)
Remarks (discriminative vs. generative approach) : (continued)
Remarks (ML principle vs. Bayes method) : (1), (2), (3)
Exploitation of Data Learning Approaches Overview
Exploitation of Data Learning Approaches Overview
Exploitation of Data Learning Approaches Overview
Exploitation of Data Learning Approaches Overview
Remarks:
Chapter ML:VII
Frequentist versus Subjectivist Logistic Regression versus Naive Bayes
Frequentist versus Subjectivist Logistic Regression versus Naive Bayes
Frequentist versus Subjectivist Logistic Regression versus Naive Bayes
Frequentist versus Subjectivist Logistic Regression versus Naive Bayes
Frequentist versus Subjectivist Logistic Regression versus Naive Bayes
Frequentist versus Subjectivist Logistic Regression versus Naive Bayes
Frequentist versus Subjectivist Logistic Regression versus Naive Bayes
Remarks:
Remarks: (continued)
Frequentist versus Subjectivist Logistic Regression versus Naive Bayes: Example
Frequentist versus Subjectivist Logistic Regression versus Naive Bayes: Example
Frequentist versus Subjectivist Logistic Regression versus Naive Bayes: Example
Frequentist versus Subjectivist Logistic Regression versus Naive Bayes: Conditional Class Probabilities
Frequentist versus Subjectivist Logistic Regression versus Naive Bayes: Conditional Class Probabilities
Frequentist versus Subjectivist Logistic Regression versus Naive Bayes: Conditional Class Probabilities
Frequentist versus Subjectivist Logistic Regression versus Naive Bayes: Conditional Class Probabilities
Frequentist versus Subjectivist Logistic Regression versus Naive Bayes: Conditional Class Probabilities
Frequentist versus Subjectivist Logistic Regression versus Naive Bayes: Conditional Class Probabilities
Frequentist versus Subjectivist Logistic Regression versus Naive Bayes: Conditional Class Probabilities
Frequentist versus Subjectivist Logistic Regression versus Naive Bayes: Conditional Class Probabilities
Remarks:
Frequentist versus Subjectivist Naive Bayes: Smoothing and Continuous Likelihoods
Frequentist versus Subjectivist Naive Bayes: Prior Probability Models
Frequentist versus Subjectivist Classification: Bayes Optimum versus MAP versus Ensemble
Frequentist versus Subjectivist Advanced Bayesian Decision Making
Frequentist versus Subjectivist Advanced Bayesian Decision Making
Frequentist versus Subjectivist Advanced Bayesian Decision Making
Frequentist versus Subjectivist Advanced Bayesian Decision Making
Frequentist versus Subjectivist Advanced Bayesian Decision Making
Remarks:
Chapter ML:IX IX. Deep Learning
Introduction to Deep Learning History
Introduction to Deep Learning History
Introduction to Deep Learning History
Introduction to Deep Learning History
Introduction to Deep Learning History
Introduction to Deep Learning History
Introduction to Deep Learning History
Introduction to Deep Learning Types of Learning Tasks
Introduction to Deep Learning Types of Learning Tasks
Remarks:
Chapter ML:IX
Autoencoder Networks Representation Learning Task: Word Embedding
Autoencoder Networks Word Embedding with Autoencoders
Autoencoder Networks Representation Learning Task: Co-Occurrence Embedding (Word2Vec)
Chapter ML:IX
Convolutional Neural Networks Image Classification Task
Convolutional Neural Networks Image Classification Task
Convolutional Neural Networks Image Classification Task
Convolutional Neural Networks Image Classification with CNNs
Convolutional Neural Networks Image Classification with CNNs
Convolutional Neural Networks Image Classification with CNNs
Convolutional Neural Networks (continued)
Convolutional Neural Networks Image Classification with CNNs
Convolutional Neural Networks Image Classification with CNNs
Remarks (computation of yc ) :
Remarks (technical variants and improvements) :
Remarks (learning strategies) :
Chapter ML:IX
Recurrent Neural Networks Notation I
Recurrent Neural Networks (continued)
Recurrent Neural Networks (continued)
Recurrent Neural Networks (continued)
Recurrent Neural Networks (continued)
Remarks:
Recurrent Neural Networks Types of Learning Tasks
Recurrent Neural Networks RNN Sequence Encoding
Recurrent Neural Networks (continued)
Recurrent Neural Networks (continued)
Recurrent Neural Networks (continued)
Remarks:
Recurrent Neural Networks RNN Sequence Encoding
Recurrent Neural Networks RNN Sequence Encoding
Recurrent Neural Networks RNN Sequence Encoding
Recurrent Neural Networks RNN Sequence Encoding
Recurrent Neural Networks RNN Sequence Encoding
Recurrent Neural Networks RNN Sequence Encoding
Recurrent Neural Networks RNN Sequence Encoding
Recurrent Neural Networks RNN Sequence Encoding
Recurrent Neural Networks RNN Sequence Encoding
Recurrent Neural Networks RNN Sequence Encoding
Recurrent Neural Networks RNN Sequence Encoding
Recurrent Neural Networks RNN Sequence Encoding
Recurrent Neural Networks RNN Sequence Encoding
Recurrent Neural Networks RNN Sequence Encoding
Recurrent Neural Networks RNN Sequence Encoding
Recurrent Neural Networks (S1) Sequence-to-Class: Sentiment Classification
Recurrent Neural Networks (S1) Sequence-to-Class: Sentiment Classification
Recurrent Neural Networks (S1) Sequence-to-Class: Sentiment Classification
Recurrent Neural Networks (S1) Sequence-to-Class Mapping with RNNs
Recurrent Neural Networks
Recurrent Neural Networks
Recurrent Neural Networks
Recurrent Neural Networks
Recurrent Neural Networks
Remarks:
Recurrent Neural Networks The IGD Algorithm for Sequence-to-Class Tasks
Recurrent Neural Networks The IGD Algorithm for Sequence-to-Class Tasks
Recurrent Neural Networks Types of Learning Tasks
Recurrent Neural Networks RNN Sequence Decoding
Recurrent Neural Networks (continued)
Recurrent Neural Networks (continued)
Recurrent Neural Networks (continued)
Remarks:
Recurrent Neural Networks RNN Sequence Decoding
Recurrent Neural Networks RNN Sequence Decoding
Recurrent Neural Networks RNN Sequence Decoding
Recurrent Neural Networks RNN Sequence Decoding
Recurrent Neural Networks RNN Sequence Decoding
Recurrent Neural Networks RNN Sequence Decoding
Recurrent Neural Networks RNN Sequence Decoding
Recurrent Neural Networks RNN Sequence Decoding
Recurrent Neural Networks RNN Sequence Decoding
Recurrent Neural Networks RNN Sequence Decoding
Recurrent Neural Networks RNN Sequence Decoding
Recurrent Neural Networks RNN Sequence Decoding
Recurrent Neural Networks RNN Sequence Decoding
Recurrent Neural Networks RNN Sequence Decoding
Recurrent Neural Networks RNN Sequence Decoding
Recurrent Neural Networks RNN Sequence Decoding
Recurrent Neural Networks RNN Sequence Decoding
Recurrent Neural Networks RNN Sequence Decoding
Recurrent Neural Networks RNN Sequence Decoding
Recurrent Neural Networks (S2) Class-to-Sequence: Text Generation
Recurrent Neural Networks (S2) Class-to-Sequence: Text Generation
Recurrent Neural Networks (S2) Class-to-Sequence: Text Generation
Recurrent Neural Networks (S2) Class-to-Sequence: Text Generation
Recurrent Neural Networks (S2) Class-to-Sequence: Text Generation
Recurrent Neural Networks (S2) Class-to-Sequence: Text Generation
Recurrent Neural Networks (S2) Class-to-Sequence: Text Generation
Recurrent Neural Networks (S2) Class-to-Sequence Mapping with RNNs
Recurrent Neural Networks (S2) Class-to-Sequence Mapping with RNNs
Recurrent Neural Networks (S2) Class-to-Sequence Mapping with RNNs
Recurrent Neural Networks (S2) Class-to-Sequence Mapping with RNNs
Recurrent Neural Networks (S2) Class-to-Sequence Mapping with RNNs
Recurrent Neural Networks (S2) Class-to-Sequence Mapping with RNNs
Recurrent Neural Networks (S2) Class-to-Sequence Mapping with RNNs
Recurrent Neural Networks (S2) Class-to-Sequence Mapping with RNNs
Recurrent Neural Networks (S2) Class-to-Sequence Mapping with RNNs
Remarks:
Recurrent Neural Networks The IGD Algorithm for Class-to-Sequence Tasks
Recurrent Neural Networks The IGD Algorithm for Class-to-Sequence Tasks
Chapter ML:IX
Long-Term Dependencies Vanishing Gradient Problem
Long-Term Dependencies RNN with Long Short-Term Memory (LSTM)
Remarks:
Long-Term Dependencies RNN with Gated Recurrent Units (GRU)
Chapter ML:IX
RNNs for Machine Translation Statistical Machine Translation (SMT)
RNNs for Machine Translation Statistical Machine Translation (SMT)
RNNs for Machine Translation Statistical Machine Translation (SMT)
RNNs for Machine Translation Statistical Machine Translation (SMT)
RNNs for Machine Translation Statistical Machine Translation (SMT)
RNNs for Machine Translation Statistical Machine Translation (SMT)
Remarks:
RNNs for Machine Translation Statistical Machine Translation (SMT)
RNNs for Machine Translation Statistical Machine Translation (SMT)
RNNs for Machine Translation Statistical Machine Translation (SMT)
RNNs for Machine Translation Statistical Machine Translation (SMT)
Remarks (statistical machine translation) :
RNNs for Machine Translation Neural Machine Translation (NMT)
RNNs for Machine Translation Neural Machine Translation (NMT)
RNNs for Machine Translation Neural Machine Translation (NMT)
RNNs for Machine Translation Neural Machine Translation (NMT)
Remarks:
RNNs for Machine Translation Types of Learning Tasks
RNNs for Machine Translation (S3) Sequence-to-Sequence: Machine Translation
RNNs for Machine Translation (S3) Sequence-to-Sequence: Machine Translation
RNNs for Machine Translation (S3) Sequence-to-Sequence: Machine Translation
RNNs for Machine Translation (S3) Sequence-to-Sequence: Machine Translation
RNNs for Machine Translation (S3) Sequence-to-Sequence: Machine Translation
RNNs for Machine Translation (S3) Sequence-to-Sequence: Machine Translation
RNNs for Machine Translation (S3) Sequence-to-Sequence: Machine Translation
RNNs for Machine Translation (S3) Sequence-to-Sequence Mapping with RNNs
RNNs for Machine Translation (S3) Sequence-to-Sequence Mapping with RNNs
RNNs for Machine Translation (S3) Sequence-to-Sequence Mapping with RNNs
RNNs for Machine Translation (S3) Sequence-to-Sequence Mapping with RNNs
RNNs for Machine Translation (S3) Sequence-to-Sequence Mapping with RNNs
RNNs for Machine Translation (S3) Sequence-to-Sequence Mapping with RNNs
RNNs for Machine Translation (S3) Sequence-to-Sequence Mapping with RNNs
RNNs for Machine Translation (S3) Sequence-to-Sequence Mapping with RNNs
Remarks:
RNNs for Machine Translation Sequence-to-Sequence RNNs are Conditional Language Models
RNNs for Machine Translation Sequence-to-Sequence RNNs are Conditional Language Models
RNNs for Machine Translation Sequence-to-Sequence RNNs are Conditional Language Models
RNNs for Machine Translation Sequence-to-Sequence RNNs are Conditional Language Models
RNNs for Machine Translation Sequence-to-Sequence RNNs are Conditional Language Models
RNNs for Machine Translation Sequence-to-Sequence RNNs are Conditional Language Models
Remarks: Each output vector y(t) corresponds to a probability distribution over Vocabularyd (recall the
Chapter ML:IX
Attention Mechanism Notation III (language model view)
Attention Mechanism Notation III (language model view)
Attention Mechanism Notation III (language model view)
Attention Mechanism Notation III (language model view)
Attention Mechanism Notation III (language model view)
Attention Mechanism Notation III (language model view)
Attention Mechanism Notation III (language model view)
Attention Mechanism Notation III (language model view)
Attention Mechanism Notation III (language model view)
Attention Mechanism Notation III (language model view)
Attention Mechanism Notation III (language model view)
Attention Mechanism Notation III (language model view)
Attention Mechanism Notation III (language model view)
Attention Mechanism Rationale of the Attention Mechanism
Attention Mechanism Rationale of the Attention Mechanism
Attention Mechanism Rationale of the Attention Mechanism
Attention Mechanism Rationale of the Attention Mechanism
Attention Mechanism Rationale of the Attention Mechanism
Attention Mechanism Query the Encoder to Tweak the Decoder Output
Remarks (observations) :
Attention Mechanism RNN with Simple Attention
Attention Mechanism (continued)
Attention Mechanism (continued)
Attention Mechanism (continued)
Attention Mechanism (continued)
Attention Mechanism
Attention Mechanism
Attention Mechanism
Attention Mechanism RNN with Parameterized Attention
Remarks (attention calculus) :
Remarks (parameterized attention) :
Kapitel ML:IX IX. Ensemble and Meta
Ensemble Methods Motivation (1): Generalisierungsfähigkeit
Ensemble Methods Motivation (2): No Free Lunch Theorem
Ensemble Methods Motivation (3): Instabilität von Lernverfahren
Ensemble Methods Motivation (4): Consequences
Ensemble Methods Bootstrap Aggregating (Bagging)
Remarks:
Ensemble Methods Bootstrap Aggregating
Ensemble Methods Bootstrap Aggregating
Ensemble Methods Bootstrap Aggregating
Ensemble Methods Boosting Weak Classifiers
Ensemble Methods Adaptive Boosting (AdaBoost)
Ensemble Methods Adaptive Boosting
Ensemble Methods Adaptive Boosting
Remarks:
Ensemble Methods Discrete AdaBoost
Ensemble Methods Cascades of Classifiers (Cascading)
Ensemble Methods Cascading
Ensemble Methods Maßzahlen für Cascading
Ensemble Methods Maßzahlen für Cascading
Ensemble Methods Cascading
Ensemble Methods Cascading for Face Detection (Main Loop)
Ensemble Methods Cascading for Face Detection (Gesamtklassifikator)
Ensemble Methods Cascading for Face Detection(Einzelklassifikator in Kaskade)
Ensemble Methods Cascading for Face Detection (Einzelklassifikator in Ensemble)
Ensemble Methods Cascading for Face Detection (alle Features)
Ensemble Methods Cascading for Face Detection (ausgewählte Features)
Ensemble Methods Cascading for Face Detection (Effizienz)
Ensemble Methods Cascading for Face Detection (Effizienz beim Training)
Ensemble Methods Trainingsmengen
Ensemble Methods Grundlage der Entscheidung
Ensemble Methods Literature