In order to facilitate the search, I wrote this blog. I’ve collected all algorithms that I learned or want to learn in Machine Learning, Deep Learning, Mathematics and Data Structure and Algorithms. I hope I can improve my skills and knowledge in these area with writing the interpretation about these algorithms.

1. Theoretical Machine Learning

1.0 Model Metrics

• Training vs Evaluation
  • Training Metrics
    • Global Training Metrics
    • Local Training Metrics
  • Evaluation Metrics
• Problem Types
  • Classification Metrics
  • Regression Metrics
  • Clustering Metrics

1.1 Supervised Learning

• Classification

  • Binary Classificaxtion
    • Logistic Regression
    • Perceptron
    • Support Vector Machines
    • Factorization Machines (FM)
    • Field-aware Factorization Machines (FFM)
  • Multi-Class Classificatin
    • Decision Tree
      • Decision Stump
      • Iterative Dichotomiser 3 (ID3)
      • C4.5 and C5.0
      • Classification and Regression Tree (CART)
      • Random Forest = Bagging + Decision Tree
      • Adaboost
      • Boosting Tree = Adaboost + Decision Tree
      • Gradient Boosting Machine (GBM)
      • Gradient Boosting Decision Tree (GBDT) = Gradient Boosting + Decision Tree
      • XGBoost
      • LightGBM
      • Chi-squared Automatic Interaction Detection (CHAID)
      • M5
      • Conditional Decision Trees
    • Neural Network
• Regression
  • Linear Regression
    • Locally Weighted Linear Regression
    • Bayesian Linear Regression
    • Polynomial Regression
    • Ridge, Lasso Regression and ElasticNet
• Instance Based
  • K-Nearest Neighbor (KNN)
  • Learning Vector Quantization (LVQ)
  • Self-Organizing Map (SOM)
  • Locally Weighted Learning (LWL)
• Probabilistic Graphical Models
  • Bayesian Network (BN)
    • Naive Bayes
    • Gaussian Naive Bayes
    • Multinomial Naive Bayes
    • Bayesian Belief Network (BBN)
    • Expectation Maximization (EM)
  • Markov Network
    • Averaged One-Dependence Estimators (AODE)
    • Hidden Markov Models
    • Conditional Random Fields (CRFs)

1.2 Unsupervised Learning

1.2.1 Clustering

• Centroid-based Clustering
  • K-Means
  • Learning Vector Quantization (LVQ)
  • Gaussian Mixed Model (GMM)
• Density-based Clustering
  • DBSCAN
• Connectivity-based clustering
  • Hierachical Clustering
  • Single-linkage Clustering
  • Complete-linkage Clustering
  • Average-linkage Clustering
  • Divisive Clustering
• Expectation Maximization (EM)
• Self-Organizing Map (SOM)
• K-Medians
• Latent Dirichlet Allocation (LDA)
• Fuzzy Clustering
• OPTICS algorithm
• Non-Negative Matrix Factorization
• Hierarchical Agglomerative Clustering (HAC)

1.2.2 Dimension Reduction / Distributed Representation

• Feature Selection
• Principal Component Analysis (PCA)
• Linear Discriminant Analysis (LDA)
• Independent Component Analysis (ICA)
• Manifold Learning
  • Isometric Mapping (Isomap)
  • Locally Linear Embedding (LLE)
• Locality Preserving Projection (LPP)
• Laplacian Eigenmaps

1.2.3 Generation

1.3 Ensemble Learning

• Ensemble Learning
• Logit Boost (Boosting)
• Bootstrapped Aggregation (Bagging)
• AdaBoost
• Stacked Generalization (blending)
• Gradient Boosting Machines (GBM)
• Random Forest

1.4 Models

• Model Evaluation
  • Bias and Variance
  • Loss Functions & Metrics
  • Overfitting and Underfitting
• Model Selection
  • Model Selection

1.5 Information Theory

• Information Entropy

2. Applied Machine Learning

• Workflow of Machine Learning
• Feature Engineering
  • Feature Extraction
    • Data Preprocessing
    • Exploratory Data Analysis (EDA)
  • Feature Selection
    • Feature Selection Methods
• Model Ensemble
  • Model Ensemble Methods
• Model Optimization
  • Model Optimization Methods
  • Result Analysis
  • Bad Case Analysis
• Applied Models
  • Click Models
  • Ranking Algorithms
• Coding
  • Exploratory Data Analysis (EDA)
  • Feature Engineering
  • Feature Selection
  • Model Training

3. Deep Learning

• Deep Neural Network
  • Regularizaton
    • Normalization
      • Batch Nomalization
  • Optimizers
• Computer Vision
  • Image Classification
    • Convolutional Neural Networks (CNN)
    • Very Deep Convolutional Networks (VGG)
    • Triplet Loss
  • Object Location
    • Non-Max Suppression
  • Object Detection
    • General Operations
      • Bounding Box Regression
      • Smooth L1 Loss
      • Attention
    • Two Stages
      • RCNN
      • Fast RCNN
      • Faster RCNN
    • One Stage
      • Single Shot MultiBox Detector (SSD)
      • YOLO
        • YOLOv1
        • YOLOv2
        • YOLOv3
  • Sequential Model
    • Recurrent Neural Networks（RNN）
    • Long Short Term Memory networks（LSTM）
• Speech Recognition
• Natural Language Processing

4. Mathematics

• Information Theory
  • Entropy
• Optimization
  • Gradient Descent
  • Monte Carlo Method
  • Singular Value Decomposition
  • Matrix Factorization
  • Newton Method
• Statistics
• Normal Distribution

References

1. Sklearn User Guide