In machine learning, pattern recognition and in image processing, feature extraction starts from an initial set of measured data and builds derived values (features) intended to be informative and non-redundant, facilitating the subsequent learning and generalization steps, and in some cases leading to better human interpretations. Feature extraction is related to dimensionality reduction. (Wiki)

• Overview

  • A survey of dimensionality reduction techniques C.O.S.Sorzano, J.Vargas, A.Pascual‐Montano
  • Feature Selection and Feature Extraction in Pattern Analysis: A Literature Review (2019) Benyamin Ghojogh, Maria N. Samad, Sayema Asif Mashhadi,Tania Kapoor, Wahab Ali, Fakhri Karray, Mark Crowley

• PCA Principal Component Analysis (Wiki)

  • On lines and planes of closest fit to systems of points in space (1901) Karl Pearson
  • Supervised PCA: Prediction by Supervised Principal Components (2006) Eric Bair, Trevor Hastie, Debashis Paul, Robert Tibshirani
  • Sparse PCA (sklearn)

• DPCA Dual Principal Component Analysis

• KPCA Kernel Principal Component Analysis (sklearn, Wiki)

  • Nonlinear Component Analysis as a Kernel Eigenvalue Problem (1998) Bernhard Scholkopf, Alexander Smola, Klaus-Robert Muller
  • Kernel PCA for Novelty Detection (2006) Heiko Hoffmann
  • Robust Kernel Principal Component Analysis Minh Hoai Nguyen, Fernando De la Torre

• IPCA Incremental (online) PCA (CRAN, sklearn)

• ICA Independent Component Analysis (Wiki)

  • Independent Component Analysis: Algorithms and Applications (2000) Aapo Hyvärinen, Erkki Oja
  • Independent Component Analysis (2001) - Free ebook Aapo Hyvarinen, Juha Karhunen, Erkki Oja
  • FastICA (sklearn)

• FLDA Fisher's Linear Discriminant Analysis (Supervised) (Wiki)

  Similar to PCA, FLDA calculates the projection of data along a direction; however, rather than maximizing the variation of data, FLDA utilizes label information to get a projection maximizing the ratio of between-class variance to within-class variance. (Source)

  • The Use of Multiple Measurements in Taxonomic Problems (1936) R. A. Fisher
  • The Utilization of Multiple Measurements in Problems of Biological Classification (1948) - require registration C. Radhakrishna Rao
  • PCA versus LDA (2001) Aleix M. Martinez, Avinash C. Kak
  • Package: MASS includes lda (CRAN)
  • Package: sda (CRAN)

• KFLDA Kernel Fisher Linear Discriminant Analysis

• MDS Multidimensional Scaling (Wiki)

  • Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis (1964) J. B. Kruskal
  • An Analysis of Classical Multidimensional Scaling (2019) Anna Little, Yuying Xie, Qiang Sun
  • Packages: sklearn

• Isomap (Homepage, Wiki)

  • A Global Geometric Framework for Nonlinear Dimensionality Reduction (2000) Joshua B. Tenenbaum, Vin de Silva, John C. Langford
  • Packages: dimRed, sklearn

• Latent Dirichlet Allocation

  • Online Learning for Latent Dirichlet Allocation (2010) Matthew D. Hoffman, David M. Blei, Francis Bach

• Factor analysys (Wiki, sklearn)

  This technique is used to reduce a large number of variables into fewer numbers of factors. The values of observed data are expressed as functions of a number of possible causes in order to find which are the most important. The observations are assumed to be caused by a linear transformation of lower-dimensional latent factors and added Gaussian noise. (Source)

• t-SNE (Homepage, Wiki, CRAN, sklearn)

  • Visualizing Data using t-SNE (2008) Laurens van der Maaten, Geoffrey Hinton
  • Accelerating t-SNE using Tree-Based Algorithms (2014) Laurens van der Maaten
  • Tree-SNE - Hieararchical t-SNE (Code)
    • Tree-SNE: Hierarchical Clustering and Visualization Using t-SNE (2020) Isaac Robinson, Emma Pierce-Hoffman
  • Let-SNE
    • Let-SNE: A Hybrid Approach to Data Embedding and Visualization of Hyperspectral Imagery (2020) Megh Shukla, Biplab Banerjee, Krishna Mohan Buddhiraju

• LLE Locally Linear Embedding

  Constructs a k-nearest neighbor graph similar to Isomap. Then it tries to locally represent every data sample x i using a weighted summation of its k-nearest neighbors. (Source)

• HLLE Hessian Eigenmapping

  Projects data to a lower dimension while preserving the local neighborhood like LLE but uses the Hessian operator to better achieve this result and hence the name. (Source)

• Laplacian Eigenmap Spectral Embedding

• Maximum Variance Unfolding

• NMF Non-negative matrix factorization

• UMAP Uniform Manifold Approximation and Projection (Code, GPU version)

  • UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction (2018) Leland McInnes, John Healy, James Melville

• Trimap (Code, PyPI)

  • Trimap: Large-scale Dimensionality Reduction Using Triplets (2019) Ehsan Amid, Manfred K. Warmuth

• Autoencoders (Wiki)

• SOM Self-Organizing Maps or Kohonen Maps (Wiki)

  • Self-Organized Formation of Topologically Correct Feature Maps (1982) Teuvo Kohonen

• Sammon’s Mapping

• SDE Semi-definite embedding

• LargeVis

  • Visualizing Large-scale and High-dimensional Data (2016) Jian Tang, Jingzhou Liu, Ming Zhang, Qiaozhu Mei

• R
  • dimRed (CRAN)
  • dyndimred (CRAN)
  • intrinsicDimemsion (CRAN)
  • Rdimtools (Paper, CRAN)
• Python
  • scikit-learn
  • umap-learn (Homepage, PyPI)
• Javascript
  • tsne (NPM)
  • umap-js (NPM)
  • dimred (NPM)
• C++
  • tapkee (Code)
• Web
  • StatSim (Vis)