Linear Discriminant Analysis (LDA): A Supervised Dimensionality Reduction Technique, Slides of Machine Learning

Download Linear Discriminant Analysis (LDA): A Supervised Dimensionality Reduction Technique and more Slides Machine Learning in PDF only on Docsity!

Linear Discriminant Analysis (LDA)

~ p.Sony

• (^) Linear Discriminant Analysis (LDA) is the most commonly used dimensionality reduction technique in supervised learning. Basically, it is a preprocessing step for pattern classification and machine learning applications. LDA is a powerful algorithm that can be used to determine the best separation between two or more classes.
• (^) LDA is a supervised learning algorithm, which means that it requires a labelled training set of data points in order to learn the linear discriminant function.
• (^) The main purpose of LDA is to find the line or plane that best separates data points belonging to different classes. The key idea behind LDA is that the decision boundary should be chosen.

• (^) Here Linear Discriminant analysis uses both the axes x and y to create a new axis and projects data onto a new axis in a way to maximize the separation of two categories and hence reducing the 2D graph into 1D graph

• LDA algorithm works based on the following steps: a) The first step is to calculate the means and standard deviation of each feature. b) Within class scatter matrix and between class scatter matrix is calculated c) These matrices are then used to calculate the eigenvectors and eigenvalues. d) LDA chooses the k eigenvectors with the largest eigenvalues to form a transformation matrix. e) LDA uses this transformation matrix to transform the data into a new space with k dimensions. f) Once the transformation matrix transforms the data into new space with k dimensions, LDA can then be used for classification or dimensionality -reduction