Download Supervised, Unsupervised, and Semi-Supervised Learning: Techniques and Applications and more Study notes Computer Science in PDF only on Docsity!
Machine Learning
Machine Learning Basics: Notes and Interview Questions
(Part 1).
About: - This document provides easy-to-understand notes and questions about Machine Learning. It is designed to help students learn the basics and prepare for interviews. You'll find explanations of key topics like supervised and unsupervised learning, how to evaluate models, feature engineering, and popular algorithms. Each section has questions to test your understanding and get you ready for real interview situations. This is Part 1, and it will help you build a strong foundation in Machine Learning.
Q1. Define Artificial Intelligence (AI).
Artificial Intelligence (AI) is a branch of computer science that focuses on creating systems capable of performing tasks that typically require human intelligence. These tasks include problem-solving, understanding natural language, recognizing patterns, learning from experience, and making decisions.
Q2. Explain the differences between Artificial Intelligence (AI), Machine Learning (ML), Deep
Learning (DL), Data Science (DS).
Concept Definition Key Features Examples
Artificial Intelligence (AI) AI is the broad field of creating intelligent systems capable of performing tasks that typically require human intelligence. Includes reasoning, problem-solving, understanding language, perception, and learning. Chatbots, Autonomous Vehicles, Recommendation Systems Machine Learning (ML) ML is a subset of AI that involves training algorithms on data to learn patterns and make decisions without explicit programming. Focuses on predictive accuracy, uses statistical methods, can handle large datasets. Spam Detection, Image Recognition, Predictive Analytics Deep Learning (DL) DL is a subset of ML that uses neural networks with many layers to analyse various factors of data. Excels with large amounts of data, automatic feature extraction, uses layers of neural networks. Speech Recognition, Image Classification, Natural Language Processing Data Science (DS) DS is an interdisciplinary field focused on extracting knowledge and insights from structured and unstructured data. Uses statistical, mathematical, and computational techniques, involves data cleaning, analysis, and visualization. Market Analysis, Fraud Detection, Personalized Recommendations
Q3. How does AI differ from traditional software development?
The process involves feeding the algorithm input-output pairs so that it can learn patterns and relationships in the data. The model's performance is evaluated by how accurately it can predict the output when given new inputs.
Q7. Provides examples of supervised learning algorithm?
1. Email Spam Detection:
Input: Emails with features like subject line, content, sender's address. Output: Labels such as "spam" or "not spam." Application: The model learns to classify incoming emails as spam or not based on patterns learned from previously labelled emails
- House Price Prediction: Input: Features such as the size of the house, number of bedrooms, location. Output: Predicted price of the house. Application: Real estate models use historical data of house sales to predict the selling price of new listings.
- Customer Churn Prediction: Input: Customer data like usage frequency, customer support interactions, and subscription history. Output: Labels like "churn" (customer leaves) or "retain" (customer stays). Application: Companies use this to identify customers at risk of leaving and take proactive measures to retain them.
- Image Classification: Input: Images with features such as pixel values. Output: Labels like "cat," "dog," or "car." Application: Models trained on labelled image datasets can classify new images into different categories.
Q8. Explain the process of supervised learning?
Supervised learning involves several key steps, from data collection to model deployment. Here's a breakdown of the process:
- Data Collection: o The first step is gathering a labelled dataset, which consists of input data (features) and corresponding output labels. For example, if you're building a model to predict house prices, your dataset might include features like the number of bedrooms, location, and square footage, along with the actual prices of the houses.
- Data Preprocessing: o The collected data often needs to be cleaned and transformed before it can be used to train the model. This involves: ▪ Handling Missing Data: Filling in or removing missing values in the dataset. ▪ Normalization/Standardization: Scaling features to ensure they contribute equally to the model. ▪ Categorical Encoding: Converting categorical data into numerical format using techniques like one-hot encoding or label encoding. ▪ Splitting Data: Dividing the dataset into training and test sets (and sometimes a validation set) to evaluate the model’s performance on unseen data.
- Choosing a Model: o Depending on the nature of the problem (classification or regression), you select an appropriate machine learning algorithm. Common algorithms include:
▪ Linear Regression for predicting continuous values. ▪ Logistic Regression for binary classification. ▪ Decision Trees and Random Forests for more complex decision-making processes. ▪ Support Vector Machines (SVM) for classification tasks. ▪ Neural Networks for handling large and complex datasets.
- Training the Model: o The model is trained on the training dataset. During this process: ▪ The algorithm learns the relationship between the input features and the output labels. ▪ The model makes predictions on the training data, and the predictions are compared to the actual labels. ▪ The model adjusts its parameters to minimize the error, typically using techniques like gradient descent.
- Evaluation: o After training, the model is evaluated using the test dataset, which the model has not seen before. This step is crucial to assess how well the model generalizes to new data. ▪ Metrics: The model’s performance is measured using metrics like accuracy, precision, recall, F1-score for classification tasks, and mean squared error (MSE) or R-squared for regression tasks. ▪ Confusion Matrix: For classification tasks, a confusion matrix helps visualize the performance by showing the true positive, true negative, false positive, and false negative rates.
- Hyperparameter Tuning: o To improve the model’s performance, you may need to adjust the hyperparameters, which are the settings that control the learning process (e.g., learning rate, number of trees in a random forest, etc.). ▪ This can be done using techniques like grid search, random search, or more advanced methods like Bayesian optimization.
- Model Validation: o If a validation set was used, the model’s performance on this set can help fine-tune the model further. Cross-validation is another technique used, where the training set is split into several subsets, and the model is trained and validated on different combinations of these subsets.
- Deployment: o Once the model is sufficiently trained and evaluated, it is deployed into a production environment where it can make predictions on new, real-world data. This might involve integrating the model into an application, API, or system where it can operate at scale.
- Monitoring and Maintenance: o After deployment, the model's performance should be continuously monitored. If the model's accuracy degrades over time (due to changes in the data or environment), it may need to be retrained or updated. This process is known as model maintenance or model lifecycle management.
- Retraining:
Q11. Describe Semi-Supervised Learning and Its Significance?
Semi-supervised learning is a type of machine learning that combines a small amount of labelled data with a large amount of unlabelled data during training. It serves as a middle ground between supervised and unsupervised learning. Significance:
- Improved Accuracy: o By leveraging the vast amounts of unlabelled data alongside labelled data, semi-supervised learning can improve model accuracy compared to using only labelled data.
- Cost-Effective: o Labelling data can be expensive and time-consuming. Semi-supervised learning allows the use of large unlabelled datasets, reducing the reliance on labelled data and thereby lowering costs.
- Handling Limited Data: o In scenarios where labelled data is scarce, semi-supervised learning can still build effective models by utilizing available unlabelled data, making it particularly useful in fields like medical imaging and natural language processing.
- Scalability: o Semi-supervised learning scales well to large datasets, as it doesn’t require extensive manual labelling efforts, making it suitable for big data applications.
Q12. Explain Reinforcement Learning and Its Applications.
Reinforcement learning (RL) is a type of machine learning where an agent learns to make decisions by taking actions in an environment to maximize a cumulative reward. The agent receives feedback in the form of rewards or penalties based on the actions it takes, and it learns to adjust its strategy to achieve the best possible outcome. Key Concepts:
- Agent: The learner or decision-maker.
- Environment: The external system that the agent interacts with.
- Actions: The choices the agent can make.
- Rewards: Feedback from the environment used to guide the learning process.
- Policy: The strategy the agent uses to determine its actions. Applications:
- Gaming: o RL has been successfully applied in games like chess, Go, and video games, where agents learn strategies to beat human or AI opponents.
- Robotics: o In robotics, RL is used to teach robots how to perform tasks such as walking, grasping objects, or navigating through environments.
- Autonomous Vehicles:
o RL is applied in self-driving cars for tasks such as path planning, obstacle avoidance, and decision- making in dynamic environments.
- Finance: o RL algorithms are used in trading strategies, portfolio management, and market-making to optimize returns based on historical data and market conditions.
- Healthcare: o RL is being explored for personalized medicine, where treatment plans are optimized based on patient responses, and for managing chronic diseases.
Q13. How Does Reinforcement Learning Differ from Supervised Learning and Unsupervised
Learning?
Reinforcement Learning vs. Supervised Learning:
- Learning Paradigm: o In supervised learning, the model is trained on labelled data where the correct output is known. The goal is to learn a mapping from inputs to outputs. o In reinforcement learning, the agent learns by interacting with an environment and receiving rewards or penalties based on its actions. There is no labelled data; instead, the agent learns through trial and error.
- Feedback: o Supervised learning provides direct feedback in the form of correct labels, allowing the model to learn from mistakes immediately. o Reinforcement learning provides feedback in the form of rewards, which may be delayed, making it more challenging to learn the correct actions.
- Objective: o The objective of supervised learning is to minimize the error between the predicted and actual outputs. o The objective of reinforcement learning is to maximize the cumulative reward over time. Reinforcement Learning vs. Unsupervised Learning:
- Data Labels: o Unsupervised learning does not use labelled data and instead seeks to find patterns or structure in the input data. o Reinforcement learning also does not rely on labelled data but focuses on learning a policy to maximize rewards based on interactions with the environment.
- Goal: o The goal of unsupervised learning is to uncover hidden patterns or groupings in the data. o The goal of reinforcement learning is to learn an optimal strategy for decision-making in a given environment to achieve maximum rewards.
- Application Context: o Unsupervised learning is often used for clustering, association, and dimensionality reduction.
▪ 10% is used for validation. ▪ 10% is used for testing.
- Dataset Size: o For large datasets, a smaller percentage can be allocated to testing and validation, as even 10% of a large dataset can provide a robust evaluation. o For smaller datasets, a higher percentage might be needed for testing and validation to ensure the results are statistically significant.
- Model Complexity: o Complex models (e.g., deep neural networks) may require more validation data to properly tune the hyperparameters and avoid overfitting. o Simpler models might perform well with a smaller validation set.
- Cross-Validation: o In cases where data is limited, techniques like k-fold cross-validation can be used, where the dataset is split into k subsets. The model is trained and validated k times, each time using a different subset as the validation set and the remaining as the training set.
Q17. What Are the Consequences of Improper Train-Test-Validation Split?
Improper splitting of data can lead to several issues:
- Overfitting: o If the validation set is too small or not representative, the model might overfit to the training data, leading to poor generalization to new data.
- Underfitting: o If the training set is too small, the model might not learn enough about the data, resulting in underfitting, where the model performs poorly on both training and test data.
- Bias in Evaluation: o An improperly split test set can lead to biased or overly optimistic performance estimates, giving a false sense of the model's ability to generalize.
- Poor Model Selection: o If the validation set does not accurately reflect the test set, hyperparameter tuning might lead to a suboptimal model that performs poorly in real-world applications.
- Data Leakage: o If data from the test set leaks into the training or validation sets, the model might inadvertently learn from the test data, leading to misleadingly high-performance metrics during testing.
Q18. Discuss the Trade-Off in Selecting an Appropriate Split Ratio.
Choosing the right split ratio involves balancing several trade-offs:
- Training Set Size vs. Model Learning: o A larger training set helps the model learn better patterns, but it reduces the size of the validation and test sets, potentially leading to less reliable performance evaluation.
- Validation Set Size vs. Model Tuning: o A larger validation set improves the reliability of hyperparameter tuning but leaves less data for training, which could affect the model's ability to learn.
- Test Set Size vs. Evaluation Confidence: o A larger test set provides more confidence in the model's performance evaluation but reduces the data available for training and validation, possibly affecting model development.
- Model Complexity vs. Split Ratio: o More complex models might require more data for training, validation, and testing to avoid overfitting and underfitting, necessitating a careful balance in the split ratio.
Q19. Define Model Performance in Machine Learning?
Model performance in machine learning refers to how well a model makes predictions or classifications based on new, unseen data. It is an indication of the model's ability to generalize from the training data to real-world scenarios.
Q20. How Do You Measure the Performance of a Machine Learning Model?
The performance of a machine learning model can be measured using various metrics, depending on the type of problem (classification, regression, etc.):
- For Classification Problems: o Accuracy: The percentage of correct predictions made by the model out of all predictions. o Precision: The proportion of true positive predictions to the total predicted positives. o Recall (Sensitivity): The proportion of true positive predictions to the total actual positives. o F1 Score: The harmonic mean of precision and recall, useful when you need to balance both metrics. o Confusion Matrix: A table that shows the true positives, true negatives, false positives, and false negatives, providing a detailed breakdown of model performance. o AUC-ROC Curve: A plot of the true positive rate (recall) against the false positive rate, used to evaluate the performance of a binary classifier.
- For Regression Problems: o Mean Squared Error (MSE): The average of the squared differences between the predicted and actual values. o Root Mean Squared Error (RMSE): The square root of the mean squared error, providing an error metric in the same units as the output variable. o Mean Absolute Error (MAE): The average of the absolute differences between the predicted and actual values. o R-squared (R²): A statistical measure that represents the proportion of the variance in the dependent variable that is predictable from the independent variables.
- For Clustering Problems (Unsupervised Learning): o Silhouette Score: Measures how similar an object is to its own cluster compared to other clusters, providing a measure of how well the data has been clustered. o Davies-Bouldin Index: Evaluates the average similarity ratio of each cluster with the cluster that is most similar to it. o Adjusted Rand Index (ARI): Measures the similarity between the clustering result and the ground truth labels, adjusted for chance.
- For Reinforcement Learning: o Cumulative Reward: The total reward accumulated by the agent over time, indicating how well the agent is performing in the environment. o Policy Performance: Evaluates how effective the policy is in making decisions that maximize long-term rewards.
Q21. What is Overfitting and Why is it Problematic?
Overfitting occurs when a machine learning model learns the training data too well, including its noise and outliers. As a result, the model performs very well on the training data but poorly on unseen data. This happens because the model becomes too complex, capturing specific details that do not generalize to new data. Why it is Problematic:
- Poor Generalization:
- Increase Model Complexity: o Use a more complex model that can capture the underlying patterns in the data. For example, switch from a linear model to a non-linear model, or add more layers and neurons in a neural network.
- Feature Engineering: o Create or select more relevant features that can help the model better understand the data.
- Reduce Regularization: o If using regularization, consider lowering the regularization parameter to allow the model to fit the data more closely.
- Increase Training Time: o Ensure that the model is trained for an adequate number of epochs or iterations to fully capture the patterns in the data.
- Use More Data: o Increase the amount of training data to provide the model with more examples, which can help it learn better.
Q25. Discuss the Balance Between Bias and Variance in Model Performance.
Bias-Variance Trade-off is a key concept in machine learning that describes the balance between two sources of error:
- Bias: o Bias refers to the error introduced by approximating a real-world problem, which may be complex, by a simplified model. High bias models are often too simple and may underfit the data, leading to systematic errors.
- Variance: o Variance refers to the error introduced by the model's sensitivity to small fluctuations in the training data. High variance models are often too complex and may overfit the data, capturing noise and leading to errors on new data. Trade-off:
- High Bias, Low Variance: A model with high bias is simple, leading to underfitting. It might perform poorly on both training and test data but is consistent in its predictions.
- Low Bias, High Variance: A model with low bias is complex, leading to overfitting. It performs well on training data but poorly on test data due to its sensitivity to noise.
Q26. What Are the Common Techniques to Handle Missing Data?
Handling missing data is crucial in maintaining the integrity and accuracy of a machine learning model. Common techniques include:
- Imputation: o Mean/Median/Mode Imputation: Replace missing values with the mean, median, or mode of the non- missing data. o K-Nearest Neighbours (KNN) Imputation: Use the values of the nearest neighbours to impute missing data based on feature similarity. o Regression Imputation: Predict missing values using a regression model trained on the non-missing data.
- Deletion: o Listwise Deletion: Remove any rows with missing data. o Pairwise Deletion: Use all available data by excluding only the missing values when performing specific analyses.
- Interpolation: o Estimate missing values by interpolating between the known values in a time series or other continuous data.
- Using Algorithms That Handle Missing Data:
o Some machine learning algorithms, like decision trees and certain ensemble methods, can handle missing data directly.
- Filling with a Specific Value: o Replace missing values with a specific constant, like 0, or a domain-specific value.
Q27. Explain the Implications of Ignoring Missing Data.
Ignoring missing data can lead to several issues:
- Bias in Results: o Ignoring missing data can introduce bias, as the remaining data may not be representative of the entire population, leading to incorrect conclusions.
- Reduced Data Size: o Removing rows with missing data (listwise deletion) reduces the dataset size, which can weaken the statistical power and the model's ability to learn.
- Inaccurate Model Predictions: o If the missing data is not handled properly, the model may produce inaccurate predictions, as it may fail to learn the correct patterns from incomplete data.
- Loss of Information: o Missing data may contain valuable information. Ignoring it outright can result in a loss of important insights.
Q28. Discuss the Pros and Cons of Imputation Methods.
Pros of Imputation:
- Preserves Data Size: o Imputation maintains the size of the dataset, which is especially important when data is scarce.
- Reduces Bias: o Imputation can help reduce bias compared to simply deleting rows with missing data, especially if the missing data is random.
- Improves Model Accuracy: o By filling in missing values, imputation can improve model accuracy by providing more complete data for learning. Cons of Imputation:
- Risk of Introducing Bias: o Poorly chosen imputation methods can introduce bias, especially if the imputed values do not accurately represent the missing data.
- Complexity: o Some imputation methods, like KNN or regression, are computationally intensive and may add complexity to the data preprocessing pipeline.
- Imputation Error: o Imputed values are estimates, not actual observations, which can lead to errors in the model's predictions, particularly if a large proportion of the data is missing.
Q29. How Does Missing Data Affect Model Performance?
Missing data can significantly impact model performance:
- Reduced Accuracy: o Missing data can lead to incomplete learning, resulting in reduced accuracy and poor generalization to new data.
- Increased Variance:
o Down-sampling (Under-sampling): Reduce the number of instances in the majority class to balance the class distribution.
- Synthetic Data Generation: o SMOTE (Synthetic Minority Over-sampling Technique): Generate synthetic samples for the minority class to balance the dataset.
- Class Weighting: o Modify the cost function of the algorithm to penalize misclassifications of the minority class more heavily, effectively giving it more importance during training.
- Ensemble Methods: o Use ensemble techniques like Balanced Random Forest or EasyEnsemble that are specifically designed to handle imbalanced data by combining multiple models.
- Anomaly Detection Algorithms: o Treat the minority class as an anomaly and use anomaly detection techniques to identify it.
- Adjusting Decision Thresholds: o Modify the decision threshold of the classifier to favour the minority class, improving its recall.
Q33. Explain the Process of Up-sampling and Down-sampling.
Up-sampling (Over-sampling):
- Definition: Up-sampling involves increasing the number of instances in the minority class to balance the dataset.
- Process:
- Random Over-sampling: Duplicate existing instances from the minority class randomly.
- SMOTE: Generate synthetic samples for the minority class by interpolating between existing minority class instances. Down-sampling (Under-sampling):
- Definition: Down-sampling involves reducing the number of instances in the majority class to balance the dataset.
- Process:
- Random Under-sampling: Randomly remove instances from the majority class.
- Cluster-based Under-sampling: Identify and remove redundant majority class instances based on clustering or other criteria.
Q34. When Would You Use Up-sampling Versus Down-sampling?
Use Up-sampling When:
- The minority class is extremely underrepresented, and removing majority class instances would result in a loss of valuable information.
- You have enough computational resources to handle the increased size of the dataset after up-sampling. Use Down-sampling When:
- The majority class dominates the dataset, and reducing its size would not result in significant loss of information.
- You want to reduce the computational cost by working with a smaller dataset.
- The dataset is large, and reducing its size helps in quicker model training.
Q35. What is SMOTE and How Does It Work?
SMOTE (Synthetic Minority Over-sampling Technique) is a popular technique for addressing class imbalance by generating synthetic samples for the minority class.
How It Works:
- Identify Neighbours: o For each instance in the minority class, SMOTE identifies its k-nearest neighbours from the same class.
- Generate Synthetic Samples: o SMOTE generates new synthetic samples by selecting a random point along the line connecting the original instance and one of its neighbours.
- Balance the Dataset: o The synthetic samples are added to the dataset, balancing the class distribution.
Q36. Explain the Role of SMOTE in Handling Imbalanced Datasets.
SMOTE plays a crucial role in handling imbalanced datasets by:
- Enhancing Minority Class Representation: o SMOTE increases the number of minority class instances, helping the model learn its patterns more effectively.
- Improving Model Performance: o By balancing the class distribution, SMOTE can lead to better model performance, particularly in terms of recall and F1-score for the minority class.
- Reducing Overfitting: o Unlike random over-sampling, SMOTE reduces the risk of overfitting by generating new, synthetic instances rather than simply duplicating existing ones.
Q 3 7. Discuss the Advantages and Limitations of SMOTE.
Advantages of SMOTE:
- Improves Class Balance: o SMOTE effectively balances the dataset, improving the model's ability to learn the minority class.
- Reduces Overfitting: o By generating synthetic samples rather than duplicating existing ones, SMOTE reduces the risk of overfitting to specific instances.
- Versatility: o SMOTE can be combined with other techniques, like down-sampling, to create more balanced and representative datasets. Limitations of SMOTE:
- Noise Amplification: o SMOTE can inadvertently amplify noise if the synthetic samples are generated from noisy minority class instances.
- Boundary Samples: o SMOTE may generate synthetic samples that fall near the class boundary, potentially leading to misclassification.
- Computational Cost: o SMOTE can increase the size of the dataset, leading to higher computational costs, especially with large datasets.
Q 3 8. Provide Examples of Scenarios Where SMOTE is Beneficial.
SMOTE is beneficial in scenarios such as:
- Fraud Detection:
However, implications include:
- Introduction of Bias: If the interpolation method is not appropriate for the data distribution, it can introduce bias, leading to incorrect model predictions.
- Overfitting: Models might overfit to the interpolated values, which do not represent real-world scenarios, reducing generalization.
- Distortion of Relationships: Interpolation may distort the true relationships between variables, especially in time-series data.
Q42. What are outliers in datasets?
Outliers are data points that significantly differ from the majority of data in a dataset. They can be unusually high or low values and can result from variability in the data, errors in data collection, or genuine anomalies.
Q43. Explain the impacts of outliers on machine learning models?
Outliers can have several impacts on machine learning models:
- Skewed Model Predictions: Outliers can disproportionately influence the model, leading to skewed predictions.
- Distorted Performance Metrics: Outliers can affect metrics like mean and variance, leading to misleading conclusions about model performance.
- Reduced Model Accuracy: In algorithms sensitive to the distance between data points (e.g., k-NN, linear regression), outliers can reduce model accuracy.
- Overfitting: Outliers might cause models to overfit to the noise in the data rather than the underlying pattern.
Q44. Discuss the techniques to identify outliers?
Common techniques to identify outliers include:
- Visual Methods: o Box Plots: Highlight outliers as data points outside the interquartile range (IQR). o Scatter Plots: Visualize the relationship between variables and identify points that deviate significantly.
- Statistical Methods: o Z-Score: Measures how many standard deviations a data point is from the mean. A z-score above 3 or below - 3 is often considered an outlier. o IQR Method: Identifies outliers as data points below Q1 - 1.5 IQR or above Q3 + 1.5 IQR.
- Machine Learning Methods: o Isolation Forest: Anomaly detection method that isolates observations by randomly selecting a feature and then randomly selecting a split value between the maximum and minimum values. o DBSCAN (Density-Based Spatial Clustering of Applications with Noise): Clusters data based on density, labelling low-density points as outliers.
Q4 5. How can outliers be handled in a dataset?
Handling outliers can be done in several ways:
- Removal: If outliers are due to data errors, they can be removed from the dataset.
- Transformation: Applying transformations like logarithms can reduce the impact of outliers.
- Capping: Setting a maximum and minimum threshold to limit the influence of extreme values.
- Imputation: Replacing outliers with mean, median, or other estimates.
- Robust Algorithms: Using algorithms that are less sensitive to outliers, such as robust regression techniques.
Q46. Compare and contrast Filter, Wrapper, and Embedded methods for feature selection?
Filter Methods:
- Description: Feature selection is done independently of the learning algorithm, based on statistical measures (e.g., correlation, chi-square).
- Advantages: Fast, scalable, and computationally efficient.
- Disadvantages: May select redundant features as they do not consider feature interactions. Wrapper Methods:
- Description: Feature subsets are evaluated by training models and selecting the subset that produces the best performance (e.g., forward selection, backward elimination).
- Advantages: Consider interactions between features and usually provide better performance.
- Disadvantages: Computationally expensive, especially with large datasets. Embedded Methods:
- Description: Feature selection occurs during the model training process (e.g., Lasso, decision trees).
- Advantages: Integrates feature selection into the model training process, often leading to better results.
- Disadvantages: Dependent on the learning algorithm, which might limit flexibility.
Q47. Provide examples of algorithms associated with these methods: Filter, Embedded,
Wrapper.
Filter Methods:
- Examples: Chi-square test, ANOVA, correlation coefficient, mutual information. Wrapper Methods:
- Examples: Recursive Feature Elimination (RFE), Genetic Algorithms, Sequential Feature Selection. Embedded Methods:
- Examples: Lasso Regression (L1 regularization), Ridge Regression (L2 regularization), Decision Trees (e.g., feature importance in Random Forest).
Q48. Discuss the advantages and disadvantages of these methods for feature selection.
Filter Methods:
- Advantages: Simple, fast, and not computationally intensive. They are independent of the learning algorithm.
- Disadvantages: Might ignore interactions between features and can select redundant features. Wrapper Methods:
- Advantages: Consider interactions between features and often provide higher accuracy.
- Disadvantages: Computationally expensive and can be prone to overfitting, especially with small datasets. Embedded Methods:
- Advantages: Feature selection is integrated into the model, often leading to a good balance between accuracy and computational efficiency.
