Docsity
Log inSign up
Docsity
Prepare for your exams
Prepare for your exams

Study with the several resources on Docsity

Earn points to download
Earn points to download

Earn points by helping other students or get them with a premium plan

Guidelines and tips
Guidelines and tips
Sell on Docsity
Sell on Docsity
Log inSign up

Prepare for your exams

Study with the several resources on Docsity

Find documents

Prepare for your exams with the study notes shared by other students like you on Docsity

Search Store documents

The best documents sold by students who completed their studies

Search through all study resources
Docsity AINEW

Summarize your documents, ask them questions, convert them into quizzes and concept maps

Explore questions

Clear up your doubts by reading the answers to questions asked by your fellow students

Earn points to download

Earn points by helping other students or get them with a premium plan

Share documents
download point icon20 Points

For each uploaded document

Answer questions
download point icon5 Points

For each given answer (max 1 per day)

All the ways to get free points
Get points immediately

Choose a premium plan with all the points you need

Study Opportunities

Choose your next study program

Get in touch with the best universities in the world. Search through thousands of universities and official partners

Community

Ask the community

Ask the community for help and clear up your study doubts

University Rankings

Discover the best universities in your country according to Docsity users

Free resources

Our save-the-student-ebooks!

Download our free guides on studying techniques, anxiety management strategies, and thesis advice from Docsity tutors

From our blog

Exams and Study
Go to the blog

Machine Learning Quiz: ROC, Precision-Recall Curves, & Evaluating Imbalanced Datasets, Slides of Data Analysis & Statistical Methods

Ohio Wesleyan University (OWU)Data Analysis & Statistical Methods

A series of questions testing the understanding of ROC curves, Precision-Recall curves, and evaluating machine learning models on imbalanced datasets. The questions cover topics such as calculating confidence intervals, identifying false statements, and comparing ROC curves.

Typology: Slides

2021/2022

Uploaded on 09/12/2022

lana23
lana23 🇺🇸

4.8

(4)

216 documents

1 / 12

Toggle sidebar

This page cannot be seen from the preview

Don't miss anything!

bg1
Q1-1: Consider following statements and choose the correct option
(True/False for all the statements A/B/C/D).
A. ROC curve summarize the trade-off between the true positive rate and the positive
predictive value for a model
B. Precision-Recall curve summarize the trade-off between the true positive rate and false
positive rate for a model
C. In both imbalanced and balanced datasets, the area under the curve (AUC) can be used as
a summary of the model performance.
D. If we decrease the false negative (select more positives), recall always increases, but
precision may increase or decrease.
1. A: True, B: False, C: True, D: False
2.A: False, B: False, C: True, D: True
3. A: True, B: True, C: True, D: True
4. A: False, B: True, C: False, D: True
pf3
pf4
pf5
pf8
pf9
pfa

Partial preview of the text

Download Machine Learning Quiz: ROC, Precision-Recall Curves, & Evaluating Imbalanced Datasets and more Slides Data Analysis & Statistical Methods in PDF only on Docsity!

Q1-1: Consider following statements and choose the correct option

(True/False for all the statements A/B/C/D).

A. ROC curve summarize the trade-off between the true positive rate and the positive predictive value for a model B. Precision-Recall curve summarize the trade-off between the true positive rate and false positive rate for a model C. In both imbalanced and balanced datasets, the area under the curve (AUC) can be used as a summary of the model performance. D. If we decrease the false negative (select more positives), recall always increases, but precision may increase or decrease.

  1. A: True, B: False, C: True, D: False

2.A: False, B: False, C: True, D: True

  1. A: True, B: True, C: True, D: True
  2. A: False, B: True, C: False, D: True

Q1-1: Consider following statements and choose the correct option

(True/False for all the statements A/B/C/D).

A. ROC curve summarize the trade-off between the true positive rate and the positive predictive value for a model B. Precision-Recall curve summarize the trade-off between the true positive rate and false positive rate for a model C. In both imbalanced and balanced datasets, the area under the curve (AUC) can be used as a summary of the model performance. D. If we decrease the false negative (select more positives), recall always increases, but precision may increase or decrease.

  1. A: True, B: False, C: True, D: False

2.A: False, B: False, C: True, D: True

  1. A: True, B: True, C: True, D: True
  2. A: False, B: True, C: False, D: True

Q1-2: Which of the following metrics is NOT specifically tailored to help with

evaluating highly imbalanced data?

  1. Precision and Recall
  2. Area Under the ROC curve
  3. Accuracy
  4. All of the above helps in evaluating highly imbalanced data

If you have an imbalanced dataset accuracy can give you false assumptions regarding the classifier’s performance, it’s better to rely on precision and recall.

Q2-1: A learned model h makes 10 errors over the 100 instances. Calculate the 95% confidence interval i.e. With approximately 95% probability, the true error lies in the interval ______. Take zC = 2

  1. 0.1 ± 0.
  2. 0.1 ± 0.
  3. 0.1 ± 0.
  4. 0.1 ± 0.

Q2-2: Which of the following statements is FALSE?

  1. The null hypothesis states that the 2 learning systems have the same accuracy
  2. Alternative hypothesis states that one of the systems is more accurate than the other
  3. If p is sufficiently small, then reject the alternative hypothesis
  4. A two tailed test asks if the accuracy of the two systems are different.

Q2-2: Which of the following statements is FALSE?

  1. The null hypothesis states that the 2 learning systems have the same accuracy
  2. Alternative hypothesis states that one of the systems is more accurate than the other
  3. If p is sufficiently small, then reject the alternative hypothesis
  4. A two tailed test asks if the accuracy of the two systems are different.

If p is sufficiently small, then reject the null hypothesis

Q3-1: The figure shows ROC curve for different models.

Select the correct option.

A. Dashed black line represents random classification. B. ROC curve for any model can’t fall below the dashed black line. C. The model represented by solid blue line is better than that represent by solid lime.

  1. Statement A is true. Statement B, C are false.
  2. Statement A, B are true. Statement C is false.
  3. Statement B, C are true. Statement A is false.
  4. All Statements are true.

Q3-2: How to avoid pitfalls while training a model?

  1. Collect test data that is true representation of real world.
  2. Don’t access the label of a test instance while training.
  3. Avoid excessive preprocessing/training on a particular dataset.
  4. All of the above.