Machine learning(5): Quiz 5

1. Testing a classifier’s performance using a 10-fold cross-validation scheme produced the following confusion matrix:
   
   Please compute the following performance measures (as ratios ∈[0,1], not as percentages)
   Estimated prior probability of class Yes:
   Recall (also known as true positive rate or sensitivity):
   Precision (also called positive predictive value) :
   Classification accuracy:

Ananlysis:
We could see that the 71 is TP, 89 is TN, 7 is FP, 14 is FN.
So, we could change this picture into :



2. Consider three information retrieval systems with the following performances: system #1 has precision p1=0.82 and recall r1=0.75; system #2 has precision p2=0.9 and recall r2=0.85; system #3 has precision p3=0.92 and recall r3=0.8. Which one of the following is true?

a. system #3 is the best of all three.
b. system #1 is the best of all three.
c. system #2 is better than #1.
d. system #3 is better than #2.
Correct answer: C
Ananysis:

3. For each one of the following functions, determine if they are convex or non-convex. Remember that a function is convex if the line segment between any two points on the graph of the function lies above the graph：

The huber loss function is： convex.

The huber loss function is： convex.

The huber loss function is： convex.

The huber loss function is： convex.

The huber loss function is： non-convex.


The huber loss function is： non-convex.


The huber loss function is： convex.
Analysis:

4. Consider a binary (2 classes) classification task and a system that achieves 95% accuracy. Which one of the following is true?
a. the classifier has an extremely high accuracy, considering that chance level is 50% for a binary classification task .
b. the information given is insufficient to assess whether the accuracy of the classifier is good or not.
c. assuming that the accuracy was assessed on a test set not used for training, it is a reasonably high value
d. the classifier has a very high accuracy
Correct answer : B
Analysis:

5. suppose you have a 1 nearest neighbour learning system that uses the number of identical attributes as its similarity measure. The system retains all training examples.
You are given the data in the table below:

You use a 10-fold cross-validation scheme to try to assess the performance of the classifier on unseen data. Performing, like in this case, N-fold cross-validation on N training examples is called leave-one-out (LOO). In this case, you simply need to consider each example in turn and see how it would be classified by a classifier trained using only the N-1 remaining examples. At the end, count how many of the examples of class Yes have been classified as Yes (True Positives), and so on…
Please compute the following performance measures (as ratios ∈[0,1], not as a percentages):

Recall (also known as true positive rate or sensitivity):0.8
Precision (also called positive predictive value) Answer:0.57
Classification accuracy：0.6

Analysis：



6. Which one of the following is true?
a. If f(θ) and g(θ) are both convex, then their product, f(θ)g(θ), is also convex .
b. If a function is convex, a local minimum is also a global minimum.
c. A function is convex if and only if it can be written as a finite sum of paraboloids.
d. Any convex function is differentiable.
Correct answer: B
Analysis:

7. Training and Testing/Validation must use different data sets:
a. because estimating the accuracy using the same set used for training would very likely underestimate the true accuracy on unseen data.
b. because splitting the original data set into two separate data sets makes training and testing computationally less demanding than training and testing using all the data available .
c. otherwise the resulting predictor is very likely to underfit the data.
d. because estimating the accuracy using the same set used for training would very likely overestimate the true accuracy on unseen data.

Correct answer: D
Analysis: