ROC曲線.ppt

1、ROC曲線,Hongjie Wu,01,Basic concept,02,Plotting an ROC curve,03,AUC,contents,Basic concept,Receiver operating characteristic (ROC) curves are a useful visual tool for comparing two classification models.(比較兩個分類模型有用的可視化工具) The curve is created by plotting the true positive rate (TPR) against the false

2、positive rate (FPR) at various threshold settings. （ROC曲線顯示了真正例和假正例之間的權(quán)衡）,Basic concept,true positive (TP真正例)診斷為有，實際上也有高血壓 eqv. with hit true negative (TN真負例)診斷為沒有，實際卻沒有高血壓 eqv. with correct rejection false positive (FP假正例)診斷為有，實際上也沒有高血壓 eqv. with false alarm, Type I error false negative (FN假負例)診斷為沒

3、有，實際卻有高血壓 eqv. with miss, Type II error,Basic concept,sensitivity or true positive rate (TPR)Y軸 eqv. with hit rate, recall 在所有實際為陽性的樣本中，被正確地判斷為陽性之比率 fall-out or false positive rate (FPR)X軸 在所有實際為陰性的樣本中，被錯誤地判斷為陽性之比率,Plotting an ROC curve,Figure 8.18 shows the probability value (column 3) returned by

4、a probabilistic classifier for each of the 10 tuples in a test set, sorted by decreasing probability order. Column 1 is merely a tuple identification number, which aids in our explanation. Column 2 is the actual class label of the tuple. There are five positive tuples and five negative tuples, thus

5、and . As we examine the known class label of each tuple,we can determine the values of the remaining columns, TP, FP, TN, FN, TPR, and FPR. We start with tuple1, which has the highest probability score, and take that score as our threshold, that is, . Thus, the classifier considers tuple 1 to be pos

6、itive, and all the other tuples are considered negative. Since the actual class label of tuple 1 is positive, we have a true positive, hence and . Among the remaining nine tuples, which are all classified as negative, five actually are negative (thus, ). The remaining four are all actually positive,

7、 thus, . We can therefore compute , while .Thus, we have the point for the ROC curve.,Plotting an ROC curve,Figure 8.18 Tuples sorted by decreasing score, where the score is the value returned by a probabilistic classifier. (元組按遞減得分排序，其中得分是概率分類器的返回值),對于二類問題， 選擇閾值t， 使得f(X)=t的 元組X視為正， 而其它元組視為負,Plottin

8、g an ROC curve,There are many methods to obtain a curve out of these points, the most common of which is to use a convex hull. The plot also shows a diagonal line where for every true positive of such a model, we are just as likely to encounter a false positive. For comparison, this line represents

9、random guessing（許多方法可以從這些點得到凸包。該圖現(xiàn)實的對角線，對模型的每個真正例元組，都恰好遇到一個假正例，為了比較，這條直線代表隨機猜測）.,AUC,trapezoid method 簡單地將每個相鄰的點以直線連接，計算連線下方的總面積。因為每一線段下方都是一個梯形，所以叫梯形法。 優(yōu)點：簡單，所以常用。 缺點：傾向于低估AUC。 ROC AUCH AUC of ROC是機器學(xué)習(xí)的社群最常使用來比較不同模型優(yōu)劣的方法 。然而近來這個做法開始受到質(zhì)疑，因為有些機器學(xué)習(xí)的研究指出，AUC的噪聲太多，并且很常求不出可信又有效的AUC值，使得AUC在模型比較時產(chǎn)生的問題比解釋的問題更多 。,AUC,To assess the accuracy of a model, we can measure the area under the curve.（為了評估模型的準(zhǔn)確率，可以測量曲線下方的面積） Several software packages are able to perform such calculation.（一些軟件包可以用來進行這些計算）,AUC,從AUC判斷分類器（預(yù)測模型）優(yōu)劣的標(biāo)準(zhǔn)： AUC = 1，是完美分類器，采用這個預(yù)測模型時，存在至少一個