




版權(quán)說(shuō)明:本文檔由用戶(hù)提供并上傳,收益歸屬內(nèi)容提供方,若內(nèi)容存在侵權(quán),請(qǐng)進(jìn)行舉報(bào)或認(rèn)領(lǐng)
文檔簡(jiǎn)介
1、On the (In)fi delity and Sensitivity of Explanations Chih-Kuan Yeh , Cheng-Yu Hsieh : , Arun Sai Suggala ; Department of Machine Learning Carnegie Mellon University David I. Inouye School of Electrical and Computer Engineering Purdue University Pradeep Ravikumar Department of Machine Learning Carneg
2、ie Mellon University Abstract We consider objective evaluation measures of saliency explanations for complex black-box machine learning models. We propose simple robust variants of two notions that have been considered in recent literature: (in)fi delity, and sensitivity. We analyze optimal explanat
3、ions with respect to both these measures, and while the optimal explanation for sensitivity is a vacuous constant explanation, the optimal explanation for infi delity is a novel combination of two popular explanation methods. By varying the perturbation distribution that defi nes infi delity, we obt
4、ain novel explanations by optimizing infi delity, which we show to out-perform existing explanations in both quantitative and qualitative measurements. Another salient question given these measures is how to modify any given explanation to have better values with respect to these measures. We propos
5、e a simple modifi cation based on lowering sensitivity, and moreover show that when done appropriately, we could simultaneously improve both sensitivity as well as fi delity. 1Introduction We consider the task of how to explain a complex machine learning model, abstracted as a function that predicts
6、 a response given an input feature vector, given only black-box access to the model. A popular approach to do so is to attribute any given prediction to the set of input features: ranging from providing a vector of importance weights, one per input feature, to simply providing a set of important fea
7、tures. For instance, given a deep neural network for image classifi cation, we may explain a specifi c prediction by showing the set of salient pixels, or a heatmap image showing the importance weights for all the pixels. But how good is any such explanation mechanism? We can distinguish between two
8、 classes of explanation evaluation measures 22,27: objective measures and subjective measures. The predominant evaluations of explanations have been subjective measures, since the notion of explanation is very human-centric; these range from qualitative displays of explanation examples, to crowd-sou
9、rced evaluations of human satisfaction with the explanations, as well as whether humans are able to understand the model. Nonetheless, it is also important to consider objective measures of explanation effectiveness, not only because these place explanations on a sounder theoretical foundation, but
10、also because they allow us to improve our explanations by improving their objective measures. One way to objectively evaluate explanations is to verify whether the explanation mechanism satisfi es (or does not satisfy) certain axioms, or properties 25,43. In this paper, we focus on quantitative obje
11、ctive measures, and provide and analyze two such measures. First, we formalize the notion of fi delity of an explanation to the predictor function. One natural approach to measure fi delity, when we have apriori information that only a particular subset of features is relevant, is to test if the cjy
12、 :chyu.hsieh ; 33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, Canada. features with high explanation weights belong to this relevant subset 10. In the absence of such apriori information, Anco
13、na et al. 3 provide a more quantitative perspective on the earlier notion by measuring the correlation between the sum of a subset of feature importances and the difference in function value when setting the features in the subset to some reference value; by varying the subsets, we get different val
14、ues of such subset correlations. In this work, we consider a simple generalization of this notion, that produces a single fi delity measure, which we call the infi delity measure. Our infi delity measure is defi ned as the expected difference between the two terms: (a) the dot product of the input p
15、erturbation to the explanation and (b) the output perturbation (i.e., the difference in function values after signifi cant perturbations on the input). This general setup allows for a varied class of signifi cant perturbations: a non-random perturbation towards a single reference or baseline value,
16、perturbations towards multiple reference points e.g. by varying subsets of features to perturb, and a random perturbation towards a reference point with added small Gaussian noise, which allows the infi delity measure to be robust to small mis-specifi cations or noise in either the test input or the
17、 reference point. We then show that the optimal explanation that minimizes this infi delity measure could be loosely cast as a novel combination of two well-known explanation mechanisms: Smooth-Grad 40 and Integrated Gradients 43 using a kernel function specifi ed by the random perturbations. As ano
18、ther validation of our formalization, we show that many recently proposed explanations can be seen as optimal explanations for the infi delity measure with specifi c perturbations. We also introduce new perturbations which lead to novel explanations by optimizing the infi delity measure, and we vali
19、date the explanations are qualitatively better through human experiments. It is worth emphasizing that the infi delity measure, while objective, may not capture all the desiderata of a successful explanation; thus, it is still of interest to take a given explanation that does not have the form of th
20、e optimal explanation with respect to a specifi ed infi delity measure and modify it to have lesser infi delity. Analyzing this question leads us to another objective measure: the sensitivity of an explanation, which measures the degree to which the explanation is affected by insignifi cant perturba
21、tions from the test point. It is natural to wish for our explanation to have low sensitivity, since that would entail differing explanations with minor variations in the input (or prediction values), which might lead us to distrust the explanations. Explanations with high sensitivity could also be m
22、ore amenable to adversarial attacks, as Ghorbani et al.13show in the context of gradient based explanations. Regardless, we largely expect explanations to be simple, and lower sensitivity could be viewed as one such notion of simplicity. Due in part to this, there have been some recent efforts to qu
23、antify the sensitivity of explanations 2,28,13. We propose and analyze a simple robust variant of these recent proposals that is amenable to Monte Carlo sampling-based approximation. Our key contribution, however, is in relating the notion of sensitivity to our proposed notion of infi delity, which
24、also allows us to address the earlier raised question of how to modify an explanation to have better fi delity. Asking this question for sensitivity might seem vacuous, since the optimal explanation that minimizes sensitivity (for all its related variants) is simply a trivial constant explanation, w
25、hich is naturally not a desired explanation. So a more interesting question would be: how do we modify a given explanation so that it has lower sensitivity, but not too much. To quantify the latter, we could in turn use fi delity. As one key contribution of the paper, we show that a restrained lower
26、ing of the sensitivity of an explanation also increases its fi delity. In particular, we consider a simple kernel smoothing based algorithm that appropriately lowers the sensitivity of any given explanation, but importantly also lowers its infi delity. Our meta-algorithm encompasses Smooth-Grad 40 w
27、hich too modifi es any existing explanation mechanism by averaging explanations in a small local neighborhood of the test point. In the appendix, we also consider an alternative approach to improve gradient explanation sensitivity and fi delity by adversarial training, which however requires that we
28、 be able to modify the given predictor function itself, which might not always be feasible. Our modifi cations improve both sensitivity and fi delity in most cases, and also provides explanations that are qualitatively better, which we validate in a series of experiments.6 2 Objective Measure: Expla
29、nation Infi delity Consider the following general supervised learning setting: input spaceX Rd, an output space Y R, and a (machine-learnt) black-box predictorf : Rd R, which at some test inputx P Rd, predicts the outputfpxq. Then a feature attribution explanation is some function : F Rd Rd, that gi
30、ven a black-box predictorf, and a test pointx, provides importance scorespf,xqfor the set of 6Implementation available at 2 input features. We let denote a given norm over the input and explanation space. In experiments, if not specifi ed, this will be set to the 2norm. 2.1 Defi ning the infi delity
31、 measure A natural notion of the goodness of an explanation is to quantify the degree to which it captures how the predictor function itself changes in response to signifi cant perturbations. Along this spirit, 4,37,43 propose the completeness axiom for explanations consisting of feature importances
32、, which states that the sum of the feature importances should sum up to the difference in the predictor function value at the given input and some specifi c baseline. 3 extend this to require that the sum of a subset of feature importance weights should sum up to the difference in the predictor func
33、tion value at the given input and to a perturbed input that sets the subset of features to some specifi c baseline value. When the subset of features is large, this would entail that explanations capture the combined importance of the subset of features even if not the individual feature importances
34、, and when the the subset of features is small, this would entail that explanations capture the individual importance of features. We note that this can be contrasted with requiring the explanations to capture the function values itself as in the causal local explanation metric of 30, rather than th
35、e difference in function values, but we focus on the latter. LettingSkdenote a subset ofkfeatures, 3 measured the discrepancy of the above desiderata as the correlation between iPSkpf,xqi andfpxq fpxrxSk“ 0sq,where xrxS“ asj“ aIpj P Sq xjIpj R Sq and I is the indicator function. One minor caveat wit
36、h the above is that we may be interested in perturbations more general than setting feature values to 0, or even to a single baseline; for instance, we might simultaneously require smaller discrepancy over a set of subsets, or some distribution of subsets (as is common in game theoretic approaches t
37、o deriving feature importances 11,42,25), or even simply a prior distribution over the baseline input. The correlation measure also focuses on second order moments, and is not as easy to optimize. We thus build on the above developments, by fi rst allowing random perturbations on feature values inst
38、ead of setting certain features to some baseline value, and secondly, by replacing correlation with expected mean square error (our development could be further generalized to allow for more general loss functions). We term our evaluation measure explanation infi delity. Defi nition 2.1.Given a blac
39、k-box functionf, explanation functional, a random variableI P Rd with probability measureI , which represents meaningful perturbations of interest, we defi ne the explanation infi delity of as: INFDp,f,xq “ EII ” ITpf,xq pfpxq fpx Iqq 2 .(1) I represents signifi cant perturbations aroundx , and can
40、be specifi ed in various ways. We begin by listing various plausible perturbations of interest. Difference to baseline: I “ x x0, the difference between input and baseline. Subset of difference to baseline: for any fi xed subsetSk rds,ISk“ xxrxSk“ px0qSks corresponds to the perturbation in the corre
41、lation measure of 3 when x0“ 0. Difference to noisy baseline:I “ x z0, wherez0“ x0 ?, for some zero mean random vector ?, for instance ? Np0,2q. Difference to multiple baselines:I “ x x0, wherex0is a random variable that can take multiple values. As we will next show in Section 2.3, many recently pr
42、oposed explanations could be viewed as optimiz- ing the aforementioned infi delity for varying perturbationsI . Our proposed infi delity measurement can thus be seen as a unifying framework for these explanations, but moreover, as a way to design new explanations, and evaluate any existing explanati
43、ons. 2.2 Explanations with least Infi delity Given our notion of infi delity, a natural question is: what is the explanation that is optimal with respect to infi delity, that is, has the least infi delity possible. This naturally depends on the distribution of the perturbations I, and its surprising
44、ly simple form is detailed in the following proposition. Proposition 2.1.Suppose the perturbationsIare such that IITdI1is invertible. The optimal explanation pf,xq that minimizes infi delity for perturbations I can then be written as pf,xq “ IITdI 1 IITIGpf,x,IqdI ,(2) 3 whereIGpf,x,Iq “ 1 t“0fpx pt
45、 1qIqdt is the integrated gradient offpqbetweenpx Iq andx43 , but can be replaced by any functional that satisfi esITIGpf,x,Iq “ fpxq fpx Iq. A generalized version of SmoothGrad can be written askpf,xq :“ rzkpx,zqs1 zpf,zqkpx,zqdz where the Gaussian kernel can be replaced by any kernel. Therefore, t
46、he optimal solution of Proposition 2.1 can be seen as applying a smoothing operation reminiscent of SmoothGrad on Integrated Gradients (or any explanation that satisfi es the completeness axiom), where a special kernelIITis used instead of the original kernelkpx,zq. When I is deterministic, the inte
47、gral ofIIT is rank-one and cannot be inverted, but being optimal with respect to the infi delity can be shown to be equivalent to satisfying the Completeness Axiom. To enhance computational stability, we can replace inverse by pseudo-inverse, or add a small diagonal matrix to overcome the non-invert
48、ible case, which works well in experiments. 2.3 Many Recent Explanations Optimize Infi delity As we show in the sequel, many recently proposed explanation methods can be shown to be optimal with respect to our infi delity measure in Defi nition 2.1, for varying perturbations I. Proposition 2.2.Suppo
49、se the perturbationI “ x x0is deterministic and is equal to the dif- ference betweenxand some baselinex0. Letpf,xqbe any explanation which is optimal with respect to infi delity for perturbationsI. Thenpf,xq d I satisfi es the completeness axiom; that is d j“1r pf,xq d Isj “ fpxq fpx Iq . Note that
50、the completeness axiom is also satisfi ed by IG 43, DeepLift 37, LRP 4. Proposition 2.3.Suppose the perturbation is given byI?“ ? ei, whereeiis a coordinate basis vector. Then the optimal explanation ?pf,xq with respect to infi delity for perturbationsI? , satisfi es: lim?0 ?pf,xq “ fpxq, so that th
51、e limit point of the optimal explanations is the gradient explanation 36. Proposition 2.4.Suppose the perturbation is given byI “ eid x, whereeiis a coordinate basis vector. Letpf,xq be the optimal explanation with respect to infi delity for perturbationsI. Then pf,xq d x is the occlusion-1 explanat
52、ion47. Proposition 2.5.Following the notation in 25, given a test inputx, suppose there is a mapping hx: t0,1ud Rd that maps simplifi ed binary inputsz P t0,1udtoRd, such that the given test input xis equal tohxpz0qwherez0is a vector with all ones andhxp0q=0where0is the zero vector. Now, consider th
53、e perturbationI “ hxpZq, whereZ P t0,1udis a binary random vector with distribution PpZ “ zq9 d1 p d z1qz1pdz1q. Then for the optimal explanation pf,xq with respect to infi delity for perturbations I, pf,xq d x is the Shapley value25. 2.4Some Novel Explanations with New Perturbations Byvaryingtheper
54、turbationsI inourinfi delitydefi nition2.1, wenotonlyrecoverexistingexplanations (as those that optimize the corresponding infi delity), but also design some novel explanations. We provide two such instances below. Noisy Baseline.The completeness axiom is one of the most commonly adopted axioms in t
55、he context of explanations, but a caveat is that the baseline is set to some fi xed vector, which does not account for noise in the input (or the baseline itself). We thus set the baseline to be a Gaussian random vector centered around a certain clean baseline (such as the mean input or zero) depend
56、ing on the context. The explanation that optimizes infi delity with corresponding perturbationsIis a novel explanation that can be seen as satisfying a robust variant of the completeness axiom. Square Removal. Our second example is specifi c for image data. We argue that perturbations that remove ra
57、ndom subsets of pixels in images may be somewhat meaningless, since there is very little loss of information given surrounding pixels that are not removed. Also ranging over all possible subsets to remove (as in SHAP 25) is infeasible for high dimension images. We thus propose a modifi ed subset dis
58、tribution from that described in Proposition 2.5 where the perturbationZhas a uniform distribution over square patches with predefi ned length, which is in spirit similar to the work of 49. This not only improves the computational complexity, but also better captures spatial relationships in the ima
59、ges. One can also replace the square with more complex random masks designed specifi cally for the image domain 29. 4 2.5Local and Global Explanations As discussed in 3, we can contrast between local and global feature attribution explanations: global feature attribution methods directly provide the change in the function value given changes in the f
溫馨提示
- 1. 本站所有資源如無(wú)特殊說(shuō)明,都需要本地電腦安裝OFFICE2007和PDF閱讀器。圖紙軟件為CAD,CAXA,PROE,UG,SolidWorks等.壓縮文件請(qǐng)下載最新的WinRAR軟件解壓。
- 2. 本站的文檔不包含任何第三方提供的附件圖紙等,如果需要附件,請(qǐng)聯(lián)系上傳者。文件的所有權(quán)益歸上傳用戶(hù)所有。
- 3. 本站RAR壓縮包中若帶圖紙,網(wǎng)頁(yè)內(nèi)容里面會(huì)有圖紙預(yù)覽,若沒(méi)有圖紙預(yù)覽就沒(méi)有圖紙。
- 4. 未經(jīng)權(quán)益所有人同意不得將文件中的內(nèi)容挪作商業(yè)或盈利用途。
- 5. 人人文庫(kù)網(wǎng)僅提供信息存儲(chǔ)空間,僅對(duì)用戶(hù)上傳內(nèi)容的表現(xiàn)方式做保護(hù)處理,對(duì)用戶(hù)上傳分享的文檔內(nèi)容本身不做任何修改或編輯,并不能對(duì)任何下載內(nèi)容負(fù)責(zé)。
- 6. 下載文件中如有侵權(quán)或不適當(dāng)內(nèi)容,請(qǐng)與我們聯(lián)系,我們立即糾正。
- 7. 本站不保證下載資源的準(zhǔn)確性、安全性和完整性, 同時(shí)也不承擔(dān)用戶(hù)因使用這些下載資源對(duì)自己和他人造成任何形式的傷害或損失。
最新文檔
- 醫(yī)療教育中模擬游戲的成效評(píng)估研究
- 抖音商戶(hù)差評(píng)回復(fù)內(nèi)容審核制度
- 八大城市物流行業(yè)物流配送體系建設(shè)研究報(bào)告
- 公交優(yōu)先政策2025年實(shí)施對(duì)城市交通擁堵治理的成本效益分析報(bào)告
- 公眾參與對(duì)2025年環(huán)境影響評(píng)價(jià)結(jié)論影響的研究報(bào)告
- 2024-2025學(xué)年河南省駐馬店市新蔡縣九上化學(xué)期末考試模擬試題含解析
- 2024年湖南省長(zhǎng)沙市明德旗艦化學(xué)九年級(jí)第一學(xué)期期末達(dá)標(biāo)檢測(cè)模擬試題含解析
- 上海邦德職業(yè)技術(shù)學(xué)院《數(shù)字媒體設(shè)計(jì)》2023-2024學(xué)年第一學(xué)期期末試卷
- 鄭州工業(yè)安全職業(yè)學(xué)院《橋梁工程D》2023-2024學(xué)年第一學(xué)期期末試卷
- 宿遷學(xué)院《建筑設(shè)備與環(huán)境》2023-2024學(xué)年第一學(xué)期期末試卷
- 《工業(yè)固廢資源化技術(shù)及應(yīng)用》課程教學(xué)大綱
- [北京]輸變電工程標(biāo)準(zhǔn)工藝應(yīng)用圖冊(cè)(圖文并茂)
- 信用修復(fù)申請(qǐng)書(shū)
- 深圳房地產(chǎn)開(kāi)發(fā)企業(yè)資質(zhì)申報(bào)表
- 美變出廠(chǎng)檢驗(yàn)記錄
- 2020年雀巢公司北京總部十周年慶典暨雀巢家庭日活動(dòng)策劃案ppt課件
- UPS電源施工方案
- 1000MW機(jī)組鍋爐長(zhǎng)伸縮式吹灰器檢修規(guī)程
- 清關(guān)發(fā)票裝箱單樣本
- 超臨界蒸汽對(duì)鋼表面的氧化作用
- GB 18382-2021 肥料標(biāo)識(shí) 內(nèi)容和要求(高清-現(xiàn)行)
評(píng)論
0/150
提交評(píng)論