ccml改進(jìn)多支持向量機(jī)遞歸特征消除在癌癥分類中應(yīng)用

SVM-RFE特征選擇算法改進(jìn)的多類特征選擇算法實(shí)驗(yàn)設(shè)計(jì)與實(shí)驗(yàn)結(jié)果總結(jié)主要內(nèi)容2支持向量機(jī)3SVM-RFE特征選擇算法支持向量機(jī)通過引入拉格朗日乘子，可以將SVM的優(yōu)化問題轉(zhuǎn)變?yōu)橐韵碌膶ε家?guī)劃問題：SVM-RFE特征選擇算法min2Ni

j

i

j

i

jiNsubject

toi

1

j

1

i

11

N

N

yiii

1

0，0

i

C，i

1,

2,...,

N

y

y

(

x

x

)

(2)4(3)(4)SVM-RFE特征選擇算法SVM-RFE特征選擇算法5多類SVM-RFE特征選擇算法目前已經(jīng)提出兩種多類SVM-RFE特征選擇算法。MSVM-RFEMSVM-RFE-KERNEL這兩種算法采用線性核時是等價的。MSVM-RFE算法先用一對一的策略把多類問題分解為多個兩類問題，每個兩類問題用一個SVM-RFE來進(jìn)行特征選擇。利用多個SVM-RFE獲得多個排序準(zhǔn)則分?jǐn)?shù)，然后把多個排序準(zhǔn)則分?jǐn)?shù)相加后得到排序準(zhǔn)則總分，以此來作為特征剔除的依據(jù)，每次迭代消去總分最小的特征，直至所有特征被刪除。MSVM-RFE的具體算法見算法2.6SVM-RFE特征選擇算法多類SVM-RFE特征選擇算法SVM-RFE特征選擇算法7改進(jìn)的多類特征選擇算法(MMSVM-RFE)問題：上述的多類SVM-RFE特征選擇方法考慮的是所有子學(xué)習(xí)機(jī)的權(quán)重融合，忽略了各子學(xué)習(xí)機(jī)自身挑選特征的能力。改進(jìn)：利用一對一的策略把多類問題分解了多個兩類問題。每個兩類問題采用一個SVM-RFE方法逐漸剔除掉冗余特征，得到一個特征子集；最后將所有特征子集合并得到最終的特征子集。改進(jìn)的多類特征選擇算法X1X

2F1F22FX

Fl(l-1)2Fl(l-1)X

l(l-1)

28改進(jìn)的多類特征選擇算法改進(jìn)的多類特征選擇算法(MMSVM-RFE)

j

l

l

12ji

i1

j

12

j

l

3l

1

lN

Ni1N

Ni1N

Ni1X， 當(dāng)vi

1時，yi

1；當(dāng)vi

j

1，yi

1,j

1,,l1, 當(dāng)vi

2時，yi

1；當(dāng)vi

j

l

3，yi

1,j

l,,2l

3,當(dāng)vi

l

1時，yi

1；當(dāng)vi

l，yi

1,xi

,

yixi

,

yi

x

,

y9改進(jìn)的多類特征選擇算法改進(jìn)的多類特征選擇算法(MMSVM-RFE)X1X

2F1F22FX

Fl(l-1)2Fl(l-1)X

l(l-1)

21改進(jìn)的多類特征選擇算法改進(jìn)的多類特征選擇算法(MMSVM-RFE)11數(shù)據(jù)集類別基因數(shù)#訓(xùn)練樣本#測試樣本CNS

tumors5

(M/

CRE/

MG/N/

S)98921(5/5/5/2/4)21(5/5/5/2/4)Leukemia3

(ALL/

MLL/

AML)99920(10/4/6)18(9/4/6)Lung

cancer4

(AD/

NL/

SQ/

COID)1000100(70/9/11/10)97(69/8/10/10)12數(shù)據(jù)集描述我們采用了三個多類基因數(shù)據(jù)集：CNS

tumors，Leukemia和Lung

cancer。數(shù)據(jù)集的詳細(xì)信息實(shí)驗(yàn)設(shè)計(jì)與實(shí)驗(yàn)結(jié)果對比實(shí)驗(yàn)：MMSVM-RFEVS.

MSVM-RFE實(shí)驗(yàn)設(shè)計(jì)與實(shí)驗(yàn)結(jié)果13對比實(shí)驗(yàn)：MMSVM-RFE

VS.

MSVM-RFE下面是在三個基因數(shù)據(jù)集上的平均精度對比圖.實(shí)驗(yàn)設(shè)計(jì)與實(shí)驗(yàn)結(jié)果050

100

150200

250

0.40.50.60.70.80.91Gene

Num

berAverage

accuracyMMS

VM-

RFE

MSVM-R

FE050

100

150

2000.65

0.70.75

0.80.85

0.90.95

1Gene

Num

ber

Average

accuracyMMSVM-

RFE

MSVM-RFE50

1502002500.7250

0100Gene

Num

ber0.750.80.850.90.951Average

accuracyMMSVM-

RFEMSVM-R

FE(a)CNS

tumors14(b)

Leukemia(c)Lung

cancer實(shí)驗(yàn)設(shè)計(jì)與實(shí)驗(yàn)結(jié)果15（a）CNS

tumorMSVM-RFEMMSVM-RFEBest

number

of

selectedgenes14798Average

accuracy

(%)90.48±7.1092.38±6.02Average

accuracy

of

classM

(%)100.00±0.00100.00±0.00Average

accuracy

of

classCRE(%)94.00±9.6692.00±10.33Average

accuracy

of

class

MG

(%)98.00±6.3298.00±6.32Average

accuracy

of

class

N

(%)85.00±24.1595.00±15.81Average

accuracy

of

classS(%)67.50±33.4475.00±31.18對比實(shí)驗(yàn)：MMSVM-RFEVS.

MSVM-RFE（b）

LeukemiaMSVM-RFEMMSVM-RFEBest

number

of

selected

genes143185Average

accuracy

(%)97.78±2.2299.44±0.56Average

accuracy

of

class

ALL

(%)97.78±4.68100.00±0.00Average

accuracy

of

class

MLL

(%)100.00±0.00100.00±0.00Average

accuracy

of

class

AML

(%)96.00±8.4398.00±6.32（c）Lung

cancerMSVM-RFEMMSVM-RFEBest

number

of

selected

genes242160Average

accuracy

(%)96.08±0.9596.29±1.00Average

accuracy

of

class

AD

(%)97.39±1.1497.54±1.53Average

accuracy

of

class

NL

(%)91.25±11.8691.25±8.44Average

accuracy

of

class

SQ

(%)87.00±9.4988.00±8.50Average

accuracy

of

class

COID

(%)100.00±0.00100.00±0.00實(shí)驗(yàn)設(shè)計(jì)與實(shí)驗(yàn)結(jié)果對比實(shí)驗(yàn)：SVM

VS.

K近鄰

VS.

隨機(jī)森林將本文方法特征提取后的結(jié)果采用K近鄰和隨機(jī)森林等分類器進(jìn)行實(shí)驗(yàn)1實(shí)驗(yàn)設(shè)計(jì)與實(shí)驗(yàn)結(jié)果(a)CNS

tumors(b)

Leukemia(c)Lung

cancer050

1502000.55100Gene

Num

ber0.60.650.70.750.80.850.90.95Average

accuracyMMSVM-KNNMMSVM-R

F

MMSVM-R

FE250

050

100

150200250

0.70.75

0.80.85

0.90.95

1Gene

Num

ber

Average

accuracyMMSVM-KNN

MMSVM-R

F

MMSVM-R

FE050

150200250

0.87

100

Gene

Num

ber0.88

0.89

0.90.91

0.92

0.93

0.94

0.95

0.96

0.97

Average

accuracyMMSVM-

KNN

MMSVM-

RFMMSVM-

RFE對比實(shí)驗(yàn)：SVM

VS.

K近鄰

VS.

隨機(jī)森林下面是在三個分類器上的平均精度對比.17實(shí)驗(yàn)設(shè)計(jì)與實(shí)驗(yàn)結(jié)果對比實(shí)驗(yàn)：MMSVM-RFE

VS.

PCA將本文MMSVM-RFE特征選擇算法與經(jīng)典的降維算法——主成分分析算法(principle

component

analysis,PCA)進(jìn)行對比1實(shí)驗(yàn)設(shè)計(jì)與實(shí)驗(yàn)結(jié)果19對比實(shí)驗(yàn)：MMSVM-RFEVS.

PCA（a）CNS

tumorMMSVM-RFEPCABest

number

of

selectedgenes9898Average

accuracy

(%)92.38±6.0291.43±5.85Average

accuracy

of

classM

(%)100.00±0.00100.00±0.00Average

accuracy

of

classCRE(%)92.00±10.3392.00±10.33Average

accuracy

of

class

MG

(%)98.00±6.3298.00±6.32Average

accuracy

of

class

N

(%)95.00±15.8185.00±24.15Average

accuracy

of

classS(%)75.00±31.1862.50±31.18（b）

LeukemiaMMSVM-RFEPCABest

number

of

selected

genes185185Average

accuracy

(%)99.44±0.5698.89±5.85Average

accuracy

of

class

ALL

(%)100.00±0.0097.78±4.68Average

accuracy

of

class

MLL

(%)100.00±0.00100.00±0.00Average

accuracy

of

class

AML

(%)98.00±6.32100.00±0.00（c）Lung

cancerMMSVM-RFEPCABest

number

of

selected

ge