第7章 算法鏈電子課件

第7章算法鏈7.1什么是管道7.2在網(wǎng)格搜索使用管道7.3通用管道模型7.4模型選擇與調(diào)優(yōu)7.5小結(jié)7.1什么是管道

在對前幾章進(jìn)行學(xué)習(xí)后知道,機(jī)器學(xué)習(xí)的構(gòu)建包括需要對數(shù)據(jù)集進(jìn)行訓(xùn)練，需要進(jìn)行預(yù)處理，需要特征選取，需要利用算法建立模型評估。上面四個步驟是一個流水式的工作，而管道機(jī)制實(shí)現(xiàn)了對全部步驟的流式化封裝和管理，那么不使用管道不行嗎？有些時候確實(shí)不行。假設(shè)利用make_blobs數(shù)據(jù)集生成為標(biāo)準(zhǔn)差為5的數(shù)據(jù)集。fromsklearn.datasetsimportmake_blobsfromsklearn.model_selectionimporttrain_test_splitfromsklearn.preprocessingimportStandardScalerimportmatplotlib.pyplotaspltX,y=make_blobs(n_samples=500,centers=2,cluster_std=5)X_train,X_test,y_train,y_test=train_test_split(X,y,random_state=9)scaler=StandardScaler().fit(X_train)X_train_scaled=scaler.transform(X_train)X_test_scaled=scaler.transform(X_test)plt.scatter(X_train[:,0],X_train[:,1])plt.scatter(X_train_scaled[:,0],X_train_scaled[:,1],marker='^',edgecolor='k')plt.title('trainingset&scaledtrainingset')plt.show()我們對數(shù)據(jù)進(jìn)行預(yù)處理是為了算法能夠更好的建立模型，加入我們使用svm算法。fromsklearn.datasetsimportmake_blobsfromsklearn.model_selectionimporttrain_test_splitfromsklearn.preprocessingimportStandardScalerfromsklearn.svmimportSVCX,y=make_blobs(n_samples=500,centers=2,cluster_std=5)X_train,X_test,y_train,y_test=train_test_split(X,y,random_state=9)scaler=StandardScaler().fit(X_train)X_train_scaled=scaler.transform(X_train)X_test_scaled=scaler.transform(X_test)svm=SVC()svm.fit(X_train_scaled,y_train)print("testscore:{}".format(svm.score(X_test_scaled,y_test)))輸出結(jié)果：testscore:0.864接下來還可以通過網(wǎng)格搜索選取最優(yōu)參數(shù)fromsklearn.datasetsimportmake_blobsfromsklearn.model_selectionimporttrain_test_splitfromsklearn.preprocessingimportStandardScalerfromsklearn.svmimportSVCfromsklearn.model_selectionimportGridSearchCVX,y=make_blobs(n_samples=500,centers=2,cluster_std=5)X_train,X_test,y_train,y_test=train_test_split(X,y,random_state=9)scaler=StandardScaler().fit(X_train)X_train_scaled=scaler.transform(X_train)X_test_scaled=scaler.transform(X_test)param_grid={'C':[0.01,0.1,1,10,100],'gamma':[0.01,0.1,1,10,100]}grid=GridSearchCV(SVC(),param_grid=param_grid,cv=5)grid.fit(X_train_scaled,y_train)print("Bestcross-validationaccuracy:{}".format(grid.best_score_))print("Bestsetscore:{}".format(grid.score(X_test_scaled,y_test)))print("Bestparameters:{}",format(grid.best_params_))輸出結(jié)果Bestcross-validationaccuracy:0.9386666666666666Bestsetscore:0.952Bestparameters:{}{'C':0.1,'gamma':0.01}經(jīng)過網(wǎng)格搜索得到的精度達(dá)到了95%，但其實(shí)上述步驟中出現(xiàn)了錯誤。我們在進(jìn)行數(shù)據(jù)預(yù)處理的時候，用StandardScaler擬合了訓(xùn)練數(shù)據(jù)集，而后用這個擬合后的的scaler去分別轉(zhuǎn)換了X_trainX_test，這一步?jīng)]有錯。

但在進(jìn)行網(wǎng)格搜索的時候，用訓(xùn)練集來擬合的GrindSearchCV。在進(jìn)行交叉驗(yàn)證的時候，我們把train_scaled進(jìn)行了拆分，這時的train_scaled都是基于訓(xùn)練集進(jìn)行StandardScaler擬合后再對自身進(jìn)行轉(zhuǎn)換，相當(dāng)于我們用交叉驗(yàn)證中生成的測試集擬合了StandardScaler后，再用這個scaler轉(zhuǎn)換這個測試集自身。從圖中可以看到，這樣的做法是錯誤的。這樣一來，交叉驗(yàn)證的得分就是不準(zhǔn)確的。為了解決這個問題，在交叉驗(yàn)證的過程中，應(yīng)該在進(jìn)行任何預(yù)處理之前完成數(shù)據(jù)集的劃分。那怎么完成在預(yù)處理之前完成數(shù)據(jù)集的劃分呢，顯然如果一次一次的劃分太麻煩，所以這是就需要用到管道模型。fromsklearn.datasetsimportmake_blobsfromsklearn.model_selectionimporttrain_test_splitfromsklearn.preprocessingimportStandardScalerfromsklearn.svmimportSVCfromsklearn.pipelineimportPipelineX,y=make_blobs(n_samples=500,centers=2,cluster_std=5)X_train,X_test,y_train,y_test=train_test_split(X,y,random_state=9)pipe=Pipeline([("scaler",StandardScaler()),("svm",SVC())])pipe.fit(X_train,y_train)print("Testscore:{}".format(pipe.score(X_test,y_test)))輸出結(jié)果Testscore:0.904

建立管道模型后，對比我們之前屬于處理以及算法建模的過程代碼量大大減少。7.2在網(wǎng)格搜索使用管道

管道配合網(wǎng)格搜索使用，我們定義一個需要搜索的參數(shù)網(wǎng)格，并利用管道和參數(shù)網(wǎng)格構(gòu)建一個GridSearchCV，然后需要為每個參數(shù)指定它在管道中所屬的步驟。fromsklearn.datasetsimportmake_blobsfromsklearn.model_selectionimporttrain_test_splitfromsklearn.svmimportSVCfromsklearn.preprocessingimportStandardScalerfromsklearn.pipelineimportPipelinefromsklearn.model_selectionimportGridSearchCVX,y=make_blobs(n_samples=500,centers=2,cluster_std=5)X_train,X_test,y_train,y_test=train_test_split(X,y,random_state=9)param_grid={'svm__C':[0.01,0.1,1,10,100],'svm__gamma':[0.01,0.1,1,10,100]}pipe=Pipeline([("scaler",StandardScaler()),("svm",SVC())])grid=GridSearchCV(pipe,param_grid=param_grid,cv=5)grid.fit(X_train,y_train)print("Bestcross-validationaccuracy:{:.2f}".format(grid.best_score_))print("Testsetscore:{:.2f}".format(grid.score(X_test,y_test)))print("Bestparameters:{}".format(grid.best_params_))輸出結(jié)果Bestcross-validationaccuracy:0.94Testsetscore:0.96Bestparameters:{'svm__C':0.1,'svm__gamma':0.1}

通過在網(wǎng)格搜索中使用管道，我們解決了進(jìn)行任何預(yù)處理之前完成數(shù)據(jù)集的劃分的問題在上述創(chuàng)建管道時還有一個簡單的方法，就是利用make_pipeline函數(shù)fromsklearn.pipelineimportmake_pipeline#標(biāo)準(zhǔn)語法pipe=Pipeline([("scaler",StandardScaler()),("svm",SVC())])#縮寫語法pipe_short=make_pipeline(StandardScaler(),SVC())7.3通用管道模型

Pipeline類不但可用于預(yù)處理和網(wǎng)格搜索，實(shí)際上還可以將任意數(shù)量的估計器連接在一起，還可以構(gòu)建一個包含特征提取、特征選擇、縮放和分類的管道。fromsklearn.preprocessingimportStandardScalerfromsklearn.preprocessingimportMinMaxScalerfromsklearn.pipelineimportmake_pipelinepipe=make_pipeline(StandardScaler(),MinMaxScaler(),StandardScaler())print("Pipelinesteps:\n{}".format(pipe.steps))Pipelinesteps:[('standardscaler-1',StandardScaler(copy=True,with_mean=True,with_std=True)),('minmaxscaler',MinMaxScaler(copy=True,feature_range=(0,1))),('standardscaler-2',StandardScaler(copy=True,with_mean=True,with_std=True))]

這里我們在管道中進(jìn)行了兩次預(yù)處理，因?yàn)閮纱晤A(yù)處理屬于同一階段，我們可以看到管道的工作步驟。根據(jù)需要我們可以將任意的估計器都構(gòu)建到一個管道里。7.4模型選擇與調(diào)優(yōu)

利用管道，還可以創(chuàng)建一個多分類器的管道。在前面的線性回歸中，我們知道容易產(chǎn)生過擬合，所以需要正則化。

接下來我們利用管道來對兩種正則化方式進(jìn)行選擇以及參數(shù)的調(diào)優(yōu)。fromsklearn.datasetsimportload_breast_cancerfromsklearn.model_selectionimporttrain_test_splitfromsklearn.preprocessingimportStandardScalerfromsklearn.linear_modelimportRidgefromsklearn.linear_modelimportLassofromsklearn.model_selectionimportGridSearchCVfromsklearn.pipelineimportPipelinepipe=Pipeline([('scaler',StandardScaler()),('classifier',Ridge())])param_grid=[{'classifier':[Ridge()],'scaler':[StandardScaler(),None],'classifier__alpha':[0.001,0.01,0.1,1,10,100]},{'classifier':[Lasso()],'scaler':[StandardScaler(),None],'classifier__alpha':[0.001,0.01,0.1,1,10,100]}]canner=load_breast_cancer()X_train,X_test,y_train,y_test=train_test_split(canner.data,canner.target,random_state=2)grid=GridSearchCV(pipe,