Chapter9Testing the System

1、1 Chapter 9 Testing the SystemNote A: unit and integration testing-by yourself or a small part of the development team（you can complete control over the testing process.） B: system testing-by the entire development team (you cant control the testing process，you must work with the entire development

2、team.） 2 Chapter 9 Testing the System9.1 Principles (原理) of System Testing Focus A: The objective of unit and integration -ensure the code implemented the design properly (程序員/測(cè)試者滿足設(shè)計(jì)師的設(shè)計(jì)要求) B: The objective of system testing -ensure the customer wants it to do (開(kāi)發(fā)團(tuán)隊(duì)確保系統(tǒng)滿足客戶的需求)軟件的最終要求3 Chapter 9 Te

3、sting the System1. Sources of Software Faults (軟件缺陷的來(lái)源) The objective(目標(biāo))of testing : eliminate all faults that might lead to failures appearance of faults : can be inserted in any phase during software process (Fig9.1) A: faults in the requirement (uncertain, incorrect misunderstand) B: faults in t

4、he system design(incorrect or unclear design, incorrect translation from SRS) 4 Chapter 9 Testing the System5 Chapter 9 Testing the System C: faults in the program design ( incorrect or unclear program design, misinterpretation of system design ) D: faults in coding ( incorrect documentation,incorre

5、ct semantics, misinterpretation of program design ) E: faults in unit/integration testing -incomplete test process -correct old mistakes generate new errors -(example P419-s4 ) 6 Chapter 9 Testing the System F: faults in system testing ( incomplete test procedure ) G: faults in maintenance (change r

6、equirement, incorrect user document, poor human factors, new faults introduced when correct old errors) demanding (要求) to testing process A: thoroughness B: satisfaction ( by user, customer, developer ) (因?yàn)樵诖笮拖到y(tǒng)中，肯定存在用戶允許的、當(dāng)時(shí)不 必修復(fù)的缺陷存在) 7 Chapter 9 Testing the System2. System Testing Process (Fig9.2

7、) Process Objectives（處理過(guò)程的各個(gè)目標(biāo)） A: function test -check by developer B: performance test -check by developer C: acceptance test -check by developer and user D: installation test -test in users environment Build Plan（構(gòu)造/集成計(jì)劃） A: system testing of large system X: phased system testing (from small to l

8、arge) Y: nested set of levels (from inner to outside)8 Chapter 9 Testing the System B: example: routes calls ( in telecommunications system5 subsystems ) (P421) advantage: X: tested one level-deliver one level Y: easy to check or correct fault C: build plan -incremental test plan for large system( i

9、nclude how,where,when,who ) example: build plan for routes calls subsystem X: table 9.1 : schedule table Y: focus on: spin(subsystem ) hardware and software resource + time/schedule + personnel + customer 9 Chapter 9 Testing the System3. Configuration Management (SCM)(配置管理) concepts A: system config

10、uration (P423) （交付給特定客戶的一系列部件的集合）AFBDGCHE用戶1用戶2ABCDEFABCDEGH 產(chǎn)品1 產(chǎn)品210 Chapter 9 Testing the System 上圖:兩個(gè)產(chǎn)品具有不同的配置,其中A、B、C、D、 E是核心部件/基本部件。B: configuration management(P423)（配置管理） （對(duì)系統(tǒng)不同軟件配置的管理及控制方法（既有開(kāi) 發(fā)，也有測(cè)試），通過(guò)控制系統(tǒng)差別以降低風(fēng) 險(xiǎn)，減少錯(cuò)誤）。C: importance: ( of configuration management ) -coordinating efforts e

11、ffective configuration 協(xié)調(diào)各方努力 get 有效配置11 Chapter 9 Testing the System 確定配置管理(SCM)計(jì)劃 配置管理是整個(gè)開(kāi)發(fā)過(guò)程中,用于管理軟件產(chǎn)品內(nèi)容的一系列措施.SCM流程旨在確保在任何時(shí)候產(chǎn)品的內(nèi)容都是已知的、可用的，以及在設(shè)計(jì)到實(shí)施的過(guò)程中，產(chǎn)品的功能都是可跟蹤的，可以完全控制和保護(hù)產(chǎn)品的內(nèi)容。（例如現(xiàn)在流行的配置管理工具clearcase等）使用配置管理計(jì)劃的必要性： 如果不使用適當(dāng)?shù)呐渲霉芾碛?jì)劃，我們就經(jīng)常無(wú)法知道哪一個(gè)模塊是被確定、增強(qiáng)或測(cè)試過(guò)的。我們會(huì)開(kāi)發(fā)出功能錯(cuò)誤或是缺少功能的產(chǎn)品來(lái)，而且我們不知道哪些測(cè)試正在進(jìn)行

12、，哪些缺陷已被確定。我們不得不重新整理已經(jīng)完成了的工作。 12 Chapter 9 Testing the System配置管理計(jì)劃關(guān)鍵的功能：每個(gè)產(chǎn)品元素（部件）版本的復(fù)件；對(duì)每一個(gè)基線修改的記錄；（修改內(nèi)容的說(shuō)明：修改位置，修改時(shí)間，修改內(nèi)容等等）基線：是指軟件文檔和其他資料的集合，它們代表了產(chǎn)品在某一時(shí)間點(diǎn)的情況。誰(shuí)進(jìn)行了這個(gè)改變；他們什么時(shí)候改變的；改變具體內(nèi)容是什么；為什么他們要進(jìn)行改變。 可能涉及的其他功能等等(從涉及的基本文檔開(kāi)始統(tǒng)計(jì))。 13 Chapter 9 Testing the System技術(shù)支持經(jīng)理負(fù)責(zé)草擬配置管理計(jì)劃，并且和全組一起對(duì)它進(jìn)行復(fù)核。 確定配置控制委員

13、會(huì)和控制過(guò)程 確定任何所需的工具和設(shè)備。和小組一起重申整個(gè)計(jì)劃，并取得一致意見(jiàn)14 Chapter 9 Testing the System軟件配置管理的任務(wù)制定軟件配置管理計(jì)劃確定配置標(biāo)識(shí)規(guī)則實(shí)施變更控制報(bào)告配置狀態(tài)進(jìn)行配置審核進(jìn)行版本管理和發(fā)行管理 15 Chapter 9 Testing the System忽視軟件配置管理可能導(dǎo)致的混亂現(xiàn)象 發(fā)錯(cuò)了版本安裝后不工作異地不能正常工作已經(jīng)解決的缺陷過(guò)后又出現(xiàn)錯(cuò)誤開(kāi)發(fā)人員把產(chǎn)品拿出去出售贏利找不到最新修改了的源程序找不到編程序的人 16 Chapter 9 Testing the System versions and releases（版本

14、和改進(jìn)版本） versions-a particular configuration for a system（針對(duì)特定系統(tǒng)的特定配置） release-improved version/system （針對(duì)舊版本的改進(jìn)版本） example: Version n.m=Version n and release m Version 3.0-Version3.1 ，release 3.2 target of configuration management ( in system testing )-accuracy and promptness in testing 對(duì)于大型系統(tǒng)，人們絕不敢在

15、忽略配置的情況下運(yùn)行針對(duì)全部版本的測(cè)試用例集，一來(lái)太龐大麻煩；二來(lái)無(wú)針對(duì)性 17 Chapter 9 Testing the System production system vs. development system A:production system: B:development system: regression testing（回歸測(cè)試） - a test applied to a new version( verify the correctness of the old and new functions) example: version m-version m+1 (P

16、425) ( test new version by old and new test cases )18 Chapter 9 Testing the SystemChange Control (變更控制) A: note: system testing -find faults-correct faults-change system (under the control by configuration management team) B: problem caused by the change X: change any part of software-effect all doc

17、uments and all developers Y: more developers make change-effect each other C: change control: 配置管理小組采取措施使不同開(kāi)發(fā) 者各自修改后的版本統(tǒng)一為一個(gè)版本.(使開(kāi)發(fā)者不 同時(shí)修改軟件同一部分, 或采取其他額外步驟) 19 Chapter 9 Testing the System4. Test team focus on: A: unit and integration test-large role is developers B: function and performance test-la

18、rge role is developers C: acceptance and installation test large role is customers D: programmers cant participate the testing of his own modules 20 Chapter 9 Testing the System several types (about testers) A: professional testers (who organize and run the tests, include former analysts, programmer

19、s, and designers) (P429) B: analysts: ( who created requirement ) C: system designers (who understand SRS and propose solution) D: configuration management specialists (who controls the changing of software) E: users: to evaluate issues that arise 21 Chapter 9 Testing the System9.2 Function Testing

20、Focus on: A: function testing -black box B: function testing -based on 1.Purpose and Roles（目的和作用）function testing : test for functional requirement ( of ) role: have a high probability of finding a fault (because one function testing only deal with the small set of components) example : water-monito

21、ring system -acknowledging change in dissolved oxygen, temperature, acidity, radioactivity.22 Chapter 9 Testing the System guidelines for function testing (P4316 dots) A: 較高的查錯(cuò)概率 B: 獨(dú)立的測(cè)試團(tuán)隊(duì) C: 了解預(yù)期的輸出結(jié)果 D: 對(duì)合法與非法的輸入都予以測(cè)試（假設(shè)是弱健壯等價(jià)類） E: 不能僅僅為了測(cè)試的方便而修改系統(tǒng) F: 停止測(cè)試應(yīng)該有前提條件 備注 A: 功能測(cè)試是在謹(jǐn)慎的受控制狀態(tài)下進(jìn)行的； B: 由于一次

22、測(cè)試一個(gè)功能，若需要，功能測(cè)試 可以早于整個(gè)系統(tǒng)的集成來(lái)進(jìn)行.(重要功能) developing test cases ( according ) example : word processing system ( P431) 23 Chapter 9 Testing the System2. Cause-and-Effect Graphs (因果圖) 說(shuō)明: A: 功能測(cè)試的測(cè)試用例是由需求定義的功能說(shuō)明部分 轉(zhuǎn)化而來(lái). B: 測(cè)試用例不應(yīng)產(chǎn)生冗余. (the test cases should not redundant) C:測(cè)試用例應(yīng)能發(fā)現(xiàn)需求中不完整和不明確的方面. 定義: 需求中

23、的INPUT稱為原因,OUTPUT稱為結(jié)果.反 映這種因果關(guān)系的布爾邏輯圖稱為因果圖. 基于因果圖的測(cè)試用例產(chǎn)生過(guò)程: A: 由布爾邏輯和約束條件產(chǎn)生因果圖. B: 將因果圖轉(zhuǎn)化為判定表. (decision table)24 Chapter 9 Testing the System C: 由判定表產(chǎn)生測(cè)試用例. 產(chǎn)生因果圖的步驟: A: 將需求分解為各個(gè)單獨(dú)的功能. (separated functions) B: 每個(gè)功能分出原因和結(jié)果. (cause + effect) C: 中間節(jié)點(diǎn) (extra nodes) 的產(chǎn)生. Example: water-level monitoring

24、system A: analysis: requirement: ”the system sends a message to the dam operator about the safety of lake level” Input: syntax of the function: LEVEL(A,B) “A”height in meters of the water “B”-the number of centimeters of rain 25 Chapter 9 Testing the System PROCESSING: The function calculates whethe

25、r the water level is within a safe range, is too high, or is too low. output: result of level(A,B): The screen shows one of the following messages:“LEVEL = SAFE”, when the result is safe or low. “LEVEL = HIGH”, when the result is high.“INVALID SYNTAX”depending on the result of the calculation.26 Cha

26、pter 9 Testing the System B: cause-and-effect graph causes: 5 (P399) 1.The first five characters of the command “LEVEL” 2.The command contains exactly two parameters separated by a comma and enclosed in parentheses. 3.The parameters A and B are real numbers such that the water level is calculated to

27、 be LOW 4.The parameters A and B are real numbers such that the water level is calculated to be SAFE. 5.The parameters A and B are real numbers such that the water level is calculated to be HIGH.27 Chapter 9 Testing the System effects: 3 (P399) 1.The message “LEVEL = SAFE” is displayed on the screen

28、. 2.The message “LEVEL = HIGH” is displayed on the screen. 3.The message “LNVALID SYNTAX” is printed out intermediate nodes: 2 (P399) 1.The command is syntactically valid. 2.The operands are syntactically valid. C-and-E: - fig9.5 (P399) C: decision table: table9.2 D: advantage of C-and-E disadvantag

29、e of C-and-E 28 Chapter 9 Testing the System29 Chapter 9 Testing the SystemTable 9.2. Decision table for cause-and-effect graph.Test 1Test 2Test 3Test 4Test 5Cause 1IIISICause 2IIIXSCause 3ISSXXCause 4SISXXCause 5SSIXXEffect 1PPAAAEffect 2AAPAAEffect 3AAAPP30 Chapter 9 Testing the System9.3 Performa

30、nce TestingFocus on: performance test nonfunctional addressesrequirement(in SRS)1.Purpose and Roles performance testing : test for nonfunctional requirement ( of ) example : calculate the trajectory of a rocket (the speed of response, accuracy, data accessibility, etc.) note : A: performance testing

31、(by a team) get result give to developer and customer B: hardware engineer test team 31 Chapter 9 Testing the System2. Types of Performance testsseveral types of performance tests A: stress tests（強(qiáng)度測(cè)試） (在短時(shí)間內(nèi)加載極限負(fù)荷, 以驗(yàn)證系統(tǒng)能力) B: volume tests（巨量數(shù)據(jù)測(cè)試 / 容量測(cè)試） (驗(yàn)證系統(tǒng)處理巨量數(shù)據(jù)的能力) C: configuration tests（配置測(cè)試）

32、 D: compatibility tests（兼容性測(cè)試） E: regression tests (by old and new test cases) F: security tests ( security requirement ) G: timing tests H: environment tests 32 Chapter 9 Testing the System I: quality tests J: recovery tests K: human factors test ( use interface-usability) focus on: A: performance

33、test -more difficult than function test B: requirement must be explicit to ensure performance test C: requirement quality is often been reflected in the ease of performance testing (需求質(zhì)量通常可以反映在性能測(cè)試的容易度上)33 Chapter 9 Testing the System9.4 Reliability,Availability,Maintainability （可靠性，可用性，可維護(hù)性） (關(guān)于性能測(cè)

34、試) focus:A: performance test-Reliability, Availability, Maintainability-is critical issues B: it is difficult to measure directly before delivery1. Definitions software reliability : ( probability : 0 1 ) -軟件系統(tǒng)在給定的時(shí)間間隔和給定條件下運(yùn)行 成功的概率 availability: ( 0/1 ) -軟件系統(tǒng)在給定的時(shí)間點(diǎn)（按SRS要求）成功 運(yùn)行的概率34 Chapter 9 Test

35、ing the System maintainability: ( 0 1 ) -是指在給定的使用條件（預(yù)定的時(shí)間 間隔、維護(hù)程序、維護(hù)資源之下進(jìn)行維 護(hù)）下，維護(hù)活動(dòng)能被執(zhí)行的概率2. Four different levels of failure severity(嚴(yán)重性) A: reason: because reliability, availability, and maintainability are defined in terms of failures, they must be measured once the system is complete and worki

36、ng B: four different levels ( P438) 3. Failure data : 根據(jù)失效時(shí)間記錄，來(lái)理解系統(tǒng)的不確定性， 以圖改進(jìn). 或者以此理解系統(tǒng)可靠性的改進(jìn)程度.35 Chapter 9 Testing the System9.5 Acceptance Testing（確認(rèn)/驗(yàn)收測(cè)試）1. Purpose and roles acceptance testing: customer check if the system meet the need in requirement definition roles: customer -play large ro

37、les ( in acceptance test，customer write, conduct, and evaluate the test result) developers-play small roles (answer or explain technical questions when the customer requests) 36 Chapter 9 Testing the System2. Types of Acceptance tests benchmark test（基準(zhǔn)測(cè)試） ( formal test ) A: customer prepare test cas

38、es (with formal cases) B: install on experimental basis C: customer evaluate the performance pilot test（引導(dǎo)測(cè)試） (informal test) A: user/developer install on a experimental basis B: pilot test rely on daily working(no formal cases) alpha test: pilot test in organization (by developer) beta test: customers pilot (in workin