Chapter9Testing the System

1、1 Chapter 9 Testing the SystemNote A: unit and integration testing-by yourself or a small part of the development team（you can complete control over the testing process.） B: system testing-by the entire development team (you cant control the testing process，you must work with the entire development

2、team.） 2 Chapter 9 Testing the System9.1 Principles (原理) of System Testing Focus A: The objective of unit and integration -ensure the code implemented the design properly (程序員/測試者滿足設計師的設計要求) B: The objective of system testing -ensure the customer wants it to do (開發(fā)團隊確保系統(tǒng)滿足客戶的需求)軟件的最終要求3 Chapter 9 Te

3、sting the System1. Sources of Software Faults (軟件缺陷的來源) The objective(目標)of testing : eliminate all faults that might lead to failures appearance of faults : can be inserted in any phase during software process (Fig9.1) A: faults in the requirement (uncertain, incorrect misunderstand) B: faults in t

4、he system design(incorrect or unclear design, incorrect translation from SRS) 4 Chapter 9 Testing the System5 Chapter 9 Testing the System C: faults in the program design ( incorrect or unclear program design, misinterpretation of system design ) D: faults in coding ( incorrect documentation,incorre

5、ct semantics, misinterpretation of program design ) E: faults in unit/integration testing -incomplete test process -correct old mistakes generate new errors -(example P419-s4 ) 6 Chapter 9 Testing the System F: faults in system testing ( incomplete test procedure ) G: faults in maintenance (change r

6、equirement, incorrect user document, poor human factors, new faults introduced when correct old errors) demanding (要求) to testing process A: thoroughness B: satisfaction ( by user, customer, developer ) (因為在大型系統(tǒng)中，肯定存在用戶允許的、當時不 必修復的缺陷存在) 7 Chapter 9 Testing the System2. System Testing Process (Fig9.2

7、) Process Objectives（處理過程的各個目標） A: function test -check by developer B: performance test -check by developer C: acceptance test -check by developer and user D: installation test -test in users environment Build Plan（構造/集成計劃） A: system testing of large system X: phased system testing (from small to l

8、arge) Y: nested set of levels (from inner to outside)8 Chapter 9 Testing the System B: example: routes calls ( in telecommunications system5 subsystems ) (P421) advantage: X: tested one level-deliver one level Y: easy to check or correct fault C: build plan -incremental test plan for large system( i

9、nclude how,where,when,who ) example: build plan for routes calls subsystem X: table 9.1 : schedule table Y: focus on: spin(subsystem ) hardware and software resource + time/schedule + personnel + customer 9 Chapter 9 Testing the System3. Configuration Management (SCM)(配置管理) concepts A: system config

10、uration (P423) （交付給特定客戶的一系列部件的集合）AFBDGCHE用戶1用戶2ABCDEFABCDEGH 產(chǎn)品1 產(chǎn)品210 Chapter 9 Testing the System 上圖:兩個產(chǎn)品具有不同的配置,其中A、B、C、D、 E是核心部件/基本部件。B: configuration management(P423)（配置管理） （對系統(tǒng)不同軟件配置的管理及控制方法（既有開 發(fā)，也有測試），通過控制系統(tǒng)差別以降低風 險，減少錯誤）。C: importance: ( of configuration management ) -coordinating efforts e

11、ffective configuration 協(xié)調各方努力 get 有效配置11 Chapter 9 Testing the System 確定配置管理(SCM)計劃 配置管理是整個開發(fā)過程中,用于管理軟件產(chǎn)品內(nèi)容的一系列措施.SCM流程旨在確保在任何時候產(chǎn)品的內(nèi)容都是已知的、可用的，以及在設計到實施的過程中，產(chǎn)品的功能都是可跟蹤的，可以完全控制和保護產(chǎn)品的內(nèi)容。（例如現(xiàn)在流行的配置管理工具clearcase等）使用配置管理計劃的必要性： 如果不使用適當?shù)呐渲霉芾碛媱?，我們就?jīng)常無法知道哪一個模塊是被確定、增強或測試過的。我們會開發(fā)出功能錯誤或是缺少功能的產(chǎn)品來，而且我們不知道哪些測試正在進行

12、，哪些缺陷已被確定。我們不得不重新整理已經(jīng)完成了的工作。 12 Chapter 9 Testing the System配置管理計劃關鍵的功能：每個產(chǎn)品元素（部件）版本的復件；對每一個基線修改的記錄；（修改內(nèi)容的說明：修改位置，修改時間，修改內(nèi)容等等）基線：是指軟件文檔和其他資料的集合，它們代表了產(chǎn)品在某一時間點的情況。誰進行了這個改變；他們什么時候改變的；改變具體內(nèi)容是什么；為什么他們要進行改變。 可能涉及的其他功能等等(從涉及的基本文檔開始統(tǒng)計)。 13 Chapter 9 Testing the System技術支持經(jīng)理負責草擬配置管理計劃，并且和全組一起對它進行復核。 確定配置控制委員

13、會和控制過程 確定任何所需的工具和設備。和小組一起重申整個計劃，并取得一致意見14 Chapter 9 Testing the System軟件配置管理的任務制定軟件配置管理計劃確定配置標識規(guī)則實施變更控制報告配置狀態(tài)進行配置審核進行版本管理和發(fā)行管理 15 Chapter 9 Testing the System忽視軟件配置管理可能導致的混亂現(xiàn)象 發(fā)錯了版本安裝后不工作異地不能正常工作已經(jīng)解決的缺陷過后又出現(xiàn)錯誤開發(fā)人員把產(chǎn)品拿出去出售贏利找不到最新修改了的源程序找不到編程序的人 16 Chapter 9 Testing the System versions and releases（版本

14、和改進版本） versions-a particular configuration for a system（針對特定系統(tǒng)的特定配置） release-improved version/system （針對舊版本的改進版本） example: Version n.m=Version n and release m Version 3.0-Version3.1 ，release 3.2 target of configuration management ( in system testing )-accuracy and promptness in testing 對于大型系統(tǒng)，人們絕不敢在

15、忽略配置的情況下運行針對全部版本的測試用例集，一來太龐大麻煩；二來無針對性 17 Chapter 9 Testing the System production system vs. development system A:production system: B:development system: regression testing（回歸測試） - a test applied to a new version( verify the correctness of the old and new functions) example: version m-version m+1 (P

16、425) ( test new version by old and new test cases )18 Chapter 9 Testing the SystemChange Control (變更控制) A: note: system testing -find faults-correct faults-change system (under the control by configuration management team) B: problem caused by the change X: change any part of software-effect all doc

17、uments and all developers Y: more developers make change-effect each other C: change control: 配置管理小組采取措施使不同開發(fā) 者各自修改后的版本統(tǒng)一為一個版本.(使開發(fā)者不 同時修改軟件同一部分, 或采取其他額外步驟) 19 Chapter 9 Testing the System4. Test team focus on: A: unit and integration test-large role is developers B: function and performance test-la

18、rge role is developers C: acceptance and installation test large role is customers D: programmers cant participate the testing of his own modules 20 Chapter 9 Testing the System several types (about testers) A: professional testers (who organize and run the tests, include former analysts, programmer

19、s, and designers) (P429) B: analysts: ( who created requirement ) C: system designers (who understand SRS and propose solution) D: configuration management specialists (who controls the changing of software) E: users: to evaluate issues that arise 21 Chapter 9 Testing the System9.2 Function Testing

20、Focus on: A: function testing -black box B: function testing -based on 1.Purpose and Roles（目的和作用）function testing : test for functional requirement ( of ) role: have a high probability of finding a fault (because one function testing only deal with the small set of components) example : water-monito

21、ring system -acknowledging change in dissolved oxygen, temperature, acidity, radioactivity.22 Chapter 9 Testing the System guidelines for function testing (P4316 dots) A: 較高的查錯概率 B: 獨立的測試團隊 C: 了解預期的輸出結果 D: 對合法與非法的輸入都予以測試（假設是弱健壯等價類） E: 不能僅僅為了測試的方便而修改系統(tǒng) F: 停止測試應該有前提條件 備注 A: 功能測試是在謹慎的受控制狀態(tài)下進行的； B: 由于一次

22、測試一個功能，若需要，功能測試 可以早于整個系統(tǒng)的集成來進行.(重要功能) developing test cases ( according ) example : word processing system ( P431) 23 Chapter 9 Testing the System2. Cause-and-Effect Graphs (因果圖) 說明: A: 功能測試的測試用例是由需求定義的功能說明部分 轉化而來. B: 測試用例不應產(chǎn)生冗余. (the test cases should not redundant) C:測試用例應能發(fā)現(xiàn)需求中不完整和不明確的方面. 定義: 需求中

23、的INPUT稱為原因,OUTPUT稱為結果.反 映這種因果關系的布爾邏輯圖稱為因果圖. 基于因果圖的測試用例產(chǎn)生過程: A: 由布爾邏輯和約束條件產(chǎn)生因果圖. B: 將因果圖轉化為判定表. (decision table)24 Chapter 9 Testing the System C: 由判定表產(chǎn)生測試用例. 產(chǎn)生因果圖的步驟: A: 將需求分解為各個單獨的功能. (separated functions) B: 每個功能分出原因和結果. (cause + effect) C: 中間節(jié)點 (extra nodes) 的產(chǎn)生. Example: water-level monitoring

24、system A: analysis: requirement: ”the system sends a message to the dam operator about the safety of lake level” Input: syntax of the function: LEVEL(A,B) “A”height in meters of the water “B”-the number of centimeters of rain 25 Chapter 9 Testing the System PROCESSING: The function calculates whethe

25、r the water level is within a safe range, is too high, or is too low. output: result of level(A,B): The screen shows one of the following messages:“LEVEL = SAFE”, when the result is safe or low. “LEVEL = HIGH”, when the result is high.“INVALID SYNTAX”depending on the result of the calculation.26 Cha

26、pter 9 Testing the System B: cause-and-effect graph causes: 5 (P399) 1.The first five characters of the command “LEVEL” 2.The command contains exactly two parameters separated by a comma and enclosed in parentheses. 3.The parameters A and B are real numbers such that the water level is calculated to

27、 be LOW 4.The parameters A and B are real numbers such that the water level is calculated to be SAFE. 5.The parameters A and B are real numbers such that the water level is calculated to be HIGH.27 Chapter 9 Testing the System effects: 3 (P399) 1.The message “LEVEL = SAFE” is displayed on the screen

28、. 2.The message “LEVEL = HIGH” is displayed on the screen. 3.The message “LNVALID SYNTAX” is printed out intermediate nodes: 2 (P399) 1.The command is syntactically valid. 2.The operands are syntactically valid. C-and-E: - fig9.5 (P399) C: decision table: table9.2 D: advantage of C-and-E disadvantag

29、e of C-and-E 28 Chapter 9 Testing the System29 Chapter 9 Testing the SystemTable 9.2. Decision table for cause-and-effect graph.Test 1Test 2Test 3Test 4Test 5Cause 1IIISICause 2IIIXSCause 3ISSXXCause 4SISXXCause 5SSIXXEffect 1PPAAAEffect 2AAPAAEffect 3AAAPP30 Chapter 9 Testing the System9.3 Performa

30、nce TestingFocus on: performance test nonfunctional addressesrequirement(in SRS)1.Purpose and Roles performance testing : test for nonfunctional requirement ( of ) example : calculate the trajectory of a rocket (the speed of response, accuracy, data accessibility, etc.) note : A: performance testing

31、(by a team) get result give to developer and customer B: hardware engineer test team 31 Chapter 9 Testing the System2. Types of Performance testsseveral types of performance tests A: stress tests（強度測試） (在短時間內(nèi)加載極限負荷, 以驗證系統(tǒng)能力) B: volume tests（巨量數(shù)據(jù)測試 / 容量測試） (驗證系統(tǒng)處理巨量數(shù)據(jù)的能力) C: configuration tests（配置測試）

32、 D: compatibility tests（兼容性測試） E: regression tests (by old and new test cases) F: security tests ( security requirement ) G: timing tests H: environment tests 32 Chapter 9 Testing the System I: quality tests J: recovery tests K: human factors test ( use interface-usability) focus on: A: performance

33、test -more difficult than function test B: requirement must be explicit to ensure performance test C: requirement quality is often been reflected in the ease of performance testing (需求質量通?？梢苑从吃谛阅軠y試的容易度上)33 Chapter 9 Testing the System9.4 Reliability,Availability,Maintainability （可靠性，可用性，可維護性） (關于性能測

34、試) focus:A: performance test-Reliability, Availability, Maintainability-is critical issues B: it is difficult to measure directly before delivery1. Definitions software reliability : ( probability : 0 1 ) -軟件系統(tǒng)在給定的時間間隔和給定條件下運行 成功的概率 availability: ( 0/1 ) -軟件系統(tǒng)在給定的時間點（按SRS要求）成功 運行的概率34 Chapter 9 Test

35、ing the System maintainability: ( 0 1 ) -是指在給定的使用條件（預定的時間 間隔、維護程序、維護資源之下進行維 護）下，維護活動能被執(zhí)行的概率2. Four different levels of failure severity(嚴重性) A: reason: because reliability, availability, and maintainability are defined in terms of failures, they must be measured once the system is complete and worki

36、ng B: four different levels ( P438) 3. Failure data : 根據(jù)失效時間記錄，來理解系統(tǒng)的不確定性， 以圖改進. 或者以此理解系統(tǒng)可靠性的改進程度.35 Chapter 9 Testing the System9.5 Acceptance Testing（確認/驗收測試）1. Purpose and roles acceptance testing: customer check if the system meet the need in requirement definition roles: customer -play large ro

37、les ( in acceptance test，customer write, conduct, and evaluate the test result) developers-play small roles (answer or explain technical questions when the customer requests) 36 Chapter 9 Testing the System2. Types of Acceptance tests benchmark test（基準測試） ( formal test ) A: customer prepare test cas

38、es (with formal cases) B: install on experimental basis C: customer evaluate the performance pilot test（引導測試） (informal test) A: user/developer install on a experimental basis B: pilot test rely on daily working(no formal cases) alpha test: pilot test in organization (by developer) beta test: customers pilot (in workin