自動化測試框架

1、ITF Intelligent Test Framework for SCOS-2000 Executive SummaryITF - StudyIntelligent Test Frameworkfor SCOS-2000PurposeNowadays testers are confronted with the challenges of increasing software complexity and shortening development cycles. Without proper testing methods and tools it has become impos

2、sible to achieve and maintain high software quality standards.The overall goal of the ITF study was to define and implement (prototype an approach to automated testing of SCOS 2000 and similar data systems in ESOC's ground segment, based on requirements defined in the ITF Statement of Work.Metho

3、ds for Test AutomationAutomated GUI Testing and Model TestingAutomated testing is based on appropriate tools, which allow stimulating the system under test (SUT without manual interaction. These tools can be categorized under the term test drivers.GUI test drivers simulate a human user by communicat

4、ing with the tested application over its user interface. Mouse and keyboard input is simulated by sending appropriate messages to the GUI manager. In addition, the GUI test driver tries to determine the results of the test by querying the GUI manager about the current status of the GUI objects. Mode

5、l-oriented testing gives focus on the business model of an application. In a multi-tier architecture access to the business model can be performed via API interfaces like CORBA, RMI, DCOM or sending requests using a network protocol like TCP/IP. Similar to GUI testing, the capabilities of model-orie

6、nted test drivers depend on the range of functions that can be accessed over the interface.Test PhasesThe following diagram shows an overview of the relations between the project documentation phases and the corresponding testing activities throughout the software life cycle (based on the guidelines

7、 of the International Software Testing Qualifications Board, ISTQB: Unit testing is performed before the integration of several modules into a complex system. The aim of the integration test is to verify that the interfaces between the modules work correctly. The system test verifies the functionali

8、ty of the completely assembled system. At this stage, the focus no longer lies on the internal structure of the system, but on the functions defined in the software requirements specification. The most common form of system testing is done via the user interface. Acceptance testing is a special form

9、 of System Testing, where the user requirements are focused.Each of the test activities can be divided into a test preparation phase and a test execution phase. The aim of the test preparation phase is the creation of test plans while the tests based on these test plans are performed during the test

10、 execution phase.Test Tool and Test Language EvaluationOne part of the ITF study was an analysis of test automation tools (test drivers with a main focus on GUI automation tools. The outcome of this analysis is that object-level capture/replay strictly required (as opposed to simple analogue, pixel-

11、oriented capturing. This in turn requires that the GUI toolkit is supported by at least one GUI automation tool. Another important point in the analysis was the evaluation of tools and languages, which can be used to define automated test cases in general. The main result of this analysis is that th

12、e nature of the test interfaces (these are the interfaces exposed by the System Under Test determine what test drivers are suitable. The choice of the test driver, in turn, determines the test language in the majority of cases. There is no "silver bullet" test tool that fulfils all needs,

13、and consequently, there is no universal test language that covers all test scenarios. For frequent testing requirements, the following test drivers have been identified:Test Interface Test Driver Test Languages CORBAOmniPy combatPython TclTTY-based HMIexpect pexpectTcl or Pythonsystem/bin/sh /bin/cs

14、h Bourne shell, C shell HTTP service WGET Shell commandTCP/IP socket Custom TD Tcl, Python, C+ Java/Swing GUIWinRunner SilkTestTSL 4Testwithout test tool supportScripting extension JavaScript,. and/or schedule files PLUTO engine PLUTOCheck-Out procedures TOPEPLUTO, Tcl/TOPE,. DatabaseDatabase Driver

15、SQLThe table above gives an overview of the test tools and test languages, which have been analyzed. From this tools and languages survey it became clear that a generic test framework that shall support a heterogeneous system under test and various classes of tests must not be restricted towards a s

16、ingle test tool or test language. Instead, the overall testing approach needs to be open and flexible, so that different tools can be integrated.Technical Solution - ITFIn the course of the study, the "Intelligent Test Framework" (ITF has been developed based on the tool survey, analysis o

17、f requirements, and relevant standards. The ITF focus on automated system testing and it consists of the following components: The ITF Framework has a clear separation between test preparation and test execution on the one hand and between an abstract level and an implementation specific level on th

18、e other hand.Test Preparation EnvironmentThe test preparation environment of ITF is adopting Siemens IDATG (Integrating Design and Automated Test Generation as a Test Specification and Generation Tool. The tool supports the specification of test cases on 3 abstraction levels:Level 1 Task Model: The

19、hierarchical task model is based on the functional requirements (use cases of a software requirements specification. Decomposition in sub-tasks is used to improve the clearness and maintainability.Level 2 Sequence Diagrams: The sequence of tests steps for each task can be defined in the Task Flow Ed

20、itor: Each step may either represent an atomic step (cyan or an entire sub task flow (yellow. This concept allows re-using sub task, which minimizes the effort for test maintenance.Level 3 Step Descriptions: A step dialog allows entering a Test Instruction that will be executed by the test execution

21、 environment. Typically, such a Test Instruction will invoke a Test Driver with the argument list required by this Test Driver.Test Execution EnvironmentThe test execution environment consists of the following components: The Test Commander is the central instance that executes test schedules that a

22、re loaded from test schedule files in XML format. Each test schedule consists of test cases which in turn consist of test steps: ITF Intelligent Test Framework for SCOS-2000 Executive Summary The Test Commander allows step-by-step execution, automatic execution, and breakpoints. Unlike usual program

23、 debuggers, the Test Commander supports that the execution cursor can be moved manually forwards or backwards. Test Drivers are the interceptors between the Test Commander and the SUT. In this sense the test drivers hide the SUT's interfaces from the Test Commander and the Test Commander does an

24、d shall not care about the implementation details of these interfaces. A compact subset of instructions from the Test Commander is used to communicate with test drivers for the provision of parameters to the test driver and for the reply of execution status back from the test driver to the Test Comm

25、ander. A server side C+ framework has been developed in ITF due to the lack of commercial test drivers for the ILOGviews based SCOS-2000 GUI applications. This framework allows embedded GUI scripting with remote access via TCP/IP: SCOS-2000 Application (SUT TestCommander TELNET Client TCP/IP Interfa

26、ce TCL Interpreter EGOS-MCS-ITF-ESUM-0001 Page 6 / 6 2005-05-31 ITF Intelligent Test Framework for SCOS-2000 Executive Summary Conclusion The ITF study has prepared a way to move from manual SCOS-2000 testing to automated system testing. Concepts therefore are clear and stable, and the implementation of the concept is available on a prototype base. The generic components of ITF are nearly complete, but the roll-out of the concept requires a lot of application specific work to implement GUI application specific scrip