Conformance Testing and TTCN 工研院無線通訊技術部林牧台 / Morton Lin 03-5912360 mtlin@itri.org.tw 1
Outline Introduction and Terminology Conformance Testing Process 3GPP conformance testing and test cases A real world example Test generation Test realization TTCN Introduction Overview part Declaration part Constraint part Dynamic Part Test execution 2
Introduction (1) A protocol describes the rules with which computer systems have to comply in their communication with other computer systems. How to make sure that these implementations really behave according to these standards protocol specifications, i.e. conform to these standards? An implementation of a protocol entity is tested with respect to its specification. The aim is to gain confidence that during normal use the system will work satisfactory. To Certify the implementation with respect to the standard. 3
Software testing Introduction (2) Structural testing = white-box testing Based on the internal structure of a computer program. The aim is to exercise thoroughly the program code. e.g. by executing each statement at least once, or trying to execute all paths through the program code. Structure tests are derived from the program code. Functional testing = black-box testing Testing the externally observed functionality of a program based on its specification. No reference is made to the internal structure of the program. Main goal is to determine whether the right (with respect to the specification) product has been built. Functional tests are derived from the specification. 4
Conformance Testing A kind of functional testing An implementation of a protocol entity is solely tested for conformance with respect to the requirements given in its specification The process of testing the extent to which implementations of protocol entities adhere to the requirements stated in the relevant standard or specification Concerned with external behavior (black box) Prerequisites for interoperability The primary objective of conformance testing is to increasing the probability that different product implementations actually interoperate. Testing of implementations But NOT with performance, reliability, fault tolerance, efficiency, etc. 5
ISO/IEC 9646 The standard 6
OSI Terminology Layer S Layer S Session layer Layer T Service provided PDU Layer T SP SAP SP Transport layer Service used Layer N Layer N Network layer SP : Service Primitive SAP : Service Access Point PDU : Protocol Data Unit 7
Conformance Testing Terminology Upper Tester ASP ASP PCO PDU IUT ASP: Abstract Service Primitive ASP ASP PCO PCO: Point of Control and Observation IUT: Implementation Under Test Lower Tester PDU: Protocol Data Unit Tester: Test script written in TTCN The PCO has two FIFO queues: Send (from tester to IUT) Receive (by tester from IUT ) TTCN: Tree and Tabular Combined Notation (TTCN-2) 8
Conformance Testing Process Standard Protocol Specification Test generation Abstract Test Case TTCN Test realization Protocol implementation Executable Test Case C System System Under Under Test Test IUT Tester Tester Test execution Verdict (result) : accept failure inconclusive error (in test HW) 9
Test Generation Standard TPs Test Purposes (in English) Abstract Test Suite (in TTCN) ATS Protocol Implementation Conformance Statement (tables) PICS Protocol Implementation extra Information for Testing (tables) PIXIT 10
Test realization ATS is non-executable test suite. has to be converted into ETS. ATS is specified independently of any real testing device. Test selection: the tests relevant to the IUT are selected based on the PICS Also information about the IUT and its environment must be supplied: PIXIT Use tools to convert the ATS in TTCN to C executable code. The C code generated has to be adapted based upon the target environment. 11
Outline Introduction and Terminology Conformance Testing Process 3GPP conformance testing and test cases A real world example Test generation Test realization TTCN Introduction Overview part Declaration part Constraint part Dynamic Part Test execution 12