Microsoft Word - Percy Thesis_VCT_ Final.doc

Transcription

1 國立交通大學 電機學院與資訊學院資訊學程 碩士論文 數位機頂盒的影像品質研究 : 維持品質指標下改進測試生產力 Video Quality in Set-Top Box: Improving Test Productivity While Retaining Quality Index 研究生 : 劉百振 指導教授 : 林盈達 蔡文錦教授 中華民國九十五年四月 -1-

2 數位機頂盒的影像品質研究 : 維持品質指標下改進測試生產力 Video Quality in Set-Top Box: Improving Test Productivity While Retaining Quality Index 研究生 : 劉百振 指導教授 : 林盈達 蔡文錦 Student:Pai-Chen Liu Advisor:Ying-Dar Lin Wen-Jiin Tsai 國立交通大學電機學院與資訊學院資訊學程碩士論文 A Thesis Submitted to Degree Program of Electrical Engineering Computer Science College of Electrical Engineering and Computer Science National Chiao Tung University in Partial Fulfillment of the Requirements for the Degree of Master of Science in Computer Science June 2006 Hsinchu, Taiwan, Republic of China 中華民國九十五年四月 -2-

3 數位機頂盒的影像品質研究 : 維持品質指標下改進測試生產力 學生 : 劉百振 指導教授 : 林盈達 蔡文錦教授 國立交通大學電機學院與資訊學院資訊學程(研究所)碩士班 摘 要 在數位家庭與多媒體產品中, 數位機頂盒扮演著極為重要的溝通橋樑, 其中讓使用者能直接感受到產品性能優劣的第一印象, 莫過於影像與聲音的品質 然而一般影像測試時, 需要使用各種不同的測試圖形, 所以我們依據影像品質指標, 提出一種組合式圖形測試方法 : 把二十四種個別單一的圖形, 構成一種組合式圖形 另外, 在數位機頂盒系統品質測試中, 需要測試影音訊號輸出品質 數位壓縮的相容性與傳輸網路所造成的影響, 若要全部測試則高達 329,726,592 項, 故我們使用一台電腦控制了影音訊號源 數位壓縮設備 傳輸網路 影像聲音品質分析儀, 實作了自動化測試系統, 此一系統可以大大地縮短測試時間 綜觀上述之研究其優點 : 降低測試複雜性 縮短測試週期 提高產品競爭力進而大幅提升測試生產力 關鍵字 : 機頂盒, 機上盒, 影像, 品質, 自動化, 測試, 多媒體, 數位電視, 量測, 測試方法, 數位家庭 -3-

4 Video Quality in Set-Top Box: Improving Test Productivity While Retaining Quality Index Student:Pai-Chen Liu Advisors:Prof. Ying-Dar Lin Prof. Wen-Jiin Tsai Degree Program of Electrical Engineering Computer Science National Chiao Tung University ABSTRACT Among many multimedia products, digital set-top box plays a key role as consumer promise equipment. The key performance index for a set-top box appears to be its video and audio quality. The video quality is benchmarked by many different types of video patterns. In this thesis, we propose a Video Combination Technique (VCT) to integrate twenty-four video sources into a single one. Thus, the benefits of VCT include lower test complexity, lower test cycles and cost saving. From the system quality point of view, there are testing network transmission layer, compression layer and video/audio output signal layer. All the tests are sum up to be 329,726,592 items. Therefore, we implement an automated test measurement system (ATMS) to save the test cycles and time. So set-top box manufacturing cost is lower and productivity is improved. Keyword:set-top box, STB, video, quality, automated test, test, multimedia, digital TV, measurement, test methodology, digital home -4-

5 誌 謝 組合式圖形的靈感來自孩子們的樂高積木, 看著他們將一堆分散的積木, 經由他們的巧手它可以變成老鷹 城堡 汽車 直昇機 蝴蝶 翼手龍 甚至更多我們想像不到的創意 這些立體造型不禁讓我聯想到工作上影像測試所使用的二十四種單一圖形, 如果可以整合成一張圖形將可獲得事半功倍的效果, 經過不斷地實驗, 終於獲得成果展現並取得專利權 而在數位電視產業多年工作經驗中, 都是使用人工來量測眾多的測試項目以驗證系統品質, 因此衍生一套自動化量測系統, 用來控制影像聲音的測試 數位壓縮設備, 大大提昇工作上的效率, 感謝目前任職的鴻海精密工業股份有限公司新竹園區分公司提供實驗中所有設備 承蒙太太鄭梅芬女士的支持與孩子們濬磊 羿欣 玟欣的體諒, 讓我能夠充分利用上班之餘, 進入嚮往已久的交通大學, 尋求新知增進見識 ; 更感謝林盈達教授與蔡文錦教授的傾囊相授大力指導, 讓我能在專業的領域裡更上層樓 最後, 再次感謝所有幫助過我的人! -5-

6 Content LIST OF FIGURES...7 LIST OF TABLES...8 CHAPTER 1. INTRODUCTION...9 CHAPTER 2. BACKGROUND SET-TOP BOX CATEGORY OVERVIEW SET-TOP BOX SYSTEM TOPOLOGY LAYERS OF TEST ITEM VIDEO QUALITY INDEX COMPOSITE VIDEO...21 CHAPTER 3. PROBLEM STATEMENT COMPLEX VIDEO PATTERNS MPEG SIDE EFFECT A LARGE NUMBER OF TEST CASES...30 CHAPTER 4. OPERATION MODEL VIDEO COMBINATION TECHNIQUE (VCT) AUTOMATION TEST MEASUREMENT SYSTEM (ATMS)...36 CHAPTER 5. IMPLEMENTATION VCT Video Scanning Line Assignment Frame Editing Flow Chart Guard Video Scanning Line ATMS Controller PC Working Flow Chart Menu Tree of ATMS System Implementation...47 CHAPTER 6. EVALUATION...51 CHAPTER 7. CONCLUSIONS AND FUTURE WORKS...53 REFERENCES...54 APPENDIX - ACRONYMS DEFINITION

7 List OF Figures FIGURE 2-1 SET-TOP BOX CATEGORY FIGURE 2-2 SPTS TOPOLOGY FIGURE 2-3 MPTS TOPOLOGY FIGURE 2-4 IP SET-TOP BOX SYSTEM TOPOLOGY FIGURE 2-5 SIGNAL VS. TEST ITEM FIGURE 2-6 RS-170A DEFINED TIMING STANDARD FIGURE 2-7 FCC DEFINED TIMING STANDARD FIGURE 2-8 UV AND IQ VECTOR DIAGRAM FOR 75% COLOR BARS FIGURE 2-9 COMPOSITE VIDEO SIGNAL FOR 75% COLOR BARS FIGURE 3-1 MPEG ENCODING PROCESS FIGURE 3-2 VIDEO CODEC TEST CASES FIGURE 3-3 AUDIO CODEC TEST CASES FIGURE 3-4 AUDIO DOLBY CODEC TEST CASES FIGURE 4-1 VIDEO COMBINATION TECHNIQUE PATTERN FIGURE 4-2 AUTOMATION TEST SYSTEM TOPOLOGY FIGURE 5-1 TELEVISION INTERLACED SCANNING FIGURE 5-2 DETAILS OF NTSC SCANNING STANDARD FIGURE 5-3 FRAME EDITING FLOW CHART FIGURE 5-4 GUARD VIDEO SCANNING PATTERN FIGURE 5-5 CONTROLLER PC WORKING FLOW FIGURE 5-6 MENU TREE OF ATMS FIGURE 6-1 VCT EVALUATION TEST TOPOLOGY

8 LIST OF TABLES TABLE 2-1 VIDEO QUALITY INDEX TABLE 2-2 SUMMARY OF LINEAR DISTORTIONS AND TEST METHODS TABLE 2-3 SUMMARY OF NONLINEAR DISTORTIONS AND TEST METHODS TABLE 2-4 SUMMARY OF NOISE AND TEST METHODS TABLE 2-5 NTSC 75% COLOR BARS SIGNAL WAVEFORM CHARACTERISTICS TABLE 3-1 VIDEO SOURCES OUTLOOK AND WAVEFORM TABLE 3-2 SET-TOP BOX TEST CASES SUMMARY TABLE 3-3 TEST CASES CONSUME YEARS TABLE 4-1 VCT PATTERN ARRANGEMENT TABLE 5-1 NTSC SYSTEM VCT ASSIGNMENT TABLE 5-2 GUARD VIDEO SCANNING PATTERN LINES ARRANGEMENT TABLE 5-3 GUARD VIDEO SCANNING LINE COLOR VECTOR PERFORMANCE TABLE 5-4 ATMS EQUIPMENT LIST TABLE 5-5 ATMS IMPLEMENTATION MENU TABLE 6-1 GENERIC PATTERN AND VCT PATTERN COMPARISON

9 Chapter 1. Introduction Digital Set-top Box (STB) presents new opportunities for network service providers to deliver revenue-generating home entertainment services. Users can enjoy video and music, browsing over Internet, playing games, and using services all through a single interface to a television provided by an STB. Set-top boxes can receive signals through the cable, terrestrial or satellite. The bi-directional IP infrastructure enables inherent support for a broader range of applications and interactive services.[1] Everyone knows what the video is it is another definition for the television. Televisions have been around in our homes; it is an integral part of modern life.[24] What is the key performance index (KPI) of a digital home product? It appears to be its video quality.[4] The quality index contains amplitude, timing, linear distortion, nonlinear distortion and noise. The quality index covers all test items and the measurement. This thesis discusses the test methodology of video quality. Since the video quality is benchmarked by many different types of video test patterns, how to simplify these complicated video sources will be an issue. We propose a Video Combination Technique (VCT) to simplify the video test pattern. The VCT combines twenty-four video sources into one. The VCT is now in patent pending. The benefits of VCT include lower test complexity, lower test cycles and cost saving. Thus, it improves the test productivity while retaining quality index. There are many test cases for system quality of STBs. For example, the numbers of video and audio codecs are 716 and 371, respectively. So we implement the automated test measurement system (ATMS) to save the test cycles and time. The ATMS consists of the video source, audio source, encoder, transmission network, video analyzer and audio analyzer. The ATMS can be applied to all STB products. -9-

10 The rest of this thesis is organized as follows. Chapter 2 describes the background knowledge. The problem statement is presented in Chapter 3. In Chapter 4, the operation model is explained. Chapter 5 gives the details of implementation. And finally, we evaluate VCT in chapter 6 and Chapter 7 concludes this work. -10-

11 Chapter 2. Background Before the problem statement, there are some background knowledge to be introduced first. The first one is STB connection overview, and leads all kinds of STB categories. The second one will have a system overview. Due to the data rate difference, digital broadcasting system has two types like single program transport stream (SPTS) and multiple program transport stream (MPTS). There is a system model discussing from signal layer, compression layer and transmission layer. Finally, it will talk about video methodology and their quality index. 2.1 Set-Top Box Category Overview There are some kinds of STB categories as shown in Figure 2-1. The STB receives the triple-play digital multimedia contents from kinds of transmission media then decodes to sorts of services and networking. The transmission media can be different types like cable, terrestrial, satellite and Ethernet. The worldwide digital TV standards include DVB-C, DVB-T, DVB-S, ISDB and ATSC. Figure 2-1 Set-Top Box Category -11-

12 2.2 Set-Top Box System Topology In order to convert an analog television to a digital one, the analog television signal inputs into an encoder and comes out a digital stream. One type of encoding that is dominant right now is called MPEG2, and the encoded digital streams are called MPEG2 Transport streams. Transport streams can be either single or multiple. A single program transport stream (SPTS) refers to Figure 2-2, like those coming out of an encoder contains only one program channel of information. For example, if you play the movie "Gone With The Wind" into an encoder, a single program transport stream (SPTS) of digital bits would come out of the encoder. This stream can go into a decoder, in order to convert the digital stream back into pictures that can be seen on a normal television.[2] Figure 2-2 SPTS Topology Figure 2-3 illustrates the topology that explains the benefit of digital television with many of these SPTS combined together into one multiple program transport stream (MPTS). This combining process is done by a multiplexer. The input to a multiplexer is several SPTS. The output of a multiplexer is one MPTS. For example, if the movies like "Gone With The Wind", "Wizard of Oz", "The Terminator", and "Star Wars" are all encoded and multiplexed together, one MPTS would carry all four of the digital television programs. This stream goes into a decoder to convert any one of the -12-

13 programs back to television pictures.[2] Figure 2-3 MPTS Topology As the video production and broadcast industry makes the transition from analog to digital video, the facilities will contain a mixture of both analog and digital video equipment. In order to verify video performance in a mixed analog and digital video transmission link, the system should account for the following categories. [3] As STB is a home user product, it requires a digital headend to distribute multimedia contents over the transmission media to home users. For the system topology, we divide them into four panels like data, network and transmission, control and test Panels. The IP STB system of topology can divide into four panels in Figure 2-4. Data Panel: Member: Satellite IRD, Terrestrial IRD, Video Server, DVD + Encoder Objective: Acquire digital contents or encoding analog A/V source to MPEG-2-13-

14 transport stream Control Panel: Member: Middleware Server, Web Server, TFTP Server, DHCP server Objective: STB user interface, EPG, Database, Subscriber Management System, Billing System Test Panel: Member: Video Source Generator, Audio Source Generator, Video Analyzer, Audio Analyzer Objective: Video/Audio Quality Measurement Networking and Transmission Panel: Member: TS/IP Gateway, Ethernet Switch, Ethernet Router, DSLAM Figure 2-4 IP Set-Top Box System Topology -14-

15 2.3 Layers of Test Item Three key testing layers can be defined as the modern television system as shown in Figure 2-5.[4] The layers simply separate the standard into three consecutive blocks that have defined interfaces. This does not only make it easier to understand the standard, but provides means for interoperability with other standard that may have equivalent layers. In broadcasting, the layers are traversed top-down, and in a receiver that is processed bottom-up. Thus, the top layer is Video Source for headend site and Video Output for receiver respectively. What we measure this layer is Video Quality. The second layer, compression, performs audio and video compression in transmission for encoder and decompression at the STB. This layer of the standard defines the usage of video and audio compression techniques. They are based on MPEG-2, MPEG-4, VC1 for video and MPEG-1 Layer 2, Dolby AC3 for audio. In the final layer, transmission, the composite data stream containing video, audio, and possibly other data are modulated into a carrier suitable for distribution by terrestrial broadcasting, satellite, cable, or Ethernet. The test item here is channel analysis.[24] Figure 2-5 Signal vs. Test Item -15-

16 2.4 Video Quality Index The overall of video quality is influenced at a number of index: amplitude, timing, linear distortion, nonlinear distortion and noise in Table 2-1. Index Description Influence How to measure NTSC defines Amplitude peak-to-peak amplitude 1 Too light or too dark 100 IRE white level volt (140 IRE) Horizontal and vertical Timing synchronization pulse widths fall within specified limits Picture breakup Any composite signal 12.5T sine-squared pulse Linear Distortion Incorrect color saturation, with 3.58 MHz Caused by imperfect color smearing, fuzzy modulation, T rise time transfer characteristics in vertical edges, brightness white bar, 18 us 100 IRE the signal path variation, flicker bar, window or field square wave Nonlinear Distortion Noise Crosstalk and intermodulation effects between luminance and chrominance High brightness areas colors not reproduced, Noise random or coherent from natural and Snowy, grainy, sparkles man-made sources Table 2-1 Video Quality Index Modulated staircase, unmodulated 5 step staircase, modulated pedestal Any line with constant level Amplitude Video amplitudes are most frequently measured in order to verify the conformity to nominal values. Measurement of the peak-to-peak amplitude is 1 voltage for NTSC systems. 1 IRE is defined into an absolute unit equal to 1/140 of 1 voltage. Signal amplitude can be measured with a waveform monitor. The amplitude of gain errors causes the picture to appear too light or too dark.[22] Timing Both RS-170A and the FCC provide the recommended limits for these timing parameters. However, the two standards have different definitions for various time -16-

17 intervals. FCC specifies sync width between the 90% points of the two transitions as shown in Figure 2-6. RS-170A specifies sync width between the 50% points as shown in Figure 2-5. The RS-170A requirements are generally more stringent.[22] Figure 2-6 RS-170A Defined Timing Standard Figure 2-7 FCC Defined Timing Standard -17-

18 Linear Distortions All systems require a specific bandwidth to transmit all significant luminance and chrominance information. Table 2-2 summarizes significant linear distortions and gives a short description of the test concept.[8] Parameter Definition Test signal Test method Gain versus Gain variation over Multiburst or sweep Measure signal level at frequency band from 500kHz to selected frequencies in db response maximum frequency wrt level at 500kHz Long-time Damped signal-level Any test signal Measure overshoot peak distortion oscillations resulting allowing a controlled as % of peak luminance from sudden change in average level picture-level changes picture level Field-time Change in shape of a Field-rate square Measure peak-to-peak tilt distortion field-time square wave wave of top of bar in % wrt centerer of bar excluding first and last 250 µs Line-time Change in shape of a Line-rate square Measure peak-to-peak tilt distortion Line-time square wave wave of top of bar in % wrt centerer of bar excluding first and last 1 µs K factor Quantified subjective 2T sine-squared pulse Read K rating in % on impairment rating Special graticule Chrominance to Change in ratio of Modulated Measure amplitude of luminance gain chrominance versus sine-squared pulse chrominance component luminance amplitude in % wrt the luminance component Chrominance to Change in timing of Modulated Measure delay of luminance delay chrominance versus sine-squared pulse chrominance component luminance in ns wrt to the luminance component Table 2-2 Summary of Linear Distortions and Test Methods Nonlinear Distortions -18-

19 Table 2-3 summaries and defines the nonlinear distortions and gives a short description of the measurement concept.[5] The most common measurable nonlinear distortions are luminance nonlinearity, differential gain and differential phase.[8] Parameter Definition Test signal Test method Luminance Output/input amplitude Luminance Measure % of largest nonlinearity proportionality change of a small staircase step amplitude variation unitstep function as the step level is shifted from blanking level to white lecel. The average picture level is kept constant. Chrominance Output/input Chrominance Modulated Measure % of steps 3 gain subcarrier amplitude pedestal and 1 chrominance nonlinearity proportionality change as the amplitude change with subcarrier amplitude is varied from reference to step 2 a minimum to a maximum specified value. Luminance and average picture levels are kept constant. Chrominance Chrominance subcarrier phase Modulated Measure largest phase phase variation when the subcarrier pedestal difference(in degrees) nonlinearity amplitude is varied from a of steps 3 or 2 with minimum to a maximum specified reference to step 1 value. Luminance and average picture levels are kept constant. Chrominance Variation of luminance signal Modulated Measure % of largest to luminance amplitude resulting from the pedestal luminance level change intermodula- superimposition of a specified due to chrominance tion amplitude chrominance signal. The level average picture level is kept constant. Differential Amplitude change of a constant Modulated Measure % of largest gain small-amplitude chrominance ramp or chrominance subcarrier subcarrier superimposed on a staircase level change with luminance signal level that changes reference to burst from blanking to white. The amplitude average picture level is kept constant. -19-

20 Differential Change in phase of a constant Modulated Measure largest chroma phase small-amplitude chrominance staircase phase change in degrees subcarrier without phase or ramp with reference to burst modulation superimposed on a phase luminance signal level that changes from blanking to white. The average picture level is kept constant. Table 2-3 Summary of Nonlinear Distortions and Test Methods Noise Table 2-4 summaries the types of noise encountered in studio environment and gives a short description of the test method.[23] The test signal for the measurement of noise is a flat field at the black level.[8] Parameter Definition Test signal Test method Continuous random noise Ratio, expressed in db, of the nominal amplitude of the luminance signal (714.3 or 700 mv) to the RMS amplitude of the noise in a frequency band extending from 10 khz to the upper frequency of the video band (4.2,5,5.5 or 6 MHz). Flat field Feed test signal to input of equipment under test and measure SNR at output with specialized RMS-reading instrument. Alternately, use an oscilloscope preceded by a band-limiting filter and synchronized to display single line. Hum Ratio, expressed in db, of the nominal amplitude of the luminance signal (714.3 or 700 mv) to the peak-to-peak amplitude of the noise after band-limiting to 10 khz. Flat field Feed signal to input of equipment under test and measure SNR at output with specialized peak-to-peak reading instrument. Alternately, use an oscilloscope synchronized to display a single field and measure peak-to-peak hum amplitude. Other Ratio, expressed in db, of the Flat field Feed test signal to equipment -20-

21 periodic nominal amplitude of the under test and use an noise luminance signal(714.3 or 700 oscilloscope to isolate, identify, mv) to the peak-to-peak and measure peak-to-peak amplitude of the noise in a amplitude of interfering single in frequency band extending the specified frequency band. from 1 khz to the upper frequency of the video band(4.2,5,5.5, or 6 MHz). Crosstalk Ratio, expressed in db, of the Sweep signal Feed sweep to unwanted channel nominal amplitude of the and flat field and flat field to wanted channel. luminance signal (714.3 or Normalize gains and measure 700 mv) to the peak-to-peak the peak-to-peak value of the amplitude of the interfering interfering signal at the wanted signal. channel output. Table 2-4 Summary of Noise and Test Methods 2.5 Composite Video The most television systems communicate by means of composite signal, which is a special format that contains all the information needed to convey a picture like brightness, color and synchronization on a single cable. [24] Luminance (Y) signal is derived by adding from red, green and blue together, but in a sum which is weighted by the relative response of the eye.[21] Thus : Y = 0.299R G B The UV and IQ vector diagram is shown in Figure 2-8. To transmit color information, U and V or I and Q color difference signals are used: U = 0.492(B - Y) V = 0.877(R - Y) Or I = 0.596R 0.275G 0.321B = V cos33 Q = 0.212R 0.532G B = V sin 33 U sin33 + U cos33-21-

22 Figure 2-8 UV and IQ Vector Diagram for 75% Color Bars I and Q (or U and V) are used to modulate a 3.58MHz color subcarrier using two balanced modulators operated in phase quadrature: one modulator is driven by the subcarrier at sine phase, the other is driven by the subcarrier at cosine phase. The outputs of the modulators are added together to form the modulated chrominance signal: C = Qsin( wt + 33 ) + I cos( wt + 33 ) where w = 2πFsc ; Fsc = MHz C = U sin wt + V cos wt With respect to Figure 2-8, a good video quality of color vector must locate on square for each vertex. We will adopt the color vector diagram to be our pass/fail criteria in Table 5-3. The modulated chrominance is added to the luminance information along with appropriate horizontal and vertical sync signals, blanking information, and color burst information, to generate the composite color video waveform shown in Figure

23 Figure 2-9 Composite Video Signal for 75% Color Bars Table 2.5 lists the details of the color bar signal luminance and chrominance values as well as the phase angles of the six colors. Luminance, Chrominance, Angle, Color IRE/mV IRE/mV degrees White 100/ / Gray 76.9/ / Yellow 69/ / Cyan 56.1/ / Green 48.2/ / Magenta 36.2/ / Red 28.2/ / Blue 15.4/ / Black 7.5/ / Burst 0/0 40/ Table 2-5 NTSC 75% Color Bars Signal Waveform Characteristics -23-

24 Chapter 3. Problem Statement While benchmarking the video quality of STB, we normally use different kinds of test patterns to measure the performance from video output. Depends on the specification s requirement, you should change the pattern source, and then measure the performance. For the video broadcasting system, the source is encoded by compression standard. We use MPEG compression as example to describe the problem. Finally, we summarize the test cases. 3.1 Complex Video Patterns Broadcasters traditionally rely on test patterns and images to check the signal quality throughout the signal path. These various formats produce different signal degradation and so naturally require different test material.[6] To measure these parameters, a series of video test signals are injected into the video transmission system. Then measure the response from the output of the transmission system with a video waveform monitor and a video chrominance vector scope.[3] Table 3-1 illustrates generic video source patterns. There are twenty-four patterns. Each video pattern on television is full screen. Each pattern is designed by their physical methodology to benchmark video performance. In Table 3-1, it shows the picture outlook and waveform. Name Picture Outlook Waveform Color Bar 75% -24-

25 IRE100 GreyWind Sin(x/x) RED100 Ramp PulseBar -25-

26 RBB 6MHzSweep IRE0 5 Step ColorBar 100% FCC Composite -26-

27 N7 Composite Mod 5 Step Mod 10 Step Multi-burst Mod Ramp 8MHz Sweep -27-

28 Mod Ped Multi-pulse Horzline IWQ Vertline Table 3-1 Video Sources Outlook and Waveform 3.2 MPEG Side Effect The starting point of the MPEG compression process is color space conversion and chroma sampling as shown in Figure 3-2 Perceptual redundancy is removed from the compressed video signal as a result of 4:2:0 chroma sampling. 4:2:0 chroma sampling -28-

29 is used by most MPEG encoders. By down-sampling from 4:4:4 (RGB) to 4:2:0 in YUV color space, 50% of the original RGB signal is lost but with minimal impact to the human eyes. The human eyes are very sensitive to the changes in luminance (Y), for this reason the luminance component is preserved at full resolution. The human eyes are much less sensitive to color changes that allow for the half-resolution color difference signals (U and V). Macro block formatting is the next stage in the compression process. Each macro block represents a 16x16 pixel area. This area is comprised of four 8x8 luminance blocks (Y) and two 8x8 chrominance blocks (U and V). The block served as the primary tool is used in removing temporal redundancy. When video is tested on a frame by frame basis, very little changes between adjacent frames. MPEG uses three frames types to remove temporal redundancy. These frames types are I, P, and B. Converting from the spatial domain to the frequency domain is performed by discrete cosine transformation (DCT). The 8x8 block of pixels is represented in the frequency domain by an 8x8 matrix of frequency coefficients. Quantization is the process of removing high frequency information. While the DCT process is completely reversible without loss, the quantization process is lossy. High frequencies are removed by applying a scale factor to the DCT coefficients. The final stage of the compression process involves run length encoding (RLE) and entropy encoding. The output of the compression process is a packetized elementary stream. The raw compressed data emerged from the RLE and entropy encoding stage is packed with MPEG header information. This information includes time stamps, start codes, quantization tables, and other information needed to generate a syntactically correct MPEG bit stream.[20] -29-

30 Figure 3-1 MPEG Encoding Process The side effect of MPEG encoding is the macro block formatting. While STB is decoding the bit stream and recovering to analog video output. The television has blocking effect. So we measure the video quality from the composite output signal that should consider the television scanning line with guard video. 3.3 A Large Number of Test Cases A robust STB should decode any kinds of compression parameters to meet customers requirements. The following Figure 3-3, 3-4, 3-5 represent Encoder s video and audio parameters. The video setting contains video standard, compression codec, chroma format, data rate, resolution and aspect ratio. The total test cases are

31 Figure 3-2 Video Codec Test Cases The audio settings of Encoder are compression codec, sound track, sample frequency and data rate. The total test cases are

32 Figure 3-3 Audio Codec Test Cases Figure 3-4 Audio Dolby Codec Test Cases The above test cases are encoding parts. For the system test cases, it covers from signal layer, compression and transmission as Table 3-2. Layer Interface Items Total Test Cases Signal CVBS NTSC/PAL 18 Amplitude Timing Linear Distortion Nonlinear Distortion Noise Stereo Output Voltage 6 Frequency Response THD+Noise Channel Separation Compression Audio Codec MPEG 533 Dolby AAC Video Codec MPEG-2 H

33 VC1 Transmission Ethernet Packet delay Both fixed and variable jitter Packet reordering Packet loss Both random and congestion-dependent Packet duplication Bandwidth limitation 8 Table 3-2 Set-Top Box Test Cases Summary If all of the test layers and test cases are considered as following Table 3-3. It will consume 329,726,592 test cases. Assume each test case takes 10 second duration. The time will be as long as years. It is impossible to do the test, so the most important thing is to do the partial test and automation. The advantage of automation saves the test cycles and time for hundreds of years. Layer Total Test Cases Test Duration Time Consume Signal (CVBS Video) sec 180 sec (Stereo Audio) 6 10 sec 60 sec Compression (Audio Codec) sec 5,330 sec (Video Codec) sec 7,160 sec Transmission 8 10 sec 80 sec Summary 329,726,592-16,486,329,600 sec ( Years) Table 3-3 Test Cases Consume Years -33-

34 Chapter 4. Operation Model Both of the video combination technique and automation test measurement system are discussed in this chapter. 4.1 Video Combination Technique (VCT) As there are so many video test patterns, the video combination technique is collecting 24 patterns into one video frame as shown in Figure 4-1. Figure 4-1 Video Combination Technique Pattern -34-

35 The specific unique patterns are assigned as Table 4-1. Each unique video source has 10 scanning lines. There are 24 specific patterns totally. The 24 patterns can apply to all of the STB s video performance test requirements. 1 Name 1 21 ~ 30 Bar ~ 40 IRE ~ 50 GreyWIND 4 51 ~ 60 Sin(x/x) 5 61 ~ 70 Red ~ 80 Ramp 7 81 ~ 90 Pulse Bar 8 91 ~ 100 RBB ~ 110 6MHz Sweep ~ 120 IRE ~ Step ~ 140 Bar ~ 150 FCC Comp ~ 160 N7 Comp ~ 170 Mod 5 Step ~ 180 Mod 10 Step ~ 190 Multi Burst ~ 200 Mod Ped ~ 210 8MHz Sweep ~ 220 Mod Red ~ 230 Multi Pulse ~ 240 Horizontal Line ~ 250 IWQ ~ 262 Vertical Line Table 4-1 VCT Pattern Arrangement -35-

36 4.2 Automation Test Measurement System (ATMS) The generic digital broadcast system contains signal source, compression, transmission network and STB, so the Automation Test Measurement System (ATMS) also follows the generic rules. The sources must have standard video source and audio source. The real-time encoder plays with different kinds of codec, bit rate, resolution. After the encoder, signal is digital bit stream then connects to transmission network. Depends on the transmission network, STB receives the signal then decodes to baseband video and audio signal to television. The controller PC is the key equipment. The controller PC connects to all equipment like Figure 4-2. The jobs of control PC are automatic changing video/audio source patterns, real-time encoder parameters, video analyzer and audio analyzer. With respect to equipment s interface, the control PC must have some interface like GPIB card, APIB card, Ethernet and infrared fixture. Figure 4-2 Automation Test System Topology -36-

37 Chapter 5. Implementation Here we implement two techniques. The first one is Video Combination Technique (VCT), and the other one is Automation Test Measurement System (ATMS). The VCT has 3 sections to describe video scanning line assignment, frame editing flow chart and guard video scanning line. Finally, ATMS implements a program to integrate all the test cases. 5.1 VCT Video Scanning Line Assignment As NTSC system has 525 scanning lines. Thus, each frame takes two vertical scans (fields), with the even and odd lines scanned on alternate fields as shown in Figure 5-1.[24] Figure 5-1 Television Interlaced Scanning The first one to 20 scanning lines are vertical blanking interval (VBI). VBI is not -37-

38 displayed by televisions. Figure 5-2 illustrates the logical of NTSC scanning standard, hence, the useful scanning lines start from 21 to 262 for odd field as summarized in Table 5-1. Figure 5-2 Details of NTSC Scanning Standard There are twenty-four video sources to be implemented by video combination technique. Therefore, each specific pattern requires 10 scanning lines. Each pattern scanning line = (262-21) / 24 = 10 Lines Useful Lines Field 1 (Odd) 1 ~ ~ 262 Field 2 (Even) 263 ~ ~ 525 Table 5-1 NTSC System VCT Assignment Frame Editing Flow Chart First of all, each video source must follow their equations as illustrated in Figure 5-3. Each scanning lines contain three parts: sync, color burst and white Level. Each of -38-

39 specific patterns then combines to a video frame as Table 4-1. While the combination frame is done then compiles the unique frame, we must assign the pattern s directory to be fetched out. Finally, download the file to video source generator. Operate the video source and play the video combination pattern. Figure 5-3 Frame Editing Flow Chart Guard Video Scanning Line According to chapter 5.1.1, each video contains 10 scanning lines logically. But the physically, we design a unique pattern as shown in Figure 5-3 to evaluate how many scanning lines are available. We do an evaluation pattern that only has IRE0 and color bar 75%. The scanning lines with color bar are from 1 to 9 group and step with 20 lines as shown in Table 5-2. The pattern passes through the digital broadcasting system as shown in Figure 2-4. We normally use color bar pattern to evaluate the picture quality as discussed in Chapter 2.5. Using video analyzer analysis the video quality then comes out the color vector result as shown in Table

40 Figure 5-4 Guard Video Scanning Pattern In Table 5-2, we define the guard video scanning lines. Due to the line limitation, we test to group nine. Name Field Field Total Lines IRE0 21 ~ ~ Color Bar 75% 42 ~ ~ IRE0 43 ~ ~ Color Bar 75% 64 ~ ~ IRE0 66 ~ ~ Color Bar 75% 87 ~ ~ IRE0 90 ~ ~ Color Bar 75% 111 ~ ~ IRE0 115 ~ ~ Color Bar 75% 136 ~ ~ IRE0 141 ~ ~ Color Bar 75% 162 ~ ~ IRE0 168 ~ ~ Color Bar 75% 189 ~ ~ IRE0 196 ~ ~ Color Bar 75% 217 ~ ~ IRE0 225 ~ ~ Color Bar 75% 246 ~ ~ IRE0 255 ~ ~ Table 5-2 Guard Video Scanning Pattern Lines Arrangement Table 5-3 describes the guard video scanning line result from group 1 to 9. We come out our observation. The background knowledge of color vector is discussed in Chapter 2.5. A good video quality of color vector must locate on square for each vertex. -40-

41 We conclude the test result while groups are below 3 then the video fails. All of top and bottom video are worse. After Group 7, the scanning line has 5 good quality of video. The Guard Video Scanning Line should have two scanning line spaces for each top and bottom video source. The video quality is good besides Guard Video Scanning Line. Scanning Line Line 42 (Group 1) Color Vector Measurement Result Line 42 Line64 ~ 65 (Group 2) Fail Line 64 Line 65 Line 87 ~ 89 (Group 3) Fail Fail Line 87 Line 88 Line 89 Fail Pass Fail -41-

42 Line 111 ~ 114 (Group 4) Line 111 Line 112 Line 113 Fail Pass Fail Line 114 Line 136 ~ 140 (Group 5) Fail Line 136 Line 137 Line 138 Fail Fail Pass Line 139 Line 140 Fail Fail -42-

43 Line 162 ~ 167 (Group 6) Line 162 Line 163 Line 164 Fail Fail Pass Line 165 Line 166 Line 167 Line 189 ~ 195 (Group 7) Pass Pass Fail Line 189 Line 190 Line 191 Fail Pass Pass Line 192 Line 193 Line 194 Pass Pass Pass -43-

44 Line 195 Line 217 ~ 224 (Group 8) Fail Line 217 Line 218 Line 219 Fail Pass Pass Line 220 Line 221 Line 222 Pass Pass Pass Line 223 Line 224 Fail Fail -44-

45 Line 246 ~ 254 (Group 9) Line 246 Line 247 Line 248 Fail Fail Pass Line 249 Line 250 Line 251 Pass Pass Pass Line 252 Line 253 Line 254 Pass Fail Fail Table 5-3 Guard Video Scanning Line Color Vector Performance 5.2 ATMS While STB has so many test cases, it is essential to implement the Automation Test Measurement System (ATMS). All components of the measurement systems allow for a remote control operation.[25] The controller PC will be the control center for all equipment. According to test cases and specification, the test engineer can define the criteria and limitation. Then everything is automatic. ATMS improves the test -45-

46 productivity to hundreds of year as shown in Table Controller PC Working Flow Chart ATMS connects video source, video analyzer, audio source, audio analyzer, encoder and device under test unit as shown in Figure 5-4. First of all, ATMS must communicate with these equipments. The next step will be parameter setting. The settings depend on customer s requirements, run the program, and then generate the test report. Figure 5-5 Controller PC Working Flow Menu Tree of ATMS The ATMS of menu hierarchy is as Figure 5-5. In the beginning, we must check the device setup, and then set the test items. Finally, ATMS will record the test results. Device Setup menu will configure the TG2000, AP2700, MV100 and VM700. So the -46-

47 program should make sure every thing connected. Figure 5-6 Menu Tree of ATMS System Implementation While implementing the ATMS, it requires many professional equipment like Table 5-4. Depends on equipment interface, the controller PC is also required to setup the same interface as well. We use Borland C++ Builder for programming. The functionality of program is Table 5-5 for details. Item Name Function Control Interface 1 Video Source Generate Video Patterns GPIB 2 Audio Source Generate Audio Patterns APIB (GPIB like) 3 Encoder MPEG-2 Realtime Encoder MIB 4 IP Gateway Transcode MPEG-2 To IP MIB 5 Cable Modulator Digital QAM Modulation MIB 6 Satellite Modulator Digital QPSK Modulation MIB 7 Terrestrial Modulator Digital COFDM Modulation RS232 8 Video Analyzer Analyze Video Performance GPIB 9 Audio Analyzer Analyze Audio Performance APIB 10 PC Control All Equipments 11 Router IP Multicast Router Ethernet 12 NIST Net WAN Emulator Table 5-4 ATMS Equipment List -47-

48 Name Objective Menu Device Setup TG2000 Connecting to video source generator Device Setup AP2700 Connecting to audio source generator Device Setup VM700T Connecting to video analyzer -48-

49 Test Item Video Codec Test Cases Setting Connecting to real-time encoder to setup video compression parameters Test Item Audio Codec Test Cases Setting Connecting to real-time encoder to setup audio compression parameters Test Item Device Under Test Unit Measure set-top box CVBS output quality performance Video Output Performance Measurement -49-

50 Test Item Device Under Test Unit Measure STB stereo output quality performance Audio Output Performance Measurement Test Log According test cases generating test report automatically Table 5-5 ATMS Implementation Menu -50-

51 Chapter 6. Evaluation With respect to Figure 6-1, we setup a VCT evaluation test environment. The video source is directly connecting to video analyzer without any encoding and decoding equipment. The video source plays generic video pattern and VCT pattern as shown in Table 3-1 and Figure 4-3 then we record the values as shown in Table 6-1. Figure 6-1 VCT Evaluation Test Topology With respect to video quality index, we measure amplitude, timing, linear distortions, nonlinear distortions and noise. Each quality index defines their test items. The comparison table totally has 33 items are matching, and the other 1 items are in machine tolerance. So it proves the Video Combination Technique is reliable and robust in the video source. No. Video Quality Index Measure Item Video Analyzer Test Item Unit Scanning Line Generic Pattern VCT Pattern 1 Bar Line Time Sync Level F1L IRE 39.8 IRE 2 Bar Line Time Sync/Bar Ratio F1L % 99.80% 3 H Timing (RS-170A) Burst Cycles F1L cycles 9.0 cycles 4 H Timing (RS-170A) Burst Level F1L IRE 39.1 IRE Amplitude Time Measurement 5 & H Timing (RS-170A) Sync Cycles F1L us 4.70 us Time 6 H Timing (FCC) Burst Cycles F1L cycles 9.0 cycles 7 H Timing (FCC) Burst Level F1L IRE 39.1 IRE 8 9 H Timing (FCC) Sync Cycles F1L us 4.84 us SCH Phase SCH_Phase SCH Phase F1L deg 1.8 deg -51-

52 Chrominance-to-Luminance Gain & Delay Short Time Distortion Chrom/Lum Gain Delay Chrom/Lum Gain Delay Short Time Distortion Short Time Distortion Chroma Gain F1L % 97.90% Chroma Delay F1L ns 1.0 ns Rising Edge F1L % SD 0.4% SD Rise Time F1L ns ns 14 Bar Line Time LineTime Dist. F1L % 0.10% 15 Line Time Distortion Bar Line Time Bar tilt F1L % 0.00% 16 Bar Line Time Bar Width F1L us 18.0 us 17 MultiBurst 0.5MHz F1L db MultiBurst 1.0MHz F1L db MultiBurst 2.0MHz F1L db Linear 20 MultiBurst 3.0MHz F1L db Distrotions Frequency Response 21 MultiBurst 3.58MHz F1L db MultiBurst 4.2MHz F1L db Amplitude at GroupDelay SinX_X 0.20MHz F1L db Group Delay at GroupDelay SinX_X 0.20MHz F1L55-1 ns -1 ns Amplitude at 25 GroupDelay SinX_X F1L db MHz Group Delay Group Delay at 26 GroupDelay SinX_X F1L55-1 ns -1 ns 0.20MHz 27 K_Factor K-2T F1L % 0.3% 28 K_Factor K-PB F1L % -0.3% K Factor Ratings 29 K_Factor PB Ratio F1L % 98.8% 30 K_Factor HAD F1L ns ns 31 Differential Phase DGDP pk-pk F1L deg 0.09 deg 32 Nonlinear Distrotions Differential Gain DGDP pk-pk F1L % Luminance NonLinearity Luminance NonLinearity pk-pk F1L % 0.3% 34 Noise Measurement Signal-to-Noise Ratio Noise Spectrum Noise 15KHz to 5.0MHz F1L dbrms dbrms Table 6-1 Generic Pattern and VCT Pattern Comparison -52-

53 Chapter 7. Conclusions and Future Works As the generic video test methodology is to display the source in turns. Our approach VCT shows the pattern one time and at least 24 patterns per frame. The benefit of VCT will save time for 24 times less than the one of traditional methodology. The operation complexity is also lower than generic one. If we estimate the shipping quantity is one million per year and use six kinds of video sources for test cases, the operation costs two NT dollars per minute. Therefore, the generic method costs twelve million NT dollars. And the VCT approach costs two million NT dollars. As a result, the VCT approach could save up to ten million NT dollars per year. The concept of guard video improves the accuracy of video measurement. Due to broadcasting system, it is essential to choose video compression standard. The side effect of MPEG compression downgrades the VCT performance between the adjacent patterns. While the pattern of VCT is over ten scanning lines, we choose the center scanning line of each pattern. The performance of video quality is reliable. The VCT has been filed as a patent application in Taiwan, US, Europe and China. It also presents in Hon-Hai Precision 3 rd R&D Technical Committee. The Automation Test Measurement System (ATMS) can apply to all STB products. The ATMS reduces the human resources and test cycles. The benefits of ATMS are also time and cost saving. In the modern STB and broadcasting network, high-definition contents are more popular. The decoder interface will build in digital video and component interface. Based on the quality index experience, we will do more research on this topic in the future. -53-

54 References [1] Chien-Chin Yen, Trends of Interactive TV Trends of Interactive TV & Triple Play, Chunghwa Telecom, The 13th Annual Wireless & Optical Communication Conference, Taipei, [2] Jack Krooss, MediaPump MediaSplice Technology Overview, Optibase Inc.,, [3] Radyne ComStream Inc., Video Performance Measurements in DTV Transmission Systems, A Tiernan White Paper, May [4] D K Fibush, Video Testing in Modern Television Systems, International Broadcasting Convention, September 1996, Conference Publication No. 428, IEE, [5] IEEE Std , IEEE Standard on Video Signal Transmission Measurement of Linear Waveform Distortion, December 11, [6] P Kavanagh, Patterns For All Formats,International Broadasting Convension, September 1995, IEE [7] Tektronix Inc. Ltd., A Guide to Picture Quality Measurements for Modern Television Systems, white paper, App_Notes/Technical_Briefs/digital_QoS/25W_14000_1.pdf [8] Michael Robin; Michael Poulin, Digital Television Fundamentals: Design and Installation of Video and Audio Systems, McGraw-Hill, [9] Nishida, Y.; Nakasu, E.; Aoki, K.; Kanda, K.; Mizuno, O.; Statistical analysis of picture quality for the digital television broadcasting, Broadcasting Convention, International (Conf. Publ. No. 428) Sept Page(s): [10] Koga, T.; Yokosawa, K.; Ashizawa, K.; Hontani, H., Improvement of degraded picture quality due to scanning frequency difference between TV camera and PC display with CRT, Circuits and Systems, MWSCAS '04. The th Midwest Symposium on Volume 1, July 2004 Page(s):I vol.1, Digital Object Identifier /MWSCAS

55 [11] Tektronix Inc. Ltd., NTSC Video Measurements, white paper. [12] Video Reference, [13] Leader Instruments Corp., Differential Phase and Gain Revised, Teleproduction Test Volume 1, Number 12. [14] Harmonic Inc., Guidelines for Video Quality Evaluations, white paper. [15] Tektronix Inc. Ltd., A Guide to Digital Television Systems and Measurements. [16] David K. Fibush, Video Testing in a DTV World, SMPTE Journal, [17] Mike Knee, Snell & Wilcox Ltd., A Single-ended Picture Quality Measurement for MPEG-2, [18] John Watkinson, Snell & Wilcox Ltd., The Engineer s Guide to Decoding & Encoding, [19] John Fletcher, Michael Prior-Jones, MPEG-2 Video Quality Versus Quantiser Scale, BBC Research & Development, UK. [20] George Gazzam, Sam Knight, USE OF DIGITAL VIDEO TECHNOLOGY FOR REAL-TIME EXERCISE MONITORING AND DEBRIEF OF COLLECTIVE TRAINING APPLICATIONS, L-3 Communications Link Training and Simulation. [21] John Watkinson, Television Fundamentals, JButterwirth-Heinemann, [22] Tektronix Inc. Ltd., NTSC Systems Television Measurements. [23] Tektronix Inc. Ltd., Measuring Noise in Video Systems. [24] Andrew F. Inglis & Arch C. Luther, Video Engineering 2 nd Edition, McGraw-Hill, [25] J. Lauterjung, A Measurement System For The Test Of DVB Receivers, International Broadcasting Convention, September 1995, Conference Publication No. 413, IEE

56 Appendix - Acronyms Definition AAC Advanced Audio Coding, developed by the MPEG group. AC3 A way of compressing audio signals to produce Dolby Digital 5.1. surround sound. ie 5 Channels. It is sometimes refered to as a AC3 Codec. APIB Audio Precision Interface Bus A/V Audio and video ATMS Automated test measurement system ATSC Advanced Television Systems Committee CVBS A format designed to provide both video and synchronizing information in one signal. CPE Consumer promise equipment DHCP Dynamic Host Configuration Protocol Dolby Dolby Laboratories, well-known in consumer audio standard DUT Device under test unit DVB Digital video broadcasting EPG Electronic program guide FCC Federal Communications Commission GPIB General Purpose Interface Bus H.264 H.264, MPEG-4 Part 10, or AVC, for Advanced Video Coding, is a digital video codec standard which is noted for achieving very high data compression. It was written by the ITU-T Video Coding Experts Group (VCEG) together with the ISO/IEC Moving Picture Experts Group (MPEG) as the product of a collective partnership effort known as the Joint Video Team (JVT). The ITU-T H.264 standard and the ISO/IEC MPEG-4 Part 10 standard (formally, ISO/IEC ) are technically identical. HDTV High Definition Television, this term describes several advanced standards proposals to allow high-resolution TV to be received in the home. IR Infrared IRD Integrated Receiver Decoder IRE Institute of Radio Engineers, a unit equal to 1/140 of the peak-to-peak amplitude of the video signal, which is typically one volt. ISDB Integrated Services Digital Broadcasting KPI Key performance index -56-

57 MIB Management Information Base MPEG Moving Picture Experts Group MPTS Multiple program transport stream NIST NIST Net is a network emulation package that runs on Linux. NTSC National Television Standards / Systems Committee, the color system used in the United States and North America. The field rate for NTSC is 60 Hz with 525 lines per screen and the subcarrier transmission method is a straight phase and amplitude modulation system for chroma using a subcarrier frequency of 3.58 MHz. RS-170 the United States standard that was used for black-and-white TV (monochrome), and defines voltage levels, blanking times, the width of the sync pulses, and so forth. SNMP Simple Network Management Protocol SPTS Single program transport stream STB Set-top box TFTP Trivial File Transfer Protocol THD Total Harmonic Distortion, Harmonic Distortion is a means for measuring Nonlinear Distortion. TS Transport stream TS/IP Transport stream over IP VBI Vertical blanking interval VC1 Windows Media Video 9 Advanced Profile codec is Microsoft's implementation of the advanced profile of VC1, Microsoft submits into Society of Motion Picture and Television Engineers (SMPTE) to establish a new video codec specification. VCT Video combination technique -57-