Audio Video coding Standard (AVS) Technology and Opportunity Prof. YU, Lu ( ) Zhejiang University Institute of Information and Communication Engineering yul@zju.edu.cn June 5. 2004 1
Outline Why target and purpose of AVS How work style of AVS experts group What technical characteristics of AVS-video Future AVS-M IEE HK Workshop June 5, 2004 2
Why IEE HK Workshop June 5, 2004 3
Digital television system Front end satellite subscriber HPA Customer site A Capture editing storage coding Terrestrial HPA Customer site B cable Harmonic Enterprise 1 Customer site C Channel coding IEE HK Workshop June 5, 2004 4
Technical standards in digital television systems Channel coding technical standard Terrestrial Tsinghua Univ. And Shanghai Jiaotong Univ. Cable Zhejiang Univ. Satellite Source coding technical standard (AVS) System (data and command) Video Audio Subscriber management and copyright management standard Management of payment Encryption and decryption IEE HK Workshop June 5, 2004 5
Applications require digital audio video coding standard Core technology of: Digital Television Storage Media (DVD, HD-DVD) Broadband Network Multimedia Applications (Video Conferencing, Video on Demand) New Generation of Mobile Communication Surveillance system IEE HK Workshop June 5, 2004 6
Vertical Vertical & Horizontal Standards Digital television broadcasting (DVB, ATSC) Streaming (ISMA) Visual disk (DVD forum) Multimedia service (MMS) and I-mode (JP) Horizontal Digital audio-visual coding standard (MPEG, H26x, AVS) Storage Media DTV Streaming Mobile multimedia Source coding Source coding Source coding Source coding AVS System Storage Media System S/T/C DTV System TCP/IP System Wireless IEE HK Workshop June 5, 2004 7
MPEG domination standard of AV coding VCD: MPEG-1 (ISO/IEC 11172) MP3 DVD: MPEG-2 MP@ ML (ISO/IEC 13818) SDTV: MPEG-2 MP@ ML (ISO/IEC 13818) HDTV: MPEG-2 MP@ HL (ISO/IEC 13818) Multimedia applications: MPEG-4 (ISO/IEC 14496) MPEG-4 AVC / H.264 Multimedia search engine: MPEG-7 (ISO/IEC 15938) Multimedia framework: MPEG-21 (ISO/IEC 18034) IEE HK Workshop June 5, 2004 8
No free lunch VCD: MPEG-1 (ISO/IEC 11172) MP3 DVD: MPEG-2 MP@ ML (ISO/IEC 13818) SDTV: MPEG-2 MP@ ML (ISO/IEC 13818) HDTV: MPEG-2 MP@ HL (ISO/IEC 13818) Multimedia applications: MPEG-4 (ISO/IEC 14496) MPEG-4 part 10 /MPEG-4 AVC / H.264 Multimedia search engine: MPEG-7 (ISO/IEC 15938) Multimedia framework: MPEG-21 (ISO/IEC 18034) Loyalty free Patent License Patent License Patent License Patent License Patent License NA NA IEE HK Workshop June 5, 2004 9
MPEG and it s Patent Pools MPEG (Motion Picture Expert Group) Technical experts group of ISO/IEC JTC1 / SC29 / WG11 MPEG LA (www.mpegla.com) an independent licensing administrator company one-stop technology standards licensing with a portfolio of essential patents in the standard Via Licensing (www.vialicensing.com) an independent subsidiary of Dolby Laboratories provides patent licensing services for third-party intellectual-property owners IEE HK Workshop June 5, 2004 10
Licensing terms of MPEG-2 (MPEG LA) For MPEG-2 encoding products in hardware or software $2.50 from January 1, 2002 MPEG-2 decoding products in hardware or software set-top boxes, DVD players and computers equipped with MPEG-2 decode units $2.50 from January 1, 2002 Consumer Products camcorders, read/write DVD players, computers and/or software $2.50 from January 1, 2002 MPEG-2 Transport or Program stream products file servers, multiplexers, demultiplexers $4.00 x the greater of the input or output transport or program streams MPEG-2 distribution encoding products real time terrestrial, cable, satellite etc. broadcast and/or distribution $2.50 from January 1, 2002 x the number of channels in parallel MPEG-2 Packaged Media $0.03 from March 1, 2003 for the first MPEG-2 Video Event (up to and including 133 minutes ) plus $0.01 for each additional 30 minutes IEE HK Workshop June 5, 2004 11
Main patent holders of MPEG Company name country No. SONY JP 130 THOMSON LICENSING S.A. USA 99 PHILIPS NL 88 GE TECHNOLOGY DEVELOPMENT, INC. USA 56 MITSUBISHI JP 50 MATSUSHITA JP 36 VICTOR COMPANY OF JAPAN, LIMITED (JVC) JP 32 GENERAL INSTRUMENT CORPORATION USA 21 SAMSUNG ELECTRONICS CO., LTD. KR 14 SCIENTIFIC ATLANTA USA 13 Updated in 1 July, 2003 From Prof. Gao IEE HK Workshop June 5, 2004 12
Licensing terms of AVC (Via Licesing) Apr. 2004 Who must sign a License Manufacturers of end-user products replicators (for physical media) and companies conducting transactions (for non-physical distribution) of Content subject to the Replication Fees Encoder/ decoder US$0.25 per Licensed Product US$0.0025 per Temporary Decoder (expire after not more than 20 decodes & not more than 30 days) Encoder/Decoder Royalty Cap US$2.5 Million per year per enterprise for sale of all products Replication Fees permanent basis Length of Title Rate 90 Minutes or more $ 0.02531 Minutes to 89 Minutes $ 0.01530 Minutes or less $ 0.005 Temporary basis IEE HK Workshop June 5, 2004 13 $ 0.0025 per Title Replicated/Sold
Licensing terms of AVC (Via Licesing) Initial fee US$15,000 Free of initial fee for Enterprises with revenues of less than US$2 Million per year Free of initial fee for Enterprises with up to 50,000 end user products, and generate less than $500,000 in revenue The companies who have participated in the development of the terms are: Apple Computer Dolby Laboratories FastVDO Fraunhofer-Gesellschaft ev IBM LSI Logic Microsoft Motorola Polycom RealNetworks IEE HK Workshop June 5, 2004 14
Loyalties per year US$ 100 million per year DVD production of a year in China is more than 30 million set-top-box production of a year in China is no less then 10 million MPEG A golden key not in our hands IEE HK Workshop June 5, 2004 15
Technical reason necessity & possibility MPEG-2 (1992) More than 10-years old Compression efficiency 4.7GB DVD 2 hours movie (5.3Mbps) 18GB 2 hours high definition movie (20Mbps) MPEG-4 AVC / H.264 (2001-2002) Advanced coding techniques Multiple reference frame inter-frame prediction Content-based adaptive binary arithmetic coding High compression efficiency 1.5~2Mbps for SD, 6-8Mbps for HD) Save of storage space, save of channel bandwidth, save of frequency spectrum resource IEE HK Workshop June 5, 2004 16
Short history of AVS Mar18-21. 2002 178th Xiangshan Science Conference, Beijing, Broad-band Network and Security Stream Media Technology June 11, 2002 Science and Technology Department of MII released a bulletin about setting up Audio Video Coding Standard Workgroup on China Electronics News June 21, 2002 Audio Video Coding Standard Working Group was set up in Beijing. Aug 23-24, 2002 first meeting of AVS, AVS united with MPEG-China. Website of AVS opened to the members formally. Dec 9, 2002 Department of Science and Technology of Ministry of Information Industry issued the notice of Setting up Audio Video Coding Standard Working Group and assigned the task of the group Dec 19, 2003 On the 7 th AVS meeting, AVS-video (1.0) and AVS-system (1.0) was finalized. IEE HK Workshop June 5, 2004 17
Authorization document IEE HK Workshop June 5, 2004 18
AVS members (May 2004) 1. LG 24. 2. LSI Logic Corporation 25. 3. TCL 26. 4. UT 27. 5. WIS Technologies, Inc 28. 6. 29. 7. 30. 8. 31. 9. 32. 10. 33. 11. 34. 12. 35. 13. 36. 14. 37. 15. 38. 16. 39. 17. 40. 18. 41. 19. 42. 20. 21. 22. 23. 43. On2 44. 45. 46. IEE HK Workshop June 5, 2004 19
AVS members (May 2004) 47. 70. 48. 71. 49. 72. 50. 73. 51. 74. (ASTRI) 52. 75. 53. 76. 54. 77. 55. 78. 56. 79. 57. 80. 58. 81. 59. 82. 60. 83. 61. 84. 62. 85. 63. 86. 64. 87. 65. 88. 66. 89. 67. 90. 68. 91. 69. IEE HK Workshop June 5, 2004 20
How IEE HK Workshop June 5, 2004 21
National standard advanced and practical technologies low-licensing-cost (conceptual loyalty fee 1.0 ) Strategy Based on international mainstream technologies Based on AVS member institutions own technologies IP management with one-stop patent pool Maxim Superiority Practicality Compatibility Independence IEE HK Workshop June 5, 2004 22
Subgroups in AVS Requirement and test System Video Audio Implementation Digital Rights Management Intellectual Property Rights IEE HK Workshop June 5, 2004 23
Time table IEE HK Workshop June 5, 2004 24
AVS working style Requirement Call for proposal & common test condition Technical proposal with technical explanation and experiment result (under common test condition) Review and cross check Adopt or not 256 technical proposals in all 8 meetings Four full meetings per year, with some additional group meetings 1. 2002 8 23-24 2. 2002 10 23-24 3. 2002 12 19-20 4. 2003 3 27-28 5. 2003 7 28-29 6. 2003 10 9-10 7. 2003 12 18-19 8. 2004 3 30-31 9. 2004 6 18-20 IEE HK Workshop June 5, 2004 25
What IEE HK Workshop June 5, 2004 26
MPEG-2 Video - Decoder Control Trans./ Quant IQ/IT Control Data Coeff. 0 Entropy Coding Inter/Intra Motion compensation MV Motion IEE HK Workshop estimation June 5, 2004 27
AVS video version 1 (AVS-HD) Control Control Data - Decoder Trans./ Quant IQ/IT Coeff. Intraframe Prediction Entropy Coding Deblock Inter/Intra Motion compensation MV Motion IEE HK Workshop estimation June 5, 2004 28
AVS video version 1 (AVS-HD) Control Control Data - Decoder Trans./ Quant IQ/IT Coeff. Intraframe Prediction Entropy Coding Deblock Inter/Intra Motion compensation MV Motion IEE HK Workshop estimation June 5, 2004 29
AVS video tools Accept 42 technical proposals from more than 200. Major tools Transform 16bit-implemented 8x8 integer transform Quantization and scaling - scaling only in encoder Motion compensation 16x16/16x8/8x16/8x8 modes Quarter-pel interpolation 4-taps interpolation filter Intra prediction 5 modes Deblocking Entropy coding Minor tools Motion vector prediction Skipped mode and Coded block pattern Alternative scan Weighted prediction 4:2:2 IEE HK Workshop June 5, 2004 30
Profile and Level 1 Profile (JiZhun Profile) 4 level 4.0 : up to Standard Definition with 4:2:0 4.2 : up to Standard Definition with 4:2:2 6.0 : up to High Definition with 4:2:0 6.2 : up to High Definition with 4:2:2 IEE HK Workshop June 5, 2004 31
Video document editing Editing meeting 2003.8.25-30 2003.11.7-9 AVS reference software ICT, Tsinghua, ZJU IEE HK Workshop June 5, 2004 32
Performance- HD (1280 x720) Harbour PSNR(dB) 41 39 37 35 33 31 29 1000 6000 11000 16000 21000 26000 31000 36000 Bitrate(kbits/s) H.264 Main H.264 Baseline AVS MPEG-4 MPEG-2 IEE HK Workshop June 5, 2004 33
Performance- HD (1280 x720) Flamingo PSNR(dB) 40 38 36 34 32 30 H.264 Main 28 AVS 26 MPEG2 10000 40000 70000 100000 130000 Bitrate(kbits/s) IEE HK Workshop June 5, 2004 34
Next step? IEE HK Workshop June 5, 2004 35
AVS-M Stage 1 Dec. 2004 Local decoding with non-real-time transmission Low resolution (up to VGA), low complexity Compression performance and complexity of decoder Stage 2 Apr. 2006 Bi-direction communication Low latency Error resilience tools Compression performance and complexity of both encoder and decoder IEE HK Workshop June 5, 2004 36
Conclusion Advantages of AVS Based on self-owned Intellectual Property Lower loyalty fee (1.0 ) Advanced compression efficiency Similar performance with H.264 2 times to MPEG-2 Well-defined applications Simplify and clear Chances 300-500 million Chips and end-user products Early join, early benefit from Website: www.avs.org.cn IEE HK Workshop June 5, 2004 37
Thank you! IEE HK Workshop June 5, 2004 38