中文模板

Size: px

Start display at page:

Download "中文模板"

杀矮严
6 years ago
Views:

1 ISSN , CODEN RUXUEW Journal of Software, Vol18, No6, June 2007, pp DOI: /jos Tel/Fax: by Journal of Software All rights reserved GPU 1,3+, 1 1,2, 1 (, ) 2 (, ) 3 (, ) Real-Time Rendering Depth Images on GPU by Forward Warping LIU Bao-Quan 1,3+, LIU Xue-Hui 1, WU En-Hua 1,2 1 (State Key Laboratory of Computer Science, Institute of Software, The Chinese Academy of Sciences, Beijing , China) 2 (Department of Computer and Information Science, Faculty of Science and Technology, University of Macau, Macao) 3 (Graduate School, The Chinese Academy of Sciences, Beijing , China) + Corresponding author: Phn: , Fax , lbq@iosaccn, Liu BQ, Liu XH, Wu EH Real-Time rendering depth images on GPU by forward warping Journal of Software, 2007,18(6): Abstract: This paper presents a new pipeline for rendering depth images entirely on GPU (graphics processing unit) The implementation exploits inherent parallelism of GPU to speed up the rendering of depth images By the scheme, a novel forward 3D warping method is proposed for vertex shader to obtain high rendering performance Furthermore, the hardware pipeline s rasterization function is utilized to conduct the image re-sampling efficiently to generate holes free rendering results Per pixel lighting effect is implemented in pixel shader to get high image quality The rendering shows rapid performance at full screen resolution, with correct self-occlusions and accurate silhouettes Moreover, a real-time walkthrough system is implemented for the objects based on cylindrical depth image rendered by view-dependent dynamic LOD (level of detail) representation at runtime Key words: : graphics hardware; GPU (graphics processing unit); real-time rendering; depth image; image-based rendering (IBR); per-pixel lighting GPU(graphics processing unit) GPU vertex shader,,,, pixel shader,,,, LOD(level of detail) : ;GPU(graphics processing unit); ; ; Supported by the National Natural Science Foundation of China under Grant Nos , ( ); the National Basic Research Program of China under Grant No2002CB ( (973)) Received ; Accepted

2 1532 Journal of Software Vol18, No6, June 2007 : TP391 : A,,,,,,,, GPU, Macedonia [1] : GPU GPU, GPU,,, vertex shader,,, Relief Texture Mapping(RTM) [2 4] RTM,, 1 RTM, 941%,, GPU ;,, RTM RTM, :, 2 1 X s, X i, 2 (a) Depth image (b) Normal map (c) Color texture (d) Rendering result (a) (b) (c) (d) Fig1 Input of our system is in (a), (b), and (c), the rendering result is shown in (d) 1 (a),(b) (c), (d) X Target image plane X i X s X t Reference image plane Fig2 C s The two-step rendering procedure: Pre-warping followed by conventional texture-mapping 2 :, C t

3 : GPU 1533,,, : 1) vertex shader, GPU 2) vertex shader 2D,, Z buffer, 3), RTM, RTM 4),,, LOD(level of detail) Bump mapping [5],, Displacement mapping [6 8] Displacement mapping,, Displacement mapping,, 4 4,,, vertex shader Relief texture mapping(rtm) [2 4] parallax mapping [9], RTM, RTM, (, ) Fujita [10] RTM, reflection mapping, RTM, RTM, RTM, RTM Relief mapping [11],,,,, [12,13],,Donnelly [14],,,, [15,16],, 3, pixel shader, Dmesh [17] (i,j) P ij R; (, P ij );,,, 4 4,,

4 1534 Journal of Software Vol18, No6, June 2007,Dmesh,,,,, 2 GPU 21 GPU, :(1) ;(2) Z-buffer 1, X s X i,, vertex shader,, pixel shader, 2, 1,, 3 Reference image Pre-Warping in vertex shader Hardware Lighting in + rasterization + pixel shader Intermediate image in Pbuffer Intermediate image in Pbuffer Texture mapping Final image Fig3 The flowcharts of the two rendering passes 3 22 vertex shader GPU,, X s, McMillan [18,19],, vetex shader vetex shader, (u s,v s ) ( u, v),, (u i,v i )=(u s + u,v s + v) 3D warping Image-Based Rendering using Image Warping, [2,18] ( u, v) 4 u s u=u i u s 4, C t FX u i u s X, u/d e =depth/(depth+dt),d e u e u s,d e =u e u s ;D t C t,, D t ;u e (epipolar),u s u e ;depth u s, u:

5 : GPU 1535 u=(u e u s ) depth/(depth+d t )=(u e u s )/(1+D t /depth)=(u e u s ) Table(depth),,Table(depth)=1/(1+D t /depth), vertex texture vertex shader depth v Dmesh,, (Z-Buffer ),, 1, ; 2,, 4, Z-Buffer, Z t =D t +depth 1, vertex shader Z t, pixel shader, Pbuffer, 2 Z t, ( 17 ), shadow map, Fig4 u s F X u D e u i Top view of depth image pre-warping, (u s,v s ) (u i,v i ),,, 23 :,, ( ) vertex shader,, pixel shader, 24 pixel shader, pixel shader Phong 1, 1(d), : 1) (VD), C t, (u e,v e ), :D t C t (u e,v e, D t ) (u s,v s,depth), VD=(u e,v e, D t ) (u s,v s,depth) 2), : 3) Phong,,, pixel shader, pixel shader Policarpo [11] Donnelly [14], pixel shader,,, C t u e D t

6 1536 Journal of Software Vol18, No6, June : 1 ( ) Pbuffer,, 2,,, 1(d) ,,, 2,, [4,20],,, 6 6, ( ), ( RTM 6 ),,,RTM pre-warping, 9 pre-warping, RTM,,,33,, DPCC(double projective cylindrical camera),, ( ), 5(a) ;,, 5(b) (a) Top view (perspective projection) (b) Side view (parallel projection) (a) ( ) (b) ( ) Fig5 Double projective cylindrical camera 5 DPCC, [19] ( 6(a) ),, DPCI(double projective cylindrical image), 6(b) DPCIDPCI 6,

7 : GPU 1537 warping DPCI u s, 7,C ;u s, B; u s u s B depth (a) Image projected by cylindrical pinhole-camera model (a) C B depth u s (b) DPCI projected by DPCC (b) (DPCI) Fig6 Comparison of different cylindrical images Fig7 The pixel s depth value of DPCI 6 32 DPCI 7 DPCI DPCI, : 1 ; 2, 8 DPCI Pre-Warping Intermediate image in tangent plane Texture mapping Final image Fig8 The flowchart of the two-pass rendering 8 9,,, E,E (epipolar) DPCI Tangent plane ; 2,, E, Pre-Warping 10 Texture mapping, DPCI X s X i, 3D warping, B X i X t C t 4, Fig9 X i C t B, X s B X s, X i C s C t Target image plane 2-D illustration of the rendering procedure 9

8 1538 Journal of Software Vol18, No6, June 2007 Case 2 X s B C s E Tangent plane Case 1 X i Target image plane Case 3 X t C t Fig10 DPCI to tangent plane pre-warping 10 DPCI Fig11 Three cases of the viewpoint s position ,, 12(a),,,,, Z-Buffer ( ), 2, DPCI, Z-Buffer ( ), 11, 3 : Case 1, DPCI, ; Case 2, DPCI, 12 ; Case 3, DPCI,,, : DPCI; (a) Rendered by a DPCI (b) Rendered by a top planar depth image (c) The composition of the first two (a) DPCI (b) (c) Fig12 34 DPCI Rendering DPCI with a top end cap 12 DPCI (meso-structure),, III (tree man) : DPCI, DPCI, pixel shader

: GPU 1539, DPCI,, 13 35 LOD Fig13 Meso-Structure added to DPCI 13 DPCI DPCI N N, LOD,,,, DPCI, Dist object2view Iter step DPCI, DPCI, Iter step, Dist object2view A, A Iter step,iter step 1,,N N;, Nd

9 : GPU 1539, DPCI,, LOD Fig13 Meso-Structure added to DPCI 13 DPCI DPCI N N, LOD,,,, DPCI, Dist object2view Iter step DPCI, DPCI, Iter step, Dist object2view A, A Iter step,iter step 1,,N N;, Nd Nd, Nd=N Iter step,nd<n, DPCI LOD, DPCI, 14, DPCI Fig14 Real-Time walk through system in a scene full of image-based objects at fps 14 DPCI (48327 fps) 4 OpenGL Cg Windows XP Pentium IV 24 GHz 512M, nvidia GeForceFx5200, 1, 15,

10 1540 Journal of Software Vol18, No6, June 2007 Table 1 Comparison of frame rate between DPCI based rendering and traditional polygon based rendering 1 DPCI based Polygon based Object Polygon number Resolution of DPCI Ratio of fps rendering (fps) rendering (fps) Venus Man s head (a) Traditional polygon method (b) DPCI based method (input DPCI is ) (c) DPCI based method (input DPCI is ) (a) (b) DPCI ( ) (c) DPCI ( ) Fig15 Comparison of the rendering quality 15 : ;, [21],, : (,Per-Pixel displacement mapping [14] relief mapping [11], DMesh [17] ), 2, : , 16, ,, 16 Table 2 Comparison of frame rate for planar depth image rendering 2 Resolution of Per-Pixel displacement input image mapping [14] Relief mapping [11] (fps) DMesh [17] (fps) Our method (fps) (fps) DPCI, ; DPCI, DPCI, [2,3],DPCI ;,,,, 5 GPU, RTM,,

11 : GPU 1541 ( 18 ) Fig16 Rendering results of planar depth image 16 Fig17 The hybrid rendering results of DPCI interpenetrated with standard triangle based geometry teapots 17 DPCI Fig18 A full screen walking-through system based on image-based objects at fps 18 (41716 fps) References: [1] Macedonia M The GPU enters computing s mainstream IEEE Computer Society, 2003,36(10): [2] Oliveira MM Relief texture mapping [PhD Thesis] North Carolina: University of North Carolina, 2000 [3] Oliveira MM, Bishop G, McAllister D Relief texture mapping In: Whitted T, ed Proc of the SIGGRAPH 2000 New Orleans: ACM Press, [4] Oliveira MM, Bishop G Image-Based objects In: Proc of the 99 ACM Symp on Interactive 3D Graphics Atlanta: ACM Press, [5] James B Simulation of wrinkled surfaces Computer Graphics (Proc of the SIGGRAPH 78), 1978,12(3):

12 1542 Journal of Software Vol18, No6, June 2007 [6] Cook RL Shade trees Computer Graphics (Proc of the SIGGRAPH 84), 1984,18(3): [7] Hirche J, Ehlert A, Guthe S, Doggett M Hardware accelerated per-pixel displacement mapping In: Proc of the Graphics Interface Waterloo: ACM Press, [8] Kautz J, Seidel HP Hardware accelerated displacement mapping for image based rendering In: Proc of the Graphics Interface 2001 Toronto: Canadian Information Processing Society, [9] Tomomichi K, Takahei T, Inami M, Kawakami N, Yanagida Y, Maeda T, Tachi S Detailed shape representation with parallax mapping In: Proc of the 11th Int l Conf on Artificial Reality and Telexistence (ICAT 2001) Tokyo: ACM Press, [10] Fujita M, Kanai T Hardware-Assisted relief texture mapping In: Navazo I, ed Proc of the Eurographics 2002 Saarbrücken: Eurographics Association, [11] Policarpo F, Oliveira MM, Comba JLD Real-Time relief mapping on arbitrary polygonal surfaces ACM Trans on Graphics (Proc of the SIGGRAPH I3D 2005), 2005,24(3): [12] Yerex K, Jagersand M Displacement mapping with ray-casting in hardware In: Proc of the SIGGRAPH 2004 Los Angeles: ACM Press, 2004 [13] Delgass LL, McGhee K Parallax searching and mesosurface shadowing In: Proc of the SIGGRAPH 2005 Los Angeles: ACM Press, 2005 [14] Donnelly W GPU Gems 2nd ed, Addison-Wesley, [15] Wang LF, Wang X, Tong X, Lin S, Hu SM, Guo BN, Shum HY View-Dependent displacement mapping ACM Trans on Graphics (Proc of the SIGGRAPH 2003), 2003,22(3): [16] Wang X, Tong X, Lin S, Hu SM, Guo BN, Shum HY Generalized displacement maps In: Proc of the Eurograph Symp on Rendering 2004 Norrköping: Eurographics Association, [17] Pajarola R, Sainz M, Meng Y DMesh: Fast depth-image meshing and warping Int l Journal of Image and Graphics (IJIG), 2004, 4(4):1 29 [18] McMillan Jr L An image-based approach to three-dimensional computer graphics [PhD Thesis] University of North Carolina, 1997 [19] McMillan L, Bishop G Plenoptic modeling: An image-based rendering system Computer Graphics (Proc of the SIGGRAPH 95), 1995, [20] Adelson EH, Bergen J The plenoptic function and the elements of early vision In: Computational Models of Visual Processing Cambridge: MIT Press, [21] Li KY, Wang WC, Wu EH Bar texture mapping Journal of Software, 2004,15: (in Chinese with English abstract) : [21],,,2004,15: (1975 ),,,,,, (1947 ),,,,CCF,,, (1968 ),,,,CCF,,

中文模板

中文模板 ISSN 1000-9825, CODEN RUXUEW E-mail jos@iscasaccn Journal of Software, Vol17, Supplement, November 2006, pp70 77 http//wwwjosorgcn 2006 by Journal of Software All rights reserved Tel/Fax +86-10-62562563