35 11 Vol. 35, No. 11 2009 11 ACTA AUTOMATICA SINICA November, 2009 1 1 1 1 1,.,,,,.,,.,,, TP391 People Tracking Based on Muli-regions Join Paricle Filers WANG Yu-Ru 1 LIU Jia-Feng 1 LIU Guo-Jun 1 TANG Xiang-Long 1 LIU Peng 1 Absrac A people racking algorihm based on muli-regions join paricle filers (MR-JPF) has been proposed in his paper o solve he occlusion problem of people racking in video. Through locaing muliple key regions on human body, he algorihm deals wih he occlusion problem by consrucing he join paricle filer, which is based on a join moion model specified by an undireced graph, and on he regions relaion based observe-and-esimae scheme. The experimenal resuls have demonsraed ha he proposed algorihm is more effecive in solving long-ime parial or oal occlusion problem han he racking mehod based on single region paricle filer. Key words Compuer vision, objec racking, muli-regions, join paricle filers,.,,.., : 1),, ; 2),,,., Zhu [1],, ; Zhou [2] Huber,, 2008-07-15 2009-03-23 Received July 15, 2008; in revised form March 23, 2009 (60672090) Suppored by Naional Naural Science Foundaion of China (60672090) 1. 150001 1. School of Compuer Science and Technology, Harbin Insiue of Technology, Harbin 150001 DOI: 10.3724/SP.J.1004.2009.01387 ; Gennari [3] (Join probabilisic daa associaive filer, JPDAF),.,.,,.,,.,. (Muliple hypohesis racker, MHT) [4] JPDAF [5], JPDAF [6],. Khan [7] (Markov chain Mone Carlo, MCMC),.. (Muli-regions join paricle filers, MR-JPF).
1388 35,,,,,., ( ),,.,., MR-JPF,. 1 MR-JPF [8] (Sequenial Mone Carlo, SMC), [9 11],,.,,,,,,. [12 14],, MR-JPF,,. P (X Z 1: ) = cp (Z X ) r ω r 1P (X X r 1) (1), P (Z X ) X Z, c, P (X X r 1). 1.1 (n ) (V, E), 1 (a),,,, φ., V = {X i } i=1,,n, E = {φ(x i, X j )} i,j=1,,n. φ P (X X 1 ), P (X X 1 ) P (X i X i( 1) ) φ(x i, X j ) i i,j E (2), X i, X j i j, φ(x i, X j ) i j. (a) (a) Consruc he undireced graph Fig. 1 1 (b) φ (b) Define funcion φ The join moion model φ : R = {R i,j = (m θ i,j, m d i,j )} i,j=1,,n, m d i,j, m θ i,j i, j, 1 (b)., 3. R i,j φ φ(x i, X j ) = k d f d + k θ f θ (3), k d k θ. f d f θ, f d = 1 di,j f d = 0, f θ = 1 θi,j f θ = 0, m d i,j, d i,j δ d m θ i,j, θ i,j δ θ m d i,j δ d m θ i,j δ θ, θ i,j (4), φ [0,1], d i,j i, j, δ d δ θ d i,j, θ i,j, δ d = m d i,j /2, δ θ = m θ i,j /2., Xi, s Xj, s m d i,j m θ i,j,.
11 : 1389 1.2 ( n) X = {X i } n i=1.,, P (X Z 1: ) cp (Z X ) φ(x i, X j ) r ω r 1 i,j E P (X i Xi( 1) r (5) φ(x i, X j ) X 1,, P (X Z 1: ) {X s, ω s } N s=1, X s q(x ) = ω r 1 P (X i Xi( 1) r (6) r i ω s P (Z X s ) φ(x i, X j ) (7) i i,j E, X = {X i } n i=1 R,. 1.3 {X s } N s=1 {ω s } N s=1, :, n ˆX = { ˆX i } n i=1, i N n ˆX i = (ω s (k s j X s j ϕ(xj, s Xi))) s (8) s=1 j=1, ˆX i n i ω s. s, s = 1,, N i, Xj, s j = 1,, n i Xj s ϕ(xj, s Xi) s kj s, ϕ(xs j, Xi) s R ij j i. (8), N, n, kj s r, j i, n j=1 ks j = 1., n,,. φ(xj, s Xi) s :, R,,. 2 MR-JPF MR-JPF, MR-JPF., MR-JPF, : 1),, ; 2),,, ; 3),,,.,, kj s,,. 1),,, ; 2),,, kj s, ; 3),,. MR-JPF,, ( ),. 3 HSV [15]. H, S, V 64, 8, 1. m = 64 8 1 = 512. p = {p(u)} u=1,,m q = {q(u)} u=1,,m, Bhaacharyya m b = 1 p(u)q(u) (9) u=1 : 1. : 1 2,, P (X i X i( 1) ) i=1,,n, Q = {q i = {q i (u)} u=1,,m } i=1,,n, 3 R 3 = {R i,j 3 = (m θ i,j 3, m d i,j 3 )} i,j=1,,n.
1390 35 2., 1 N {X 1, r ω 1} r N r=1. 1 P (X 1 Z 1 ), N. N : 1) 1 {X 1, r ω 1}. r 2) P (X i X i( 1) ) X s. 3) X s, ωs. a) P s = {p s i = {p s i (u)} u=1,,m } i=1,,n b) Q, (9), p s i q i b s i, P (Z X s ) = n i=1 bs i ; c) i, j, θ i,j, (3) φ, (V, E); d) (7). 3. N, {X s, ω s } N s=1. (8), ˆX i, i = 1,, n. 1) j, j = 1,, n C j = N s=1 bs j, d i,j C = n N j=1 s=1 bs j, kj s = C j /C. 2) R i,j = (m θ i,j, m d i,j ), j = 1,, n, j i ϕ(xj, s Xi). s 3) (8) i ˆX i., ˆX = { ˆX i } n i=1. 4.. 1) ˆX, i, j p d i,j, p θ i,j ; 2) (7) ˆX R s ; 3) R + 1 k 1 k : R +1 = {R i,j +1 = (m θ i,j +1, m d i,j +1)} i,j=1,,n m d i,j +1 = k 1 m d i,j + k p d i,j m θ i,j +1 = k 1 m θ i,j + k p θ i,j (15) 2 4,,. 4, CAVIAR [16],., ( )., n = 2, k d k θ 0.6 0.4., 2. 16 20 30 43, 2,, (7),, ; (8), 2 1 2., 24 34, 1 2. 3.,,,. s k = s + s 1 k 1 = s 1 (14) s + s 1 4) + 1 : 2 ( ) Fig. 2 The resuls of he simulaing experimen
11 期 王玉茹等: 基于多区域联合粒子滤波的人体运动跟踪 为了验证算法在多区域均出现严重遮挡情况下 对被遮挡区域的正确预测, 进行了三组实验. 其中图 4 (a) 所示视频序列中, 有一个人与跟踪的客体沿同 一个方向并肩行走, 在行进的过程中另一个人从跟 踪目标的背后交叉穿过, 其中从第 12 帧开始, 被跟 踪的两个区域被另一个人不同程度地同时遮挡, 但 是系统能够有效地预测被遮挡区域的状态. 分析其 原因, 是因为当两个区域被同时遮挡时, 整个过程 图3 Fig. 3 1391 并未被完全遮挡, 而是小部分或者大部分被严重遮 挡, 在这两种情况下, 按照式 (7) 位于未被遮挡部分 的粒子概率大, 那么两个区域的状态能通过未被遮 挡部分的粒子进行准确预测. 图 4 (b) 和 4 (c) 的视 频序列分别为在室内拍摄的存在大面积遮挡的视频, 跟踪过程中出现了某个区域被完全遮挡的情况, 此 s 时按照式 (8), 被遮挡区域的粒子概率和小, 导致 kj 小, 因此该区域的状态将由未被遮挡区域来预测. 图 2 所示跟踪轨迹与实际运动轨迹的比较, 左右两个图分别为 x 和 y 方向 The comparison beween he racking resuls and he real rajecories in x (lef) and y direcions (righ) (a) 视频的第 2 12 18 40 帧, 其中连续遮挡 27 帧 (a) Frames 2, 12, 18, and 40 in he video, in which occlusion coninues 27 frames (b) 视频的第 15 21 28 帧, 其中连续遮挡 13 帧 (b) Frames 15, 21, and 28 in he video, in which occlusion coninues 13 frames (c) 视频的第 44 52 58 帧, 其中连续遮挡 14 帧 (c) Frames 44, 52, and 58 in he video, in which occlusion coninues 14 frames 图4 Fig. 4 各区域被严重遮挡时本文算法跟踪结果 The racking resuls when oal occlusion happens
自 1392 动 化 学 报 35 卷 (a) 多区域联合粒子滤波器 (a) MR-JPF (b) 单独粒子滤波器 (b) Independen paricle filers 图 5 多区域联合粒子滤波跟踪结果对比 (视频共发生了 31 帧的连续遮挡), 图中为第 38 54 63 67 帧 Fig. 5 The racking resuls comparison (occlusion happens in consecuive 31 frames), and here are frames 38, 54, 63, and 67 图6 图 5 所示的视频流在 x (左图) 和 y (右图) 方向上采用 MR-JPF 和单独粒子滤波跟踪结果与真实轨迹的对比 Fig. 6 The racking resuls comparison beween he real rajecories and he MR-JPF and IPF in x (lef) and y (righ) direcions 此外, 我们还在发生遮挡的情况下, 将 MR-JPF 方法和对每个区域单独采用粒子滤波器的方法进行 了对比实验. 如图 5 所示, 在没有发生遮挡之前的第 38 帧, 两种滤波器均能正确跟踪, 而在遮挡后的第 54 帧和 63 帧, 采用单独跟踪的方法由于各区域间 没有关联, 当对某个区域的跟踪偏离时, 不能由正确 跟踪区域对其进行修正, 导致跟踪失败. 而本文方法 由于建立了基于无向图的联合运动模型, 并应用了 区域关联的评估策略, 在一个区域被遮挡的情况下, 仍能通过其他区域对其状态进行预测, 从而在遮挡 前和遮挡过程中都能准确跟踪目标. 从图 6 中可以 明显地看出本文方法和单独对两个区域进行跟踪的 对比结果, 对于单独跟踪中的遮挡区域 (腿部区域), 从被遮挡的第 25 帧开始即偏离了初始位置, 但仍在 腿部, 当跟踪至第 54 帧则完全丢失, 相比之下, 本文 所采用方法则能够稳定地对两个区域进行跟踪. 5 结论 本文针对视频中遮挡情况下的目标跟踪, 提出 了基于多区域联合粒子滤波的方法. 其特点有: 1) 把人体划分为多个关键区域, 运用区域关联的联合 粒子滤波器进行跟踪; 2) 把多区域的关联表示为无 向图, 进而建立联合运动模型; 3) 构造联合粒子滤 波器对目标状态进行预测; 4) 建立了基于区域间关 联的评估策略, 对人体各区域状态进行联合预测. 所 提出的 MR-JPF 方法解决了人体运动跟踪中的部 分遮挡问题, 甚至对长时间的严重遮挡视频也能进 行准确跟踪, 提高了人体运动跟踪的可靠性. 然而, 联合粒子滤波器最大的缺点就是其计算复杂度会随 着划分区域数目的增加而呈指数增长, 因此在我们 的后续工作中, 将从提高多区域的联合粒子滤波器 的计算效率着手, 以期高效地解决遮挡问题.
11 : 1393 References 1 Zhu L, Zhou J, Song J Y. Tracking muliple objecs hrough occlusion wih online sampling and posiion esimaion. Paern Recogniion, 2008, 41(8): 2447 2460 2 Zhou S, Chellappa R, Moghaddam B. Appearance racking using adapive models in a paricle filer. In: Proceedings of he 6h Asian Conference on Compuer Vision. Jeju Island, Korea: Misubishi Elecric Research Laboraories, 2004. 1 6 3 Gennari G, Hager G D. Probabilisic daa associaion mehods in visual racking of groups. In: Proceedings of IEEE Compuer Sociey Conference on Compuer Vision and Paern Recogniion. California, USA: IEEE, 2004. 876 881 4 Blackman S S. Muliple hypohesis racking for muliple arge racking. IEEE Aerospace and Elecronic Sysems Magazine, 2004, 19(1): 5 18 5 Gorji A, Menhaj M B. Muliple arge racking for mobile robos using he JPDAF algorihm. In: Proceedings of he 19h IEEE Inernaional Conference on Tools wih Arificial Inelligence. California, USA: IEEE, 2007. 137 145 6 Schulz D, Burgard W, Fox D, Cremers A B. Tracking muliple moving arges wih a mobile robo using paricle filers and saisical daa associaion. In: Proceedings of he IEEE Inernaional Conference on Roboics and Auomaion. New York, USA: IEEE, 2001. 1665 1670 7 Khan Z, Balch T, Dellaer F. MCMC-based paricle filering for racking a variable number of ineracing arges. IEEE Transacions on Paern Analysis and Machine Inelligence, 2005, 27(11): 1805 1819 8 Gordon N J, Salmond D J, Smih A F M. Novel approach o nonlinear/non-gaussian Bayesian sae esimaion. IEE Proceedings on Radar and Sonar Processing, 1993, 140(2): 107 113 9 Czyz J, Risic B, Macq B. A paricle filer for join deecion and racking of color objecs. Image and Vision Compuing, 2007, 25(8): 1271 1281 10 Panrigo J J, Sanchez A, Monemayor A S, Duare A. Mulidimensional visual racking using scaer search paricle filer. Paern Recogniion Leers, 2008, 29(8): 1160 1174 11 Ryu H R, Huber M. A paricle filer approach for muliarge racking. In: Proceedings of he 2007 IEEE/RSJ Inernaional Conference on Inelligen Robos and Sysems. San Diego, USA: IEEE, 2007. 2753 2760 12 Tweed D, Calway A. Tracking many objecs using subordinae condensaion. In: Proceedings of Briish Machine Vision Conference. Cardiff, UK: Springer, 2002. 283 292 13 Vermaak J, Douce A, Perez P. Mainaining muli-modaliy hrough mixure racking. In: Proceedings of he 9h Inernaional Conference on Compuer Vision. Washingon D. C., USA: IEEE, 2003. 1110 1116 14 Okuma K, Taleghani A, de Freias N, Lile J J, Lowe D G. A boosed paricle filer: muliarge deecion and racking. In: Proceedings of European Conference on Compuer Vision. Prague, Czech Republic: Springer, 2004. 28 39 15 Comaniciu D, Ramesh V, Meer P. Kernel-based objec racking. IEEE Transacions on Paern Analysis and Machine Inelligence, 2003, 25(5): 564 577 16 CAVIAR es case scenarios [Online], available: hp:// groups.inf.ed.ac.uk/vision/caviar/caviardata1/, March 9, 2009... E-mail: yuru 765@163.com (WANG Yu-Ru Ph. D. candidae a he School of Compuer Science and Technology, Harbin Insiue of Technology. Her main research ineres is objec racking. Corresponding auhor of his paper.).,. E-mail: jefferyliu@hi.edu.cn (LIU Jia-Feng Associae professor a he School of Compuer Science and Technology, Harbin Insiue of Technology. His research ineres covers image processing, paern recogniion, and compuer vision.).. E-mail: hiliu@hi.edu.cn (LIU Guo-Jun Ph. D. candidae a he School of Compuer Science and Technology, Harbin Insiue of Technology. His main research ineres is objec racking.).,. E-mail: xl60@public.hr.hl.cn (TANG Xiang-Long Professor a he School of Compuer Science and Technology, Harbin Insiue of Technology. His research ineres covers auomaic inelligence and informaion processing, inelligen human-compuer inerface.).. E-mail: pengliu@hi.edu.cn (LIU Peng Lecurer a he School of Compuer Science and Technology, Harbin Insiue of Technology. His research ineres covers image processing and inelligen human-compuer inerface.)