交通标志检测调研报告 学生 : 杨怡指导老师 : 吴福朝 2012 3 16 Institute of Automation, Chinese Academy of Sciences 1
项目简介 项目名称 : 鲁棒性交通标志检测与识别研究 研究目标 : 为我国现存的交通标志检测与识别提供一套完整的解决方案 项目要求 : 检测 识别算法简单易实现 对光照变化和几何形变具有较高鲁棒性满足实时性 特色与创新 : 检测 : 提出交通标志的颜色概率模型, 使交通标志检测算法对光照变化和 几何形变具有很好的鲁棒性 识别 : 引进点线特征模板 形状特征模板 以及点线特征匹配方法, 使交 通标志检测算法对光照变化和几何形变具有很好的鲁棒性 2012 3 16 Institute of Automation, Chinese Academy of Sciences 2
交通标志分类 警告标志 (23 种 ) 颜色 : 黄底 黑边 黑图案形状 : 顶角朝上的等边三角形 禁令标志 (35 种 ) 颜色 : 除个别标志外, 为白底 红圈 红杠 黑图案 图案压杠形状 : 圆形和顶角朝下的等边三角形 指示标志 (17 种 ) 颜色 : 蓝底 白图案形状 : 圆形 长方形 正方形 指路标志 (20 种 ) 颜色 : 一般为蓝底 白图案 ; 高速公路为绿底 白图案形状 : 一般为长方形和正方形 2012 3 16 Institute of Automation, Chinese Academy of Sciences 3
影响交通标志检测效果的问题 Lightingconditions lightingisdifferentaccordingtothetimeofthedayandweather conditions brightspots,shadows Geometricdeformations whentheviewinganglesaredifferent,thesignswillappeardifferently withsomedegreeofdistortion Scenecomplexity manyotherobjectswiththesimilarcoloroccurinthescenewith clutteredbackground Partialocclusions trees,commerciallogos,buildings Deterioration thepaintonsignsoftendeteriorateswithage 2012 3 16 Institute of Automation, Chinese Academy of Sciences 4
交通标志检测 Color-based Shape-based Themixofcolorandshape ROI:RegionofInterest 2012 3 16 Institute of Automation, Chinese Academy of Sciences 5
Color-basedtrafficsigndetection RGBspace lowcomputationalcost verysensitivetolightingcondition 1. color threshold (author:arturodelaescalera,1997) therelationbetweenthecomponents ifthesignhasaredborder: 2012 3 16 6 Institute of Automation, Chinese Academy of Sciences in any other case 2 ), ( ), ( ), ( ), ( ), ( ), ( 1 ), ( k y x g b TB y x r f y x b f a TB b TG y x r f y x g f a TG b R y x r f a R k y x g RGB
Color-basedtrafficsigndetection RGB RGBspace 2. Color standardization (Shuangdong Zhu,China,2008) purpose:reducethecolorcomplexity coloroftrafficsigns:red,yellow,blue,white,black defineacolorspecificationslibraryofthesefivecolors thedistortedcolorsofrealsignsarestandardizedtoabovefive colorsafterpreprocessing,soastoreducethecolorinformation significantly essence:acomplexoriginalspace:16,777,216kindsofcolorof24 bitimage asimplespace:5elements BPneuralnetwork: input:3neurons,thergbvaluesofthepixel output:5neurons,5colors procedure:scananimagepixelbypixelusingthetrainedbp network,soastoperformcolorstandardizationoneverypixel,and completethecolorstandardizationofanimagefinally invarianttolighting,deformation,occlusiondependonthetrainset 2012 3 16 Institute of Automation, Chinese Academy of Sciences 7
Color-basedtrafficsigndetection RGB RGBspace 3. Comparison of the RGB values (MohamedBenallal,2003) real-time,lowcomputationalcost studythebehaviorofthergbcomponentsofmanyroadsigns fromsunrisetosunset,thenthesimplecomparisonofthergb componentstakentwobytwoissufficienttosegmentroadsignsin real-time. invarianttolighting,weather thexaxisisnumberedinhalfhourunits 2012 3 16 Institute of Automation, Chinese Academy of Sciences 8
Color-basedtrafficsigndetection HSI HSIspace moreimmunetolightingchanges can tguaranteeaperfectcolorsegmentationtoo 1. Lookuptable(author:ArturodelaEscalera,2003) twolook-up-tables:oneforhue,anotherforsaturation oncebothlutsareapplied,theimagesaremultipliedandnormalized tothemaximumvalueof255 invarianttolighting 2012 3 16 Institute of Automation, Chinese Academy of Sciences 9
Color-basedtrafficsigndetection HSI HSIspace 2.Threshold(G.Piccioli,1996) Eachimageelementisclassifiedas1(red)or0(non-red)according toitshueandsaturation The512*512imageissubdividedinto16*16regions.Eachregionis classifiedas1or0accordingtowhetherthenumberofpixelslabelled as 1exceedsathreshold. Asearchregionisassociatedwitheveryclusterof 1 -regions. Theregionofinterestisthe rectangularhull ofthecluster. invarianttolighting,weatherconditions,deformation 2012 3 16 Institute of Automation, Chinese Academy of Sciences 10
Color-basedtrafficsigndetection HSI HSIspace 3.Threshold(UniversidaddeAlcal,spain,2007) chromaticimages:hueandsaturation new:whiteimages:achromaticdecomposition discardsomeblobsaccordingtotheirsize,aspectratio eachcandidateblobisrotatedtoareferenceposition invarianttolighting,rotation,size,translation 2012 3 16 Institute of Automation, Chinese Academy of Sciences 11
Color-basedtrafficsigndetection HSIspace 4.Neuralnetworks(CYFang,Taipei,2003) hueischosenforthecolorfeatures NN(inputandoutputlayer),red h Inputneuronq(k,l):input:theRGBvalueofpixel(k,l) hue(h) 360 h h c output: x kl 360 thelarger,themoresimilartored x kl Weight:circular,triangular,octagonalsigns Outputneuronp(i,j):input: output: invarianttolighting,deformations,occlusions c net ij x w kl, y ij kl k l f ( net ij ) ij HSI 2012 3 16 Institute of Automation, Chinese Academy of Sciences 12
Color-basedtrafficsigndetection CIECAM97 CIECAM97colormodel(X.W.Gao,2006) isamodelformeasuringcolorappearanceundervariousviewing conditions.ittakesweatherconditionsintoaccountandsimulates human sperception. findtherangesofhueandchroma underdifferentweather conditionsandthedaylightviewingconditions. thetestimagesaretransformedfromrgbspacetociexyzvaluesand thentolch(lightness,chroma,hue)spaceusingciecam97model. basedontherangeofthecolors,trafficsign-to-bearesegmented usingquad-treehistogrammethod. invarianttolighting,weather 2012 3 16 Institute of Automation, Chinese Academy of Sciences 13
Color-basedtrafficsigndetection Eigen color Eigencolor(L.-WTsai,Taiwan) eigen colormodel:(onedetector) designanovelcolortransformmodelfordetectingthepixelswith higherreflectancefromtheirbackground greencolordetector: 3eigenvectors: (0.3396,0.3392,0.3212) (0.4896,0.0923,-0.4181) (0.2898,-0.4823,0.2279) Color-to-gery transform: 1 1 C 1 R G 3 3 1 3 B thecolorplane(u,v)perpendiculartotheaxis(1/3,1/3,1/3) 2( R B) R B - expandedbytheothertwoeigenvectors: u and v RBFnetwork candidateverification:size,aspectratio,arearatio,distance transform rectification:dltalgorithm invarianttoperspectivedistortions,occlusions,lighting 2 * G 2012 3 16 Institute of Automation, Chinese Academy of Sciences 14
Color-basedtrafficsigndetection Otherspaces:YIQ,L*a*bspace 2012 3 16 Institute of Automation, Chinese Academy of Sciences 15
Shape-basedtrafficsigndetection Corner detection Cornerdetection (author:arturodelaescalera,1997,page6:color threshold) cornerdetector:theconvolutionoftheimageandthemask differentmask,differenttypeofcorner cornerextraction:(triangle) asignwillbepresentwhen3cornersarefoundformingan equilateraltriangle. notinvarianttoocclusion,deformation 2012 3 16 Institute of Automation, Chinese Academy of Sciences 16
Shape-basedtrafficsigndetection Radialsymmetry Radial symmetry color-to-gery transform edgedetection findthegradientdirectionandvalueofthepixelsonthegreyimage weightthepotentialcenterbymultiplyingthegradientdirectionand afixedradiusaccordingtothesizeofthesignneedtodetect thepointwiththehighestweightisthecenter \ iftherearetwo(ormore)signsinoneimage,justusingtwodifferent radius expandthecenteraccordingtothefixedradius,thewecanfindthe candidatesign.itisradialsymmetry notinvarianttodeformation,clutteredbackground 2012 3 16 Institute of Automation, Chinese Academy of Sciences 17
Shape-basedtrafficsigndetection Slope (G.Piccioli,1996,page10,threshold)(triangle) edgedetection polygonalapproximation remainthesegmentswiththeslopeof[-ƹ,ƹ],[60-ƹ,60+ƹ], [-60-Ƹ,-60+Ƹ] findthesegmentswhichcanformanequilateraltriangle Slope, Annuli Annuli(circle) lookforcirclesofvariableradiusbyinspectingthesearchregion withannuliofvariableradiusandfixedthickness notinvarianttodeformation,occlusion,cluttered 2012 3 16 Institute of Automation, Chinese Academy of Sciences 18
Shape-basedtrafficsigndetection DtBs DistancetoBorders(UniversidaddeAlcal,spain,2007,page11) linearsvms inputfeature:distancetoborders invarianttotranslation,scale,rotation,occlusion 2012 3 16 Institute of Automation, Chinese Academy of Sciences 19
Shape-basedtrafficsigndetection FOSTS Foveal systemfortraffic(x.w.gao,2006,page13) representationofshapefeaturesusingfostsmodel eachimageinthefostsmodelisrepresentedbyviewingtrajectory andspecificdescriptionofimagefragmentsinthevicinityofeachfixation point invarianttoman-madenoise,scale,perspectivedistortion theredtriangularsignssharplydecreasesattheincreaseofdistortion levels:someparticularpropertiesthataresensitivetothemodel algorithms. 2012 3 16 Institute of Automation, Chinese Academy of Sciences 20
Shape-basedtrafficsigndetection NN, Fuzzy Neuralnetworks(shape)(CYFang,Taipei,2003,page12:NN) Inputneuronq(k,l):input:theRGBvalueofpixel(k,l) output::anyedgedetection,roberts,sobel,laplacian x kl Weight:circular,triangular,octagonalsigns Outputneuronp(i,j):input: net ij x kl w kl, ij k l output: Featureintegrationanddetection:afuzzymethod u(i,j):themembershipdegreeofthepixelbelongingtoaroadsign ifthecolorandshapefeaturesarebothatthesameposition(i,j),then thevalueofmembershipfunctionu(i,j)willbeverylarge. onthecontrary,u(i,j)willbesmall verification:size,aspectratio,colorarearatio,symmetry invarianttodeformation,occlusion,clutteredbackground y ij f ( net ij ) 2012 3 16 Institute of Automation, Chinese Academy of Sciences 21
Shape-basedtrafficsigndetection Othermethod: GA,SA,AdaBoost 2012 3 16 Institute of Automation, Chinese Academy of Sciences 22
Colorandshapebasedtrafficsign detection Color shape Invariant to Not invariant to RGB: color threshold corner detection deformation, cluttered background Occlusion HSI: hue, saturation (1 or 0) HSI: Chromatic image: hue, saturation threshold white: achromatic decomposition slope, annuli lighting deformation, occlusion, cluttered background DtBs lighting, scale, rotation, occlusion cluttered perspective distortion CIECAM97 FOSTS man made noise, lighting, scale, occlusion, perspective distortion HSI: NN, edge deformation NN: hue detection occlusion cluttered, lighting 2012 3 16 Institute of Automation, Chinese Academy of Sciences 23
本项目中拟采用的方法 建立交通标志的颜色概率模型, 用于检测交通标志在图像中的区域 1 根据主颜色 ( 红 黄 ) 将交通标志分为若干类, 记为 k 1, k2,..., k N 2 根据类 ki 的模板以及不同光照条件下的实际图像, 估计类 ki 主颜色的条件密度 p(( r, g, b) ki ) 3 确定类 k 的先验密度 p k ) 4 根据 Bayes 规则, 类主颜色的后验密度为 p( ki ( r, g, b)) [ p(( r, g, b) ki ) * p( ki )]/ p( r, g, b), 其中 p( r, g, b) 表示某像素点的颜色概率. 在确定的图像中 p( r, g, b) 是确定的, 因此直接将 p k ( r, g, b)) p(( r, g, b) k ) * p( k ) 作为类主颜色的后验概率, 不会影响交通标 志的检测效果. 即类 交通标志的颜色概率模型 条件密度 : 如何估计? 现有的非参数估计方法中, 哪种方法对交通标志检测有最好的效 果 i k i ( i ( i i i k i 先验密度 : 选择大量典型路况的标志, 统计各类交通标志出现的次数来确定各类的先验密度 p k ) ( i k i 2012 3 16 Institute of Automation, Chinese Academy of Sciences 24
下一步计划 进一步对交通标志检测进行调研 各种算法的实现, 看一下结果 本项目拟采用方法在其他领域现有的应用 探索本项目拟采用的方法该如何实现 2012 3 16 Institute of Automation, Chinese Academy of Sciences 25
Thank you! 2012 3 16 Institute of Automation, Chinese Academy of Sciences 26