Microsoft Word - 09-徐伟_oee180198_xeyz.doc

Size: px

Start display at page:

Download "Microsoft Word - 09-徐伟_oee180198_xeyz.doc"

縻簪柏
4 years ago
Views:

Opto-Electronic Engineering 光电工程 Article 2018 年, 第 45 卷, 第 12 期扫描电子显微镜图像漂移的实时矫正方法徐伟 1, 谷森 1, 储成智 1, 靳振伟 1, 汝长海 1,2* 1 215021 2 215021 摘要 : 为解决扫描电子显微镜 (SEM) 由于电子束漂移电磁干扰等原因导致的图像漂移问题, 提出基于 ORB 结合

1 Opto-Electronic Engineering 光电工程 Article 2018 年, 第 45 卷, 第 12 期扫描电子显微镜图像漂移的实时矫正方法徐伟 1, 谷森 1, 储成智 1, 靳振伟 1, 汝长海 1,2* 摘要 : 为解决扫描电子显微镜 (SEM) 由于电子束漂移电磁干扰等原因导致的图像漂移问题, 提出基于 ORB 结合 PROSAC 的图像漂移矫正算法首先采用 ORB 算法对基准图像和实时图像进行特征检测, 然后利用汉明距离与交叉匹配实现特征的初匹配, 再结合 RANSAC 的优化算法 PROSAC 计算帧间的单应矩阵, 利用单应矩阵映射剔除外点后重新迭代计算出最终的精确单应矩阵, 最后利用单应矩阵的透视变换实现 SEM 图像漂移实时矫正通过实验证明, 该算法不仅精度高, 而且能够满足 SEM 实时处理的要求关键词 : 扫描电子显微镜 ; 图像漂移 ; 单应矩阵 ; 矫正中图分类号 :O436.3;TP391 文献标志码 :A 引用格式 : 徐伟, 谷森, 储成智, 等. 扫描电子显微镜图像漂移的实时矫正方法 [J]. 光电工程,2018,45(12): Real-time correction of image drift in scanning electron microscope Xu Wei 1, Gu Sen 1, Chu Chengzhi 1, Jin Zhenwei 1, Ru Changhai 1,2* 1 Robotics and Microsystems Center of Soochow University, Suzhou, Jiangsu , China; 2 Collaborative Innovation Center of Suzhou Nano Science and Technology, Suzhou, Jiangsu , China Abstract: In order to solve the problem of imaging drift in scanning electron microscope (SEM) that caused by electron beam drift, electromagnetic interference and other reasons, an image shift correction algorithm based on ORB (oriented FAST and rotated BRIEF) combing the PROSAC (progressive sample consensus) is proposed in this paper. Firstly, the ORB algorithm is used to detect the feature between the reference image and real-time image. Then the initial matching of the feature is implemented by using the Hamming distance and cross-matching. Moreover, the RANSAN (random sample consensus) optimization algorithm PROSAC is used to calculate the homography matrix between frames and the final exact homography matrix is re-iterated after eliminating exterior point. Finally, the SEM image drift is corrected in real time using the perspective transformation of the homography matrix. The experiments show that the proposed algorithm is high precision and satisfies the requirement of SEM real-time processing. Keywords: scanning electron microscopy; imaging drift; homography matrix; correction Citation: Xu W, Gu S, Chu C Z, et al. Real-time correction of image drift in scanning electron microscope[j]. Opto-Electronic Engineering, 2018, 45(12): 收稿日期 : ; 收到修改稿日期 : 基金项目 :( ) 作者简介 :(1994-) @qq.com 通信作者 :(1976-) rzh@suda.edu.cn

2 1 引言 [1] (scanning electron microscope SEM) 3D SEM IC [2] [3] [4] SEM SEM SEM [5] Cizmar SEM [6] Naresh ORB RANSAC [7] RANSAC [8] SEM ORB(oriented FAST and rotated BRIEF) PROSAC(progressive sample consensus) RANSAC(random sample consensus) RANSAC RANSAC RANSAC PROSAC ORB SEM SEM 2 SEM 成像分析 1 SEM ξ(r) [9] 理想情况下, 强度值 ξ(r) 仅仅取决于电子束在样图 1 扫描电镜工作原理图 Fig. 1 The working principle of SEM

3 品上的停留位置 r r (t) 但由于 SEM 本身结构的缺陷, 往往引起图像的静态失真和动态失真, 导致图像的实际停留位置 r 为 r() t = r () t + Dd() t + Ds( r ), (1) 其中 : r()= t M( x() t e + y() t e ), (2) r 其中 :t 为时间,M 为常数, e x, e y 为 x 和 y 方向上的单位向量, D s 为静态失真, D d 为动态失真研究表明 : 在低倍率下, 由于观察的样品区域和电子束扫描区域范围比较大,SEM 的成像模型可近似为透视投影模型此时, 电磁透镜固有缺陷往往引起图像的静态失真, 但随着倍率的不断增加, 静态失真逐渐消失由于静态失真为常量, 可以通过畸变参数的计算来完成矫正然而在高倍率下,SEM 的投影中心移动到无限远处, 所以成像模型近似认为是平行投影模型此时, 静态失真完全消失, 但由电子束漂移电磁干扰等原因导致图像的动态失真常常引起 SEM 图像进行无规律的连续运动 ( 即图像的漂移现象 ), 如图 2 所示, 红色标记区域随着时间的推移发生了位移, 从而影响 [10] SEM 对纳米材料的测量与表征 (a) (b) 3 漂移矫正方法图 2 不同时刻 SEM 图像 Fig. 2 SEM images at different times I 0 I t SEM x y t0 t 1 SEM t 1 P 0 ( x, y) I 0 P (, ) 1 x y It H [11] P0 H P1, (3) 其中 : H 是 3 3 的满秩矩阵, 为 H11 H12 H13 H = H21 H22 H 23 (4) H31 H32 1 I I = H P 1 (5) H SEM 3 ORB H X = ( x1, x2,, x n ) Y y1 y2 y n = (,,, ) n k k = 1 DXY (, ) = x y, (6) xk {0,1} yk {0,1} H RANSAC [11] PROSAC H U n [12] 1) 0 2) n 3) n m H 1 2) U n I n I n 5% I n U n K 5% H k

4 ORB SEM SEM PROSAC 图 3 SEM 图像漂移矫正流程图 Fig. 3 The flow chart of SEM image drift correction 4 实验与结果分析 Windows 7 Intel(R)Core i5-2450m CPU 2.5 GHz 4 G VS2010+OpenCV FEI Quanta ESEM 300 SU ORB+RANSACORB+D_RANSAC(RANSAC+RAN SAC) SEM 4(a)5(a)6(a) 7(a) , SEM (ROI) 4(b)5(b)6(b) 7(b) 4(c)5(c) 6(c) 7(c) 1~4 H 11 H 12 H 21 H 22 H 13 H 23 (ROI) (MAE)MAE (a) (b) (c) 图 4 FEI 在倍率下漂移矫正 (a) 基准图 ;(b) 实时图 ;(c) 矫正图 Fig. 4 FEI drift correction at magnification. (a) Reference image; (b) Real-time image; (c) Corrective image

(a) (b) (c) 图 5 FEI 在 35000 倍率下漂移矫正 (a) 基准图 ;(b) 实时图 ;(c) 矫正图 Fig.

(a) Reference image; (b) Real-time image; (c) Corrective image (a) (b) (c) 图 6

6 SU8010 drift correction at 30000 magnification.

SU8010 在 35000 倍率下漂移矫正 (a) 基准图 ;(b) 实时图 ;(c) 矫正图 Fig.

(a) Reference image; (b) Real-time image; (c) Corrective image 表 1 FEI 在 30000

5 (a) (b) (c) 图 5 FEI 在倍率下漂移矫正 (a) 基准图 ;(b) 实时图 ;(c) 矫正图 Fig. 5 FEI drift correction at magnification. (a) Reference image; (b) Real-time image; (c) Corrective image (a) (b) (c) 图 6 SU8010 在倍率下漂移矫正 (a) 基准图 ;(b) 实时图 ;(c) 矫正图 Fig. 6 SU8010 drift correction at magnification. (a) Reference image; (b) Real-time image; (c) Corrective image (a) (b) (c) 图 7 SU8010 在倍率下漂移矫正 (a) 基准图 ;(b) 实时图 ;(c) 矫正图 Fig. 7 SU8010 drift correction at magnification. (a) Reference image; (b) Real-time image; (c) Corrective image 表 1 FEI 在倍率下单应矩阵参数 Table 1 Homography parameters computed at magnification in FEI H 11 H 12 H 21 H 22 H 13 H 23 ORB+RANSAC ORB+D_RANSAC

6 表 2 FEI 在倍率下单应矩阵参数 Table 2 Homography parameters computed at magnification in FEI H 11 H 12 H 21 H 22 H 13 H 23 ORB+RANSAC ORB+D_RANSAC 表 3 SU8010 在倍率下单应矩阵参数 Table 3 Homography parameters computed at magnification in SU8010 H 11 H 12 H 21 H 22 H 13 H 23 ORB+RANSAC ORB+D_RANSAC 表 4 SU8010 在倍率下单应矩阵参数 Table 4 Homography parameters computed at magnification in SU8010 H 11 H 12 H 21 H 22 H 13 H 23 ORB+RANSAC ORB+D_RANSAC MAE = I ˆ ROI IROI (7) MN M N 5~8 ORB+RANSAC MAE ORB+D_RANSAC SEM 表 5 FEI 漂移矫正平均绝对误差 Table 5 The MAE of FEI drift correction ORB+RANSAC ORB+D_RANSAC 表 6 FEI 漂移矫正帧率 Table 6 The frame rate of FEI drift correction (f/s) ORB+RANSAC ORB+D_RANSAC 表 7 SU8010 漂移矫正平均绝对误差 Table 7 The MAE of SU8010 drift correction ORB+RANSAC ORB+D_RANSAC 表 8 SU8010 漂移矫正帧率 Table 8 The frame rate of SU8010 drift correction (f/s) ORB+RANSAC ORB+D_RANSAC

7 5 结论 SEM ORB PROSAC SEM SEM SEM SEM 参考文献 [1] Cao Y Y, Xie F, Zhang P D, et al. Dual-beam super-resolution direct laser writing nanofabrication technology[j]. Opto-Electronic Engineering, 2017, 44(12): 曹耀宇, 谢飞, 张鹏达, 等. 双光束超分辨激光直写纳米加工技术 [J]. 光电工程,2017,44(12): [2] Gong Z, Chen B K, Sun Y, et al. Robotic Probing of Nanostructures inside Scanning Electron Microscopy[J]. IEEE Transactions on Robotics,2014,30(3): [3] Ye X T, Zhang Y, Ru C H, et al. Automated Pick-Place of Silicon Nanowires[J]. IEEE Transactions on Automation Science and Engineering, 2013, 10(3): [4] Ru C H, Zhang Y, Sun Y, et al. Automated Four-Point probe Measurement of Nanowires Inside a Scanning Electron Microscope[J]. IEEE Transactions on Nanotechnology, 2011, 10(4): [5] Gong Z, Chen B K, Sun Y, et al. Fluorescence and SEM correlative microscopy for nanomanipulation of subcellular structures[j]. Light: Science and Application, 2014, 3(11): e224. [6] Cizmar P, Vladar A E, Postek M T. Real-Time Scanning Charged-Particle Microscope Image Composition with Correction of Drift[J]. Microscopy and Microanalysis, 2011, 17(2): [7] Marturi N, Demele S, Piat N. Fast Image Drift Compensation in Scanning Electron Microscope using Image Registration[C]// Proceedings of 2013 IEEE International Conference on Automation Science and Engineering, 2013: [8] Xiao C B, Feng D Z, Feng X W. Fast RANSAC algorithm with resample optimization[j]. Journal of Computer-Aided Design and Computer Graphics, 2016, 28(4): 肖春宝, 冯大政, 冯祥卫. 重抽样优化的快速随机抽样一致性算法 [J]. 计算机辅助设计与图形学学报,2016,28(4): [9] Sutton M A, Li N, Li X D. Metrology in a scanning electron microscope: theoretical developments and experimental validation[j]. Measurement Science and Technology, 2006, 17(10): [10] Malti A C, Dembele S, Piat N, et al. Magnification-continuous static calibration model of a scanning- electron microscope[j]. Journal of Electronic Imaging, 2012, 21(3): [11] Marturi N. Vision and visual servoing for nanomanipulation and nanocharacterization in scanning electron microscope[d]: France: Univerite De France-comte, [12] Liu W, Zhao W J, Li C, et al. Detecting small moving target based on the improved ORB feature matching[j]. Opto-Electronic Engineering, 2015, 42(10): 刘威, 赵文杰, 李成, 等. 基于改进 ORB 特征匹配的运动小目标检测 [J]. 光电工程,2015,42(10):

8 Real-time correction of image drift in scanning electron microscope Xu Wei 1, Gu Sen 1, Chu Chengzhi 1, Jin Zhenwei 1, Ru Changhai 1,2* 1 Robotics and Microsystems Center of Soochow University, Suzhou, Jiangsu , China; 2 Collaborative Innovation Center of Suzhou Nano Science and Technology, Suzhou, Jiangsu , China The correction of image drift in SEM Overview: With the development of nanoscience, new nanomaterials and their excellent properties have been continuously discovered and recognized, showing a broad application prospect. Compared with the traditional optical microscope, the scanning electron microscope (SEM) has been widely used in the characterization of nanomaterials, micro deformation measurement with the advantages of its simple preparation, wide range of adjustable magnification, high resolution and great depth of field. In addition, the use of SEM images as visual sensors has merit in the development of automated nanomanipulations, such as automatic detection of IC chips, automatic pick-up of nanowires, and automatic measurement of nanowire impedance characteristics by four-point probes, making humans liberated from tedious nano-manipulations, greatly improving work efficiency. However, the drift of images caused by electron beam drift and electromagnetic interference at high magnification will affect the size measurement and characterization of nanomaterials in SEM. At present, domestic and foreign scholars have actively studied the correction method of SEM image drift, but there are still many obvious shortcomings. Cizmar et al. aligned the images correctly to form a single image by using larges of image frames, but which can t meet the requirements of the real-time in SEM. In order to solve the problem of real time, Naresh took the ORB combing the RANSAC to correct the drift image, but it does not eliminate the external points. The increase of the external points lead to the rapid increase of the sampling times of RANSAC, and then affects the efficiency of the algorithm and the accuracy of the measurement. Although AFM compensation algorithm has been able to solve the problem of image drift, it is not suitable for SEM because of the different principles. An image shift correction algorithm based on ORB (oriented FAST and rotated BRIEF) combing the PROSAC (progressive sample consensus) is proposed in this paper. Firstly, the ORB algorithm is used to detect the feature between the reference image and real-time image. Then the initial matching of the feature is implemented by using the Hamming distance and cross-matching. Moreover, the RANSAN (random sample consensus) optimization algorithm PROSAC is used to calculate the homography matrix between frames and the final exact homography matrix is re-iterated after eliminating exterior point. Finally, the SEM image drift is corrected in real time using the perspective transformation of the homography matrix. The experiments show that the proposed algorithm is high precision and satisfies the requirement of SEM real-time processing. This method can meet the requirements of the drift correction in SEM under different magnifications, and provide help for SEM measurement, characterization and other applications. Citation: Xu W, Gu S, Chu C Z, et al. Real-time correction of image drift in scanning electron microscope[j]. Opto-Electronic Engineering, 2018, 45(12): Supported by National Natural Science Foundation of China ( ) * rzh@suda.edu.cn

2 ( 自然科学版 ) 第 20 卷波 ). 这种压缩波空气必然有一部分要绕流到车身两端的环状空间中, 形成与列车运行方向相反的空气流动. 在列车尾部, 会产生低于大气压的空气流

2 ( 自然科学版 ) 第 20 卷波 ). 这种压缩波空气必然有一部分要绕流到车身两端的环状空间中, 形成与列车运行方向相反的空气流动. 在列车尾部, 会产生低于大气压的空气流第 20 卷第 3 期 2014 年 6 月 ( 自然科学版 ) JOURNAL OF SHANGHAI UNIVERSITY (NATURAL SCIENCE) Vol. 20 No. 3 June 2014 DOI: 10.3969/j.issn.1007-2861.2013.07.031 基于 FLUENT 测轨道交通隧道中电波折射率结构常数张永