by

1 2

3 4

5 6 7

x 1,,x n n n X Y Z t T t T Y Y (X) X Z Z (X) X f (Y ) f : Y R g(z) g : Z R Y Ŷ Z Ẑ d( ) δ M N

1 1.1

X X Y Y Z Z t t T Y f : Y R Y Z g : Z R Z X X Ŷ = arg max f (Y ) Y (X) (X) X f (Y ) (X) = Y (X) Y (X) X X f (X,Y ) Y X X Ẑ = arg max g(z) Z (X) (X) X g(z) (X) = Z (X) Z (X) X X Y (X) = Z (Y ) Z (Y ) Y

1.2 1.2.1 X Y (X) = 2 X 1 Z (X) = ( T + 1) X 1 f (Y ) g(z) M M X Y Y

原始中文句子 X 原始中文句子 X 词性标注结果 Z 压缩表示 M 词性标注结果 Z 1.2.2 1.2.3

1.2.4

1.3 1.3.1 = = = 2 +

1.3.2

1.3.3

n 1.3.4

1.3.5 1.3.5.1 X = x 1,x 2,...x X x i X X X =,,,,, A = x 1,a 1, x 2,a 2,... x X,a X a i x i

a {,,, } A =,,,,,,,,,,, X Y A a = b,t b b {,,, } t t T A =,,,,,,,,,,,,,,,,, X Y A A(Y ) Y A(Z) Z X Ŷ Ẑ Ŷ = arg max f (Y ) = arg max Y Y (X) Y Y (X) ΛT Φ(A(Y )) = arg max λ p φ p (A(Y )) Y Y (X) p P Ẑ = arg max g(z) = arg max Z Z (X) Z Z (X) ΛT Φ(A(Z)) = arg max λ p φ p (A(Z)) Z Z (X) p P P p φ p (A) λ P f p (A) λ X Ŷ Ẑ

1,x i 1,a i 2,x i,a i 3,x i+1,a i 4,x i 2,x i 1,a i 5,x i 1,x i,a i 6,x i,x i+1,a i 7,x i+1,x i+2,a i 8,a i 1,a i 1.3.5.2 p A i = 6 1,, A f p (A) A p f 1,, (A) = 1 i = 6 1.3.5.3 f (A) λ n

X 1,Y1... X m,ym Φ n Λ Λ = i [1...n] j [1...m] Ŷ argmax Y Y (Xj ) Λ T Φ(Y ) Λ Λ + Φ(Y j ) Φ(Ŷ ) Λ i, j Λ Λ Λ i, j 1.3.5.4 i a i i α(i,a i )

1.4

2 原始中文句子 X 分词二叉树压缩表示 分词结果 Y 2.1

n y 2.1.1 y l 2 l 1 r 1 r 2,y y p(x,y) = p(y)p(x y) x p(y) p(x y) p(x ) = p (l 1,r 1 )p (l 2 l 1,r 1 )p (r 2 l 1,r 1 ) p(x ) = p (l 1,r 1 )p (l 2 l 1,r 1 )p (r 2 l 1,r 1 ) l 1 r 1

y l 2 l 1 r 1 r 2 l 2 r 2 n p(y) p(x ) p(x ) x ŷ g(x) = log p( x) p(x ) p( ) = log p( x) p(x ) p( ) p( x) p( x) x, g(x) > 0 ŷ =, g(x) < 0 2.1.2 p(x ) = p (l 2,l 1 )p (r 2,r 1 ) p(x ) = p (l 1,r 1 )p (l 2 l 1,r 1 )p (r 2 l 1,r 1 )

2.1.3 Kneser-Ney n

p(l 1,r 1 ) p(l 2 r 1 l 1 ) p(r 2 l 1 r 1 ) n x p(x) = P (l 1,r 1 )Pr 1,l 1 (l 2 )Pl 1,r 1 (r 2 ) P r 1,l 1 (l 2 ) P l 1,r 1 (r 2 ) c P (c) = max(0,c c d ) C + d t Pπ( ) C (c) C t = {c c c > 0} d i π( ) Pπ( ) (c) 1 c c t c c = c > 0 0 c c c = 0 C c = t c = c t c c = /0 P /0 (c) = max(0,c c d 0 ) C + d 0t P (c) C P (c) = 1 N

N P (l 1,r 1 ) P (l 1,r 1 ) = max(0,c l 1,r 1 d 2 ) c + d 2 t c l1 l1,r,r 1 P/0 (l 1 )P/0 (r 1 ) 1 n P /0 (l 1 ) P /0 (r 1 ) l 1 r 1 d i 2.1.4 Gibbs x 1,...,x I ψ p(y) p(x y) k := 1000 n := 0 n s := 0 t := 1...T i := 1...I (x i,y t 1 i ) (x i ) y t i := (x i) y t i := (p(x i )e k(ns nψ) p(x i ) (x i,y t i ) n := n + 1 y t i = n := n + 1

x i x i x i () () () () p(x y) p(x y) () p(x y) () p(y) p(y) n n s e k(ns nψ) p( ) ψ k () p(x i ) p(x i ) 2.1.5 2.1.5.1 d 0 d 1 d 2 d 0 d 1

2.1.5.2 g(x) g(x) Freq. 6000 4000 2000 s c 0 20 15 10 5 0 5 10 15 20 25 g(x)

2.1.5.3 ψ 2.2 2.2.1 g( ) g ( ) = g( ) + δ δ δ

c 1 c 2...c n q(i) c i q(i) = g(c i 1,c i,c i+1,c i+2 ) c a...c b δ min(q(a 1),q(b)) > δ > max(q(a),q(a + 1),...,q(b 1)) g ( ) = g( ) + δ c a...c b min(q(a 1),q(b)) > 0 > max(q(a),q(a + 1),...,q(b 1)) δ q(1) = 2.517 q(2) = 2.194 q(3) = 2.027 q(4) = 1.791 q(5) = 1.644

2.2.2 c i c j (c i c j ) i = j c i m := argmax k=i,, j 1 q(k) c i c j (c i c m ) (c m+1 c j )

q() m c i c j X O( X log 2 X ) 2 X 1 X O(2 1 2 X ) p (Y X) p ( i X) p ( i X) i q(i) = log p ( i X) log p ( i X)

2.2.3 1, p(, ) = 0, 1, q(i) < 0,i p q (, ) = 0, p(, ) = 1 p(, ) = 0

(T ) T. = T T L := T. R := T. p(l.,r.) = 1 T. T. (L) (R) T. T. T. T 2.2.4

(T ) T. = T T L := (T.) R := (T. ) L. = L R. = R p(l,r) = 1 T. T. L R

1, p (, ) = 0,

2.2.5 2.3 2.3.1 p () S S S S p () 0.95 > q(i) > 0.05 p () p (, ) 1 +1

s s s s P( X) P( a X) P( a X) a l r S l r m l 1,l l,r r,r +1 l 1,m m,r +1 χ 2 l,r a = m a + b = l a + c = r a + b + c + d χ 2 l 1,l r,r +1 l 1,m m,r +1 (ad bc) 2 (a+b)(a+c)(b+d)(c+d) log n log( n max n (n) n ) (n) n n n n 2.3.2 2.3.2.1

c i 1 t i,c i t i,c i+1 t i c i 2 c i 1 t i c i 1 c i t i c i c i+1 t i c i+1 c i+2 t i t i 1 t i p q(i) S S 2.3.2.2

2.4

3 原始中文句子 X 分词压缩表示 M 词性标注结果 Z 3.1 3.1.1

X f : Y R Ŷ = arg max f (Y ) Y (X) M (X) = {Y 1,...} M (X) = {Ŷ } d : Y 2 V Y d(y ) = {v 1,...,v d(y ) } V V Y (X) V M (X) = {Y d(y ) (X)} δ (X) = {v i Y (X)v i d(y ) f (Y ) f (Ŷ ) δ} d() Y δ 3.1.2 d() 3.1.2.1

d (Y ) = { 1,v 1,..., X 1,v X 1 } v i {, } v i = i i + 1 v i = i i + 1 { 1,, 2,, 3,, 4,, 5, } (X) (X) = { i,v Y Y (X) i,v d (Y ) f (Y ) f (Ŷ ) δ} M (X) = {Y d (Y ) (X)} Ẑ = arg max g(z) Z M (X) (X) = { 1,, 2,, 3,, 4,, 4,, 5, } ( 用 )com( 率 ) ( 材 )com( 料 ) ( 料 )sep( 利 ) ( 利 )com( 用 ) ( 率 )sep( 高 ) ( 用 )sep( 率 ) M (X)

i i, i, i, i, δ = 0 (X) = d (Ŷ ) M (X) = {Ŷ } δ = (X) M (X) = Y (X) 3.1.2.2 A d (Y ) = { 1,a 1,..., X,a X } a i i { 1,, 2,, 3,, 4,, 5,, 6, } ( 材 )B ( 料 )E ( 利 )B ( 用 )M ( 用 )E ( 率 )E ( 率 )S ( 高 )S

(X) M (X) M (X) M (X) Y M (X) M (X) d (Y ) (X) i,v d (Y ) Y i,v Y f (Y ) f (Ŷ ) δ i,v Y Y i,v d (Y ) i,a i Y M (X) Y i,a i Y f (Y ) f (Ŷ ) δ i,a i Y i,v Y Y Y 3.1.2.3 d (Y ) = { b 1,e 1,..., b d (Y ),e d (Y ) } b i,e i b i + 1 e i { 0,2, 2,5, 5,6 } ( 利用率 ) ( 材料 ) ( 利用 ) ( 率 ) ( 高 ) (X) M (X) M (X) M (X)

f (Y ) 3.1.2.4 n-best d (Y ) = {Y } (X) M (X) (X) = M (X) (X) (X) n n n ( 材料 利用 率 高 ) ( 材料 利用率 高 ) n n = 1 n = 3.1.3

(X) = { i,a i f (Ŷ ) max Y, i,a i d (Y ) f (Y ) δ} i,a i max Y, i,ai d (Y ) f (Y ) f (Ŷ ) α(i,a i ) β(i,a i ) i a i e(i,a i ) max f (Y ) = α(i,a i) + β(i,a i ) e(i,a i ) Y, i,a i d (Y ) i,a i (X) O(4 2 X ) α f (Ŷ ) β O(4 2 X ) O(4 X ) O( X ) O( X ) X 2 O( X 2 ) n n O(nlnn X ) δ n n n

O( X ) O( X ) O( X 2 ) n O(nlnn X ) 3.1.4 δ δ 3.2 3.2.1

F1 1.000 0.995 0.990 0.985 0.980 0.975 0.970 0.965 δ =10 δ =5 δ =0 δ =20 δ =30 δ =40 δ =50 0.960 0.5 1.0 1.5 2.0 2.5 3.0 3.5 / F1 1.000 0.995 0.990 0.985 0.980 0.975 0.970 0.965 δ =20 δ =10 δ =5 δ =0 δ =30 δ =40 δ =50 0.960 0.5 1.0 1.5 2.0 2.5 3.0 3.5 / 1.00 0.99 0.98 δ =30 δ =20 δ =40 δ =50 1.00 0.99 0.98 δ =20 δ =40 δ =50 δ =30 F1 0.97 0.96 δ =10 δ =5 F1 0.97 0.96 δ =10 δ =5 0.95 δ =0 0.95 δ =0 0.94 0.5 1.0 1.5 2.0 2.5 3.0 3.5 / 0.94 0.5 1.0 1.5 2.0 2.5 3.0 3.5 /

Ŷ = arg max Y Y (X) f (Y ) Ŷ Ẑ = arg max g(z) Z Z (Ŷ ) Ẑ = arg max Z Z (X) g(z) Z Z 4 O(4 2 X ) 4 T O(4 2 T 2 x ) 3.2.2

原始句子 分词模型 唯一分词结果 词性标注模型 词性标注结果 原始句子 分词模型 适当数量的分词结果 词性标注模型 词性标注结果 原始句子 分词模型 所有分词结果 词性标注模型 词性标注结果

δ n 3.2.3 n 3.2.3.1

δ 3.2.3.2 n δ n n δ = 0 n = 1 δ = n =

n n F1 0.936 0.935 0.934 0.933 0.932 0.931 0.930 0.929 0.928 0 5 10 15 20 25 δ x δ y δ δ = 15 x y n n δ n

F1 0.936 0.935 0.934 0.933 0.932 0.931 0.930 0.929 n-best 0.928 0.9730.974 0.9750.9760.9770.9780.9790.9800.981 F1 2 p(1 p) n p n = 8008

± ± ± ± ± ± ± ± δ = 15 ± ± c t c t c i c j t c i c j t c i t c j c i c i t c j c i t i t j 3.3 3.3.1

c c i t t i 3.3.2 3.3.2.1 3.3.2.2

δ n n δ

0.888 0.886 F1 0.884 0.882 0.880 n-best 0.936 0.938 0.940 0.942 0.944 F1 3.4

4 原始中文句子 X 词性标注压缩表示 N 词性标注结果 Z 4.1 4.1.1 δ

X f (Y ) g(z) δ Ẑ f (Y ) X δ (X) g(z) (X) Ẑ 4.1.2 Y (X) Z (X) O( X 3 T 2 ) f (Y ) g(z) g(z) g(z) f (Y ) f (Y ) f (Y ) g(z) n i

n δ f (X) g(x) n 1,..., n i [1...n] i f i (X) f i (X) δ i i g(x) f (X) i δ δ δ = 0 δ =

δ δ δ δ 4.2 n g g g d(z) g X N (X) Ẑ Ẑ = arg max Z N (X) g (d(z))

d(z) N (X) g (d(z)) 4.2.1 d(z) = { b 1, 1,t 1,..., b d(z), d(z),t d(z) } d(z) d(z) b i, i,t i b i i i t i { 0,,, 2,,, 5,, } Z Z (X) = { b i, i,t i Z Y (X) b i, i,t i d(z) f (Z) f (Ẑ ) δ} N (X) = {Z d(z) (X)} { 0,,, 2,,, 2,,, 4,,, 5,, } N (X) = {, }

利用率,NN 材料,NN 利用,VV 率,NN 高,VA 4.2.2 O( X 2 ) O( T 2 X ) O( T X 2 ) O( T 2 X + T X 2 ) d (A) = { b 1,a 1,..., b X,a X } b i,a i b i a i O( T X ) d d ({ 2,, }) = { 2,,, 3,,, 4,, }

i i,t i i =? 1 h(m i ) h(m i ), i =? 1 t i t i, i =? 1 t i 1,t i t i 1, i 1 =? 1,t i, i =? 1 (X) = {v i d (v i ) (X) Y Y (X)v i d(y ) f (Y ) f (Ŷ ) δ} v i d (v i ) (X) (X) v i 4.2.3 v i δ m i = f (Ŷ ) max f (Y ) Y,v i d(y ) m i [0,δ] i m i = 0 v i v i d(ŷ ) i t i i,t i

h(m i ) h( ) i =? 1 i O( T 2 X 3 ) 4.2.4 4.2.4.1 = X (X) X d(ŷ ) = X (X) d(ŷ ) X d(ŷ ) δ

1.000 0.995 0.990 0.985 0.980 δ =5 δ =10 δ =15 δ =20 δ =25 δ =30 1.00 0.99 0.98 0.97 0.96 δ =10 δ =5 δ =20 δ =15 δ =25 δ =30 0.975 0.95 0.970 δ =0 0.965 1.0 1.2 1.4 1.6 1.8 2.0 2.2 2.4 2.6 2.8 0.94 δ =0 0.93 0 5 10 15 20 (X) δ δ = 10 δ = 15 δ = 15 4.2.4.2

± ± ± ± ± ± ± ± δ = 15 ± ± δ = 15 ± ± m i 4.2.4.3

4.3 4.3.1

4.3.2

x = [x 1,...x n ] T x = x n δ 2 = (x x)2 n 1

2 δ/n = (x x) 2 n(n 1) 4.3.3 n 4.3.3.1 δ

0.964 0.918 0.962 0.960 F1 0.916 0.914 0.912 F1 0.958 0.910 0.956 0.954 0.908 0.906 0.904 0.952 0 5 10 15 20 25 δ 0.902 0 5 10 15 20 25 δ d F1 0.0018 0.0016 0.0014 0.0012 0.0010 0.0008 0.0006 10 15 20 δ d F1 0.0010 0.0009 0.0008 0.0007 0.0006 0.0005 0.0004 0.0003 0.0002 0.0001 10 15 20 δ

11 / 10 9 8 7 δ =5 δ =10 δ =15 δ =20 δ =25 6 5 0.904 0.906 0.908 0.910 0.912 0.914 0.916 F1 4.3.3.2 δ

4.3.3.3 # #

4.3.4 δ

δ = 15 4.3.4.1 δ = δ δ = 0 δ δ δ

0.966 0.920 0.965 0.919 0.964 0.918 F1 0.963 0.962 F1 0.917 0.916 0.915 0.961 0.914 0.960 0.913 0.959 0 1 3 5 7 9 11 13 15 17 19 21 δ 0.912 0 1 3 5 7 9 11 13 15 17 19 21 δ 0.0035 0.0030 0.0025 0.0045 0.0040 0.0035 d F1 0.0020 d F1 0.0030 0.0015 0.0025 0.0010 11 13 15 17 19 21 δ 0.0020 11 13 15 17 19 21 δ

/ 70 60 50 40 30 δ =0 δ =1 δ =3 δ =5 δ =7 δ =9 δ =11 δ =13 20 δ =15 10 δ =17 δ =19 δ =21 0 0.913 0.914 0.915 0.916 0.917 0.918 0.919 F1 δ = δ = δ δ 4.3.4.2 δ δ = δ δ δ δ δ = 15 δ = 21 δ δ δ δ δ δ δ δ = 17 δ = 19

/ 60 50 40 30 20 10 δ 2 =0 δ 2 =1 δ 2 =3 δ 2 =5 δ 2 =7 δ 2 =9 δ 2 =11 δ 2 =13 δ 2 =15 0 0.913 0.914 0.915 0.916 0.917 0.918 0.919 F1 δ = 15 δ = δ 2 / 60 50 40 30 δ 2 =0 δ 2 =1 δ 2 =3 δ 2 =5 δ 2 =7 δ 2 =9 δ 2 =11 20 δ 2 =13 δ 2 =15 10 δ 2 =17 2 =19 δ 2 =21 0 0.913 0.914 0.915 0.916 0.917 F1 0.918 0.919 δ = 21 δ = δ 2 δ δ = δ 2 δ δ 4.3.4.3 δ = δ = 21

δ = δ = 21 4.4

5 原始中文句子 X 分词压缩表示 M 中文词判定任务 5.1

5.1.1

5.1.2

5.1.3

5.2 ( ) = ( ) =

5.3 5.3.1, 1,, (,, ) = 0,,, P P ( ) = (, i, i ) i, i P

,, 5.3.2 P S P S (, ) = ( i,, ) i P S S 5.3.3 ( ) = λ ( ) + (1 λ) ( )

( ) ( ) λ 5.4 5.4.1 5.4.1.1

10 9 10 8 2000 10 7 20 10 6 2000 10 5 10 4 10 3 10 2 10 1 10 0 10 0 10 1 10 2 10 3 10 4 10 5 10 6 10 9 10 8 2000 10 7 20 10 6 2000 10 5 10 4 10 3 10 2 10 1 10 0 0 200 400 600 800 10001200140016001800

5.4.1.2 = = 5.4.2

0.8 0.7 0.6 GWSR-2000 WSR-2000 WSR-20 WSR-2000 0.5 0.4 0.3 0.2 0.15 0.20 0.25 0.30 0.35 0.40 0.45 0.50 0.55 λ

,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, 5.4.3 P S

,,,,

0.6 0.5 0.4 0.3 RAV-2000 RAV-20 RAV-2000 AV-2000 AV-20 AV-2000 0.2 0.1 0.0 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 5.4.4

0.8 0.7 0.6 RAV+GWSR RAV+WSR GWSR WSR RAV 0.5 0.4 0.3 0.2 0.1 0.2 0.3 0.4 0.5 0.6 0.7 5.5

6 原始中文句子 大规模无标注语料 词性标注压缩表示词性标注结果 6.1 6.1.1

n 6.1.1.1 n-gram n n n n n n 6.1.1.2 n n n

n n n,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, 6.1.1.3 n-gram n n n 6.1.2

i i,t i i > 1 h(m i ) h(m i ), i > 1 t i t i, i > 1 t i 1,t i t i 1, i 1 > 1,t i, i > 1 t i,τ ( i ) t i,( i ) t i, ( i )/2 ( ) = {τ i } τ i t τ i t i,τ ( ) 1 ( ) = 0 ( i ) i

± ± 6.1.3 6.1.3.1 CTB5 6.1.3.2 δ = 21

0.972 0.928 0.970 0.926 F1 0.968 0.966 F1 0.924 0.922 0.920 0.964 0.918 0.962 RAV 0.916 RAV 6.2

6.2.1 n P P Q 1,...,Q q q ( ) Q i,...,q q 6.2.1.1 n Q 1,...,Q q n Q 1,...,Q q χ 2 p i p j (p i, p j ) = a b c d (ad bc) 2 (a + b)(a + c)(b + d)(c + d) G = V,E V p i e i, j E (p i, p j ) 1

p j n p j n p i n a b p i n c d,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Q 1,...,Q 246,,,,,, Q i ( ) 6.2.1.2 ( )

δ = 22 ( ) k k = 30 ( )

0.974 0.932 0.972 0.970 F1 0.930 0.928 0.926 F1 0.968 0.924 0.966 0.964 0.922 0.920 0.918 0.962 RAV RAV + RAV + + 0.916 RAV RAV + RAV + + 6.2.2 ( i ) t i, ( i )/2, j, q j ( i ) i q j ( i ) j ( i )/2 w i 6.2.3 6.2.3.1

6.2.3.2

6.2.4

6.3

7 7.1

0.975 0.970 F1 0.965 0.960 0.955 0.950 RAV + 0.935 0.930 0.925 F1 0.920 0.915 0.910 0.905 0.900 RAV +

7.2 7.2.1 7.2.2

7.2.3