- PDF Free Download

Size: px

Start display at page:

Download ""

瞻韦
5 years ago
Views:

3 by

10 1 2

11 3 4

12 5 6 7

14 x 1,,x n n n X Y Z t T t T Y Y (X) X Z Z (X) X f (Y ) f : Y R g(z) g : Z R Y Ŷ Z Ẑ d( ) δ M N

15 1 1.1

16 X X Y Y Z Z t t T Y f : Y R Y Z g : Z R Z X X Ŷ = arg max f (Y ) Y (X) (X) X f (Y ) (X) = Y (X) Y (X) X X f (X,Y ) Y X X Ẑ = arg max g(z) Z (X) (X) X g(z) (X) = Z (X) Z (X) X X Y (X) = Z (Y ) Z (Y ) Y

17 X Y (X) = 2 X 1 Z (X) = ( T + 1) X 1 f (Y ) g(z) M M X Y Y

18 原始中文句子 X 原始中文句子 X 词性标注结果 Z 压缩表示 M 词性标注结果 Z

19 1.2.4

20 = = = 2 +

21 1.3.2

22 1.3.3

23 n 1.3.4

24 X = x 1,x 2,...x X x i X X X =,,,,, A = x 1,a 1, x 2,a 2,... x X,a X a i x i

25 a {,,, } A =,,,,,,,,,,, X Y A a = b,t b b {,,, } t t T A =,,,,,,,,,,,,,,,,, X Y A A(Y ) Y A(Z) Z X Ŷ Ẑ Ŷ = arg max f (Y ) = arg max Y Y (X) Y Y (X) ΛT Φ(A(Y )) = arg max λ p φ p (A(Y )) Y Y (X) p P Ẑ = arg max g(z) = arg max Z Z (X) Z Z (X) ΛT Φ(A(Z)) = arg max λ p φ p (A(Z)) Z Z (X) p P P p φ p (A) λ P f p (A) λ X Ŷ Ẑ

26 1,x i 1,a i 2,x i,a i 3,x i+1,a i 4,x i 2,x i 1,a i 5,x i 1,x i,a i 6,x i,x i+1,a i 7,x i+1,x i+2,a i 8,a i 1,a i p A i = 6 1,, A f p (A) A p f 1,, (A) = 1 i = f (A) λ n

27 X 1,Y1... X m,ym Φ n Λ Λ = i [1...n] j [1...m] Ŷ argmax Y Y (Xj ) Λ T Φ(Y ) Λ Λ + Φ(Y j ) Φ(Ŷ ) Λ i, j Λ Λ Λ i, j i a i i α(i,a i )

28 1.4

30 2 原始中文句子 X 分词二叉树压缩表示分词结果 Y 2.1

31 n y y l 2 l 1 r 1 r 2,y y p(x,y) = p(y)p(x y) x p(y) p(x y) p(x ) = p (l 1,r 1 )p (l 2 l 1,r 1 )p (r 2 l 1,r 1 ) p(x ) = p (l 1,r 1 )p (l 2 l 1,r 1 )p (r 2 l 1,r 1 ) l 1 r 1

32 y l 2 l 1 r 1 r 2 l 2 r 2 n p(y) p(x ) p(x ) x ŷ g(x) = log p( x) p(x ) p( ) = log p( x) p(x ) p( ) p( x) p( x) x, g(x) > 0 ŷ =, g(x) < p(x ) = p (l 2,l 1 )p (r 2,r 1 ) p(x ) = p (l 1,r 1 )p (l 2 l 1,r 1 )p (r 2 l 1,r 1 )

33 2.1.3 Kneser-Ney n

34 p(l 1,r 1 ) p(l 2 r 1 l 1 ) p(r 2 l 1 r 1 ) n x p(x) = P (l 1,r 1 )Pr 1,l 1 (l 2 )Pl 1,r 1 (r 2 ) P r 1,l 1 (l 2 ) P l 1,r 1 (r 2 ) c P (c) = max(0,c c d ) C + d t Pπ( ) C (c) C t = {c c c > 0} d i π( ) Pπ( ) (c) 1 c c t c c = c > 0 0 c c c = 0 C c = t c = c t c c = /0 P /0 (c) = max(0,c c d 0 ) C + d 0t P (c) C P (c) = 1 N

35 N P (l 1,r 1 ) P (l 1,r 1 ) = max(0,c l 1,r 1 d 2 ) c + d 2 t c l1 l1,r,r 1 P/0 (l 1 )P/0 (r 1 ) 1 n P /0 (l 1 ) P /0 (r 1 ) l 1 r 1 d i Gibbs x 1,...,x I ψ p(y) p(x y) k := 1000 n := 0 n s := 0 t := 1...T i := 1...I (x i,y t 1 i ) (x i ) y t i := (x i) y t i := (p(x i )e k(ns nψ) p(x i ) (x i,y t i ) n := n + 1 y t i = n := n + 1

36 x i x i x i () () () () p(x y) p(x y) () p(x y) () p(y) p(y) n n s e k(ns nψ) p( ) ψ k () p(x i ) p(x i ) d 0 d 1 d 2 d 0 d 1

37 g(x) g(x) Freq s c g(x)

38 ψ g( ) g ( ) = g( ) + δ δ δ

39 c 1 c 2...c n q(i) c i q(i) = g(c i 1,c i,c i+1,c i+2 ) c a...c b δ min(q(a 1),q(b)) > δ > max(q(a),q(a + 1),...,q(b 1)) g ( ) = g( ) + δ c a...c b min(q(a 1),q(b)) > 0 > max(q(a),q(a + 1),...,q(b 1)) δ q(1) = q(2) = q(3) = q(4) = q(5) = 1.644

40 2.2.2 c i c j (c i c j ) i = j c i m := argmax k=i,, j 1 q(k) c i c j (c i c m ) (c m+1 c j )

41 q() m c i c j X O( X log 2 X ) 2 X 1 X O(2 1 2 X ) p (Y X) p ( i X) p ( i X) i q(i) = log p ( i X) log p ( i X)

42 , p(, ) = 0, 1, q(i) < 0,i p q (, ) = 0, p(, ) = 1 p(, ) = 0

43 (T ) T. = T T L := T. R := T. p(l.,r.) = 1 T. T. (L) (R) T. T. T. T 2.2.4

44 (T ) T. = T T L := (T.) R := (T. ) L. = L R. = R p(l,r) = 1 T. T. L R

45 1, p (, ) = 0,

46 p () S S S S p () 0.95 > q(i) > 0.05 p () p (, ) 1 +1

47 s s s s P( X) P( a X) P( a X) a l r S l r m l 1,l l,r r,r +1 l 1,m m,r +1 χ 2 l,r a = m a + b = l a + c = r a + b + c + d χ 2 l 1,l r,r +1 l 1,m m,r +1 (ad bc) 2 (a+b)(a+c)(b+d)(c+d) log n log( n max n (n) n ) (n) n n n n

48 c i 1 t i,c i t i,c i+1 t i c i 2 c i 1 t i c i 1 c i t i c i c i+1 t i c i+1 c i+2 t i t i 1 t i p q(i) S S

49 2.4

50 3 原始中文句子 X 分词压缩表示 M 词性标注结果 Z

51 X f : Y R Ŷ = arg max f (Y ) Y (X) M (X) = {Y 1,...} M (X) = {Ŷ } d : Y 2 V Y d(y ) = {v 1,...,v d(y ) } V V Y (X) V M (X) = {Y d(y ) (X)} δ (X) = {v i Y (X)v i d(y ) f (Y ) f (Ŷ ) δ} d() Y δ d()

52 d (Y ) = { 1,v 1,..., X 1,v X 1 } v i {, } v i = i i + 1 v i = i i + 1 { 1,, 2,, 3,, 4,, 5, } (X) (X) = { i,v Y Y (X) i,v d (Y ) f (Y ) f (Ŷ ) δ} M (X) = {Y d (Y ) (X)} Ẑ = arg max g(z) Z M (X) (X) = { 1,, 2,, 3,, 4,, 4,, 5, } ( 用 )com( 率 ) ( 材 )com( 料 ) ( 料 )sep( 利 ) ( 利 )com( 用 ) ( 率 )sep( 高 ) ( 用 )sep( 率 ) M (X)

53 i i, i, i, i, δ = 0 (X) = d (Ŷ ) M (X) = {Ŷ } δ = (X) M (X) = Y (X) A d (Y ) = { 1,a 1,..., X,a X } a i i { 1,, 2,, 3,, 4,, 5,, 6, } ( 材 )B ( 料 )E ( 利 )B ( 用 )M ( 用 )E ( 率 )E ( 率 )S ( 高 )S

54 (X) M (X) M (X) M (X) Y M (X) M (X) d (Y ) (X) i,v d (Y ) Y i,v Y f (Y ) f (Ŷ ) δ i,v Y Y i,v d (Y ) i,a i Y M (X) Y i,a i Y f (Y ) f (Ŷ ) δ i,a i Y i,v Y Y Y d (Y ) = { b 1,e 1,..., b d (Y ),e d (Y ) } b i,e i b i + 1 e i { 0,2, 2,5, 5,6 } ( 利用率 ) ( 材料 ) ( 利用 ) ( 率 ) ( 高 ) (X) M (X) M (X) M (X)

55 f (Y ) n-best d (Y ) = {Y } (X) M (X) (X) = M (X) (X) (X) n n n ( 材料利用率高 ) ( 材料利用率高 ) n n = 1 n = 3.1.3

56 (X) = { i,a i f (Ŷ ) max Y, i,a i d (Y ) f (Y ) δ} i,a i max Y, i,ai d (Y ) f (Y ) f (Ŷ ) α(i,a i ) β(i,a i ) i a i e(i,a i ) max f (Y ) = α(i,a i) + β(i,a i ) e(i,a i ) Y, i,a i d (Y ) i,a i (X) O(4 2 X ) α f (Ŷ ) β O(4 2 X ) O(4 X ) O( X ) O( X ) X 2 O( X 2 ) n n O(nlnn X ) δ n n n

57 O( X ) O( X ) O( X 2 ) n O(nlnn X ) δ δ

58 F δ =10 δ =5 δ =0 δ =20 δ =30 δ =40 δ = / F δ =20 δ =10 δ =5 δ =0 δ =30 δ =40 δ = / δ =30 δ =20 δ =40 δ = δ =20 δ =40 δ =50 δ =30 F δ =10 δ =5 F δ =10 δ = δ = δ = / /

59 Ŷ = arg max Y Y (X) f (Y ) Ŷ Ẑ = arg max g(z) Z Z (Ŷ ) Ẑ = arg max Z Z (X) g(z) Z Z 4 O(4 2 X ) 4 T O(4 2 T 2 x ) 3.2.2

60 原始句子分词模型唯一分词结果词性标注模型词性标注结果原始句子分词模型适当数量的分词结果词性标注模型词性标注结果原始句子分词模型所有分词结果词性标注模型词性标注结果

61 δ n n

62 δ n δ n n δ = 0 n = 1 δ = n =

63 n n F δ x δ y δ δ = 15 x y n n δ n

64 F n-best F1 2 p(1 p) n p n = 8008

65 ± ± ± ± ± ± ± ± δ = 15 ± ± c t c t c i c j t c i c j t c i t c j c i c i t c j c i t i t j

66 c c i t t i

67 δ n n δ

68 F n-best F1 3.4

69 4 原始中文句子 X 词性标注压缩表示 N 词性标注结果 Z δ

70 X f (Y ) g(z) δ Ẑ f (Y ) X δ (X) g(z) (X) Ẑ Y (X) Z (X) O( X 3 T 2 ) f (Y ) g(z) g(z) g(z) f (Y ) f (Y ) f (Y ) g(z) n i

71 n δ f (X) g(x) n 1,..., n i [1...n] i f i (X) f i (X) δ i i g(x) f (X) i δ δ δ = 0 δ =

72 δ δ δ δ 4.2 n g g g d(z) g X N (X) Ẑ Ẑ = arg max Z N (X) g (d(z))

73 d(z) N (X) g (d(z)) d(z) = { b 1, 1,t 1,..., b d(z), d(z),t d(z) } d(z) d(z) b i, i,t i b i i i t i { 0,,, 2,,, 5,, } Z Z (X) = { b i, i,t i Z Y (X) b i, i,t i d(z) f (Z) f (Ẑ ) δ} N (X) = {Z d(z) (X)} { 0,,, 2,,, 2,,, 4,,, 5,, } N (X) = {, }

74 利用率,NN 材料,NN 利用,VV 率,NN 高,VA O( X 2 ) O( T 2 X ) O( T X 2 ) O( T 2 X + T X 2 ) d (A) = { b 1,a 1,..., b X,a X } b i,a i b i a i O( T X ) d d ({ 2,, }) = { 2,,, 3,,, 4,, }

75 i i,t i i =? 1 h(m i ) h(m i ), i =? 1 t i t i, i =? 1 t i 1,t i t i 1, i 1 =? 1,t i, i =? 1 (X) = {v i d (v i ) (X) Y Y (X)v i d(y ) f (Y ) f (Ŷ ) δ} v i d (v i ) (X) (X) v i v i δ m i = f (Ŷ ) max f (Y ) Y,v i d(y ) m i [0,δ] i m i = 0 v i v i d(ŷ ) i t i i,t i

76 h(m i ) h( ) i =? 1 i O( T 2 X 3 ) = X (X) X d(ŷ ) = X (X) d(ŷ ) X d(ŷ ) δ

77 δ =5 δ =10 δ =15 δ =20 δ =25 δ = δ =10 δ =5 δ =20 δ =15 δ =25 δ = δ = δ = (X) δ δ = 10 δ = 15 δ =

78 ± ± ± ± ± ± ± ± δ = 15 ± ± δ = 15 ± ± m i

80 4.3.2

81 x = [x 1,...x n ] T x = x n δ 2 = (x x)2 n 1

82 2 δ/n = (x x) 2 n(n 1) n δ

83 F F δ δ d F δ d F δ

84 11 / δ =5 δ =10 δ =15 δ =20 δ = F δ

85 # #

86 4.3.4 δ

87 δ = δ = δ δ = 0 δ δ δ

88 F F δ δ d F d F δ δ

89 / δ =0 δ =1 δ =3 δ =5 δ =7 δ =9 δ =11 δ =13 20 δ =15 10 δ =17 δ =19 δ = F1 δ = δ = δ δ δ δ = δ δ δ δ δ = 15 δ = 21 δ δ δ δ δ δ δ δ = 17 δ = 19

90 / δ 2 =0 δ 2 =1 δ 2 =3 δ 2 =5 δ 2 =7 δ 2 =9 δ 2 =11 δ 2 =13 δ 2 = F1 δ = 15 δ = δ 2 / δ 2 =0 δ 2 =1 δ 2 =3 δ 2 =5 δ 2 =7 δ 2 =9 δ 2 =11 20 δ 2 =13 δ 2 =15 10 δ 2 =17 2 =19 δ 2 = F δ = 21 δ = δ 2 δ δ = δ 2 δ δ δ = δ = 21

91 δ = δ =

93 5 原始中文句子 X 分词压缩表示 M 中文词判定任务 5.1

94 5.1.1

95 5.1.2

96 5.1.3

97 5.2 ( ) = ( ) =

98 , 1,, (,, ) = 0,,, P P ( ) = (, i, i ) i, i P

99 ,, P S P S (, ) = ( i,, ) i P S S ( ) = λ ( ) + (1 λ) ( )

100 ( ) ( ) λ

101

102 = = 5.4.2

103 GWSR-2000 WSR-2000 WSR-20 WSR λ

104 ,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, P S

105 ,,,,

106 RAV-2000 RAV-20 RAV-2000 AV-2000 AV-20 AV

107 RAV+GWSR RAV+WSR GWSR WSR RAV

108

109 6 原始中文句子大规模无标注语料词性标注压缩表示词性标注结果

110 n n-gram n n n n n n n n n

111 n n n,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, n-gram n n n 6.1.2

112 i i,t i i > 1 h(m i ) h(m i ), i > 1 t i t i, i > 1 t i 1,t i t i 1, i 1 > 1,t i, i > 1 t i,τ ( i ) t i,( i ) t i, ( i )/2 ( ) = {τ i } τ i t τ i t i,τ ( ) 1 ( ) = 0 ( i ) i

113 ± ± CTB δ = 21

114 F F RAV RAV 6.2

115 6.2.1 n P P Q 1,...,Q q q ( ) Q i,...,q q n Q 1,...,Q q n Q 1,...,Q q χ 2 p i p j (p i, p j ) = a b c d (ad bc) 2 (a + b)(a + c)(b + d)(c + d) G = V,E V p i e i, j E (p i, p j ) 1

116 p j n p j n p i n a b p i n c d,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Q 1,...,Q 246,,,,,, Q i ( ) ( )

117 δ = 22 ( ) k k = 30 ( )

118 F F RAV RAV + RAV RAV RAV + RAV ( i ) t i, ( i )/2, j, q j ( i ) i q j ( i ) j ( i )/2 w i

119

120 6.2.4

121 6.3

122 7 7.1

123 F RAV F RAV +

124

125 7.2.3

126

127

128

129

130

131

132

133

134

135

) & ( +,! (# ) +. + / & 6!!!.! (!,! (! & 7 6!. 8 / ! (! & 0 6! (9 & 2 7 6!! 3 : ; 5 7 6! ) % (. ()

) & ( +,! (# ) +. + / & 6!!!.! (!,! (! & 7 6!. 8 / ! (! & 0 6! (9 & 2 7 6!! 3 : ; 5 7 6! ) % (. () ! # % & & &! # % &! ( &! # )! ) & ( +,! (# ) +. + / 0 1 2 3 4 4 5 & 6!!!.! (!,! (! & 7 6!. 8 / 6 7 6 8! (! & 0 6! (9 & 2 7 6!! 3 : ; 5 7 6! ) % (. () , 4 / 7!# + 6 7 1 1 1 0 7!.. 6 1 1 2 1 3