В.В. Лидовский - Теория Информации

  • Published on
    28-Jul-2015

  • View
    377

  • Download
    4

Embed Size (px)

Transcript

www.kodges.ru wbooks.ifolder.ru

www.kodges.ru

. .

www.kodges.ru

. .

2003www.kodges.ru

. . : . .: ?????????, 2003. 112 . ISBN ?-????-????-?.

. , . , , , . , , , Internet. . . . 23 . . 28. : (. -. . . . ), . .

plain TEX, AMS-Fonts, PICTEX TreeTEX

www.kodges.ru

108 , - 5 . . . . 1995 . (220200). (2, 9, 3336) , . : (...), , (..) .. , , . 1 9 , , .. , , , , , , , . 1018 . (-, , ), -. . . , . 19 . () , , . , , , . 2027 3

www.kodges.ru

. . 23- . 2832 . , , . . Internet. , , . , , . , . , , , . , , , . ., . . . . 1. . , , . , , . : , , , , . , . . . ( ) : , , , , . ( 1948 , ) (Wiener, ) (Shannon, ). 4

www.kodges.ru

, , , , , , , . , . , , . : . , , . . . . . 50- , , , ( ) . , , ( ). , , . . . , , .. () . , , , . ., , , . , , . , 5

www.kodges.ru

, , . , , . 2. . ( ). , , . ASCII (American Standard Code for Information Interchange), . ( , , , ) 0 127. :var i: byte; begin for i := 32 to 126 do write(i:6, chr(i):2); writeln end.

ASCII, ASCII (ASCII+), 256 ( 0 255). 128 ASCII , 128 . , ASCII . , , . , 65 ASCII A, , . A, , A, 9.5 , A . (, Fine Reader). 6

www.kodges.ru

1 W w ASCII? 3. : () (). , . , , . : , .. , , () . ( ) : - . , (T = 1/) (. . 1)................. ................... .... ..... .... ... ... .. . ... .... ... ... ... ... .. ... .. .. .. ... ... .. ... ... .. ... ... .. ... ... ... . ... .. .. .. ... . ............................................................................................................................... ........................................................... ............................................... .................... . .. .. .. . .. . ... ... ... .. .. . ........... ... .............. ... . ... ... ... ... . .. .. .. .. ... . . .. . . . .. ... ... .. ... ... ... ... ... .... .... ... ... .. ...... ....... .......... ........... .. ......

t

T

... .. . .............................................................................................................................. ............................................................................................................................. .. . . ... ... ... . .............. ............... .. . . ... . . ..

T . 1

t

, . , , , , , . . 7

www.kodges.ru

, (Nyquist). , , A sin(t + ), A , , t . , , [17]. -, . , , 20 , . 40 ( 44.1 ). , : , . , . , 3 . . (- ) ADC (Analog to Digital Convertor, A/D), (- ) DAC (Digital to Analog Convertor, D/A). 2 DAT 48 . , ? 4. , , . , . . . , . 8

www.kodges.ru

- - . , , . , , , , . . (bit, binary digit). . , . . . . , 8 (byte), 4- . B . , (K), (M), (G ), (T), (P ) . 10, : 210 = 1024 103 , 220 106 , 230 109 , 240 1012 , 250 1015 . , 1 KB = 8 bit = 1024 B = 8192 bit, 1 = 1024 = 1 048 576 = 8192 . , : ( ) , ( ) . , , . . , , . , f (t), F (t) f (t). , . , . , , . . 2 . , , , . 9

www.kodges.ru

. -: ( ), . , , . . P. 2 (baud): 1 = 1 / (bps). , , 10 Kbaud = 10240 baud. , . . , , .. . , . , , 5 . 3 ? 5. , (), () . . , ( ). . : , , . . 3. , , . , , . , , 1000 , 1 . : 140 , 3.1 , (10100 ) 10

www.kodges.ru

330 , (110 ) 30300 , ( ) 30 , ( ) 0.15 400 , ( ) 400700 , ( ) 0.71.75 . . . 3 : . , : () (ISDN, Integrated Services Digital Networks) 57128 . ( 110 ). . 50 ! 6. , , : 1. a = b, , a b. a2 = b2 , , , .. , . a3 = b3 , ; 2. . , ; 3. .. .. .. .., , .. ..; 4. . . . X, - .. Y = X + Z, Z .., . , .. Y , X. ( Z ), 11

www.kodges.ru

Y . Y X. 1865 . . 1921 . , , . 1948 . . , , , , . 4 x = 5 x > 3? 7. , .. . . . ... X Y , P (X = Xi ) = pi , P (Y = Yj ) = qj P (X = Xi , Y = Yj ) = pij , , X Y , I(X, Y ) =i,j

pij log2

pij . pi q j

. . X Y , pX (t1 ), pY (t2 ) pXY (t1 , t2 ), I(X, Y ) =R2

pXY (t1 , t2 ) log2

pXY (t1 , t2 ) dt1 dt2 . pX (t1 )pY (t2 )

, P (X = Xi , X = Xj ) = 0, i = j P (X = Xi ), i = j 12

www.kodges.ru

, , I(X, X) =i

pi log2

pi = p i pi

pi log2 pi .i

... X H(X) = HX = I(X, X). : 1) I(X, Y ) 0, I(X, Y ) = 0 X Y ; 2) I(X, Y ) = I(Y, X); 3) HX = 0 X ; 4) I(X, Y ) = HX + HY H(X, Y ), H(X, Y ) = i,j pij log2 pij ; 5) I(X, Y ) I(X, X). I(X, Y ) = I(X, X), X Y . 1) x ex1 x ( x = 1) x1 ln x x1 log2 x. ln 2 I(X, Y ) =i,j

pi q j pij log2 piji

piji,j

pi qj pij

1

ln 2

=

=i,j

pi qj pij = ln 2

pi

j

qj ln 2

i,j

pij

=

11 = 0, ln 2

. . I(X, Y ) = 0 pij = pi qj i j , . . X Y . X Y , pij = pi qj , , 1 , , 0, , I(X, Y ) = 0; 2) ; 3) HX = 0, , HX , , , X ; 4)

pij = pi ,j i

pij = qj ,

HX = i

pi log2 pi = i,j

pij log2 pi , pij log2 qji,j

HY = j

qj log2 qj = 13

www.kodges.ru

HX + HY H(X, Y ) =i,j

pij (log2 pij log2 qj log2 pi ) = I(X, Y ); HX

5) I(X, Y ) = HX + HY H(X, Y ) HY H(X, Y ) 0.

HY H(X, Y ) = i,j

pij log2 qj +i,j

pij log2 pij =i,j

pij log2 (pij /qj ),

pij = P (X = Xi , Y = Yj ) qj = P (Y = Yj ), 1 , , 0, , 0. HX = I(X, X) = I(X, Y ), i pij qj , 0. pij = P (X = Xi , Y = Yj ) = P (X = Xi /Y = Yj )P (Y = Yj ) {qj , 0} P (X = Xi /Y = Yj ) {0, 1}, , X Y .

. . X Y , , .. I(X, Y ) = 0. . . . . X1 , X2 Y . X1 X2 , 1- 2- , Y = X1 +X2 . I(Y, X1 ), I(X1 , X1 ), I(Y, Y ). ... X1 X2 , .. . X1 1 2 3 4 5 6 p 1/6 , .. j = 1...6 qj = P (X1 = j) = 1/6. ... Y , P (Y = i) = P (X1 + X2 = i), i = 2...12,

, X1 , X2 P (X1 = n, X2 = m) = P (X1 = n)P (X2 = m), pi = P (X1 + X2 = i) =1 n+m=i n,m 6

P (X1 = n)P (X2 = m) =1 n+m=i n,m 6

1/36.

, Y : 14

www.kodges.ru

X2 \

X1

1 2 3 4 5 6

1 2 3 4 5 6 7

2 3 4 5 6 7 8

3 4 5 6 7 8 9

4 5 6 7 8 9 10

5 6 6 7 7 8 8 9 9 10 10 11 11 12,

Y = X1 + X2 2 3 4 5 6 7 8 9 10 11 12 1 2 3 4 5 6 5 4 p /36 /36 /36 /36 /36 /36 /36 /36 3/36 2/36 1/36 , .. i = 2...12, pi = P (Y = i) = (6 |7 i|)/36. ... X1 Y pij = P (Y = i, X1 = j) = P (Y = i/X1 = j)P (X1 = j), , P (Y = 2, X1 = 1) = P (Y = 2/X1 = 1)P (X1 = 1) = = P (X2 = 1)P (X1 = 1) = 1/36. pij = P (Y = i, X1 = j) =X1 \ Y

1/36, 1 i j 0, . 7 1 /36 1 /36 1 /36 1 /36 1 /36 1 /36 8 0 1 /36 1 /36 1 /36 1 /36 1 /36 9 0 0 1 /36 1 /36 1 /36 1 /36 10 0 0 0 1 /36 1 /36 1 /36

6,

1 2 3 4 5 6

2 1 /36 0 0 0 0 0

3 1 /36 1 /36 0 0 0 0

4 1 /36 1 /36 1 /36 0 0 0

5 1 /36 1 /36 1 /36 1 /36 0 06

6 1 /36 1 /36 1 /36 1 /36 1 /36 0

11 0 0 0 0 1 /36 1 /36

12 0 0 0 0 0 1 /36

I(Y, X1 ) =j=1 1 ij 6

pij log26

pij = pi q j

1 = 367 8

log2j=1 1 ij 6

1 = 6pi12

1 1 1 1 1 = ( log2 + log2 + + log2 + log2 )= 36 i=2 6pi i=3 6pi 6pi i=7 6pi i=6 15

11

www.kodges.ru

=

1 6 6 6 6 6 6 ((log2 +log2 + +log2 )+ +(log2 +log2 + +log2 )) = 36 1 2 6 6 5 1 3 6 1 = (2 log2 6 + 4 log2 3 + 6 log2 2 + 8 log2 + 10 log2 + 6 log2 1) = 36 2 5

= (2 + 2 log2 3 + 4 log2 3 + 6 + 8 log2 3 8 + 10 log2 3 + 10 10 log2 5)/36 = = (10 + 24 log2 3 10 log2 5)/36 0.69 /. I(X1 , X1 ) = I(X2 , X2 ) = 2.58 /. 12 I(Y, Y ) = i=2 pi log2 pi = =6 j=1 qj

log2 qj = log2 6 = 1 + log2 3

1 36 (2 log2 36 + 4 log2 18 + 6 log2 12 + 8 log2 9 + 10 log2 + 6 log2 6)