The information characteristics of sources and channels

The average quantity of information, which gives out a source in unit of time, is named as a source productivity

H =H/ t, bit per second,(bit/sec). (2.5)

For channels of transmitting information there is existing the similar characteristic - speed of information transmitting on the channel. It is possible to consider this speed as average quantity of information, which is transmitting on the channel in unit of time,

R=WH,bitper second, (2.6)

where W is the transmitting speed for electrical code signals;

Н is the average quantity of information, which carries one code signal.

Joint entropy of the message symbols.

The information characteristics of sources of the discrete messages are determined in item 10.1. Their analysis shows, that a source entropy of the discrete messages is its basic information characteristic, through which express the majority others. Therefore it is expedient to consider properties entropy of several next symbols, to show, as the average information quantity for several next symbols is formed, as influence this size non-uniformity of distribution of occurrence probabilities of the messages symbols and static connections between symbols.

Let's consider a syllable from two letters a_ia_i_.. If the occurrence of a symbol depends only on what was in the message the previous symbol, formation of syllables from two letters describes by Marcovian simple circuit. Entropy of joint occurrence of two symbols define, applying operation of averaging on all volume of the alphabet:

(2.7)

where is probability of formation of a syllable a_ia_i_. as probability of joint occurrence of symbols and ;

is the information quantity, which is necessary on a syllable.

(2.8)

where , - probability of occurrence ;

- probability of occurrence provided that before it has appeared ;

- probability of occurrence provided that before it has appeared , (10) is possible to present as

The conditional entropy can be determined through a syllable entropy:

, (2.9)

. (2.10)

The conditional entropy has the following property: if the symbols and also are independent,

<== попередня сторінка	\|	наступна сторінка ==>
Module control work №1	\|	The redundancy factor is determined on such formula

Не знайшли потрібну інформацію? Скористайтесь пошуком google:

Генерація сторінки за: 0.003 сек.