6.4: Channel Model

Last updated
Save as PDF

Page ID: 50192

Paul Penfield, Jr.
Massachusetts Institute of Technology via MIT OpenCourseWare

\( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \) \( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)\(\newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\) \( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\) \( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\) \( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\) \( \newcommand{\Span}{\mathrm{span}}\) \(\newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\) \( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\) \( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\) \( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\) \( \newcommand{\Span}{\mathrm{span}}\)\(\newcommand{\AA}{\unicode[.8,0]{x212B}}\)

A communication channel accepts input bits and produces output bits. We model the input as the selection of one of a finite number of input states (for the simplest channel, two such states), and the output as a similar event. The language of probability theory will be useful in describing channels. If the channel perfectly changes its output state in conformance with its input state, it is said to be noiseless and in that case nothing affects the output except the input. Let us say that the channel has a certain maximum rate \(W\) at which its output can follow changes at the input (just as the source has a rate \(R\) at which symbols are selected).

We will use the index \(i\) to run over the input states, and \(j\) to index the output states. We will refer to the input events as \(A_i\) and the output events as \(B_j\). You may picture the channel as something with inputs and outputs, as in Figure 6.3, but note that the inputs are not normal signal inputs or electrical inputs to systems, but instead mutually exclusive events, only one of which is true at any one time. For simple channels such a diagram is simple because there are so few possible choices, but for more complicated structures there may be so many possible inputs that the diagrams become impractical (though they may be useful as a conceptual model). For example, a logic gate with three inputs, each of which could be 0 or 1, would have eight inputs in a diagram of this sort. The binary channel has two mutually exclusive input states and is the one pictured in Figure 6.3.

For a noiseless channel, where each of \(n\) possible input states leads to exactly one output state, each new input state (\(R\) per second) can be specified with \(\log_2 n\) bits. Thus for the binary channel, \(n\) = 2, and so the new state can be specified with one bit. The maximum rate at which information supplied to the input can

Screen Shot 2021-05-05 at 11.11.25 PM.png — Figure 6.4: Channel Loss Diagram. For an input data rate \(D\), either less than or greater than the channel capacity \(C\), the minimum possible rate at which information is lost is the greater of 0 and \(D − C\)

affect the output is called the channel capacity \(C = W \log_2 n\) bits per second. For the binary channel, \(C = W\).

If the input is changed at a rate \(R\) less than \(W\) (or, equivalently, if the information supplied at the input is less than \(C\)) then the output can follow the input, and the output events can be used to infer the identity of the input symbols at that rate. If there is an attempt to change the input more rapidly, the channel cannot follow (since \(W\) is by definition the maximum rate at which changes at the input affect the output) and some of the input information is lost.