9.6.2: Probability Formula
- Page ID
- 51695
\( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)
\( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)
\( \newcommand{\dsum}{\displaystyle\sum\limits} \)
\( \newcommand{\dint}{\displaystyle\int\limits} \)
\( \newcommand{\dlim}{\displaystyle\lim\limits} \)
\( \newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\)
( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\)
\( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\)
\( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\)
\( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\)
\( \newcommand{\Span}{\mathrm{span}}\)
\( \newcommand{\id}{\mathrm{id}}\)
\( \newcommand{\Span}{\mathrm{span}}\)
\( \newcommand{\kernel}{\mathrm{null}\,}\)
\( \newcommand{\range}{\mathrm{range}\,}\)
\( \newcommand{\RealPart}{\mathrm{Re}}\)
\( \newcommand{\ImaginaryPart}{\mathrm{Im}}\)
\( \newcommand{\Argument}{\mathrm{Arg}}\)
\( \newcommand{\norm}[1]{\| #1 \|}\)
\( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\)
\( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\AA}{\unicode[.8,0]{x212B}}\)
\( \newcommand{\vectorA}[1]{\vec{#1}} % arrow\)
\( \newcommand{\vectorAt}[1]{\vec{\text{#1}}} % arrow\)
\( \newcommand{\vectorB}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)
\( \newcommand{\vectorC}[1]{\textbf{#1}} \)
\( \newcommand{\vectorD}[1]{\overrightarrow{#1}} \)
\( \newcommand{\vectorDt}[1]{\overrightarrow{\text{#1}}} \)
\( \newcommand{\vectE}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash{\mathbf {#1}}}} \)
\( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)
\(\newcommand{\longvect}{\overrightarrow}\)
\( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)
\(\newcommand{\ket}[1]{\left| #1 \right>}\)
\(\newcommand{\bra}[1]{\left< #1 \right|}\)
\(\newcommand{\braket}[2]{\left< #1 \vphantom{#2} \right| \left. #2 \vphantom{#1} \right>}\)
\(\newcommand{\braopket}[3]{\left< #1 \vphantom{#2}\vphantom{#3} \right| #2 \vphantom{#1}\vphantom{#3} \left| #3 \vphantom{#1}\vphantom{#2} \right>}\)
\(\newcommand{\qmvec}[1]{\mathbf{\vec{#1}}}\)
\(\newcommand{\op}[1]{\hat{\mathbf{#1}}}\)
\(\newcommand{\expect}[1]{\langle #1 \rangle}\)
\(\newcommand{\dfn}[1]{\emph{\textbf{#1}}}\)
The probability distribution \(p(A_i)\) we want has been derived by others. It is a function of the dual variable \(\beta\):
\(p(A_i) = 2^{-\alpha} 2^{-\beta g(A_i)} \tag{9.12}\)
which implies
\(\log_2 \Big(\dfrac{1}{p(A_i)} \Big) = \alpha + \beta g(A_i) \tag{9.13}\)
where \(\alpha\) is a convenient abbreviation\(^2\) for this function of \(\beta\):
\(\alpha = \log_2 \Big( \displaystyle \sum_{i} 2^{-\beta g(A_i)} \Big) \tag{9.14}\)
Note that this formula for \(\alpha\) guarantees that the \(p(A_i)\) from Equation 9.12 add up to 1 as required by Equation 9.8.
If \(\beta\) is known, the function \(\alpha\) and the probabilities \(p(A_i)\) can be found and, if desired, the entropy \(S\) and the constraint variable \(G\). In fact, if \(S\) is needed, it can be calculated directly, without evaluating the \(p(A_i)\)—this is helpful if there are dozens or more probabilities to deal with. This short-cut is found by multiplying Equation 9.13 by \(p(A_i)\), and summing over \(i\). The left-hand side is \(S\) and the right-hand side simplifies because \(\alpha\) and \(\beta\) are independent of \(i\). The result is
\(S = \alpha + \beta G \tag{9.15}\)
where \(S\), \(\alpha\), and \(G\) are all functions of \(\beta\).
\(^2\)The function \(\alpha(\beta)\) is related to the partition function \(Z(\beta)\) of statistical physics: \(Z = 2^{\alpha}\) or \(\alpha = \log_2 Z\).