10: Physical Systems

Until now we have ignored most aspects of physical systems by dealing only with abstract ideas such as information. Although we assumed that each bit stored or transmitted was manifested in some physical object, we focused on the abstract bits, and ignored any limitations caused by the laws of physics. This is the fundamental mantra of the information age.

It has not always been that way, and it will not be that way in the future. In past centuries, the physical manifestation of information was of great importance because of its great cost. To preserve or communicate information, books had to be written or even words cut into stone. For example, think of the process of creating a medieval manuscript during the middle ages. Pages were laboriously copied and illustrated. The results may be viewed today with great admiration for their artistry and cultural importance, in part because they were so expensive to create—society could only afford to deal with what it considered the most important information, and the cost of superb art work was not high compared with the other costs of production.

Advances over the years have improved the efficiency of information storage and transmission—think of the printing press, telegraph, telephone, radio, television, digital signal processing, semiconductors, and fiber optics. These technologies have led to sophisticated systems such as computers and data networks, and have shaped the methods used for the creation and distribution of information-intensive products by companies such as those in the entertainment business. As the cost of processing and distributing data drops, it is relevant to consider the case where that cost is small compared to the cost of creating, maintaining, and using the information. It is in this domain that the abstract ideas of information theory, bits, coding, and indeed all of computer science are dominant. All sectors of modern society are coping with the increasing amount of information that is available. Fundamental ideas of intellectual property, copyrights, patents, and trade secrets are being rethought in light of the changing economics of information processing. Welcome to the information age.

The model of information separated from its physical embodiment is, of course, an approximation of reality. Eventually, as we make microelectronic systems more and more complicated, using smaller and smaller components, we will need to face the fundamental limits imposed not by our ability to fabricate small structures, but by the laws of physics. All physical systems are governed by quantum mechanics.

Quantum mechanics is often believed to be of importance only for small structures, say the size of an atom. Although it is unavoidable at that length scale, it also governs everyday objects. When dealing with information processing in physical systems, it is pertinent to consider both very small systems with a small number of bits of information, and large systems with large amounts of information.

The key ideas we have used thus far that need to be re-interpreted in the regime where quantum mechanics is important include

• The digital abstraction made practical by devices that can restore data with small perturbations
• Use of probability to express our knowledge in the face of uncertainty
• The Principle of Maximum Entropy as a technique to estimate probabilities without bias