22.1: Why do all this

Last updated
Save as PDF

Page ID: 88717

\( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \) \( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)\(\newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\) \( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\) \( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\) \( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\) \( \newcommand{\Span}{\mathrm{span}}\) \(\newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\) \( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\) \( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\) \( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\) \( \newcommand{\Span}{\mathrm{span}}\)\(\newcommand{\AA}{\unicode[.8,0]{x212B}}\)

It turns out there’s a lot of syntactic nonsense involved to get all the wiring right when you do this. It can cause students to pull their hair out. So it’s worth asking at the outset: what do we get for all this pain?

The answer is subtle, and can seem underwhelming at first, but it’s crucial. It essentially boils down to this lesson: any complex creative work (including a computer program) should be modular in its design. This means that it should be composed of smaller building blocks, which are in turn composed of still smaller building blocks. The entire thing should comprise an organized whole.

Any other way of doing it leads to madness.

Think of a car engine. When a mechanic opens up the hood, he or she doesn’t see just one big monolithic thing called “The Engine,” but rather piston assemblies, spark plugs, water pumps, drive shafts, and lots of other subsystems. It’s what allows piece by piece investigation of problems, and piece by piece replacement of bad parts.

Or think of a rock ’n’ roll tune. It’s not just an impenetrable mass of sound. Instead, it’s a collection of recognizable bass lines, drum sequences, vocal patterns, and variations on common guitar riffs. I don’t mean to minimize the creativity involved in its orchestration; in fact, the novel combination of the myriad possible building blocks is the creativity. If the song were just an impermeable wall of sound, it would be noise, not music.

In the same way, once your data analysis code approaches a certain size, it really must be written in a modular way or it will become a hopelessly tangled mess, what programmers refer to as “spaghetti code.” And the way to achieve this is by writing it in terms of functions that you then call at the appropriate time.

One other advantage to this approach, by the way, is that functions are reusable. Think of how many programmers all over the world have had reason to call np.sort(), or scipy.stats.pearsonr()! The same function becomes applicable in a variety of different contexts, so that nobody has to reinvent the wheel.