2.6: Memory Hierarchy

Last updated
Save as PDF

Page ID: 19867

\( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \) \( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)\(\newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\) \( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\) \( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\) \( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\) \( \newcommand{\Span}{\mathrm{span}}\) \(\newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\) \( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\) \( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\) \( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\) \( \newcommand{\Span}{\mathrm{span}}\)\(\newcommand{\AA}{\unicode[.8,0]{x212B}}\)

In order to fully understand the various different memory levels and associated usage, it is useful to review the memory hierarchy(For more information, refer to: http://en.Wikipedia.org/wiki/Memory_hierarchy). In general terms, faster memory is more expensive and slower memory blocks are less expensive. The CPU registers are small, fast, and expensive. Secondary storage devices such as disk drives and Solid State Drives (SSD's) are larger, slower, and less expensive. The overall goal is to balance performance with cost.

An overview of the memory hierarchy is as follows:

截屏2021-07-18 下午3.49.56.png

Where the top of the triangle represents the fastest, smallest, and most expensive memory. As we move down levels, the memory becomes slower, larger, and less expensive. The goal is to use an effective balance between the small, fast, expensive memory and the large, slower, and cheaper memory.

Some typical performance and size characteristics are as follows:

Memory Unit	Example Size	Typical Speed
Registers	16, 64-bit registers	~1 nonoseconds(For more information, refer to: http://en.Wikipedia.org/wiki/Nanosecond)
Cache Memory	4-8+ Megabytes(For more information, refer to: http://en.Wikipedia.org/wiki/Megabyte) (L1 and L2)	~5-60 nonoseconds
Primary Storage (i.e., main memory)	2-32+ Gigabytes (For more information, refer to: http://en.Wikipedia.org/wiki/Gigabyte)	~100-150 nanoseconds
Secondary Stroage (i.e., disk, SSD's, etc.)	500 Gigabytes- 4+ Terabytes(For more information, refer to: http://en.Wikipedia.org/wiki/Terabyte)	~3-15 milliseconds(For more information, refer to: http://en.Wikipedia.org/wiki/Millisecond)

Based on this table, a primary storage access at 100 nanoseconds (\(100 \times 10^{-9}\)) is 30,000 times faster than a secondary storage access, at 3 milliseconds (\(3 \times 10^{-3}\)).

The typical speeds improve over time (and these are already out of date). The key point is the relative difference between each memory unit is significant. This difference between the memory units applies even as newer, faster SSDs are being utilized.