17.1: Why Buffer?

In order to fully understand the importance of buffering, it is useful to review the memory hierarchy as outlined in Chapter 2, Architecture. As noted, accesses to secondary storage are significantly more expensive in terms of run-time than accesses to main memory. This would strongly encourage us to limit the number of secondary storage accesses by using some temporary storage, referred to as a buffer, in main memory.

Such a buffer would also help reduce the overhead associated with system calls. For example, a file read system call would involve pausing our program and turning over control to the OS. The OS would validate our request (e.g., ensure file descriptor is valid) and then pass the request to the secondary storage control via the system bus. The controller would do whatever is necessary to obtain the requested data and place it directly into main memory location as instructed by the OS, again accessing the system bus to perform the transfer. Note, this is referred to as Direct Memory Access(For more information, refer to: http://en.Wikipedia.org/wiki/Direct_memory_access) (DMA). Once the secondary storage controller has completed the transfer, it notifies the OS. The OS will then notify and resume our program. This process represents system overhead since our program is just waiting for completion of the system service request. It would make sense to obtain more data rather than less data on each system service call. It should be fairly obvious that limiting the number of system service requests will help the overall performance of our program.

Additionally, as described in Chapter 13, System Services, the low-level file read and write operations require the number of characters to read. A typical operation desired by a programmer would be to read a line as is provided by high-level language functions such as the C++ getline() function. It is unknown ahead of time how many characters might be on that line. This makes the low-level file read operation more difficult since the exact number of characters to read is required.

We, as humans, place special meaning and significance to the LF character (end of line). In the computer, the LF is just another ASCII character, no different than any other. We see a file as a set of lines, but in the computer the file is just a collection of bytes, one after another.

The process used for reading interactive input involved reading one character at a time. This could be used to address the issue of not knowing ahead of time how many characters are on a line. As was done for interactive input, the program could just read one at a time until an LF is found and stop.

Such a process will be unduly slow for reading large volumes of information from a file. Interactive input expects the user to type characters. The computer is easily able to keep up with a human. Even the fastest typist will not be able to out-type the computer (assuming efficiently written code). Further, interactive input is generally limited to relatively small amounts. Input from a file is not awaiting user action and is generally not limited to small amounts of data. For a sufficiently large file, single character I/O would be quite slow. Note, testing and quantizing the performance difference is left as an exercise.

To address this, the input data will be buffered. In this context, we will read a chunk of the file, say 100,000 characters, into a large array which is referred to as a buffer or input buffer. When a “line” of text is required, the line will be obtained from the input buffer. The first time a line is requested, the file is read and the buffer is filled. After the file read completes, the line is returned from the buffer (all characters up to and including the LF). Successive calls to get the next line are satisfied by obtaining the characters from the buffer without needing to read the file again. This can keep happening until all the characters in the buffer have been returned at which point the file will need to be read again. When there are no more characters, the input is completed and the program can be terminated.

This process helps improve performance, but is more complicated. High-level language functions, such as getline, hide this complexity from the user. When working at the low-level we will need to write the function to get the next line ourselves.

It is important to understand the cause of the performance degradation in order to fully understand why buffering will improve performance.