05-E.8.1: File Output Manipulation - pipe

Last updated
Save as PDF

Page ID: 32558

\( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \) \( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)\(\newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\) \( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\) \( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\) \( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\) \( \newcommand{\Span}{\mathrm{span}}\) \(\newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\) \( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\) \( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\) \( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\) \( \newcommand{\Span}{\mathrm{span}}\)\(\newcommand{\AA}{\unicode[.8,0]{x212B}}\)

EXAM OBJECTIVES COVERED
2.3 Given a scenario, create, modify, and redirect files.

The Pipe

A pipe is a form of redirection (transfer of standard output to some other destination) used to send the output of one command/program/process to another command/program/process for further processing. Linux systems allow stdout of a command to be connected to stdin of another command. You can make it do so by using the pipe character ‘|.’

Pipe is used to combine two or more commands, and in this, the output of one command acts as input to another command, and this command’s output may act as input to the next command and so on. It can also be visualized as a temporary connection between two or more commands/programs/processes. The command line programs that do the further processing are referred to as filters.

This direct connection between commands/programs/processes allows them to operate simultaneously and permits data to be transferred between them continuously rather than having to pass it through temporary text files or through the display screen.
Pipes are unidirectional i.e., data flows from left to right through the pipeline.

Syntax :

command_1 | command_2 | command_3 | .... | command_N

If we were to issue the ls -l command against the /dev directory we would get a long list of the files. If we wanted to see a screen of those files, we can use the less command, which would then hand us a screen of filenames, then we hit the SPACE bar and we get the next screen. We can use all of the less options to view the output:

pbmac@pbmac-server $ ls -l /dev | less 
total 0
crw-rw-rw-  1 root  root       14,  12 Aug  4 12:59 adsp
crw-r--r--  1 root  root       10, 235 Aug  4 12:59 autofs
drwxr-xr-x  2 root  root           380 Aug  4 12:59 block
drwxr-xr-x  2 root  root            80 Aug  4 12:58 bsg
crw-------  1 root  root       10, 234 Aug  4 12:58 btrfs-control
drwxr-xr-x  3 root  root            60 Aug  4 12:58 bus

So the stdio of the ls -l command is sent as input to the less command. Below is another example that shows piping multiple commands together:

pbmac@pbmac-server $ more stcap.txt 
Montana Helena MT
Arizona Phoenix AZ
South Dakota Pierre SD
Missouri Jefferson City MO
Alaska Juneau AK
Wyoming Cheyenne WY
Nevada Carson City NV
Nevada Reno NV
Nevada Las Vegas NV
pbmac@pbmac-server $ cat stcap.txt | grep NV | wc -l
3
pbmac@pbmac-server $

We have added 2 more cities from Nevada to the file. The entire content is output from the cat command, and used as input to the grep command, then that output is used as input to the wc command with a -l option. One more example combines pipes and redirection:

pbmac@pbmac-server $ cat stcap.txt | grep NV > NVcities.txt
pbmac@pbmac-server $ cat NVcities.txt 
Nevada Carson City NV
Nevada Reno NV
Nevada Lasa Vegas NV
pbmac@pbmac-server $

Instead of being piped to the wc command, the output of the grep command is redirected to a file name NVcities.txt.

Adapted from:
"Piping in Unix or Linux" by Rishabh132, Geeks for Geeks is licensed under CC BY-SA 4.0