Process Substitution
Piping the stdout of a command into the stdin of another is a powerful technique. But, what if you need to pipe the stdout of multiple commands? This is where process substitution comes in.
Process substitution feeds the output of a process (or processes) into the stdin of another process.
Template- Command list enclosed within parentheses
>(command_list)
<(command_list)
Process substitution uses /dev/fd/<n> files to send the results of the process(es) within parentheses to another process.
 | There is no space between the the "<" or ">" and the parentheses. Space there would give an error message. |
bash$ echo >(true)
/dev/fd/63
bash$ echo <(true)
/dev/fd/63
|
Bash creates a pipe with two file descriptors, --fIn and fOut--. The stdin of true connects to fOut (dup2(fOut, 0)), then Bash passes a /dev/fd/fIn argument to
echo. On systems lacking /dev/fd/<n> files, Bash may use temporary files. (Thanks, S.C.)
Process substitution can compare the output of two different commands, or even the output of different options to the same command.
bash$ comm <(ls -l) <(ls -al)
total 12
-rw-rw-r-- 1 bozo bozo 78 Mar 10 12:58 File0
-rw-rw-r-- 1 bozo bozo 42 Mar 10 12:58 File2
-rw-rw-r-- 1 bozo bozo 103 Mar 10 12:58 t2.sh
total 20
drwxrwxrwx 2 bozo bozo 4096 Mar 10 18:10 .
drwx------ 72 bozo bozo 4096 Mar 10 17:58 ..
-rw-rw-r-- 1 bozo bozo 78 Mar 10 12:58 File0
-rw-rw-r-- 1 bozo bozo 42 Mar 10 12:58 File2
-rw-rw-r-- 1 bozo bozo 103 Mar 10 12:58 t2.sh |
Using process substitution to compare the contents of two directories (to see which filenames are in one, but not the other):
diff <(ls $first_directory) <(ls $second_directory) |
Some other usages and uses of process substitution:
read -a list < <( od -Ad -w24 -t u2 /dev/urandom )
# Read a list of random numbers from /dev/urandom,
#+ process with "od"
#+ and feed into stdin of "read" . . .
# From "insertion-sort.bash" example script.
# Courtesy of JuanJo Ciarlante. |
cat <(ls -l)
# Same as ls -l | cat
sort -k 9 <(ls -l /bin) <(ls -l /usr/bin) <(ls -l /usr/X11R6/bin)
# Lists all the files in the 3 main 'bin' directories, and sorts by filename.
# Note that three (count 'em) distinct commands are fed to 'sort'.
diff <(command1) <(command2) # Gives difference in command output.
tar cf >(bzip2 -c > file.tar.bz2) $directory_name
# Calls "tar cf /dev/fd/?? $directory_name", and "bzip2 -c > file.tar.bz2".
#
# Because of the /dev/fd/<n> system feature,
# the pipe between both commands does not need to be named.
#
# This can be emulated.
#
bzip2 -c < pipe > file.tar.bz2&
tar cf pipe $directory_name
rm pipe
# or
exec 3>&1
tar cf /dev/fd/4 $directory_name 4>&1 >&3 3>&- | bzip2 -c > file.tar.bz2 3>&-
exec 3>&-
# Thanks, Stéphane Chazelas |
A reader sent in the following interesting example of process substitution.
# Script fragment taken from SuSE distribution:
# --------------------------------------------------------------#
while read des what mask iface; do
# Some commands ...
done < <(route -n)
# ^ ^ First < is redirection, second is process substitution.
# To test it, let's make it do something.
while read des what mask iface; do
echo $des $what $mask $iface
done < <(route -n)
# Output:
# Kernel IP routing table
# Destination Gateway Genmask Flags Metric Ref Use Iface
# 127.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0 lo
# --------------------------------------------------------------#
# As Stéphane Chazelas points out,
#+ an easier-to-understand equivalent is:
route -n |
while read des what mask iface; do # Variables set from output of pipe.
echo $des $what $mask $iface
done # This yields the same output as above.
# However, as Ulrich Gayer points out . . .
#+ this simplified equivalent uses a subshell for the while loop,
#+ and therefore the variables disappear when the pipe terminates.
# --------------------------------------------------------------#
# However, Filip Moritz comments that there is a subtle difference
#+ between the above two examples, as the following shows.
(
route -n | while read x; do ((y++)); done
echo $y # $y is still unset
while read x; do ((y++)); done < <(route -n)
echo $y # $y has the number of lines of output of route -n
)
More generally spoken
(
: | x=x
# seems to start a subshell like
: | ( x=x )
# while
x=x < <(:)
# does not
)
# This is useful, when parsing csv and the like.
# That is, in effect, what the original SuSE code fragment does. |
