Chaining Linux Commands Like a Pro

When I was reading Jason Cannon’s Linux for Beginners, I learned that if you want a program to run in the background, you just add an ampersand ("&") at the end of the line you want to invoke. That proved to be quite handy over time but recently I have learned that the single ampersand is not only useful for moving programs to the background, but chaining Linux commands as well!

The concept of command chaining is fairly common knowledge, but for my more novice readers, command chaining can manifest itself in two forms:

1. Piping (Using the “|” symbol) - This means the output of a program is given as input to the next program in the chain, for example cat ./myfile.txt | grep "some text" This will redirect the output from the cat command (which is the contents of the myfile.txt) to the grep command, where it will be checked for containing the text “some text”

2. Chaining (Using “;”, “&”, “&&” or “||”) - This means different things depending on which symbol is used but generally the commands don’t interact with each other as much.

Here’s a quick reference before we dive in:

The Semicolon (";") Operator- This is the most straightforward and basic operator out of the bunch. It tells the shell to execute commands one after the other. If there’s an error, no biggie, it’ll just continue. Let’s try running cat on a non-existent file, and echo some text to the terminal.

$ cat non_existent_file ; echo "Hello World"
cat: non_existent_file: No such file or directory
Hello World

As you can see, the first command failed and exited with an error, but the second command still got executed, just as expected.

The Double Ampersand ("&&") Operator - This one acts as a conditional. if the first command exits without an error then run the second command. Let’s try that same line with a double ampersand instead of a semicolon.

$ cat non_existent_file && echo "Hello World"
cat: non_existent_file: No such file or directory

$ echo "Hello World" && cat non_existent_file
Hello World
cat: non_existent_file: No such file or directory

As you can see, the intended behavior checks out here as well. In the first line, the first command exits with an error and therefore the second command is not executed. If we flip them around, however, the second command does execute. If we add a third command chained with &&, it would not be executed. Try to figure out a way to make this work, it’s a good thinking exercise. You already have the tools you need!

The Logical Or ("||") Operator - you probably get a sense of the intended behavior of this operator from its name. This operator will execute the second command only if the first one exited with an error.

$ cat non_existent_file || echo "Hello World"
cat: non_existent_file: No such file or directory
Hello World

$ echo "Hello World" || echo "This won't print"
Hello World

The || operator is the opposite of the && operator.

The Single Ampersand ("&") Operator - This one is a special. Unlike the other operators, this one doesn’t wait for a command to finish. If I chain 2 commands with this operator, they will both be executed at the same time and sent to the background. Obviously, the exit code doesn’t matter here. One thing you should remember is that the last command will not be sent to the background, unless you’d add another & at the end of it. Here’s an example:

$ sleep 5 & sleep 3 & echo "Hello World"
[1] 12345
[2] 12346
Hello World
$ [2]-  Done    sleep 3
[1]+  Done    sleep 5

Let’s break this down. In the first line, I chain 3 commands, and I do not add & to the end of the line. As a result, you can see both the first and the second commands get sent to the background, followed by an immediate “Hello World”. This command was not sent to the background, you can see in the next lines that it’s followed by the second command and the first command exiting respectively.

Next, I add & to the end of the line and run it again. You can see that now we have created 3 background jobs, followed by an “Hello World”; This doesn’t make much of a difference in our case, because of the simple nature of this example. yet can definitely come in handy in more complex chaining scenarios.

What do you think will be the behavior of this line?

$> sleep 2 && echo I waited & echo Life is too short to sleep

Did you figure it out? Let’s take a look.

$ sleep 2 && echo "I waited" & echo "Life is too short to sleep"
Life is too short to sleep
[1] 12348
$ I waited
[1]+  Done    sleep 2 && echo "I waited"

Hmmm… Interesting. I issued 3 commands but only 1 job was created in the background. The first 2 commands were actually backgrounded as one job, and then the third command immediately executed. Why?

Operator Precedence

Not all chaining operators were created equal. Just like in math multiplication is done before addition; different operators are processed in the different order/manner.

Essentially, && and || bind more tightly than & and ;. So in our example, && grouped sleep 2 and echo "I waited" together first, and then & sent that whole pair to the background. Let’s replace && with ; in the last example.

$ sleep 2 ; echo "I waited" & echo "Life is too short to sleep"
Life is too short to sleep
[1] 12349
$ I waited
[1]+  Done    echo "I waited"

This time, since ; and & are at the same precedence level, sleep 2 runs first (blocking for 2 seconds), then echo "I waited" gets backgrounded while echo "Life is too short to sleep" runs in the foreground.

To summarize - Linux command chaining embodies a lot of concepts and offers versatile options. It can be a very powerful tool for bash scripting and terminal one-liners.