Standard Input/Ouput

Standard streams are how terminal programs interact with the user. Specifically, once a terminal program is running, and it shows text on the terminal, that text is what is being output through Standard Input (stdin). When it asks you to type something like a file name, and you type something, that text is sent to the program through Standard Ouput (stdout). There is a third standard stream, called Standard Error (stderr) but that is generally used for debugging and you don't have to worry about that.

Standard I/O on Linux and Mac
To see the standard streams in action, just type: cat into the terminal. It doesn't look like anything happened, right? Now type "asdf" and press enter. It spits it right back at you. (PRESS CTRL+C TO EXIT.) What "cat" does is read anything from Standard Input and direct it to Standard Output, line by line. What does this mean? When you type something, it's sent to the standard input, but not until you press enter. When enter is pressed, cat takes this line, and sends this to the standard output, which by default, is just output on the terminal. The end effect is that cat will just copy what you typed and spit it, whenever you press enter.

Redirecting stdin and stdout
You can redirect standard input/output on the terminal by using the ">" and "<" keys. If you want a program to read from a file instead of typing the same thing every time, you can type "< file1". Try it out: cat < file1 What happened here? cat does exactly the same thing as before, but stdin acts different. Instead of getting lines typed from the keyboard, stdin will get lines from file1, until it runs out of lines. cat takes the lines and sends them to stdout just as before. The end result is the contents of file1 are output to the terminal. (But cat 's mission in life is to show the contents of a file(s), and thus, you don't need to use the "<" sign like this.)

WARNING: REDIRECTING STDOUT TO A FILE WILL IRREVERSIBLY ERASE AND OVERWRITE THE FILE

If you want a program to record the output into a file instead of outputting to the terminal, you use "> file1". Try typing this into the terminal, BUT FIRST MAKE SURE THAT THERE ISN'T A FILE IN THE FOLDER NAMED "asdf": cat > asdf fdsa fdsa fdsa  cat asdf Where "" means to press the control and the c buttons at the same time, not to type that whole thing out. You can remove the asdf file afterwards by typing "rm asdf". (Normally you would type control-d, rather than control-c to finish typing into a file, as in this example, or into a command interface, like the python console. Control-c is generally used to interrupt an interactive process.)

What you should have gotten out after you typed that last line should look like this: fdsa fdsa fdsa What happened here? You opened cat with stdout redirected to a file named "asdf". When you typed "fdsa" three times, each time you pressed enter, cat took that in through stdin and sent it to stdout. Normally, stdout would simply go to the terminal, however since stdout is redirected to "asdf", it didn't show up on the terminal, but instead was written, line by line to the file "asdf".

WARNING: REDIRECTING STDOUT TO A FILE WILL IRREVERSIBLY ERASE AND OVERWRITE THE FILE

If you want stdout to output to both a file named "asdf" and the terminal, you would type: cat | tee file1 and tee will split stdout to both the terminal and to the file "file1". What's that vertical bar you ask? That's a pipe. Obviously. Jeez.

WARNING: REDIRECTING STDOUT TO A FILE WILL IRREVERSIBLY ERASE AND OVERWRITE THE FILE (do it anyway)

Piping
The vertical bar (|) is called a pipe. It looks like a pipe. But more importantly, it's used for making a pipeline from one program to the other. A pipeline means that the stdout for one program will be used as the stdin for another program.

For example, from the previous section: cat | tee file1 cat takes lines from stdin (from the keyboard), and sends them to stdout, but since it's piped, the stdout from cat immediately becomes the stdin for the next program in the pipe, which is tee. tee takes stdin and sends it both to stdout and to file1. Stdout then outputs the lines back to the terminal by default, since we didn't pipe or redirect it. We could continue to pipe stdout to another program using more pipes, but this is just a simple example with two programs.

An experienced linux/unix user will know these fun and exciting basic programs, and be able to pipe them together with no problem:
 * grep
 * sed
 * awk
 * wc
 * xargs

You don't need to know these, these are just examples of what pipe is used for.

Here's an example of something I piped on the fly that took 20 seconds to write: cat test.fasta | grep \t | wc -l cat reads from test.fasta, and outputs to stdout. Stdout from cat is piped to grep, which filters for lines with a tab, and sends lines with a tab to stdout. Stdout from grep is piped to wc, which counts the number of lines (because of the "-l" option). The end result is the number of lines in test.fasta that contain a tab.

Person to Ask

 * Bryant