Using I/O, Pipes, Redirection, etc in the Unix/Linux Environment.


Jim Jackson <jj@franjam.org.uk>


In this, any reference to a SHELL is assumed to be either the Bourne shell or one of its descendants, e.g. BASH.

This document can be found at http://www.franjam.org.uk/linux/cli2.html


Program Input and Output

The Conventional Model

                     -------------       stdout
                     |           | ----- OUTPUT ----->
    ---- INPUT ----> |  Program  |
         stdin       |           | ----- ERRORS ----->
                     -------------       stderr

Each program when executed has 3 files preset for it:

STDIN is usually the keyboard

STDOUT is usually the console, terminal or shell window

STDERR is usually the console, terminal or shell window

Remember: In Unix/Linux "everything is a file" - even the keyboard/screen

The program can of course completely ignore these and get its inputs, and set its outputs to anything it wants. But most of the standard Unix/Linux command line tools follow a convention in the use of these preset files.

OK Why is this useful? ...


Redirection

... Because when executing programs, the SHELL can change what "file" is connected to STDIN, STDOUT and STDERR.

Redirecting STDOUT

Use the '>' or '>>' symbols to change where the normal output goes. '>>' causes the output to be appended to any previous contents of the file.

e.g.  % ls -l /bin > /tmp/bin_list
      % ls -l /usr/bin >> /tmp/bin_list

Redirecting STDIN

Use the '<' symbol to change where the normal input comes from

e.g.  % wc -l < /tmp/etc_list

mmmmm....... compare with "wc -l /tmp/etc_list", it's only a small change. A less forced example.....

      % /usr/sbin/sendmail -t -oem -oi < /tmp/mailfile

Redirecting STDERR

Use '2>' to change where the error output goes, 2 because STDERR is file 2. To append use '2>>'

e.g.  % ls -l Z* 2> /dev/null

Redirecting ALL output to same place

Use the construct '> outputfile 2>&1'. That is, redirect STDOUT to file and redirect file 2 (STDERR) to file 1 (&1)

e.g.       % ( ./configure ; make ) > /tmp/make.output 2>&1
or better  % ( ./configure && make ) > /tmp/make.output 2>&1

Joining programs together - PIPES

The shell can use a PIPE (see man pipe) to feed the output of one program into the input of another.

Joining STDOUT of one command with STDIN of a second command

Use the '|' symbol between the 2 commands.

e.g.  % ls -l /etc | less

Joining STDERR and STDOUT of one command to STDIN of a second command

Use the construct '2>&1 |'

e.g.       % ( ./configure ; make) 2>&1 | less
or better  % ( ./configure && make) 2>&1 | less

Because programs can be so joined together, the UNIX convention of making utilities do essentially one thing, but do it well, has developed. More complex actions can be produced by joining commands together with pipes.


Some of the main CLI utilities

some programs that generate useful info

miscellany


Some of the main CLI "filters"

These all work on files, which can be specified on the command line. If no filesnames are specified then they take their input from STDIN


Example pipelines

show the mozilla processes running...

     % ps wuax | grep mozill

show the number of files in the _Docs_ directory...

     % ls Docs | wc -l

mail yourself a list of files in your archive directory...

     % ls -l archive | mail -s "List of Current Archive files" jj

browse the kernel messages from the syslog file...

     % grep kernel /var/log/messages | less

watch for tcpwrapper connect messages in syslog file and save seperately...

     % tail -f /var/log/syslog | grep "connect from"

HomePage http://www.franjam.org.uk



This document was generated using AFT v5.095