stream editing

Any program in unix can be described as the following:

input | transform | output

This section is about transforming data.

stdin

This will output all data passed to stdin.

# transform.sh
while read stdin; do
  echo "$stdin"
done

Unix pipes now work:

$ ./transform.sh < ./my-file.log
$ echo 'hello world' | ./transform.sh

tr

[tbi]

awk

select a column

$ echo 'hi there' | awk '{print $2}'    # single column
there
$ echo 'hi there' | awk '{print $2 $1}' # multi column
therehi

See Also

• idiomatic awk
• world's best introduction to sed

grep

inverse grep / exclude data

$ grep -v hello < ./file.txt   # exclude 'hello'

file minification

tar

$ tar xzf out.tgz file1 file2
$ git ls-files | grep gateway | tar cfz target.tgz -T -

zip

$ zip out.zip file1 file2