Very Computer

I'm trying to come up with over 500 "short and sweet" bash tips tricks. So
far I have 63, but this includes other Linux tricks as well.  I'd like to
get more bash "useful" tricks.   Anyone have a useful script  they'd be
willing to share?

Although TIP 44 below seems useless.  I use it a lot for converting text
data in a file to MySQL insert statements; I pivot the text, then, use an
awk script to add in quotes and "insert into ..".

Anyway, hopefully the following will help -- maybe you have something
better?

The following was taken from
http://osdn.dl.sourceforge.net/sourceforge/souptonuts/How_to_Linux_an...

TIP 43:

     "cat" the Contents of Files Listed in a File, in That Order.

       SETUP (Assume you have the following)

              $ cat file_of_files
                  file1
                  file2

              $ cat file1
                  This is the data in file1

              $ cat file 2
                  This is the data in file2

       So there are 3 files here "file_of_files" which contains the name of
       other files.  In this case "file1" and "file2". And the contents of
       "file1" and "file2" is shown above.

               $ cat file_of_files|xargs cat
                    This is the data in  file1
                    This is the data in  file2

TIP 44:

     Columns and Rows -- getting anything you want.

     Assume you have the following file.

        $ cat data
           1 2 3
           4 5
           6 7 8 9 10
           11 12
           13 14

     How to you get everything in  2 columns?

        $ cat data|tr ' ' '\n'|xargs -l2
           1 2
           3 4
           5 6
           7 8
           9 10
           11 12
           13 14

    Three columns?

        $ cat data|tr ' ' '\n'|xargs -l3
           1 2 3
           4 5 6
           7 8 9
           10 11 12
           13 14

    What's the row sum of the "three columns?"

        $ cat data|tr ' ' '\n'|xargs -l3|tr ' ' '+'|bc
           6
           15
           24
           33
           27

Thanks,

Mike Chirico
http://souptonuts.sourceforge.net/chirico/index.php

[...]

Quote:> TIP 43:
>      "cat" the Contents of Files Listed in a File, in That Order.
[...]
>        So there are 3 files here "file_of_files" which contains the name of
>        other files.  In this case "file1" and "file2". And the contents of
>        "file1" and "file2" is shown above.

>                $ cat file_of_files|xargs cat
>                     This is the data in  file1
>                     This is the data in  file2

This can be shortened to

     xargs cat < file_of_files

As a general rule,

     cat one_file | program

can always be rewritten as

     program < one_file

Heiner
--
  ___ _

\__ \  _/ -_) V / -_) ' \    Shell Script Programmers: visit
|___/\__\___|\_/\___|_||_|   http://www.shelldorado.com/

Thanks!  I like it.  I'm trying to figure out how to do it with stdin.
I've always done...

$ cat -|tr ' ' '\n'|xargs -l3|tr ' ' '+'|bc

But, I'll play with it a bit. Seems like it should be something obvious.

Regards,

Mike Chirico

"cat -" is even one step further ;-) Since "cat" reads standard
input as a default, there's no need to specify "-" to make it
read stdin.

Well, if you really *like* cat you of course can do

     $ cat - | cat - | tr x u | tr u x | sed -n p |
          awk '{print}' | nl -ba | sort -rn | sort -n | cut -f2-

where a simple "cat" would be sufficient ;-)

     http://www.shelldorado.com/articles/ignorantsguide.html

Quote:> But, I'll play with it a bit. Seems like it should be something obvious.

     cat - | tr ... | ...

can be rewritten by just omitting the "cat":

     tr ... | ...

In a similar way that you can write

     1 * 1 * 1 * 1

you generally can write

     cat | cat | cat | cat

without changing a thing (or adding functionality). "cat" is
only needed if there is no command reading standard input (e.g.
just "cat file" -- no pipe), or if you want to conCATenate
several files:

     cat file1 file2 | ...

or (in shell scripts) need the functionality to potentially
process more than one file, e.g.

Heiner
--
  ___ _

\__ \  _/ -_) V / -_) ' \    Shell Script Programmers: visit
|___/\__\___|\_/\___|_||_|   http://www.shelldorado.com/

2004-06-23, 01:38(+02), Heiner Steven:
[...]

Quote:>> $ cat -|tr ' ' '\n'|xargs -l3|tr ' ' '+'|bc

> "cat -" is even one step further ;-) Since "cat" reads standard
> input as a default, there's no need to specify "-" to make it
> read stdin.

> Well, if you really *like* cat you of course can do

>      $ cat - | cat - | tr x u | tr u x | sed -n p |
>           awk '{print}' | nl -ba | sort -rn | sort -n | cut -f2-

tr x u | tr u x
translates u into x

Note that tr, sed, awk may fail on files containing '\0'
characters. sed and awk have unspecified behaviors if the input
doesn't end in a '\n' character (or to sum up, cat works for
binary and text files, text utilities such as sed or awk work
only for text files).

Quote:

> where a simple "cat" would be sufficient ;-)

No cat at all is sufficient.

--
Stephane

To elaborate a little: my command line was intended to not change
the input. But

     tr x u | tr u x

which was intended as "change all 'x' to 'u' and back" *can*
change the input. "tr u x" is not the reverse command to "tr x u",
because all "u" will be changed to "x", even when they were not an
"x" initially. So Stephane's concerns are valid.

Quote:> Note that tr, sed, awk may fail on files containing '\0'
> characters. sed and awk have unspecified behaviors if the input
> doesn't end in a '\n' character (or to sum up, cat works for
> binary and text files, text utilities such as sed or awk work
> only for text files).

Right. To add to this list: "nl" interprets certain characters in the
input as control codes, and may (if I remember correctly) insert e.g.
a page number.

I hope my point still was clear: you can always make things more
complicated than necessary, however the elegant way is to use as many
commands as necessary, but not a single more than that.

Heiner
--
  ___ _

\__ \  _/ -_) V / -_) ' \    Shell Script Programmers: visit
|___/\__\___|\_/\___|_||_|   http://www.shelldorado.com/

[snip]

Quote:> > tr x u | tr u x
> > translates u into x

Interesting, so it does
x u -> x x

Is there anyway, just using just a combination of "tr" to change all x to u,
and u to x?

x u -> u x

Regards,

Mike Chirico

To clarify, I realize tr "xu" "ux"  will do this.  And it looks like

tr x k|tr u x|tr k u

works a well. I was looking for a way, working through the pipe, without
sacrificing a temp variable  "k", but I don't think it's possible using
single replacement at each stage.

Regards,

Mike Chirico

2004-06-23, 22:28(-04), Mike Chirico:
[...]

Quote:> tr x k|tr u x|tr k u

> works a well. I was looking for a way, working through the pipe, without
> sacrificing a temp variable  "k", but I don't think it's possible using
> single replacement at each stage.

[...]

That translates "k"s into "u"s.

--
Stephane

...

Quote:>     cat - | tr ... | ...

>can be rewritten by just omitting the "cat":

>     tr ... | ...

>In a similar way that you can write

>     1 * 1 * 1 * 1

>you generally can write

>     cat | cat | cat | cat

>without changing a thing (or adding functionality). "cat" is
>only needed if there is no command reading standard input (e.g.
>just "cat file" -- no pipe), or if you want to conCATenate ...

Yes, we all know that using cat like this is unnecessary and the mark of an
inexperienced user.  But, the real questions: Can you think of an instance
where one of "dumb uses of cat" is actually there for a reason?

Here's an idea, that, believe it or not, I've actually seen posted to
a newsgroup at some point in the past - the idea was that some newbie was
trying to defend his use of multiple cats - for this reason:

        cat file | cat | cat | somecommand > file

The idea is that with enough cats in the pipeline (*), you could slow things
down enough so that you can effect "editing in place" - i.e., writing to
the same file as you were reading from.  Rather natty, wouldn't you say?

(*) And, yes, there has to be some joke about stray felines in here
somewhere...

I once fixed a script that was failing on a line like:

  prog <src >dest

by writing it as

  cat src | prog | cat >dest

The problem was that "prog" didn't understand quite how
big a file could get, and promptly died if it saw either
stdin or stdout exceed 2 gig.

   How did that help? It is still reading from stdin and writing to
   stdout.

--
    Chris F.A. Johnson                  http://cfaj.freeshell.org/shell
    ===================================================================
    My code (if any) in this post is copyright 2004, Chris F.A. Johnson
    and may be copied under the terms of the GNU General Public License

2004-06-28, 13:25(+00), Chris F.A. Johnson:
[...]

Quote:>>   prog <src >dest

>> by writing it as

>>   cat src | prog | cat >dest

>> The problem was that "prog" didn't understand quite how
>> big a file could get, and promptly died if it saw either
>> stdin or stdout exceed 2 gig.

>    How did that help? It is still reading from stdin and writing to
>    stdout.

In that case, presumably, "cat" is large file aware, and not
"prog".

--
Stephane

A program can tell if stdin or stdout is actually a file or if it's
and interactive source or a pipe. If his program perform said check
(possibly for optimizing buffer sizes), then it could fail if it received
an overly large file. However, using cat at both ends forced to program
to see that its input and output were pipes and therefore it couldn't make
any assumptions about how large either was.

That can also be done

cat $(< file_of_files)

$(< X ) is replaced by the content of file X.

(of course $() is a synonym of `` and more visible)

joe