Very Computer

I am trying to write a script that for each text file in a directory,
print the file, pipe the result to awk, pipe the result to sed, then
append the data to an output file.

In a script, I have:

#!/bin/sh

for FILE in ./*TXT
 do
  cat $FILE
 done

Initially, I was piping the result within the script
( e.g., cat $FILE | awk { instructions } | sed "instructions" >>
file.foo ), but that errored.

Next, I tried calling the script like this:

$ ./fileread.sh | awk -F"%%" {print $1 "\n"} | etc... >> file.foo

which results in a broken pipe.

I am probably missing something simple.  Can someone point me in the
right direction?

Try

#!/bin/sh

for FILE in `ls *.txt`
do
    echo $FILE
    cat $FILE | awk 'Instructions' | sed Instructions
# eg     cat $FILE | awk '{print $1}' | sed s/h/hello/
done

Quote:> I am trying to write a script that for each text file in a directory,
> print the file, pipe the result to awk, pipe the result to sed, then
> append the data to an output file.

> In a script, I have:

> #!/bin/sh

> for FILE in ./*TXT
>  do
>   cat $FILE
>  done

> Initially, I was piping the result within the script
> ( e.g., cat $FILE | awk { instructions } | sed "instructions" >>
> file.foo ), but that errored.

> Next, I tried calling the script like this:

> $ ./fileread.sh | awk -F"%%" {print $1 "\n"} | etc... >> file.foo

> which results in a broken pipe.

> I am probably missing something simple.  Can someone point me in the
> right direction?

Quote:>Try

>#!/bin/sh

>for FILE in `ls *.txt`
>do
>    echo $FILE
>    cat $FILE | awk 'Instructions' | sed Instructions
># eg     cat $FILE | awk '{print $1}' | sed s/h/hello/
>done

This will work, but it is a useless use of cat, try this:
#!/usr/local/bin/bash

for file in `ls *.txt`
do
        awk '{ print $1 }' $file | sed 's/h/hello/' >> converted.file
done

This will put the output to converted.file, which is what the op was
ultimately going for.  And please, for the love of $DEITY, don't
overwork the cat! :o)

HTH

Trev

Any man whose errors take ten years to correct is quite a man.
          - J. Robert Oppenheimer, speaking of Albert Einstein

Quote:>This will work, but it is a useless use of cat, try this:
>#!/usr/local/bin/bash

>for file in `ls *.txt`
>do
>        awk '{ print $1 }' $file | sed 's/h/hello/' >> converted.file
>done

>This will put the output to converted.file, which is what the op was
>ultimately going for.  And please, for the love of $DEITY, don't
>overwork the cat! :o)

Unless it's a homework assignment on using pipes, you don't need the sed
either.
awk '{sub(/h/, "hello",$1);print $1)}' >> converted.file

Submitted by Salvador Peralta to comp.unix.shell:

Quote:> I am trying to write a script that for each text file in a directory,
> print the file, pipe the result to awk, pipe the result to sed, then
> append the data to an output file.

> In a script, I have:

> #!/bin/sh

> for FILE in ./*TXT
>  do
>   cat $FILE
>  done

> Initially, I was piping the result within the script
> ( e.g., cat $FILE | awk { instructions } | sed "instructions" >>
> file.foo ), but that errored.

> Next, I tried calling the script like this:

> $ ./fileread.sh | awk -F"%%" {print $1 "\n"} | etc... >> file.foo

> which results in a broken pipe.

> I am probably missing something simple.  Can someone point me in the
> right direction?

Your script should look like this (you forgot a ;, which is why
you got a broke pipe; the script teminated with a syntax error):

for FILE in *TXT; do
    cat $FILE
done

This is basicly the same as

cat *TXT

Hence, you should be able to

cat *TXT | awk -F"%%" {print $1 "\n"} | etc... >> file.foo

which is just the same as

awk -F"%%" {print $1 "\n"} < *TXT | etc... >> file.foo

--
Andreas K?h?ri
----------------------------------------------------------------
NetBSD: En Vax i handen ?r b?ttre ?n tio p? skroten.
        http://www.netbsd.org/

Bill Marcum on Tuesday 12 March 2002 11:26:

Quote:> Unless it's a homework assignment on using pipes, you don't need the
> sed either.

lol... Nope.  Not a homework assignment. I'm just trying to use
something other than Perl to prepare a bunch of text files for
insertion into a relational database.

Here's an Awk question. Let's say I've got a text file that is
delimited like so:

%%File: Foo.dll
%%Name:  foo library function
%%Features: Here are some features
, some of which span multiple lines
        like so.
 test text
%%Notes: This is some text
that will continue on many
lines

I can use Awk like this:

#!/bin/sh

awk '
BEGIN {
    FS="\n%%"
    RS=""
    ORS=""

Quote:}

{  
        x=1
        while ( x<NF ) {
                print $x "%%"
                x++
        }
        print $NF "\n"

Quote:} '

to (in theory) print out each field within the record on a single
line.  Unfortunately, my script fails when a line has a newline that
is not a delimiter.  I tried to use sub(/\n/, "", $x) as the first
argument in the body of the while loop, and sub(/\n/, "", $NF) as the
first argument after the while loop, but these had no effect.  
Newlines not associated with the delimiter are still printed.

How should I resolve this issue?

Submitted by colt to comp.unix.shell:

Quote:> Try

> #!/bin/sh

> for FILE in `ls *.txt`

Useless use of ls.  If he wrote '*TXT' I suspect he knows that
his files ends with 'TXT' and not '.txt'.

Your example above is the same as

for FILE in *.txt

Quote:> do
>     echo $FILE
>     cat $FILE | awk 'Instructions' | sed Instructions

Useless use of cat. This is doing the same:

awk 'Instructions' < $FILE | sed Instructions

--
Andreas K?h?ri (on a slow news feed)
----------------------------------------------------------------
NetBSD: En Vax i handen ?r b?ttre ?n tio p? skroten.
        http://www.netbsd.org/

Not always.

Think about running scripts like this on an SMP machine.  You gain by
seperating your programs, sometimes even trivial things like cat.

But certainly not in this case.  cat (file) | awk (program) is certainly
less efficient than awk (program) (file) whether you're on an SMP
machine or not.

--

 __ San Jose, CA, US / 37 20 N 121 53 W / ICQ16063900 / &tSftDotIotE
/  \ Laws are silent in time of war.
\__/ Cicero
    Esperanto reference / http://www.alcyone.com/max/lang/esperanto/
 An Esperanto reference for English speakers.

No it's not.
ls prints an error message if the files do not exist.
For will just set the loop variable to the wildcard pattern.

The following two loops give different results (and I don't mean the
prefix on the echo)

for file in *.no-such-file
do
        echo "##" $file
done

for file in `ls *.no-such-file`
do

done

Submitted by "Tim Vernum" to comp.unix.shell:

In both cases you should check for existance of the file $file
before doing anything on it.

--
Andreas K?h?ri
----------------------------------------------------------------
NetBSD: Because you're worth it.    http://www.netbsd.org/