Why does: "tail -f /var/log/messages | grep eth0 >> /var/log/eth0.log" create an empty log ?

Why does: "tail -f /var/log/messages | grep eth0 >> /var/log/eth0.log" create an empty log ?

Post by Pete » Sat, 24 Jan 2004 12:54:23



bash scripting question:

if:           "cat /var/log/messages"       yeilds:

it87-isa-0290
Adapter: ISA adapter
Algorithm: ISA algorithm
VCore 1:   +1.79 V  (min =  +1.48 V, max =  +2.01 V)
VCore 2:   +1.24 V  (min =  +1.02 V, max =  +1.37 V)
+3.3V:     +3.21 V  (min =  +2.80 V, max =  +3.79 V)
+5V:       +4.97 V  (min =  +4.23 V, max =  +5.75 V)
+12V:     +12.00 V  (min = +10.16 V, max = +13.80 V)
Stdby:     +4.80 V  (min =  +4.23 V, max =  +5.75 V)
VBat:      +0.00 V
fan1:     5578 RPM  (min = 3000 RPM, div = 2)
fan2:     2922 RPM  (min =    0 RPM, div = 2)
M/B Temp:    +43C  (low =  +10C, high =  +55C)
CPU Temp:    +44C  (low =  +10C, high =  +60C)
-------------------------------------------------------------------------------------
Jan 22 21:44:14 bruley kernel: I-DROP-IN=eth0 OUT=
MAC=00:50:bf:5e:a2:49:00:0a:42:6b:48:70:08:00 SRC=68.119.147.51
DST=66.191.74.157 LEN=56 TOS=0x00 PREC=0x00 TTL=115 ID=50038 PROTO=ICMP
TYPE=3 CODE=3 [SRC=66.191.74.157 DST=68.119.147.51 LEN=719 TOS=0x00
PREC=0x00 TTL=50 ID=0 DF PROTO=UDP SPT=27960 DPT=2477 LEN=699 ]

and if:    "tail -f /var/log/messages | grep eth0"     yeilds:

Jan 22 21:44:14 bruley kernel: I-DROP-IN=eth0 OUT=
MAC=00:50:bf:5e:a2:49:00:0a:42:6b:48:70:08:00 SRC=68.119.147.51
DST=66.191.74.157 LEN=56 TOS=0x00 PREC=0x00 TTL=115 ID=50038 PROTO=ICMP
TYPE=3 CODE=3 [SRC=66.191.74.157 DST=68.119.147.51 LEN=719 TOS=0x00
PREC=0x00 TTL=50 ID=0 DF PROTO=UDP SPT=27960 DPT=2477 LEN=699 ]

Why does: "tail -f /var/log/messages | grep eth0 >> /var/log/eth0.log"
create an empty log ?

What happens to the standard out ?

(I am running as root and I have full write permision for eth0.log)

Thank You

--
Peter
http://tkvoice.netfirms.com/

 
 
 

Why does: "tail -f /var/log/messages | grep eth0 >> /var/log/eth0.log" create an empty log ?

Post by Icarus Sparr » Sat, 24 Jan 2004 16:28:27



> bash scripting question:
> and if:    "tail -f /var/log/messages | grep eth0"     yeilds:

> Jan 22 21:44:14 bruley kernel: I-DROP-IN=eth0 OUT=
> MAC=00:50:bf:5e:a2:49:00:0a:42:6b:48:70:08:00 SRC=68.119.147.51
> DST=66.191.74.157 LEN=56 TOS=0x00 PREC=0x00 TTL=115 ID=50038 PROTO=ICMP
> TYPE=3 CODE=3 [SRC=66.191.74.157 DST=68.119.147.51 LEN=719 TOS=0x00
> PREC=0x00 TTL=50 ID=0 DF PROTO=UDP SPT=27960 DPT=2477 LEN=699 ]

> Why does: "tail -f /var/log/messages | grep eth0 >> /var/log/eth0.log"
> create an empty log ?

> What happens to the standard out ?

You are being hit by the buffering provided by the standard I/O library.
The output you are interested in is sitting in a buffer until either there
is a certain amount (typically 2k or 4k) or until it is flushed.

If output is going to a 'terminal', then the buffer is flushed on every
newline character, but this does not happen if it is going to a file.

Output is also flushed just before grep exits, but this will only happen
if it gets EOF, and that will only happen if the 'tail' process is killed
in some way.

If you have 'expect' on your system, one of the example programs is 'unbuffer'
which fools the standard I/O library into thinking it is writing to a terminnal
so you can say
  tail -f /var/log/messages | unbuffer grep eth0 >> /var/log/eth0.log
and you will get what you are expecting in your log file. Other programs
exist which do the same thing, I am keen on Dan Bernstein's 'pty' program
myself but it is not as common to find on systems as expect.

Icarus

 
 
 

Why does: "tail -f /var/log/messages | grep eth0 >> /var/log/eth0.log" create an empty log ?

Post by Pete » Sun, 25 Jan 2004 03:13:30


Thank you for the quick answer

I have expect version 5.38.0 and I have the command "unbuffer"

but I am still not seeing anything in my eth0.log file when using:

tail -f /var/log/messages | unbuffer grep eth0 >> /var/log/eth0.log

Did you test this on your system?

Are you sure this is the correct syntax ?

Thank You

Peter


> If you have 'expect' on your system, one of the example programs is 'unbuffer'
> which fools the standard I/O library into thinking it is writing to a terminnal
> so you can say
>   tail -f /var/log/messages | unbuffer grep eth0 >> /var/log/eth0.log
> and you will get what you are expecting in your log file. Other programs
> exist which do the same thing, I am keen on Dan Bernstein's 'pty' program
> myself but it is not as common to find on systems as expect.

> Icarus

--
Peter
http://tkvoice.netfirms.com/
 
 
 

Why does: "tail -f /var/log/messages | grep eth0 >> /var/log/eth0.log" create an empty log ?

Post by Ed Morto » Sun, 25 Jan 2004 05:57:25



> Thank you for the quick answer

> I have expect version 5.38.0 and I have the command "unbuffer"

> but I am still not seeing anything in my eth0.log file when using:

> tail -f /var/log/messages | unbuffer grep eth0 >> /var/log/eth0.log

You could do this if you don't get a better alternative:

_pat="$1"
_file="$2"
_start=0
while :
do
         _end=`wc -l < "${_file}"`
         _end="${_end##* }"
         if (( ${_end} > ${_start} ))
         then
                 _start=$(( $_start + 1 ))
                 sed -n "${_start},${_end}p" "${_file}" | grep "${_pat}"
                 _start=${_end}
         fi
         sleep 1
done

Put the above in a script (I call mine "tgrep" for "tail and grep") and
call it as

        tgrep eth0 eth0.log

and it'll grep the end of the file every second.

        Ed.

 
 
 

Why does: "tail -f /var/log/messages | grep eth0 >> /var/log/eth0.log" create an empty log ?

Post by Pete » Sun, 25 Jan 2004 07:18:42


Thanks Ed

This script works perfect :)

I really should take the time and learn how to use sed one day ;)

Quote:>> tail -f /var/log/messages | unbuffer grep eth0 >> /var/log/eth0.log

> You could do this if you don't get a better alternative:

> _pat="$1"
> _file="$2"
> _start=0
> while :
> do
>         _end=`wc -l < "${_file}"`
>         _end="${_end##* }"
>         if (( ${_end} > ${_start} ))
>         then
>                 _start=$(( $_start + 1 ))
>                 sed -n "${_start},${_end}p" "${_file}" | grep "${_pat}"
>                 _start=${_end}
>         fi
>         sleep 1
> done

> Put the above in a script (I call mine "tgrep" for "tail and grep") and
> call it as

>     tgrep eth0 eth0.log

> and it'll grep the end of the file every second.

>     Ed.

--
Peter
http://tkvoice.netfirms.com/