Board index Unix shell

help help help to write a script in korn shell

help help help to write a script in korn shell

Post by Charles Dem » Wed, 10 Jun 1998 04:00:00





>Hi,

>My name Van Nguyen. I'm knowlegable about korn shell but never write a script.
>Now, I have to write one. The following is the purpose of the script:

>I got a access log which records every hit to my company website. What I want
>to do is to calculate all browsers that hit our site every day. The following
>is the example of the access log:

>199.76.206.71 - - [20/May/1998:23:59:20 -0400] "GET
>/banners/4708_17_Jul_28_1997_151916.gif HTTP/1.1" 200 9220
>"http://yp.bellsouth.com/?mkt=MCMS" "Mozilla/4.0 (compatible; MSIE 4.01; MSN
>2.5; Windows 95)" 199.76.206.71 - - [20/May/1998:23:59:21 -0400] "GET
>/GFX/imap.gif HTTP/1.1" 200 1690 "http://yp.bellsouth.com/?mkt=MCMS"
>"Mozilla/4.0 (compatible; MSIE 4.01; MSN 2.5; Windows 95)" 199.76.206.71 - -
>[20/May/1998:23:59:19 -0400] "GET /GFX/nresad.gif HTTP/1.1" 200 1467
>"http://yp.bellsouth.com/?mkt=MCMS" "Mozilla/4.0 (compatible; MSIE 4.01; MSN
>2.5; Windows 95)" 199.76.206.71 - - [20/May/1998:23:59:16 -0400] "GET
>/GFX/nnarrow.gif HTTP/1.1" 200 1765 "http://yp.bellsouth.com/?mkt=MCMS"
>"Mozilla/4.0 (compatible; MSIE 4.01; MSN 2.5; Windows 95)" 199.76.206.71 - -
>[20/May/1998:23:59:28 -0400] "GET /GFX/opinion.gif HTTP/1.1" 200 722
>"http://yp.bellsouth.com/?mkt=MCMS" "Mozilla/4.0 (compatible; MSIE 4.01; MSN
>2.5; Windows 95)" 209.136.1.82 - - [20/May/1998:23:59:37 -0400] "GET
>/GFX/flap2.gif HTTP/1.0" 304 16501
>"http://www.gamesville.com/art_ad/92/bellsouth-go.htm" "Mozilla/4.04 [en]
>(Win95; I)" 152.202.15.144 - - [20/May/1998:23:59:46 -0400] "GET
>/GFX/retry.gif HTTP/1.0" 200 1775 "http://yp.bellsouth.com/?mkt=MBAL"
>"Mozilla/4.04 [en] (Win95; I)" 152.168.133.44 - - [20/May/1998:23:59:55
>-0400] "GET /GFX/anifinal.gif HTTP/1.0" 200 3075
>"http://www.yp.bellsouth.com/" "Mozilla/3.0C-GZone  (Win95; I)" 209.136.1.82
>- - [20/May/1998:23:59:49 -0400] "GET /banners/50695_2_Dec_18_1997_083453.gif
>HTTP/1.0" 200 8394 "http://www.gamesville.com/art_ad/92/bellsouth-go.htm"
>"Mozilla/4.04 [en] (Win95; I)" 209.136.1.82 - - [20/May/1998:23:59:49 -0400]
>"GET /GFX/hobsrf.gif HTTP/1.0" 200 2189
>"http://www.gamesville.com/art_ad/92/bellsouth-go.htm" "Mozilla/4.04 [en]
>(Win95; I)" 206.49.117.90 - - [20/May/1998:23:59:32 -0400] "GET
>/GFX/honad.gif HTTP/1.0" 200 1584 "http://www.yp.bellsouth.com/"
>"Mozilla/4.04 [en] (Win95; I)"

>By knowing the kinds of the browser which are located at the end of each
>line,I want to figure out how many times the same type of  browser hitting us
>daily. The output should be something like:  Browser Type  # Times used
>----------------------------------------------------------------------------
>"(compatible; MSIE 4.01; MSN 2.5; Windows 95)"    37,000  "[en] (Win95; I)"
>19,999      .  .  .  .  ect

>I know that there is about more than one thousand types of browser hitting us
>daily.

>The following command is what I thought might work but still missing a lot of
>pieces to accomplish the result:

>gzcat access.gz | cat access.gz | fgrep "type of browser" | wc -l ( this is
>not enough to do what I want).

I put your sample data above in a file I called temp22

This Bourne script:

#!/bin/sh
#
sed 's/[0-9][0-9]*\.[0-9][0-9]*\.[0-9][0-9]*\.[0-9][0-9]/#&/g' temp22 |
tr '\012#' ' \012' | sed '1d;s/.*\("[^"]*"\)/\1/' | sort | uniq -c
# end of script

produced this output from your sample data:

   1 "Mozilla/3.0C-GZone  (Win95; I)"
   5 "Mozilla/4.0 (compatible; MSIE 4.01; MSN 2.5; Windows 95)"
   4 "Mozilla/4.04 [en] (Win95; I)"

I'll leave it to you to figure it out and adjust it to fit your needs.

Personally, I thought the other free package was a better long term
approach.  YMMV

Chuck Demas
Needham, Mass.

--
  Eat Healthy    |   _ _   | Nothing would be done at all,

  Die Anyway     |    v    | That no one could find fault with it.

 
 
 
Top

help help help to write a script in korn shell

Post by Ken Pizzi » Wed, 10 Jun 1998 04:00:00




>I got a access log which records every hit to my company website. What I want
>to do is to calculate all browsers that hit our site every day.
...
>199.76.206.71 - - [20/May/1998:23:59:20 -0400] "GET /banners/4708_17_Jul_28_1997_151916.gif HTTP/1.1" 200 9220 "http://yp.bellsouth.com/?mkt=MCMS" "Mozilla/4.0 (compatible; MSIE 4.01; MSN 2.5; Windows 95)"
...
>By knowing the kinds of the browser which are located at the end of each
>line,I want to figure out how many times the same type of  browser hitting us
>daily. The output should be something like:  Browser Type  # Times used

How about:
  gzcat access.gz | sed 's/.*"\([^"]*\)"$/\1/' | sort | uniq -c
?

                --Ken Pizzini

 
 
 
Top

help help help to write a script in korn shell

Post by Ken Pizzi » Thu, 11 Jun 1998 04:00:00




> I tried the following but got an error:

>sed 's/[0-9][0-9]*\.[0-9][0-9]*\.[0-9][0-9]*\.[0-9][0-9]/#&/g' temp22 | tr
>'\012#' ' \012' | sed '1d;s/.*\("[^"]*"\)/\1/' | sort
>sort:sort: write error while sorting: No space left on device

>my file's size is 1.6GB.

>Can you tell me what's wrong?

Well, offhand, I'd say you don't have enough disk space on /tmp
(or /var/tmp or /usr/tmp; wherever your "sort" command puts its
temp files).  One solution is to tell sort to put its temp file
somewhere else:
  .... | sort -T/some/dir/on/fs/with/space ...

But a perhaps better way would be to rewrite the script in perl:
  gzcat access.gz |
  perl -e 'while (<>) {s/.*"([^"]*)"$/$1/ && ++$c{$_} || print "Bad: ", $_}
  while (($k,$v) = each %c) {print $v, "\t", $k}'

This only uses space proportional to the number of *distinct* entries.

                --Ken Pizzini

 
 
 
Top

1. Need Help writing a complex Korn Shell Script

Hi there,

I'm a first year student in a UNIX class and I'm on my 3rd and final
assignment and don't have a clue how to start it.

Here's the problem I need to write a Korn Shell scipt to do a Histogram on
any text file.

Example: Histogram -a abc (-a alphabetic characters)

a ####
b ##
c #######

so it counts the number of occurences of each character and then uses #
to state how many of each character there is.

Can anybody out there suggest how I go about doing this.

It was suggested to me to use a whole bunch of commands in one with 'awk'
as the main filter.

eg. cat abc | sort | uniq -c | awk `{for (i=0; i<$1; i++) print "#" }`

something like that.

Any help would be greatly appreciated.

Cheers

2. mount iso files

3. Help to write a script in korn shell

4. java/43236: relation between "submit" and my E-mail.

5. Newbie help with Shell scripts (using Korn shell)

6. Help with landscape/portrait printer options

7. Help w/awk, korn shell & writing to a file

8. Syslogd and Linksys Router

9. I need help writing a Korn script to create 500 users.

10. Help on Korn Shell Script

11. need help with a chmod script in korn shell

12. can anyone help me with my korn shell script?

13. Urgent! Please help with basic Korn shell script!


All times are UTC
Board index