reorganizing a delimited file in bash

reorganizing a delimited file in bash

Post by juicym.. » Thu, 14 Jul 2005 03:40:26



hello,

I have a file that appears to be delimited by three spaces between
fields.   I tried using cut to seperate the fields, but cut will only
take single character delimiters.   What I need to do is extract
columns 2,3, and 8, then sort the columns based on column 2.   There's
also a file header, but I don't want to keep it...   Is there a way to
do this from the bash prompt?

thanks.

 
 
 

reorganizing a delimited file in bash

Post by Ed Morto » Thu, 14 Jul 2005 03:49:33



> hello,

> I have a file that appears to be delimited by three spaces between
> fields.   I tried using cut to seperate the fields, but cut will only
> take single character delimiters.   What I need to do is extract
> columns 2,3, and 8, then sort the columns based on column 2.   There's
> also a file header, but I don't want to keep it...   Is there a way to
> do this from the bash prompt?

If you really don't care how many spaces are between the fields (i.e.
the fields themselves contain no spaces), then this:

        awk '{print $2,$3,$8}' file

or this:

        while read line
        do
                set -- "$line"
                echo "$2 $3 $8"
        done < file

will give you the columns you want. If you need to work with exactly 3
spaces as the field separator, this will do it:

        gawk -F"   " '{print $2,$3,$8}' file

In either case, then just use "sort" as normal.

        Ed.

 
 
 

reorganizing a delimited file in bash

Post by johngnu » Thu, 14 Jul 2005 03:55:50


TR with the -s option.  Tr can squeeze a set of char to one char, thus
two, three, four spaces can become one space. Then the cut will be
happy with a space bar for a delimit char. ( cut -d ' ' -f2 )
For example.
ls -l
drwxr-xr-x   2 jbenn02 users       48 2005-06-21 08:27 bin
-rw-r--r--   1 jbenn02 users     1047 2005-07-07 19:26 callcy.txt

With tr: ( that is a quote spacebar quote )
ls -l | tr -s ' '
drwxr-xr-x 2 jbenn02 users 48 2005-06-21 08:27 bin
-rw-r--r-- 1 jbenn02 users 1047 2005-07-07 19:26 callcy.txt

Now I can cut with ease:
ls -l |tr -s ' ' |cut -d ' ' -f8
bin
callcy.txt

It sounds like then you should be able to pipe your info to sort.
Example lines ??

Ponders the JB.

 
 
 

reorganizing a delimited file in bash

Post by Chris F.A. Johnso » Thu, 14 Jul 2005 04:10:36



> I have a file that appears to be delimited by three spaces between
> fields.   I tried using cut to seperate the fields, but cut will only
> take single character delimiters.   What I need to do is extract
> columns 2,3, and 8, then sort the columns based on column 2.   There's
> also a file header, but I don't want to keep it...   Is there a way to
> do this from the bash prompt?

    Are there spaces in the fields? If not, you can use awk:

{
  read ## Discard header (assuming it's just the first line)
  awk '{print $2, $3, $8}'

Quote:} < FILENAME | sort

     If the fields contain spaces, you can convert the 3-space
     delimiters to tabs with sed, then pipe it to awk:

{
  read ## Discard header (assuming it's just the first line)
  sed "s/   /\t/g" | awk 'BEGIN {FS = "\t"} {print $2, $3, $8}'

Quote:} < FILENAME | sort

--
    Chris F.A. Johnson                     <http://cfaj.freeshell.org>
    ==================================================================
    Shell Scripting Recipes: A Problem-Solution Approach, 2005, Apress
    <http://www.torfree.net/~chris/books/cfaj/ssr.html>
 
 
 

1. BASH BASH BASH BASH BASH BASH BASH BASH BASH BASH

Is there a proper fixed bash on any of the FTP sites out there?

I know there bash is on the usual sites but I don't know if they are
bugged or not :(

Regards,

Neil.

--


------------------------------------| Edinburgh, EH14 2DE, United Kingdom
**Domino: There`s nothing you can do when you`re the next in line: Domino**

2. gethostbyname

3. Reorganize file systems in ISC 2.2?

4. prob w/unix filepath on website

5. Need help writing script to reorganize files

6. K6 and ASUS

7. Reorganizing file system

8. Diamond Stealth II G460 supported????

9. Tool to transform/reorganize ASCII files

10. Need help on creating comma delimited file

11. Importing delimited and xls files into StarCalc ???

12. FORM CGI Available?-- to export comma delimited data to file

13. Parse irregular data, dump into delimited text file