question: Unix commands for splitting a file

question: Unix commands for splitting a file

Post by sepa » Fri, 05 Jan 2001 09:38:10



Hello,

I have a data file that contains redundant parts and I wonder whether
spliiting the file based on the data can be done.  For example, the input
file looks as follows:

[input file]

45 35 33; 22 56 33; aaa
45 35 33; 22 56 32; aaa
45 35 33; 22 56 33; ddd
45 35 33; 22 56 33; ddd
55 35 33; 22 56 33; eee

What I would like to achieve is to insert a blank line each time the first
six numbers are different from the six numbers of the next line.  So, the
output of the above is as follows:

[output file]

45 35 33; 22 56 33; aaa

45 35 33; 22 56 32; aaa

45 35 33; 22 56 33; ddd
45 35 33; 22 56 33; ddd

55 35 33; 22 56 33; eee

Are there Unix commands that can be used for this?

Thank you very much for your time.

Regards,
Seongbin Park

 
 
 

question: Unix commands for splitting a file

Post by Barry Margoli » Fri, 05 Jan 2001 11:58:45



>Hello,

>I have a data file that contains redundant parts and I wonder whether
>spliiting the file based on the data can be done.  For example, the input
>file looks as follows:

>[input file]

>45 35 33; 22 56 33; aaa
>45 35 33; 22 56 32; aaa
>45 35 33; 22 56 33; ddd
>45 35 33; 22 56 33; ddd
>55 35 33; 22 56 33; eee

>What I would like to achieve is to insert a blank line each time the first
>six numbers are different from the six numbers of the next line.

awk '{ new6=$1 $2 $3 $4 $5 $6;
       if (NR > 1 && last6 != new6) {print ""};
       print; last6=new6 }' inputfile

--

Genuity, Burlington, MA
*** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups.
Please DON'T copy followups to me -- I'll assume it wasn't posted to the group.

 
 
 

question: Unix commands for splitting a file

Post by annel.. » Sat, 06 Jan 2001 10:39:07




>>Hello,

>>I have a data file that contains redundant parts and I wonder whether
>>spliiting the file based on the data can be done.  For example, the input
>>file looks as follows:

>>[input file]

>>45 35 33; 22 56 33; aaa
>>45 35 33; 22 56 32; aaa
>>45 35 33; 22 56 33; ddd
>>45 35 33; 22 56 33; ddd
>>55 35 33; 22 56 33; eee

>>What I would like to achieve is to insert a blank line each time the first
>>six numbers are different from the six numbers of the next line.
> awk '{ new6=$1 $2 $3 $4 $5 $6;
>        if (NR > 1 && last6 != new6) {print ""};
>        print; last6=new6 }' inputfile

If the parts are redundant, uniq will filter out the duplicate lines
for you.  

        Annelise

- Show quoted text -

> --

> Genuity, Burlington, MA
> *** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups.
> Please DON'T copy followups to me -- I'll assume it wasn't posted to the group.

 
 
 

question: Unix commands for splitting a file

Post by Barry Margoli » Sun, 07 Jan 2001 01:15:23






>>>Hello,

>>>I have a data file that contains redundant parts and I wonder whether
>>>spliiting the file based on the data can be done.  For example, the input
>>>file looks as follows:

>>>[input file]

>>>45 35 33; 22 56 33; aaa
>>>45 35 33; 22 56 32; aaa
>>>45 35 33; 22 56 33; ddd
>>>45 35 33; 22 56 33; ddd
>>>55 35 33; 22 56 33; eee

>>>What I would like to achieve is to insert a blank line each time the first
>>>six numbers are different from the six numbers of the next line.

>> awk '{ new6=$1 $2 $3 $4 $5 $6;
>>        if (NR > 1 && last6 != new6) {print ""};
>>        print; last6=new6 }' inputfile

>If the parts are redundant, uniq will filter out the duplicate lines
>for you.  

That's true, but it's not even close to what he asked for.

--

Genuity, Burlington, MA
*** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups.
Please DON'T copy followups to me -- I'll assume it wasn't posted to the group.

 
 
 

1. two unix questions: split and inserting text in each file.

Dear Unix gurus,

I have to solve the following types of problem - which I'm hoping can
be solved more efficiently than my crude and inefficient ways of
solving. Basically I have lists in the order 20,000 records which I
need to run a SQL query involving 'WHERE' clause. My files/environment
is a Sun (OS 5.6) unix box.
SQL 'WHERE' clause can only accept 1000 records at a time - hence, I
need to split the list in individual files of 1000 records each.

Here is a sample input file - its a list, so each record is separated
by a new line character and there are no duplicates in the list.

InputFile
---------
100789A10
100789A11
100789A12
100789A13
...
...
887648A55
887648A56
887648A57

When I use the split command as shown below
split -1000 InputFile
The resulting files get named as
xaa
xab
xac
..
..
xgh
question 1: how can I make the starting and ending line numbers appear
in the resulting output files. For example
if my Input file name is InputFile and has 20000 records, I would like
the resulting Output files named as
InputFile_00000-01000
InputFile_01001-02000
..
..
InputFile_19001-20000

Upon running the split command, I now have 20 output files - each has
1000 records. I need to insert some text in each output file so to
convert them to .sql files. When I run the split command, the
resulting xaa, xab..etc files will obviously be in the same directory?
question - can anyone show me how to do this. An illustration is
below.

xaa (sample file)
100789A10
100789A11
100789A12
100789A13
...
...
887648A55
887648A56
887648A57

xaa.sql (resulting .sql file)
SELECT PART_NUMBER '|' DESCRIPTION FROM PARTS_TABLE where PART_NUMBER
IN (
'100789A10',
'100789A11',
'100789A12',
'100789A13',
'...      ',
'...      ',
'887648A55',
'887648A56',
'887648A57'
);

Thank you,
Sri

2. Solaris on PC P200 vs Ppro20?

3. source code for Unix "split" command

4. What's the best way to save/load configuration?

5. Ho to log access_log to split file and based on split per-virtual-host !

6. XDR problem (on Linux): xdr_hyper (and xdr_uint) nonexistent.

7. trivial question on the split command

8. help on some libs

9. Unix slot and mcapos and hwconfig command files and questions????

10. Unix slot and mcapos and hwconfig commands files and questions???

11. unix slot and mcapos and hwconfig command files and questions.

12. Help W/Unix Shell Script - File Splitting Based on Text String

13. Splitting a unix-file