selecting duplicate lines in file

selecting duplicate lines in file

Post by S. Mottah » Sun, 24 Mar 1996 04:00:00



I'd appreciate your help with the following:
I have a file and wish to select and print the lines that occur more
than once.

Thank you.
Steve

--
--=====================================================--
=   Mercato Dental Centre       (Dr. S. Mottahed Inc.)  =
=   Vancouver, BC, CANADA       (604) 254-5900          =

=========================================================

 
 
 

selecting duplicate lines in file

Post by Shawn D. McPe » Sun, 24 Mar 1996 04:00:00


: I'd appreciate your help with the following:
: I have a file and wish to select and print the lines that occur more
: than once.

sort <file> | uniq -d

 
 
 

selecting duplicate lines in file

Post by David and Rachel Weintra » Sun, 24 Mar 1996 04:00:00




> I'd appreciate your help with the following:
> I have a file and wish to select and print the lines that occur more
> than once.

> Thank you.
> Steve

Take a look at the uniq command (man uniq). This may do what you want. I'm
not sure if you are only looking for duplicate lines that follow each
other or any line that has a duplicate:

uniq -d myFile          #If two or more lines together are duplicated
sort myFile | uniq -d   #If any line has a duplicate

The second form sorts your file, then prints out duplicate lines. You can
try a "-c" with uniq to get the exact count, but it doesn't work with the
"-d" on HP-UX (although the man pages state otherwise).

--
David Weintraub                    _/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/
System Configuration Management   _/                                      _/
Dow Jones/Telerate - NSG         _/    I AM THE GREAT AND POWERFUL OZ*   _/


                           *Pay no attention to the man behind the curtains

 
 
 

selecting duplicate lines in file

Post by Phil Abercromb » Tue, 26 Mar 1996 04:00:00





> > I'd appreciate your help with the following:
> > I have a file and wish to select and print the lines that occur more
> > than once.

> > Thank you.
> > Steve

> Take a look at the uniq command (man uniq). This may do what you want. I'm
> not sure if you are only looking for duplicate lines that follow each
> other or any line that has a duplicate:

> uniq -d myFile          #If two or more lines together are duplicated
> sort myFile | uniq -d   #If any line has a duplicate

> The second form sorts your file, then prints out duplicate lines. You can
> try a "-c" with uniq to get the exact count, but it doesn't work with the
> "-d" on HP-UX (although the man pages state otherwise).

If you want to preserve the original order of the file, but your
duplicate lines are not adjacent, you may want to resort to

perl -ne 'print unless $a{$_}++' myFile

This would be a Bad Thing if your file were very large (actually
probably no worse than the solution using sort)

        Phil
--
 Phil Abercrombie         |   The nail which |    __O   | Bike

 +64 4 496 9097           | be hammered flat | (*)/-(*) | Work

 
 
 

1. select line from file, and then the next line a week later

Let's say I have a file of 5 email addresses. What if I wanted to email
one person on the list, in sequence, every friday.  So on the first
friday, I would email the first person. The following friday I would
email the second person, and so on....

Now I could cron it for friday, but how can I get the script to grab
line one the first time I run it, then grab line 2 the next time I run
it. Is this even possible?

I don't think 'case' can do this for me. I am guessing awk might be
able to, but I am no good at awk.

2. How do I *not* initialize IPX, etc...?

3. Perl script log file output has duplicate copies of output lines

4. HELP! Cannot initialize X for Apollo running HPUX v9.0

5. Removing duplicate lines from a text file

6. Pipe-able vi.

7. Remove duplicated lines in a file

8. How can I implement to get system infomation?

9. deleting duplicate lines in a file.

10. duplicate lines in passwd file

11. How can I prepend comments into selected lines of a data file?

12. select alternative lines from a file

13. randomly select a line from a file