Simple sorting question

Simple sorting question

Post by Gregory Rayne » Sun, 06 Jul 1997 04:00:00



I know that this is a really stupid and basic question, but here goes:

I read somewhere how to remove similar lines from a file. So that is you
have say

pear
apple
pear
banana
apple

it can be sorted to become

apple
banana
pear

I can't for the life of me remember how to do this. Is this part of "sort"
or "diff" or yet another command?

Any help would be appreciated!

Greg

 
 
 

Simple sorting question

Post by Nick Russ » Sun, 06 Jul 1997 04:00:00


cat file | sort | uniq > newfile

uniq will get rid of adjacent identical lines...

Nick Russo


> I know that this is a really stupid and basic question, but here goes:

> I read somewhere how to remove similar lines from a file. So that is you
> have say

> pear
> apple
> pear
> banana
> apple

> it can be sorted to become

> apple
> banana
> pear

> I can't for the life of me remember how to do this. Is this part of "sort"
> or "diff" or yet another command?

> Any help would be appreciated!

> Greg


 
 
 

Simple sorting question

Post by Randal Schwart » Sun, 06 Jul 1997 04:00:00


This Week's Useless Use of Cat Award goes to

Nick> cat file | sort | uniq > newfile

And of course, if you've been following along for a week or two, you know
that this (BING!) is a Useless Use of Cat!

Rememeber, nearly all cases where you have:

        cat file | some_command and its args ...

you can rewrite it as:

        <file some_command and its args ...

and in some cases, such as this one, you can move the filename
to the arglist as in:

        some_command and its args ... file

Of course, the real tragedy in this post is that sort also has a -u
switch, meaning this could have been simply:

        sort -u file >newfile

Oh well.

Just another Useless Use of Usenet,
--
Name: Randal L. Schwartz / Stonehenge Consulting Services (503)777-0095
Keywords: Perl training, UNIX[tm] consulting, video production, skiing, flying

Web: <A HREF="http://www.stonehenge.com/merlyn/">My Home Page!</A>
Quote: "I'm telling you, if I could have five lines in my .sig, I would!" -- me

 
 
 

Simple sorting question

Post by Nick Russ » Sun, 06 Jul 1997 04:00:00


As soon as I posted this, I realized that I didn't need cat. Then later,
reading another message, I realized sort had a u flag for unique. I now
realize there's a reason pine says "Posted message may go to thousands of
readers. Really post?" And now, I slink away into the background...

I'll go back to quietly reading more thought-out posts, at least until I
wake up...

Nick Russo


> This Week's Useless Use of Cat Award goes to


> Nick> cat file | sort | uniq > newfile

> And of course, if you've been following along for a week or two, you know
> that this (BING!) is a Useless Use of Cat!

> Rememeber, nearly all cases where you have:

>    cat file | some_command and its args ...

> you can rewrite it as:

>    <file some_command and its args ...

> and in some cases, such as this one, you can move the filename
> to the arglist as in:

>    some_command and its args ... file

> Of course, the real tragedy in this post is that sort also has a -u
> switch, meaning this could have been simply:

>    sort -u file >newfile

> Oh well.

> Just another Useless Use of Usenet,
> --
> Name: Randal L. Schwartz / Stonehenge Consulting Services (503)777-0095
> Keywords: Perl training, UNIX[tm] consulting, video production, skiing, flying

> Web: <A HREF="http://www.stonehenge.com/merlyn/">My Home Page!</A>
> Quote: "I'm telling you, if I could have five lines in my .sig, I would!" -- me

 
 
 

Simple sorting question

Post by Michael P. Di Fulvi » Mon, 07 Jul 1997 04:00:00



> cat file | sort | uniq > newfile

> uniq will get rid of adjacent identical lines...

> Nick Russo

> > I know that this is a really stupid and basic question, but here goes:

> > I read somewhere how to remove similar lines from a file. So that is you
> > have say

> > pear
> > apple
> > pear
> > banana
> > apple

> > it can be sorted to become

> > apple
> > banana
> > pear

> > I can't for the life of me remember how to do this. Is this part of "sort"
> > or "diff" or yet another command?

> > Any help would be appreciated!

> > Greg

Try the direct route with % sort -fu foo >foo_sorted

This will remove the dup's and sort the file for you.

--
Michael P. Di Fulvio         W3 Solutions Inc.
PO Box 451011                Sunrise, Florida 33345              

Unix/Internet Consultants    Web Hosting,Security,Integration
--

 
 
 

Simple sorting question

Post by Philippe Cad » Tue, 08 Jul 1997 04:00:00



> I read somewhere how to remove similar lines from a file. So that is you
> have say

> pear
> apple
> pear
> banana
> apple

> it can be sorted to become

> apple
> banana
> pear

> I can't for the life of me remember how to do this. Is this part of "sort"
> or "diff" or yet another command?

try "uniq"

Philippe

--
i

:wq

 
 
 

Simple sorting question

Post by A.K.Srikant » Tue, 08 Jul 1997 04:00:00




> > I read somewhere how to remove similar lines from a file. So that is you

"uniq" will work.. However, be sure to sort the list first before using
"uniq". uniq assumes a sorted list and except similar lines to be
adjacent to each other.

Srikanth

> > have say

> > pear
> > apple
> > pear
> > banana
> > apple

> > it can be sorted to become

> > apple
> > banana
> > pear

> > I can't for the life of me remember how to do this. Is this part of "sort"
> > or "diff" or yet another command?

> try "uniq"

> Philippe
> --
> i

> :wq

--
-------------------------------------------------------------------------------
Ayikudy K. Srikanth                      Data Link (DP)- Protocol
Development
Bay Networks Inc     2,Federal Street    Billerica, MA 01821         O-

-------------------------------------------------------------------------------
 
 
 

Simple sorting question

Post by Charles Stevenso » Tue, 08 Jul 1997 04:00:00




> (Erik Hensema) posted to comp.unix.user-friendly:
>  >> pear
>  >> apple
>  >> pear
>  >> banana
>  >> apple
>  >> it can be sorted to become
>  >> apple
>  >> banana
>  >> pear
>  > cat filename|sort|uniq does the trick.

> Since this is the newbie group (right?) it's probably best to avoid
> sarcasm when awarding an Useless Use of Cat Award. If you want it with
> sarcasm, post your suggestion to comp.unix.shell and wait for Randal
> to wake up. :-)

> Anything written 'cat single_file | whatever' can be reformulated as
> 'whatever < single_file' or even '< single_file whatever'.
>   The raison d'etre for cat is to concatenate files. When you want to
> feed the contents of several files to a pipeline, you do have to cat
> them.

> Also note that many implementations of sort have a -u switch which
> will discard duplicates on the fly. So we go from three processes to
> only one.

>     sort -u < filename

> Hope this helps,

> /* era */

> --
> Defin-i-t-e-ly. Sep-a-r-a-te. Gram-m-a-r.  <http://www.iki.fi/~era/>
>  * Enjoy receiving spam? Register at <http://www.iki.fi/~era/spam.html>

1. Any advantage to
    sort -u < filename
over
   sort -u filename  ?

2.  How about a further complication:

I have blank lines interspersed with the good data:

pear
apple
              <--- null
pear
banana
              <--- null
apple

"sort -u"   yields:

            <--- null
apple
banana
pear

I want to get rid of that null line.
I can pipe the whole thing through grep or awk:

   sort -u filename | grep -v '^$' >
   sort -u filename | awk '$0 !~ /^$/ {print$0}'

But that means every line in the file is checked for null,  when, in fact, only the first line will (might) be
null.   This seems to approximately triple processing time ( For long lists 45 vs. 15 seconds on my box).

What is a better way?
What if I did someting like
   "cat one.linefeed filename | sort -u | [some command to always remove 1st line]"

With gratitude in advance,
Charles Stevenson

 
 
 

Simple sorting question

Post by Al A » Wed, 09 Jul 1997 04:00:00


                        sort -u filename

(-u for unique)

: > I read somewhere how to remove similar lines from a file. So that is you
: > have say
: >
: > pear
: > apple
: > pear
: > banana
: > apple
: >
: > it can be sorted to become
: >
: > apple
: > banana
: > pear
: >
: > I can't for the life of me remember how to do this. Is this part of "sort"
: > or "diff" or yet another command?

: try "uniq"

: Philippe

: --
: i

: :wq
--
=-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
al aab, seders moderator                                      sed u soon
               it is not zat we do not see the  s o l u t i o n          
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-+

 
 
 

1. Simple SORT question

Hello,

I am using a sunsparc 10

Can I use the sort command to sort on field 1 first and then field 3?

Problem is I need to sort both fields using the -n option.

Any suggestions are appreciated.

e-mail preferred


Thanks in advance

Bonnie

2. Bus Mouse?

3. simple sort question

4. licq

5. Simple "sort" question

6. S70 tty's issues

7. simple (sort of stupid) software raid question

8. S: Product for connection to IBM mainframe with SNA and RJE

9. SORT simple question?

10. sort sort: 0653-657 A write error occurred while sorting (4.1.3)

11. Simple mail sorting without procmail

12. sort of Network Block Device, open, simple, and especially cross-platform?

13. Simple but challenging sort problem