Linux vs. NT Benchmarks - Put your money where your mouth is!

Linux vs. NT Benchmarks - Put your money where your mouth is!

Post by Will » Fri, 15 May 1998 04:00:00



Someone had mentioned how fast Linux/Unix can search the hard drive
for a text string.

Well, that's a pretty simple and straight forward process that was
optimized long before Linux and NT were around.  And since I happen to
have to make a search, why not benchmark it?

So here goes.
I can search for "www.ivillage.com"....
through 6511 files...
totaling 337,055,891 bytes...
in 310 seconds.

That works out to just over 1.087 MBps.

The machine is a 6x86MX PR200+ with 64MB RAM running NT with SP3 and
IE4.

Okay Linux dudes...what have you got?

 
 
 

Linux vs. NT Benchmarks - Put your money where your mouth is!

Post by Mark Hamstr » Fri, 15 May 1998 04:00:00



> Someone had mentioned how fast Linux/Unix can search the hard drive
> for a text string.

> Well, that's a pretty simple and straight forward process that was
> optimized long before Linux and NT were around.  And since I happen to
> have to make a search, why not benchmark it?

> So here goes.
> I can search for "www.ivillage.com"....
> through 6511 files...
> totaling 337,055,891 bytes...
> in 310 seconds.

> That works out to just over 1.087 MBps.

> The machine is a 6x86MX PR200+ with 64MB RAM running NT with SP3 and
> IE4.

> Okay Linux dudes...what have you got?

You need to check your math.  337,055,891 bytes/310 seconds = 1.037MB/s.

For comparison, i just grep'd through 819,091,456 bytes in my source
tree in 745 seconds (=> 1.049MB/s).  That's on a PPro200, 128MB RAM.

Oh, and it also had 83 processes running at the time, including two
open telnet sessions, a remote X session, several server daemons...
and the real kicker: a similar PPro200 box running NT and building
from the same Samba-exported source tree that I was grep'ing locally.

I'd dare bet that's a fair sight more load (and particularly filesystem
load) than your NT box was seeing, and Linux still comes out ahead.

Next.

--
Mark Hamstra
Bentley Systems, Inc.

 
 
 

Linux vs. NT Benchmarks - Put your money where your mouth is!

Post by Mike Kondrato » Fri, 15 May 1998 04:00:00


You use Cyrix :) HEHAH HA HHAERAA HA  ( Real performace processor )
I would never put such junk into my PC..
such a junk.. you buy 20 of them and at least one will be bad.


> Someone had mentioned how fast Linux/Unix can search the hard drive
> for a text string.

> Well, that's a pretty simple and straight forward process that was
> optimized long before Linux and NT were around.  And since I happen to
> have to make a search, why not benchmark it?

> So here goes.
> I can search for "www.ivillage.com"....
> through 6511 files...
> totaling 337,055,891 bytes...
> in 310 seconds.

> That works out to just over 1.087 MBps.

> The machine is a 6x86MX PR200+ with 64MB RAM running NT with SP3 and
> IE4.

> Okay Linux dudes...what have you got?

 
 
 

Linux vs. NT Benchmarks - Put your money where your mouth is!

Post by Will » Sat, 16 May 1998 04:00:00


On Thu, 14 May 1998 18:52:18 +0100, Mike Kondratov


>You use Cyrix :) HEHAH HA HHAERAA HA  ( Real performace processor )
>I would never put such junk into my PC..
>such a junk.. you buy 20 of them and at least one will be bad.

You Linux dudes like to talk about how great Linux runs on a 386, and
now you make fun of my CPU.  I guess that must mean that you Linux
machine didn't do as well as mine did on the benchmark.

I knew it all along.


>> Someone had mentioned how fast Linux/Unix can search the hard drive
>> for a text string.

>> Well, that's a pretty simple and straight forward process that was
>> optimized long before Linux and NT were around.  And since I happen to
>> have to make a search, why not benchmark it?

>> So here goes.
>> I can search for "www.ivillage.com"....
>> through 6511 files...
>> totaling 337,055,891 bytes...
>> in 310 seconds.

>> That works out to just over 1.087 MBps.

>> The machine is a 6x86MX PR200+ with 64MB RAM running NT with SP3 and
>> IE4.

>> Okay Linux dudes...what have you got?

 
 
 

Linux vs. NT Benchmarks - Put your money where your mouth is!

Post by Will » Sat, 16 May 1998 04:00:00





>> Someone had mentioned how fast Linux/Unix can search the hard drive
>> for a text string.

>> Well, that's a pretty simple and straight forward process that was
>> optimized long before Linux and NT were around.  And since I happen to
>> have to make a search, why not benchmark it?

>> So here goes.
>> I can search for "www.ivillage.com"....
>> through 6511 files...
>> totaling 337,055,891 bytes...
>> in 310 seconds.

>> That works out to just over 1.087 MBps.

>> The machine is a 6x86MX PR200+ with 64MB RAM running NT with SP3 and
>> IE4.

>> Okay Linux dudes...what have you got?

>You need to check your math.  337,055,891 bytes/310 seconds = 1.037MB/s.

No, we should just decide on what we mean by MB.  How about this.
 337,055,891bytes / 310seconds = 1087277bytes per second.

Quote:>For comparison, i just grep'd through 819,091,456 bytes in my source
>tree in 745 seconds (=> 1.049MB/s).  That's on a PPro200, 128MB RAM.

>Oh, and it also had 83 processes running at the time, including two
>open telnet sessions, a remote X session, several server daemons...
>and the real kicker: a similar PPro200 box running NT and building
>from the same Samba-exported source tree that I was grep'ing locally.

>I'd dare bet that's a fair sight more load (and particularly filesystem
>load) than your NT box was seeing, and Linux still comes out ahead.

I run a proxy server, and, at the time, NT was tracking about 28
processes.  But that doesn't mean anything if it's all sitting idle.

Frankly, I'm surprised that a Pentium Pro was barely faster.  The
processor itself is much faster than my Cyrix.

I don't know what the load was from your other processes so as far as
I'm concerned Linux was slower.  Shut down all your processes and try
it again so you can get an accurate number.

 
 
 

Linux vs. NT Benchmarks - Put your money where your mouth is!

Post by Mark Jackso » Sat, 16 May 1998 04:00:00


: I don't know what the load was from your other processes so as far as
: I'm concerned Linux was slower.  Shut down all your processes and try
: it again so you can get an accurate number.

Since NT and Linux both run on similar hardware, any benchmark achieved by
one is theoretically achievable by the other.  To claim NT is so much
faster than Linux to justify its price is absurd.  In an ideal world, with
both OS's operating at 100% efficiency on the same hardware, the benchmarks
should be identical.  It's the same hardware!

The relevant number here is price/performance.  With a price of $0, Linux
wins anytime, anywhere.  Even if a certain NT system is able to run
comparably in speed, what price have you paid to get there?

Where do I want to go today?  Hmm.  With the money I saved on Linux instead
of buying NT I can fly anywhere in the US round trip and still have something
left over...  And then some...

 
 
 

Linux vs. NT Benchmarks - Put your money where your mouth is!

Post by David M. Co » Sat, 16 May 1998 04:00:00



>On Thu, 14 May 1998 18:52:18 +0100, Mike Kondratov

>>You use Cyrix :) HEHAH HA HHAERAA HA  ( Real performace processor )
>>I would never put such junk into my PC..
>>such a junk.. you buy 20 of them and at least one will be bad.
>You Linux dudes

Mr. Kondratov was posting from NT.

Dave Cook

 
 
 

Linux vs. NT Benchmarks - Put your money where your mouth is!

Post by Ed Cogbur » Sat, 16 May 1998 04:00:00






> >> Someone had mentioned how fast Linux/Unix can search the hard drive
> >> for a text string.

> >> Well, that's a pretty simple and straight forward process that was
> >> optimized long before Linux and NT were around.  And since I happen to
> >> have to make a search, why not benchmark it?

> >> So here goes.
> >> I can search for "www.ivillage.com"....
> >> through 6511 files...
> >> totaling 337,055,891 bytes...
> >> in 310 seconds.

> >> That works out to just over 1.087 MBps.

> >> The machine is a 6x86MX PR200+ with 64MB RAM running NT with SP3 and
> >> IE4.

> >> Okay Linux dudes...what have you got?

> >You need to check your math.  337,055,891 bytes/310 seconds = 1.037MB/s.

> No, we should just decide on what we mean by MB.  How about this.
>  337,055,891bytes / 310seconds = 1087277bytes per second.

> >For comparison, i just grep'd through 819,091,456 bytes in my source
> >tree in 745 seconds (=> 1.049MB/s).  That's on a PPro200, 128MB RAM.

> >Oh, and it also had 83 processes running at the time, including two
> >open telnet sessions, a remote X session, several server daemons...
> >and the real kicker: a similar PPro200 box running NT and building
> >from the same Samba-exported source tree that I was grep'ing locally.

> >I'd dare bet that's a fair sight more load (and particularly filesystem
> >load) than your NT box was seeing, and Linux still comes out ahead.

> I run a proxy server, and, at the time, NT was tracking about 28
> processes.  But that doesn't mean anything if it's all sitting idle.

> Frankly, I'm surprised that a Pentium Pro was barely faster.  The
> processor itself is much faster than my Cyrix.

> I don't know what the load was from your other processes so as far as
> I'm concerned Linux was slower.  Shut down all your processes and try
> it again so you can get an accurate number.

    Oh good grief.  Look Willy, if you want to pick a fight with a
"linux dude" you can certainly find several good subjects to bait them
with.  This comparison is meaningless unless its done on the same
hardware, and the search is done across the same files, i.e. the total
bytes searched is the same (Mark grepped almost 2x your data).  Neither
of you even mentioned your hard drive subsystem which is just as, or
more, important than the processor used for a test like this.

--
Life is a Rogue-like game:  It hits! You resist.. It hits! -more-
                            You feel confused.. It hits! -more-
                            You feel weaker.. It hits! It hits! -more-
                            You die.. Dump char file[y/n]?

 
 
 

Linux vs. NT Benchmarks - Put your money where your mouth is!

Post by Will » Sat, 16 May 1998 04:00:00


On Fri, 15 May 1998 06:44:27 -0500, Ed Cogburn


>    Oh good grief.  Look Willy, if you want to pick a fight with a
>"linux dude" you can certainly find several good subjects to bait them
>with.  This comparison is meaningless unless its done on the same
>hardware, and the search is done across the same files, i.e. the total
>bytes searched is the same (Mark grepped almost 2x your data).  Neither
>of you even mentioned your hard drive subsystem which is just as, or
>more, important than the processor used for a test like this.

First, almost every "Linux is better than NT" argument invariably
claims Linux is *much* faster.  I just want some proof on this point.

Second, the time needed to search through 1 MB of data is far below
the 1 MB transfer rate of just about every hard disk made in the last
10 years, so the disk system is spending most its time waiting.
That's not to say that it isn't important; just far less important
than the processor.  On my machine, the performance of a process
reading from the disk, and the same process reading from memory,
differ by 9%.

Third, the amount of data search means nothing since all results, so
far, have been reduced to a MBps rate.

Finally, I was hoping someone would try this on a Pentium 200 Classic
or, even better, a Cyrix chip.  However, I'm interested in any
results.

 
 
 

Linux vs. NT Benchmarks - Put your money where your mouth is!

Post by Will » Sat, 16 May 1998 04:00:00




Quote:>Since NT and Linux both run on similar hardware, any benchmark achieved by
>one is theoretically achievable by the other.  To claim NT is so much
>faster than Linux to justify its price is absurd.  In an ideal world, with
>both OS's operating at 100% efficiency on the same hardware, the benchmarks
>should be identical.  It's the same hardware!

If only the world was this simple...

I never claimed that NT is faster than anything.  I just wanted proof
that Linux is *much* faster, as claimed by RedHat and other Linux
advocates.

As for the results being identical, that is simply not the case.  The
piece of the OS being tested is not it's ability to search through
files.  I chose that test because I believe neither system would have
a method that could be considered "better" for searching through
files.  It's the OS properties that (at least in NT) we have no
control over that I'm testing; the file system, the multi-tasking
managment capabilities, etc; all the part that make the search happen.
It's these parts that make a system fast or slow.

It is a fact that OS/2 and NT run DOS programs faster than DOS.  It is
a fact that NT will run the same software faster than Win95.  The OS
has a great deal to do with the speed of processing.  We don't live in
al ideal world.

Quote:>The relevant number here is price/performance.  With a price of $0, Linux
>wins anytime, anywhere.  Even if a certain NT system is able to run
>comparably in speed, what price have you paid to get there?

How, exactly do you get Linux for free?  If you download it, that
means you have a working computer to begin with so there a cost
associated with simply being able to download it.  A CD version always
cost at least a few bucks (Red Hat is 49.95).  Anyway, there are a lot
of other factors that are important when you consider the true value
of what you're getting.  Your needs will determine which one is the
better value.

Quote:>Where do I want to go today?  Hmm.  With the money I saved on Linux instead
>of buying NT I can fly anywhere in the US round trip and still have something
>left over...  And then some...

On what...a paper airplane?  NT is only 280 bucks (and even that is in
the expensive stores.)
 
 
 

Linux vs. NT Benchmarks - Put your money where your mouth is!

Post by Mark Hamstr » Sat, 16 May 1998 04:00:00






> >> Someone had mentioned how fast Linux/Unix can search the hard drive
> >> for a text string.

> >> Well, that's a pretty simple and straight forward process that was
> >> optimized long before Linux and NT were around.  And since I happen to
> >> have to make a search, why not benchmark it?

> >> So here goes.
> >> I can search for "www.ivillage.com"....
> >> through 6511 files...
> >> totaling 337,055,891 bytes...
> >> in 310 seconds.

> >> That works out to just over 1.087 MBps.

> >> The machine is a 6x86MX PR200+ with 64MB RAM running NT with SP3 and
> >> IE4.

> >> Okay Linux dudes...what have you got?

> >You need to check your math.  337,055,891 bytes/310 seconds = 1.037MB/s.

> No, we should just decide on what we mean by MB.  How about this.
>  337,055,891bytes / 310seconds = 1087277bytes per second.

> >For comparison, i just grep'd through 819,091,456 bytes in my source
> >tree in 745 seconds (=> 1.049MB/s).  That's on a PPro200, 128MB RAM.

> >Oh, and it also had 83 processes running at the time, including two
> >open telnet sessions, a remote X session, several server daemons...
> >and the real kicker: a similar PPro200 box running NT and building
> >from the same Samba-exported source tree that I was grep'ing locally.

> >I'd dare bet that's a fair sight more load (and particularly filesystem
> >load) than your NT box was seeing, and Linux still comes out ahead.

> I run a proxy server, and, at the time, NT was tracking about 28
> processes.  But that doesn't mean anything if it's all sitting idle.

> Frankly, I'm surprised that a Pentium Pro was barely faster.  The
> processor itself is much faster than my Cyrix.

Guess again.  This kind of text search is almost completely I/O
bound.  If you watch the CPU load during such a search, you'll
find that the CPU is more than 90% idle throughout the run.  In
such a case, processor benchmark speed (particularly on such
things as floating point operations, where the PPro far exceeds
your Cyrix) means very little above a certain low minimum.
What's more important is clock speed (particularly on the system
bus, where you may well be ahead) and, even more so, disk speed.

Thus my running of another fairly disk intensive operation on
the same filesystem while performing your text search should
have been a significant penalty.

These comparisons are pretty meaningless because of the wide
variations possible in disk subsystems, but they certainly don't
give you reason to claim NT performs better than Linux.

Quote:> I don't know what the load was from your other processes so as far as
> I'm concerned Linux was slower.  Shut down all your processes and try
> it again so you can get an accurate number.

I did.  The numbers are still as meaningless as before, but they
are about 10% faster with little more than the single search
running.  Of course, if you want to see the numbers jump, scale
down to a 486 and scale up to a cacheing RAID 5 subsystem; but
that will hardly prove that whichever OS you choose to run on
such a rig is faster than any other OS running a similar search
on a 1GHz Alpha 21264 attached to nothing but a tape drive.

--
Mark Hamstra
Bentley Systems, Inc.

 
 
 

Linux vs. NT Benchmarks - Put your money where your mouth is!

Post by Mark Hamstr » Sat, 16 May 1998 04:00:00



> On Fri, 15 May 1998 06:44:27 -0500, Ed Cogburn

> >    Oh good grief.  Look Willy, if you want to pick a fight with a
> >"linux dude" you can certainly find several good subjects to bait them
> >with.  This comparison is meaningless unless its done on the same
> >hardware, and the search is done across the same files, i.e. the total
> >bytes searched is the same (Mark grepped almost 2x your data).  Neither
> >of you even mentioned your hard drive subsystem which is just as, or
> >more, important than the processor used for a test like this.

> First, almost every "Linux is better than NT" argument invariably
> claims Linux is *much* faster.  I just want some proof on this point.

> Second, the time needed to search through 1 MB of data is far below
> the 1 MB transfer rate of just about every hard disk made in the last
> 10 years, so the disk system is spending most its time waiting.

Bullshit.  There is a wide difference between the mean throughput
you will see across such a multi-file search and the maximum burst
or sustained throughput of the drive --all that head movement takes
an enternity in processor cycles.  It's your CPU that is sitting
there twiddling its thumbs the vast majority of the time.

Quote:> That's not to say that it isn't important; just far less important
> than the processor.  On my machine, the performance of a process
> reading from the disk, and the same process reading from memory,
> differ by 9%.

If I actually believed you, you would have succeeded in doing little
more than proving NT's memory management horribly deficient.  There
are huge differences between the speed of main memory read/writes
and disk read/writes, not to even mention the gargantuan differences
for L2 or L1 cache read/writes.  If this weren't the case, why would
we spend so much money for comparatively expensive RAM and cache when
we could just as quickly and more cheaply do all the operations
directly on disk?

Quote:> Third, the amount of data search means nothing since all results, so
> far, have been reduced to a MBps rate.

More or less true, but reduction to a MBps figure certainly doesn't
account for a large number of other variables.  This result is a
very gross estimate of I/O performance and not at all useful for
any fine-grained comparison.

Quote:> Finally, I was hoping someone would try this on a Pentium 200 Classic
> or, even better, a Cyrix chip.  However, I'm interested in any
> results.

As I explained before, raw processor speed will make little
difference in such a benchmark.

--
Mark Hamstra
Bentley Systems, Inc.

 
 
 

Linux vs. NT Benchmarks - Put your money where your mouth is!

Post by Mark Hamstr » Sat, 16 May 1998 04:00:00



> It is a fact that NT will run the same software faster than Win95.

Wrong again, Willy.  There is a very large class of software (in fact,
probably most single-user application software) that runs faster under
Win95 than under NT.  At least while it's not crashing.

The major advantage of NT over 95 is in terms of stability, not raw
performance.

--
Mark Hamstra
Bentley Systems, Inc.

 
 
 

Linux vs. NT Benchmarks - Put your money where your mouth is!

Post by Will » Sat, 16 May 1998 04:00:00





>> It is a fact that NT will run the same software faster than Win95.

>Wrong again, Willy.  There is a very large class of software (in fact,
>probably most single-user application software) that runs faster under
>Win95 than under NT.  At least while it's not crashing.

Name it.
 
 
 

Linux vs. NT Benchmarks - Put your money where your mouth is!

Post by Will » Sat, 16 May 1998 04:00:00





>> On Fri, 15 May 1998 06:44:27 -0500, Ed Cogburn

>> >    Oh good grief.  Look Willy, if you want to pick a fight with a
>> >"linux dude" you can certainly find several good subjects to bait them
>> >with.  This comparison is meaningless unless its done on the same
>> >hardware, and the search is done across the same files, i.e. the total
>> >bytes searched is the same (Mark grepped almost 2x your data).  Neither
>> >of you even mentioned your hard drive subsystem which is just as, or
>> >more, important than the processor used for a test like this.

>> First, almost every "Linux is better than NT" argument invariably
>> claims Linux is *much* faster.  I just want some proof on this point.

>> Second, the time needed to search through 1 MB of data is far below
>> the 1 MB transfer rate of just about every hard disk made in the last
>> 10 years, so the disk system is spending most its time waiting.

>Bullshit.  There is a wide difference between the mean throughput
>you will see across such a multi-file search and the maximum burst
>or sustained throughput of the drive --all that head movement takes
>an enternity in processor cycles.  It's your CPU that is sitting
>there twiddling its thumbs the vast majority of the time.

>> That's not to say that it isn't important; just far less important
>> than the processor.  On my machine, the performance of a process
>> reading from the disk, and the same process reading from memory,
>> differ by 9%.

>If I actually believed you, you would have succeeded in doing little
>more than proving NT's memory management horribly deficient.  There
>are huge differences between the speed of main memory read/writes
>and disk read/writes, not to even mention the gargantuan differences
>for L2 or L1 cache read/writes.  If this weren't the case, why would
>we spend so much money for comparatively expensive RAM and cache when
>we could just as quickly and more cheaply do all the operations
>directly on disk?

>> Third, the amount of data search means nothing since all results, so
>> far, have been reduced to a MBps rate.

>More or less true, but reduction to a MBps figure certainly doesn't
>account for a large number of other variables.  This result is a
>very gross estimate of I/O performance and not at all useful for
>any fine-grained comparison.

>> Finally, I was hoping someone would try this on a Pentium 200 Classic
>> or, even better, a Cyrix chip.  However, I'm interested in any
>> results.

>As I explained before, raw processor speed will make little
>difference in such a benchmark.

Then how should it be done?
 
 
 

1. Putting your money where you mouth is

Ok,

So lots of people are in love with Linux.  I think that it will eventually
be THE NT killer everyone has been praying for.  I'd  like to put my vote
where it counts.  I wnat to invest in Linux based computing companies.  Is
there a Mutual fund, venture capitol firm, or any other investing product
out there which SPECIFICALLY targets Linux/GNU/Open software/hardware
developers.  If we invest in htese people, they will have the capitol to
go fourth and slay the micro$oft giants of the world.  

Any Ideas?

--
"I'm always baffled by the number of software people who think fast code is better than correct code!"  Me.  


Senior Software Engineer and jack of all trades.  

2. Building network with private IP addresses

3. Put your money where your mouth is...

4. Motorola sees the light!

5. Ok, putting money where my mouth is...

6. Win95/NT and samba-1.9.18p7

7. Put your money where your mouth is

8. Usenet/Web based technical support for Linux

9. Linux vs OS2 vs NT vs Win95 vs Multics vs PDP11 vs BSD geeks

10. Linux vs. NT: Benchmarks

11. A useless NT vs Linux benchmark

12. [Fwd: Linux vs. NT: Benchmarks]

13. Benchmarks of software on NT vs. linux