Benchmarking Software Found

Benchmarking Software Found

Post by Christopher Lloyd - not the act » Sun, 04 Sep 1994 04:23:18




>I would normally make the source available, but there's the problem
>of vendors tampering with the code.

 Any vendor dumb enough to tamper with publicly available benchmarks
 will get what they deserve when someone else runs the real benchmarks
 on said vendors machine.

 IMO, It's a good idea to have benchmarks open for public scrutiny,
 makes for good checks all around.

--

|: "If you find this world bad, you should see some of the others" -PKD :|

 
 
 

Benchmarking Software Found

Post by Darcy Brockba » Sun, 04 Sep 1994 03:10:32


The mysterious benchmarking software has been located and sent to me. I
now have a question:

Should I make the source available to everyone, or just a binary?

I would normally make the source available, but there's the problem
of vendors tampering with the code. I don't know if this is too much
an issue, but I'd like to see a public debate here before making
a decision.

I am verifying it, and will be testing our systems. We're in the process
of getting some HPs, and do already have a sun so it will be tri-fat
and quad-fat when possible.

Comments please!

- db

--
Trust me... trust me... trust me! -- Tool -- I just decided I don't trust you
anymore -- The Wedding Present -- Darcy is too politically correct. -- Joe --
You have to think for both of us, for all of us. -- Ilsa -- Here's looking at
you, kid -- Rick -- That's the price of heroes -- REM -- Ack Phbbt! -- BTC --

 
 
 

Benchmarking Software Found

Post by Garance A. Drose » Thu, 08 Sep 1994 01:05:37




> >I would normally make the source available, but there's the problem
> >of vendors tampering with the code.

>  Any vendor dumb enough to tamper with publicly available benchmarks
>  will get what they deserve when someone else runs the real benchmarks
>  on said vendors machine.

It isn't that people will tamper with the benchmarks themselves,
it's that they'll look at the benchmarks and tamper their hardware
so the hardware (maybe) does great on the benchmarks without really
being all that much faster in normal usage.

Quote:>  IMO, It's a good idea to have benchmarks open for public scrutiny,
>  makes for good checks all around.

I'd tend to agree with this, even with the danger of vendor's
optimizing their hardware for the benchmark.

--

ITS Systems Programmer            (handles NeXT-type mail)
Rensselaer Polytechnic Institute;           Troy NY    USA