Ext3 vs Reiserfs benchmarks

Ext3 vs Reiserfs benchmarks

Post by Dax Kelso » Sun, 14 Jul 2002 01:30:09



Tested:

ext3 data=ordered
ext3 data=writeback
reiserfs
reiserfs notail

http://www.gurulabs.com/ext3-reiserfs.html

Any suggestions or comments appreciated.

Dax Kelson
Guru Labs

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

Ext3 vs Reiserfs benchmarks

Post by Andreas Dilge » Sun, 14 Jul 2002 02:10:13



Quote:> ext3 data=ordered
> ext3 data=writeback
> reiserfs
> reiserfs notail

> http://www.gurulabs.com/ext3-reiserfs.html

> Any suggestions or comments appreciated.

Did you try data=journal mode on ext3?  For real-life workloads sync-IO
workloads like mail (e.g.  not benchmarks where the system is 100% busy)
you can have considerable performance benefits from doing the sync IO
directly to the journal instead of partly to the journal and partly to
the rest of the filesystem.

The reason why "real life" is important here is because the data=journal
mode writes all the files to disk twice - once to the journal and again
to the filesystem, so you must have some "slack" in your disk bandwidth
in order to benefit from this increased throughput on the part of the
mail transport.

Cheers, Andreas
--
Andreas Dilger
http://www-mddsp.enel.ucalgary.ca/People/adilger/
http://sourceforge.net/projects/ext2resize/

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

Ext3 vs Reiserfs benchmarks

Post by kwij.. » Sun, 14 Jul 2002 02:30:09


I compared reiserfs with notails and with tails to
ext3 in journaled mode about a month ago.
Strangely enough the machine that was being
built is eventually slated for a mail machine.  I used
postmark to simulate the mail environment.

Benchmarks are available here:
http://labs.zianet.com

Let me know if I am missing any info on there.

Steven



>>ext3 data=ordered
>>ext3 data=writeback
>>reiserfs
>>reiserfs notail

>>http://www.gurulabs.com/ext3-reiserfs.html

>>Any suggestions or comments appreciated.

>Did you try data=journal mode on ext3?  For real-life workloads sync-IO
>workloads like mail (e.g.  not benchmarks where the system is 100% busy)
>you can have considerable performance benefits from doing the sync IO
>directly to the journal instead of partly to the journal and partly to
>the rest of the filesystem.

>The reason why "real life" is important here is because the data=journal
>mode writes all the files to disk twice - once to the journal and again
>to the filesystem, so you must have some "slack" in your disk bandwidth
>in order to benefit from this increased throughput on the part of the
>mail transport.

>Cheers, Andreas
>--
>Andreas Dilger
>http://www-mddsp.enel.ucalgary.ca/People/adilger/
>http://sourceforge.net/projects/ext2resize/

>-
>To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

>More majordomo info at  http://vger.kernel.org/majordomo-info.html
>Please read the FAQ at  http://www.tux.org/lkml/

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/
 
 
 

Ext3 vs Reiserfs benchmarks

Post by Andreas Dilge » Sun, 14 Jul 2002 02:50:04



Quote:> I compared reiserfs with notails and with tails to
> ext3 in journaled mode about a month ago.
> Strangely enough the machine that was being
> built is eventually slated for a mail machine.  I used
> postmark to simulate the mail environment.

> Benchmarks are available here:
> http://labs.zianet.com

> Let me know if I am missing any info on there.

Yes, I saw this benchmark when it was first posted.  It isn't clear
from the web pages that you are using data=journal for ext3.  Note
that this is only a benefit for sync I/O workloads like mail and
NFS, but not other types of usage.  Also, for sync I/O workloads
you can get a big boost by using an external journal device.

Cheers, Andreas
--
Andreas Dilger
http://www-mddsp.enel.ucalgary.ca/People/adilger/
http://sourceforge.net/projects/ext2resize/

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

Ext3 vs Reiserfs benchmarks

Post by Chris Maso » Sun, 14 Jul 2002 05:40:07



> Tested:

> ext3 data=ordered
> ext3 data=writeback
> reiserfs
> reiserfs notail

> http://www.gurulabs.com/ext3-reiserfs.html

> Any suggestions or comments appreciated.

postmark is an interesting workload, but it does not do fsync or renames
on the working set, and postfix does lots of both while delivering.
postmark does do a good job of showing the difference between lots of
files in one directory (great for reiserfs) and lots of directories with
fewer files in each (better for ext3).

Andreas Dilger already mentioned -o data=journal on ext3, you can try
the beta reiserfs patches that add support for data=journal and
data=ordered at:

ftp.suse.com/pub/people/mason/patches/data-logging

They improve reiserfs performance for just about everything, but
data=journal is especially good for fsync/O_SYNC heavy workloads.

Andrew Morton sent me a benchmark of his that tries to simulate
postfix.  He has posted it to l-k before but a quick google search found
dead links only, so I'm attaching it.  What I like about his synctest is
the results are consistent and you can play with various
fsync/rename/unlink options.

-chris

  synctest.c
7K Download
 
 
 

Ext3 vs Reiserfs benchmarks

Post by Daniel Phillip » Sun, 14 Jul 2002 13:50:04



Quote:> Any suggestions or comments appreciated.

"it is clear that IF your server is stable and not prone to crashing, and/or
you have the write cache on your hard drives battery backed, you should
strongly consider using the writeback journaling mode of Ext3 versus ordered."

You probably want to suggest UPS there rather than battery backed disk
cache, since the writeback caching is pre*ly on the cpu side.

--
Daniel
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://www.veryComputer.com/
Please read the FAQ at  http://www.veryComputer.com/

 
 
 

Ext3 vs Reiserfs benchmarks

Post by Dax Kelso » Tue, 16 Jul 2002 05:50:10



> Any suggestions or comments appreciated.

Thanks for the feedback. Look for more testing from us soon addressing
the suggestions brought up.

Dax

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

Ext3 vs Reiserfs benchmarks

Post by Sam Vilai » Tue, 16 Jul 2002 17:30:18



> > Any suggestions or comments appreciated.
> Thanks for the feedback. Look for more testing from us soon addressing
> the suggestions brought up.

One more thing - can I just make the comment that testing freshly formatted filesystems is not going to show up ext2's real weaknesses, that happen to old filesystems - particularly those where the filesystem has been allowed to fill up.

I timed *15 minutes* for a system I admin to unlink a single 1G file on a fairly old ext2 filesystem the other day (perhaps ext3 would have improved this, I'm not sure).  It took 30 minutes to scan a snort log directory log on ext2, but less than 2 minutes on reiser - and only 3 seconds once it was in the buffercache.

You are testing for a mail server - how many mailboxes are in your spool directory for the tests?  Try it with about five to ten thousand mailboxes and see how your results vary.
--

    7D74 2A09 B2D3 C30F F78E      GPG: http://sam.vilain.net/sam.asc
    278A A425 30A9 05B5 2F13

Although Mr Chavez 'was democratically elected,' one had to bear in
mind that 'Legitimacy is something that is conferred not just by a
majority of the voters'"
 - The office of George "Dubya" Bush commenting on the Venezuelan
   election
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

Ext3 vs Reiserfs benchmarks

Post by Alan Co » Tue, 16 Jul 2002 20:30:11



> You are testing for a mail server - how many mailboxes are in your spool
> directory for the tests?  Try it with about five to ten thousand
> mailboxes and see how your results vary.

If your mail server can't get heirarchical mail spools right, get one
that can.

Alan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

Ext3 vs Reiserfs benchmarks

Post by Alan Co » Tue, 16 Jul 2002 21:20:08




>    "Yes, we know that there is no directory hashing in ext2/3.  You'll have
> to find another solution to the problem, I'm afraid.  Why not ease the
> burden on the filesystem by breaking up the task for it, and giving it
> to it in small pieces.  That way it's much less likely to choke."

Actually there are several other reasons for it. It sucks a lot less
when you need to use ls and friends to inspect part of the spool. It
also makes it much easier to split the mail spool over multiple disks as
it grows without having to backup/restore the spool area

Quote:> Sure, you could set up hierarchical mail spools.  But it sure stinks of a
> temporary solution for a long-term problem.  What about the next
> application that grows to massive proportions?

JFS ?

Quote:> Hey, while I've got your attention, how do you go about debugging your
> kernel?  I'm trying to add fair scheduling to the new O(1) scheduler,
> something of a token bucket filter counting jiffies used by a
> process/user/s_context (in scheduler_tick()) and tweaking their
> priority accordingly (in effective_prio()).  It'd be really nice if I
> could run it under UML or something like that so I can trace through
> it with gdb, but I couldn't get the UML patch to apply to your tree.
> Any hints?

The UML tree and my tree don't quite merge easily. Your best bet is to
grab the Red Hat Limbo beta packages for the kernel source, which if I
remember rightly are both -ac based and include the option to build UML

Alan

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

Ext3 vs Reiserfs benchmarks

Post by Matti Aarni » Tue, 16 Jul 2002 21:20:05




> > You are testing for a mail server - how many mailboxes are in your spool
> > directory for the tests?  Try it with about five to ten thousand
> > mailboxes and see how your results vary.

> If your mail server can't get heirarchical mail spools right, get one
> that can.

   Long ago (10-15 internet-years ago..) I followed testing of
   FFS-family of filesystems in Squid cache.

   We noticed at Solaris machines using UFS, than when the directory
   data size grew above the number of blocks directly addressable by
   the direct-index pointers in the i-node, system speed plummeted.
   (Or perhaps it was something a bit smaller, like 32 kB)

   Consider:  4 kB block size, 12 direct indexes: 48 kB directory size.

   Spend 16 bytes for each file name + auxiliary data: 3000 files/subdirs

   Optimal would be to store the files inside only the first block,
   e.g. the directory shall not grow over 4k (or 1k, or ..)

   Name subdirs as:  00 thru 7F (128+2, 12 bytes ?)
   Possibly do that in 2 layers:  128^2 = 16384 subdirs, each
   with 50 long named users (even more files?): 820 000 users.

   Tune the subdir hashing function to suit your application, and
   you should be happy.

   Putting all your eggs in one basket (files in one directory)
   is not a smart thing.

Quote:> Alan

/Matti Aarnio
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/
 
 
 

Ext3 vs Reiserfs benchmarks

Post by Sam Vilai » Tue, 16 Jul 2002 21:10:09



> > You are testing for a mail server - how many mailboxes are in your spool
> > directory for the tests?  Try it with about five to ten thousand
> > mailboxes and see how your results vary.
> If your mail server can't get heirarchical mail spools right, get one
> that can.

Translation

   "Yes, we know that there is no directory hashing in ext2/3.  You'll have to find another solution to the problem, I'm afraid.  Why not ease the burden on the filesystem by breaking up the task for it, and giving it to it in small pieces.  That way it's much less likely to choke."

 :-)

Sure, you could set up hierarchical mail spools.  But it sure stinks of a temporary solution for a long-term problem.  What about the next application that grows to massive proportions?

Hey, while I've got your attention, how do you go about debugging your kernel?  I'm trying to add fair scheduling to the new O(1) scheduler, something of a token bucket filter counting jiffies used by a process/user/s_context (in scheduler_tick()) and tweaking their priority accordingly (in effective_prio()).  It'd be really nice if I could run it under UML or something like that so I can trace through it with gdb, but I couldn't get the UML patch to apply to your tree.  Any hints?
--

    7D74 2A09 B2D3 C30F F78E      GPG: http://sam.vilain.net/sam.asc
    278A A425 30A9 05B5 2F13

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

Ext3 vs Reiserfs benchmarks

Post by Chris Maso » Tue, 16 Jul 2002 22:50:05





> >    "Yes, we know that there is no directory hashing in ext2/3.  You'll have
> > to find another solution to the problem, I'm afraid.  Why not ease the
> > burden on the filesystem by breaking up the task for it, and giving it
> > to it in small pieces.  That way it's much less likely to choke."

> Actually there are several other reasons for it. It sucks a lot less
> when you need to use ls and friends to inspect part of the spool. It
> also makes it much easier to split the mail spool over multiple disks as
> it grows without having to backup/restore the spool area

Another good reason is i_sem.  If you've got more than one process doing
something to that directory, you spend lots of time waiting for the
semaphore.  I think it was andrew that reminded me i_sem is held on
fsync, so fync(dir) to make things safe after a rename can slow things
down.

reiserfs only needs fsync(file), ext3 needs fsync(anything on the fs).
If ext3 would promise to make fsync(file) sufficient forever, it might
help the mta authors tune.

-chris

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

Ext3 vs Reiserfs benchmarks

Post by Andrea Arcangel » Wed, 17 Jul 2002 00:20:08



> Hey, while I've got your attention, how do you go about debugging your
> kernel?  I'm trying to add fair scheduling to the new O(1) scheduler,
> something of a token bucket filter counting jiffies used by a
> process/user/s_context (in scheduler_tick()) and tweaking their
> priority accordingly (in effective_prio()).  It'd be really nice if I
> could run it under UML or something like that so I can trace through
> it with gdb, but I couldn't get the UML patch to apply to your tree.
> Any hints?

-aa ships with both uml and o1 scheduler. I need uml for everything non
hardware related so expect it to be always uptodate there. However since
I merged the O(1) scheduler there is the annoyance that sometime wakeup
events don't arrive at least until kupdate reschedule or something
like that (of course only with uml, not with real hardware).  Also
pressing keys is enough to unblock it. I didn't debugged it hard yet.
Accoring to Jeff it's a problem with cli that masks signals.

Andrea
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

Ext3 vs Reiserfs benchmarks

Post by Patrick J. LoPrest » Wed, 17 Jul 2002 00:30:10


Consider this argument:

  Given: On ext3, fsync() of any file on a partition commits all
         outstanding transactions on that partition to the log.

  Given: data=ordered forces pending data writes for a file to happen
         before related transactions are committed to the log.

  Therefore: With data=ordered, fsync() of any file on a partition
             syncs the outstanding writes of EVERY file on that
             partition.

Is this argument correct?  If so, it suggests that data=ordered is
actually the *worst* possible journalling mode for a mail spool.

One other thing.  I think this statement is misleading:

    IF your server is stable and not prone to crashing, and/or you
    have the write cache on your hard drives battery backed, you
    should strongly consider using the writeback journaling mode of
    Ext3 versus ordered.

This makes it sound like data=writeback is somehow unsafe when
machines crash.  I do not think this is true.  If your application
(e.g., Postfix) is written correctly (which it is), so it calls
fsync() when it is supposed to, then data=writeback is *exactly* as
safe as any other journalling mode.  "Battery backed caches" and the
like have nothing to do with it.  And if your application is written
incorrectly, then other journalling modes will reduce but not
eliminate the chances for things to break catastrophically on a crash.

So if the partition is dedicated to correct applications, like a mail
spool is, then data=writeback is perfectly safe.  If it is faster,
too, then it really is a no-brainer.

 - Pat
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

1. Ext3 vs Reiserfs benchmarks (whither dump?)

Hi Folks,

As an old dump user (dumpster?) I have to admit that we've
avoided ext3 and Reiserfs because of this issue.  We couldn't
live without the "Tower of Hanoi".

I remember using, many years ago (SunOS 3.4), a patched
dump binary that allowed safe dumps from live UFS filesystems.
I don't remember all the details (it was 16-years ago) but
this dump would compare somehow, files before and after writing
to tape.  If there was a difference it would back out the
dumped file and preserve the consistency of the tape.  I don't
remember if it would go back and try the file again.

I haven't the foggest notion if this would work in these
modern times, I'm just offering it as food for thought.

Regards,
Lew Wolfgang

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

2. Reading is much more interesting than TV (0172/1704)

3. Filesystem benchmarks: ext2 vs ext3 vs jfs vs minix

4. Zip COnfig

5. Description: EIDE vs. RAID-SCSI Performance, ext3 vs. reiserfs

6. newbie question

7. ext3 vs reiserfs

8. Why can't I ping outside my segment SCO openserver

9. ReiserFS vs. EXT3 (most space)

10. ReiserFS VS Ext3

11. ext3 htree vs. reiserfs on 2.5.40-mm1

12. BENCHMARKS - SCO vs Solaris vs Unixware vs etc...

13. ext3 vs resiserfs vs xfs