disk throughput

disk throughput

Post by Linus Torvald » Wed, 07 Nov 2001 09:10:12




> A five-time slowdown on real work _is_ pure hell. You've not shown a
> credible argument that the slow-growth behaviour would ever result in a
> five-time slowdown for _anything_.

There might also be heuristics that explicitly _notice_ slow growth, not
necessarily as a function of time, but as a function of the tree structure
itself.

For example, spreading out (and the inherent assumption of "slow growth")
might make sense for the root directory, and possibly for a level below
that. It almost certainly does _not_ make sense for a directory created
four levels down.

                Linus

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

disk throughput

Post by Linus Torvald » Wed, 07 Nov 2001 13:10:08



> OK, some digging had brought another one:

> a) if it's first-level directory - get it the fsck out of root's cylinder
> group.

Hey, now that you've read it in a paper you like it, but when I suggest it
in email you shoot it down?

<Whiny mode on>  I thought you loved me, Al.  <Whiny mode off>

Quote:> b) if we just keep creating directories in a cylinder group and do not
> create any files there - stop, it's no good (i.e. there's a limit on
> number of back-to-back directory creations in the same group).

The current code actually has some vestiges that _seem_ to be trying to do
something like this: see the commented-out

        if (tmp && le16_to_cpu(tmp->bg_used_dirs_count) << 8) <
                   le16_to_cpu(tmp->bg_free_inodes_count)) {

which _seems_ to want to play games with "number of directories allocated
vs nr of free inodes".

But it's commented out with "I am not yet convinced that this next bit is
necessary". I don't know if the code has ever been active, or whether it
showed other problems.

Quote:> c) try putting it into the parent's CG, but reserve some number of inodes
> and data blocks in it.  If we can't - tough, get the fsck out of there.

Hmm.. Maybe this is actually closer to what we try to do above..

                Linus

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

disk throughput

Post by Alexander Vir » Wed, 07 Nov 2001 17:40:13



> Surely the answer if you want short term write speed and long term balancing
> is to use ext3 not ext2 and simply ensure that the relevant stuff goes to
> the journal (which will be nicely ordered) first. That will give you some
> buffering at least.

Alan, the problem is present in ext3 as well as in all other FFS derivatives
(well, FreeBSD had tried to deal that stuff this Spring).

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

disk throughput

Post by Wojtek Pilor » Wed, 07 Nov 2001 18:20:13


[...]

Quote:

> find(1).  Stuff that runs every damn night.  We end up doing getdents() +
> bunch of lstat() on the entries we had found.  And that means access to
> inodes refered from them.  Try that on a large directory with child
> inodes spread all over the disk.

[...]

Part of the problem is that, as far as I know, there is no sane way to
request the kernel to execute a number of lstat-s or similar calls
in the order that would be convenient to the system (I do not consider
creating threads for this purpose a sane way).
If a program (say find, or tar, or anything) needs an information from
5000 lstats of fstats or reads, it has to specify them one-by-one in some
order, without knowledge which order would be best.
If we had a call to execute a vector of lstats, or open, or reads (from
different handles), program which would use such calls would allow kernel
to take decision what is the best order or individual operations.

I remember that using similar 'vector' functions in old days on IBM OS/MVT
gave dramatical performance improvements (maybe for different
reasons; machine memories were often half a megabyte, so opening a file
required the system to read and execute several modules loaded from system
disks; when opening 10 files each of the modules had to be loaded once
rather than 10 times).

Would it be possible and a good idea to add such 'vector' operations to
the Linux kernel?

Best regards,

Wojtek
--------------------
Wojtek Pilorz

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

disk throughput

Post by Stephen Tweedi » Thu, 08 Nov 2001 07:30:16


Hi,


> Another heuristic to try make be to only use a different blockgroup
> for when the mkdir()s are seperate in time.

If I'm creating a hierarchy with "tar xvzf", why am I doing it?  I
might be unpacking a source tree where all the files are going to be
used together.  But I might be restoring a backup of /home.  Assuming
that we want a compact unpacking is not always going to be correct.

Cheers,
 Stephen
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

1. Disk Throughput

How do I determine disk throughput, adapter throughput and stuff.

Actually I know by experience that for SSA you can reach a tpf of 150
while on SCSI you can reach a tps of 100. I want to validate it.
Now lets say that my adapter has some throughput how can I validate
that.

Arshad.

Sent via Deja.com http://www.deja.com/
Before you buy.

2. fewer_hairs=( Pump || NetConf_problem_in_RH6.2 ) +telnet question;

3. Interesting disk throughput performance problem

4. Automatic filename expansion - passing paramaters (file*) to script

5. disk throughput down in 2.5.8

6. PCNFS & IBM ethernet card?

7. hard disk throughput: important factors?

8. How to recongfig network for RH6?

9. Checking Disk throughput

10. disk throughput

11. Measuring disk throughput

12. Disk Throughput Capacity Measures