Disk caching, thrashing, and the kernel

Disk caching, thrashing, and the kernel

Post by Jyrin » Wed, 31 Oct 2001 14:41:05



I'm toying with a 75-MHz Pentium with 32MB RAM and Red Hat Linux 7.1. I
would admit outright that my machine is underpowered, but I also work with a
very similar machine that works just darned fine, and the performance
difference is great enough that I think I may have a problem.

The issue appears to be with memory usage. Whenever I try to do something
big and slow and memory-intensive (such as start up Mozilla or compile the
Glasgow Haskell Compiler, to recall from recent memory), the poor thing
starts thrashing very badly. From another shell, I then check what "free"
tells me,
and it reports that, indeed, the system is very short on memory - but, in
fact, well over half the available memory is taken up by buffers and cache
(?!).

Is this typical? It seems awful wasteful to me, not to mention futile - a
disk cache large enough to induce thrashing would seem to defeat its own
purpose. Do I just have too darned little RAM, or is there something I can
tweak? (I swear, on that other machine I can surf the Web and compile the
kernel while an MP3 plays at good quality without skipping; on my box, just
surfing the Web brings everything down.)

Jyrinx
jyrinx_list at mindspring dot com

P.S. This newsgroup is rather busy ... is there a more specific group for
this question? Perhaps someone (with more time/expertise/equipment than me)
should start a comp.os.linux.kernel ...

 
 
 

Disk caching, thrashing, and the kernel

Post by Michael Lee Yoh » Thu, 01 Nov 2001 05:57:01


Quote:> and it reports that, indeed, the system is very short on memory - but, in
> fact, well over half the available memory is taken up by buffers and cache
> (?!).

> Is this typical? It seems awful wasteful to me, not to mention futile - a
> disk cache large enough to induce thrashing would seem to defeat its own

It's extremely useful on machine with much memory. And as such, in your
case - I'd recommend biting the bullet and buy more memory (since it's
_SO_ cheap).

Otherwise, you should use 2.4.10+ (2.4.13 is latest) LT-series kernel
which implements an improved 2.2 VM subsystem versus the Riel VM
subsystem introduced with 2.4.

--


Software Developer, Engineering Services
Red Hat, Inc.

QUIPd 0.20: (61 of 531)
-> If you are going to sin, sin against God, not the bureaucracy. God
-> will forgive you but the bureaucracy won't.
-> - Hyman Rickover

 
 
 

Disk caching, thrashing, and the kernel

Post by Jyrin » Thu, 01 Nov 2001 15:45:00


Well, call me stubborn, cheap, or poor, but I'd rather save the money for my
next computer ... :-)

I'll try getting the newest kernel. Thanks for your help!

Jyrinx
jyrinx_list at mindspring dot com



> > and it reports that, indeed, the system is very short on memory - but,
in
> > fact, well over half the available memory is taken up by buffers and
cache
> > (?!).

> > Is this typical? It seems awful wasteful to me, not to mention futile -
a
> > disk cache large enough to induce thrashing would seem to defeat its own

> It's extremely useful on machine with much memory. And as such, in your
> case - I'd recommend biting the bullet and buy more memory (since it's
> _SO_ cheap).

> Otherwise, you should use 2.4.10+ (2.4.13 is latest) LT-series kernel
> which implements an improved 2.2 VM subsystem versus the Riel VM
> subsystem introduced with 2.4.

> --


> Software Developer, Engineering Services
> Red Hat, Inc.

> QUIPd 0.20: (61 of 531)
> -> If you are going to sin, sin against God, not the bureaucracy. God
> -> will forgive you but the bureaucracy won't.
> -> - Hyman Rickover

 
 
 

Disk caching, thrashing, and the kernel

Post by Stewart Honsberg » Fri, 02 Nov 2001 00:40:36



>Well, call me stubborn, cheap, or poor, but I'd rather save the money for my
>next computer ... :-)

Why not upgrade your existing computer as the need arises, and make your
second computer out of the parts that are left over?

I'm still running the same computer I was running more than five years ago.
It's just had all of its guts (and its casing) replaced ... a few times. ;>

--


 
 
 

Disk caching, thrashing, and the kernel

Post by Mats Wichma » Sat, 03 Nov 2001 07:56:16


On Tue, 30 Oct 2001 14:57:01 -0600, Michael Lee Yohe

:> and it reports that, indeed, the system is very short on memory - but, in
:> fact, well over half the available memory is taken up by buffers and cache
:> (?!).
:>
:> Is this typical? It seems awful wasteful to me, not to mention futile - a
:> disk cache large enough to induce thrashing would seem to defeat its own
:
:It's extremely useful on machine with much memory. And as such, in your
:case - I'd recommend biting the bullet and buy more memory (since it's
:_SO_ cheap).

Mmmm... in a machine of that age, the memory is not likely to be as
cheap as the newer memory sticks are.
Mats Wichmann

 
 
 

Disk caching, thrashing, and the kernel

Post by cbbro.. » Sat, 03 Nov 2001 08:32:48



> On Tue, 30 Oct 2001 14:57:01 -0600, Michael Lee Yohe

> :> and it reports that, indeed, the system is very short on memory -
> :> but, in fact, well over half the available memory is taken up by
> :> buffers and cache (?!).
> :> Is this typical? It seems awful wasteful to me, not to mention
> :> futile - a disk cache large enough to induce thrashing would seem
> :> to defeat its own
> :It's extremely useful on machine with much memory. And as such, in your
> :case - I'd recommend biting the bullet and buy more memory (since it's
> :_SO_ cheap).
> Mmmm... in a machine of that age, the memory is not likely to be as
> cheap as the newer memory sticks are.  Mats Wichmann

But regardless, the _point_ is that if the system has spare memory
kicking around, it might as well be used to cache stuff.

It's quite typical for a system that has been running for a while to
have _no_ unused memory; memory that's not being used for _anything_
is just sitting there consuming electricity.  If there's RAM in use as
cache, then obviously the system _does_ have enough memory for what
it's being used for, because the system will throw cache away before
it allocates swap space.

The only way you're likely to have both (memory free|used as cache)
and (swap consumed) is if some program wound up consuming enough RAM
to fill memory (reducing cache to 0), and swapped part of either
data/program out to swap space.  Subsequently, other memory got
deallocated, and since the swapped-out data has never been pulled in,
it just gets left there...

Long and short:

  It's normal _and good_ for the amount of free memory to get drawn to
  zero by using it to cache disk accesses.
--

http://www.ntlug.org/~cbbrowne/unix.html
"Laugh-a while you can, Monkey Boy."  -- Dr. Lizardo - Buckaroo Banzai

 
 
 

1. Disk I/O to kernel memory, but NOT buffer cache

I'm trying to implement a module in which I do disk I/O to kernel
memory, but without using the buffer cache.

I've played around with using getblk to setup a buffer_head, and
then tweaking the b_data field before passing it into ll_rw_block,
but this doesn't work (as I'd no doubt have realized if I'd looked
into the memory map related fields before I tried it!!)

Can anyone help me with a way to do this?? I'm working with the 2.2.5
kernel at the moment, but could switch to any other version if this
can work.

Thanks,

Jon Flower

Sent via Deja.com http://www.deja.com/
Before you buy.

2. Software Program Error

3. How can I disable disk caching altogether on kernel 2.4?

4. X-newbie(sorta)

5. Problem in IDE Disks cache handling in kernel 2.4.XX

6. Changing nic settings

7. Disk cache kernel problem or deficiant hardware?

8. XFree3.2 support for TGUI9420Dgi

9. linux kernel unable to identify harddisk disk cache

10. How Inode cache and Directory Cache are maintained By VFS of Linux Kernel

11. 2.4.7-ac4 disk thrashing

12. Disk Thrashing

13. Solaris disk thrashing