Dynamic Disk Caching

Dynamic Disk Caching

Post by Cliff Lass » Sat, 29 Apr 1995 04:00:00



   From: Kyler Jones <redman>
   Date: Fri, 28 Apr 1995 01:19:06 GMT

   I'm very concerned about the way Linux handles disk caching.  I
   have 16 megs of RAM.  The problem is when I load a largish program
   (such as Netscape under X) and then exit the prgram, most of my RAM
   has been claimed by buffers.  If I then run a new largish program
   (such as xdoom) the buffers aren't properly returning memory for
   application use.  What's more, sometimes the system will go into a
   seemingly endless loop of disk activity.

   Is there some way to flush buffers on command or even disable
   dynamic disk caching??

I seem to see the exact opposite, which is a problem for my group of
Linux users: Memory consumed by running processes never gets swapped
out if does not exceed the amount of RAM in the box.  This causes the
buffers to get progressively smaller which has a very bad effect on
compiling of large source trees.

In my case, my system has 3 C++ hackers using Emacs and X, and before
you know it most of our 32MB of RAM is all taken up and compiles start
thrashing.  To reduce this problem, every so often I run a little
program that allocates and zeroes up to close to 32MB (in stages).
This causes everything to get swapped out and only what is actually
being used gets swapped back in.  This is painful for about 30 seconds
but then system performance improves dramatically.  The next obvious
thing to do is to upgrade to 64MB!

It seems to me that main memory pages consumed by existing processes
should be aged and swapped out on a continuous basis (up to a certain
point) so that the buffers can grow to a sysadmin set-able amount.
Does Linux have anything to help in this area?

 
 
 

Dynamic Disk Caching

Post by Kyler Jone » Sat, 29 Apr 1995 04:00:00


I'm very concerned about the way Linux handles disk caching.  I have 16 megs of
RAM.  The problem is when I load a largish program (such as Netscape under X)
and then exit the prgram, most of my RAM has been claimed by buffers.  If I
then run a new largish program (such as xdoom) the buffers aren't properly
returning memory for application use.  What's more, sometimes the system will
go into a seemingly endless loop of disk activity.  

Is there some way to flush buffers on command or even disable dynamic disk
caching??

Thanks in advance.



 
 
 

Dynamic Disk Caching

Post by Bob Hau » Sat, 29 Apr 1995 04:00:00



Quote:>I'm very concerned about the way Linux handles disk caching.  I have 16 megs of
>RAM.  The problem is when I load a largish program (such as Netscape under X)
>and then exit the prgram, most of my RAM has been claimed by buffers.  If I
>then run a new largish program (such as xdoom) the buffers aren't properly
>returning memory for application use.  What's more, sometimes the system will
>go into a seemingly endless loop of disk activity.  

You didn't say what kernel you're running.  There were some
problems relative to swapping and buffers with early 1.2.x
versions (not sure about 1.1.x, but I think the problem appeared
sometime after 1.1.90 when the swap algorithms were changed).

I believe the problems were fixed in 1.2.2 and higher.  I'm
running 1.2.6 now and it seems to be fine when running Netscape
and other "largish" programs.

---
 Bob Hauck                                 Wasatch Communications Group

 
 
 

Dynamic Disk Caching

Post by Henry Wert » Sun, 30 Apr 1995 04:00:00



writes:
Quote:>   From: Kyler Jones <redman>
>   Date: Fri, 28 Apr 1995 01:19:06 GMT

(i cut all your stuff 8-)
     Try update --help.. there is *something* you can do from there to adjust
the swapping and the caching.. actually there is quite a bit.. I haven't
adjusted it since it runs decently on my system, but.. there is a lot there
*to* adjust though.

---

for this .sig 8-)
It's reported that Canter & Siegel search for and archive all articles
that contain their names or "Green Card".  This .sig is to help them.  

 
 
 

Dynamic Disk Caching

Post by Kyler Jon » Mon, 01 May 1995 04:00:00



: >I'm very concerned about the way Linux handles disk caching.  I have 16 megs of
: >RAM.  The problem is when I load a largish program (such as Netscape under X)
: >and then exit the prgram, most of my RAM has been claimed by buffers.  If I
: >then run a new largish program (such as xdoom) the buffers aren't properly
: >returning memory for application use.  What's more, sometimes the system will
: >go into a seemingly endless loop of disk activity.  

: You didn't say what kernel you're running.  There were some
: problems relative to swapping and buffers with early 1.2.x
: versions (not sure about 1.1.x, but I think the problem appeared
: sometime after 1.1.90 when the swap algorithms were changed).

: I believe the problems were fixed in 1.2.2 and higher.  I'm
: running 1.2.6 now and it seems to be fine when running Netscape
: and other "largish" programs.

: ---
:  Bob Hauck                                 Wasatch Communications Group

Indeed I was using Linux v1.2.1, I have now updated to v1.2.6.  I'm not sure if the
problem is fixed or not, however, I have recently installed a 12 MB swap partition
and I am getting much better performance. Thanks for the help.

I am still interested, however, in being able to temporarily disable disk
caching.  The reason is that i don't believe that wine and dosemu are able
to free memory used by buffers and I think this is a source of many crashes
with these packages.

Thanks


 
 
 

Dynamic Disk Caching

Post by roo » Mon, 01 May 1995 04:00:00



: >I'm very concerned about the way Linux handles disk caching.  I have 16 megs of
: >RAM.  The problem is when I load a largish program (such as Netscape under X)
: >and then exit the prgram, most of my RAM has been claimed by buffers.  If I
: >then run a new largish program (such as xdoom) the buffers aren't properly
: >returning memory for application use.  What's more, sometimes the system will
: >go into a seemingly endless loop of disk activity.  

: You didn't say what kernel you're running.  There were some
: problems relative to swapping and buffers with early 1.2.x
: versions (not sure about 1.1.x, but I think the problem appeared
: sometime after 1.1.90 when the swap algorithms were changed).

: I believe the problems were fixed in 1.2.2 and higher.  I'm
: running 1.2.6 now and it seems to be fine when running Netscape
: and other "largish" programs.

: ---
:  Bob Hauck                                 Wasatch Communications Group

 
 
 

1. Dynamic Disk Caching

Alright, more of the topic that I WON'T LET DIE!.  I just can't seem to get
the point of the way Linux uses it's disk caching system.  

I am convinced now that my system is running properly, I've tried kernels
1.21, 1.25, 1.26 and find no difference in the way the disk caching works.

As I write this, my buffers total almost 13 MB.  I have 20 MB total and a
12 MB swap partition.  What performance gain is possible with such a large
cahe, if the system has to search such a large cache every time it reads
from the disk?  What's more, every time I run a program, it has to free memory
from the buffers and write this back to disk.  Am I missing the point?
Is this a feature which people find desirable?.  There must be a way to
set a maximum on the amount of RAM the system will use for disk caching.

  Conan, just before him and Subitai take on Thulsa Doom and his army:

        "And if you don't listen, then to Hell with you"


2. in.named MEMDEBUG OVERFLOW

3. Dynamic Disk Cache

4. LATEST Version out!

5. Web cache: how to force caching dynamic pages?

6. Pb installing linux on ibm mca with RAID

7. No users can login

8. Disc caching problems with shared SCSI disc

9. Pragma "no-cache" and Cache-Control "no-cache"

10. Dynamic caching? Is this possible?

11. serving dynamic documents with Apache - Pragma: no-cache, Last-Modified, etc.

12. disable cache, change text modes, dynamic redirection